Resizing really high res images hangs Drupal

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
k6pski
Posts: 5
Joined: 2018-05-28T12:01:11-07:00
Authentication code: 1152

Resizing really high res images hangs Drupal

Post by k6pski »

Hi!

I'm co-maintaining this Drupal module https://www.drupal.org/project/batch_resize_image and I'm having trouble getting all of our 10 000+ images resized. In the background it's using the ImageMagic module https://www.drupal.org/project/imagemagick

At some point the script will just hang. I think I've now come to conclusion that it fails with really high resolution images like this:

Code: Select all

identify das_str_2.png
das_str_2.png PNG 16535x8184 16535x8184+0+0 8-bit DirectClass 4.666MB 0.000u 0:00.000
Locally on my Macbook Pro it seems ok but the images need to eventually resized on Amazon Beanstalk instance and it has little bit less resources. So while the php stops at the big images and just doesn't do anything then manually running the command takes almost 4 minutes but finishes:

Code: Select all

time convert das_str_2.png -resize 5000x2475! -quality 75 das_str_2-2.png

real	3m40.560s
user	0m20.804s
sys	0m1.184s
The version of convert is:

Code: Select all

convert --version
Version: ImageMagick 6.7.8-9 2016-06-22 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP
There's about 2GB ram on the system.

This is the image I'm talking about https://www.upload.ee/image/8493734/das_str_2.png

I'm running out of ideas. How should I solve this? Is it problem from php side if the convert runs ok on bash?

Thank you!

edit:
I've made error in my calculation as the image is converted on a network drive mount.

Code: Select all

time convert das_str_2.png -resize 5000x2475! -quality 75 das_str_2-2.png

real	5m3.598s
user	0m21.407s
sys	0m1.307s
Last edited by k6pski on 2018-05-28T12:51:40-07:00, edited 1 time in total.
Bonzo
Posts: 2971
Joined: 2006-05-20T08:08:19-07:00
Location: Cambridge, England

Re: Resizing really high res images hangs Drupal

Post by Bonzo »

php has a standard time limit of 30 seconds. Normally you can adjust it in the php.ini file but that may not be possible on Amazon.

I assume as Imagemagick is running you must have "Safe mode" disabled so you could look at ini_set() and set_time_limit() but these options may be disabled by Amazon.
k6pski
Posts: 5
Joined: 2018-05-28T12:01:11-07:00
Authentication code: 1152

Re: Resizing really high res images hangs Drupal

Post by k6pski »

I can change the settings but shouldn't it just die with timeout error then? I'll check tomorrow.
Also the image convert takes longer as I didn't run the convert on the network mount as it should. It takes 5 minutes to convert that image.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Resizing really high res images hangs Drupal

Post by fmw42 »

Check your tmp directory to be sure it is not filling up or filled up with left over imagemagick files.
k6pski
Posts: 5
Joined: 2018-05-28T12:01:11-07:00
Authentication code: 1152

Re: Resizing really high res images hangs Drupal

Post by k6pski »

I just killed the instance with this 12 000 x 12 000 image https://wallportal.com/uploads/posts/ea ... on_026.jpg

With this 12858x6534 image it took only 12 seconds to resize http://hanedanrpg.com/photos/hanedanrpg/12/55931.jpg

And my problematic image linked in the first post which is 16535x8184 takes 3-5 minutes to resize.

Code: Select all

12000x12000 = 144000000 killed server
12858x6534  =  84014172 resize in 12 seconds
16535x8184  = 135322440 resize in 3 minutes
Server doesn't have swap.

edit: sry, it didn't kill the server:

Code: Select all

time convert earth_wallpaper_high_resolution_026.jpg -resize 5000x2475! -quality 75 earth_wallpaper_high_resolution_026-2.jpg
Killed

real	24m49.255s
user	0m13.913s
sys	0m0.767s
k6pski
Posts: 5
Joined: 2018-05-28T12:01:11-07:00
Authentication code: 1152

Re: Resizing really high res images hangs Drupal

Post by k6pski »

fmw42 wrote: 2018-05-28T13:20:42-07:00 Check your tmp directory to be sure it is not filling up or filled up with left over imagemagick files.
Thre's one tmp file

Code: Select all

ls -lah /tmp/magick-*
-rw------- 1 root root 95M May 29 05:33 /tmp/magick-hZidmMde
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Resizing really high res images hangs Drupal

Post by fmw42 »

You have a 95 MB temp file there from ImageMagick. You can delete it to save space.
k6pski
Posts: 5
Joined: 2018-05-28T12:01:11-07:00
Authentication code: 1152

Re: Resizing really high res images hangs Drupal

Post by k6pski »

Thank you!
I think I also found the problem. When we increased the ram to 4GB the problematic image was resized within 12 seconds instead of 3-5 minutes. Will do more tests.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Resizing really high res images hangs Drupal

Post by fmw42 »

You cannot upload images to this forum. Please upload to some free hosting service and put the URL(s) here. Also please always provide your ImageMagick version and platform and your command line or code.

Often resizing an image to smaller dimensions creates large file sizes when the input is type palette (8-bit per pixel). The resize adds new colors that are more that 256 colors. Thus the output file size must be 24-bit or 32-bit per pixel. So you have to reduce colors again. You may need to use +dither -colors 256 and/or add -type palette or if PNG add PNG8: to your output image as PNG8:ouput.png
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Resizing really high res images hangs Drupal

Post by snibgo »

[I've removed spam from the thread.]
snibgo's IM pages: im.snibgo.com
Post Reply