Memory Allocation Error

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
matthew
Posts: 4
Joined: 2013-01-03T07:53:10-07:00
Authentication code: 6789

Memory Allocation Error

Post by matthew »

I have a number of TIF files that I am trying to convert to Group4 compressed TIF files. My processed worked fine on the smaller files but on the larger (~1GB) files I keep getting the following memory allocation error:
convert.exe: Memory allocation failed `f:\images\12015_52737_202124.TIF' @ error
/quantize.c/QuantizeImage/2726.
The easy answer is that I'm just running out of memory but I'm running this on a machine with 32GB of memory and when I monitor the memory during execution it never seems to request more than a few GB. It also uses quite a bit of disk space but I have over 300GB free on the partition for the temporary files and it does not seem to be filling up.

Code: Select all

Version: ImageMagick 6.8.0-4 2012-10-30 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP

Code: Select all

File         Area       Memory          Map         Disk    Thread         Time
-------------------------------------------------------------------------------
1536     34.355GB    15.998GiB    31.996GiB    unlimited         8    unlimited
I've read http://www.imagemagick.org/Usage/files/#massive and have tried playing with the limits but I haven't had any luck.

Any help or insight would be much appreciated.
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: Memory Allocation Error

Post by magick »

If you can grab ImageMagick 6.8.1-8, the latest release, that may help. There are some patches that address large memory / files. Now try this command:
  • convert -limit memory 0 -limit map 0 image.tif -compress group4 new.tif
If that fails, let us know and we have a couple more recommendations.
matthew
Posts: 4
Joined: 2013-01-03T07:53:10-07:00
Authentication code: 6789

Re: Memory Allocation Error

Post by matthew »

magick wrote:-limit memory 0 -limit map 0
This seems to have helped, however now some of them are causing me to run out of disk space.
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: Memory Allocation Error

Post by magick »

Set your temporary disk path to a partition with plenty of free space with the MAGICK_TMPDIR environment variable or the -define registry:temporary-path=/data/tmp command line option.
matthew
Posts: 4
Joined: 2013-01-03T07:53:10-07:00
Authentication code: 6789

Re: Memory Allocation Error

Post by matthew »

How much disk space is "plenty"? It filled up a partition with 300+ GB free.
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: Memory Allocation Error

Post by magick »

The pixel cache grows as required by the image or image sequence. For 300 GB you would need a 50000 by 50000 pixel image or larger or perhaps you have an image sequence with modest size frames but hundreds of them. Or perhaps you have a corrupt image that incorrectly returns a huge image size. Post a URL to your image and we will download it and try to reproduce the problem.
matthew
Posts: 4
Joined: 2013-01-03T07:53:10-07:00
Authentication code: 6789

Re: Memory Allocation Error

Post by matthew »

I am not able to post the images because they contain sensitive information. They vary but as an example one is 1764x2248 grayscale with 179 pages; another is 1721x2216 grayscale with 441 pages. It may partly be because it doesn't always clear the cache after it finishes processing a given image.
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: Memory Allocation Error

Post by magick »

ImageMagick opens each of the frames in an image sequence. With that many frames, 300GB of disk seems reasonable. If that's a problem you can read just a frame or two at a time, e.g. convert 'image.tif[3-5]'.
Post Reply