File size / resolution limits for MagickReadImage?

Questions and postings pertaining to the development of ImageMagick, feature enhancements, and ImageMagick internals. ImageMagick source code and algorithms are discussed here. Usage questions which are too arcane for the normal user list should also be posted here.
168gr
Posts: 47
Joined: 2013-01-09T22:46:19-07:00
Authentication code: 6789

File size / resolution limits for MagickReadImage?

Post by 168gr »

Is there a size limit to the images MagickReadImage() can load? Some setting I'm missing somewhere?

I have a JPG image that is 3264 x 2448 (2.34 MB) and MagickReadImage() won't load it. It complains:

Code: Select all

test3.c main 22 Empty input file `input.jpg' @ error/jpeg.c/JPEGErrorHandler/318
If I resize it to 1024 x 768 with mogrify, it still won't load.

If I resize it to 800 x 600 with mogrify, it will load.


Here's a simple test program and output illustrating the problem:

Code: Select all

pgg@g550 ~/test
$ ls -l
total 2404
-rw-r--r-- 1 pgg None 2454865 Feb 21 22:13 input.jpg
-rwxr-xr-x 1 pgg None     888 Feb 21 22:13 test3.c

pgg@g550 ~/test
$ cat test3.c
#include <stdio.h>
#include <wand/magick_wand.h>

#define ThrowWandException(wand) \
{ \
        char *description; \
        ExceptionType severity; \
        description=MagickGetException(wand,&severity); \
        printf("\n\n-----\n%s %s %lu %s\n",GetMagickModule(),description); \
        description=(char *) MagickRelinquishMemory(description); \
        exit(-1); \
}

int main() {

  MagickWand *mw1 = NULL;

  MagickWandGenesis();
  mw1 = NewMagickWand();

  if( !MagickReadImage( mw1, "input.jpg" ) )
    ThrowWandException( mw1 );

  MagickWriteImage( mw1, "output.jpg" );

  mw1 = DestroyMagickWand(mw1);

  MagickWandTerminus();

  return 0;
}

/*
gcc test3.c -o test3 -lMagickWand-Q16 -lgdi32 -L/home/pgg/ImageMagick-6.8.1/lib -I/home/pgg/ImageMagick-6.8.1/include/ImageMagick -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16
*/

pgg@g550 ~/test
$ gcc test3.c -o test3 -lMagickWand-Q16 -lgdi32 -L/home/pgg/ImageMagick-6.8.1/lib -I/home/pgg/ImageMagick-6.8.1/include/ImageMagick -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16

pgg@g550 ~/test
$ ./test3.exe


-----
test3.c main 22 Empty input file `input.jpg' @ error/jpeg.c/JPEGErrorHandler/318

pgg@g550 ~/test
$ mogrify -geometry 1024x768 input.jpg

pgg@g550 ~/test
$ ./test3.exe


-----
test3.c main 22 Empty input file `input.jpg' @ error/jpeg.c/JPEGErrorHandler/318

pgg@g550 ~/test
$ mogrify -geometry 800x600 input.jpg

pgg@g550 ~/test
$ ./test3.exe

pgg@g550 ~/test
$ ls -l
total 488
-rw-r--r-- 1 pgg None 220768 Feb 21 22:14 input.jpg
-rw-r--r-- 1 pgg None 220870 Feb 21 22:14 output.jpg
-rwxr-xr-x 1 pgg None    888 Feb 21 22:13 test3.c
-rwxr-xr-x 1 pgg None  52766 Feb 21 22:14 test3.exe
el_supremo
Posts: 1015
Joined: 2005-03-21T21:16:57-07:00

Re: File size / resolution limits for MagickReadImage?

Post by el_supremo »

ImageMagick can handle images that are much larger than 3264 x 2448. There may be something wrong with the original image.
Do an identify -verbose on it and see if either "identify" complains about the file or something odd shows up in the output.
Failing that, you may have to post the image so that others can try it.

Pete
Sorry, my ISP shutdown all personal webspace so my MagickWand Examples in C is offline.
See my message in this topic for a link to a zip of all the files.
168gr
Posts: 47
Joined: 2013-01-09T22:46:19-07:00
Authentication code: 6789

Re: File size / resolution limits for MagickReadImage?

Post by 168gr »

Thanks. It fails with all images taken with my iPhone 5.

Here's one I took of the transient quarters of the Air Force base in sunny but freakishly cold Manas, Kyrgyzstan. Same problem/error with MagickReadImage.

Code: Select all

$ identify -verbose input.jpg
Image: input.jpg
  Format: JPEG (Joint Photographic Experts Group JFIF format)
  Class: DirectClass
  Geometry: 3264x2448+0+0
  Resolution: 72x72
  Print size: 45.3333x34
  Units: PixelsPerInch
  Type: TrueColor
  Endianess: Undefined
  Colorspace: sRGB
  Depth: 8-bit
  Channel depth:
    red: 8-bit
    green: 8-bit
    blue: 8-bit
  Channel statistics:
    Red:
      min: 0 (0)
      max: 255 (1)
      mean: 138.107 (0.541598)
      standard deviation: 59.0705 (0.231649)
      kurtosis: -0.918534
      skewness: 0.275063
    Green:
      min: 0 (0)
      max: 255 (1)
      mean: 148.689 (0.583094)
      standard deviation: 53.773 (0.210874)
      kurtosis: -0.534919
      skewness: -0.0280879
    Blue:
      min: 0 (0)
      max: 255 (1)
      mean: 165.899 (0.650583)
      standard deviation: 53.9804 (0.211688)
      kurtosis: -0.34041
      skewness: -0.670298
  Image statistics:
    Overall:
      min: 0 (0)
      max: 255 (1)
      mean: 150.898 (0.591758)
      standard deviation: 55.6619 (0.218282)
      kurtosis: -0.677985
      skewness: -0.142582
  Rendering intent: Perceptual
  Gamma: 0.45455
  Chromaticity:
    red primary: (0.64,0.33)
    green primary: (0.3,0.6)
    blue primary: (0.15,0.06)
    white point: (0.3127,0.329)
  Interlace: None
  Background color: white
  Border color: srgb(223,223,223)
  Matte color: grey74
  Transparent color: black
  Compose: Over
  Page geometry: 3264x2448+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: JPEG
  Quality: 96
  Orientation: TopLeft
  Properties:
    date:create: 2013-02-23T12:06:22+05:00
    date:modify: 2013-02-23T12:06:22+05:00
    exif:ApertureValue: 4845/1918
    exif:BrightnessValue: 12166/1105
    exif:ColorSpace: 1
    exif:ComponentsConfiguration: 1, 2, 3, 0
    exif:Compression: 6
    exif:DateTime: 2012:12:26 12:33:19
    exif:DateTimeDigitized: 2012:12:26 12:33:19
    exif:DateTimeOriginal: 2012:12:26 12:33:19
    exif:ExifImageLength: 2448
    exif:ExifImageWidth: 3264
    exif:ExifOffset: 192
    exif:ExifVersion: 48, 50, 50, 49
    exif:ExposureMode: 0
    exif:ExposureProgram: 2
    exif:ExposureTime: 1/3390
    exif:Flash: 24
    exif:FlashPixVersion: 48, 49, 48, 48
    exif:FNumber: 12/5
    exif:FocalLength: 413/100
    exif:FocalLengthIn35mmFilm: 33
    exif:ISOSpeedRatings: 50
    exif:JPEGInterchangeFormat: 676
    exif:JPEGInterchangeFormatLength: 9595
    exif:Make: Apple
    exif:MeteringMode: 5
    exif:Model: iPhone 5
    exif:Orientation: 1
    exif:ResolutionUnit: 2
    exif:SceneCaptureType: 0
    exif:SensingMethod: 2
    exif:ShutterSpeedValue: 7775/663
    exif:Software: 6.0.1
    exif:SubjectArea: 1631, 1223, 881, 881
    exif:WhiteBalance: 0
    exif:XResolution: 72/1
    exif:YCbCrPositioning: 1
    exif:YResolution: 72/1
    jpeg:colorspace: 2
    jpeg:sampling-factor: 2x2,1x1,1x1
    signature: 64c0ef4471419eec082278d92fd43142cf3b65a864a41b960ccbd9e9991a8464
  Profiles:
    Profile-exif: 12284 bytes
  Artifacts:
    filename: input.jpg
    verbose: true
  Tainted: False
  Filesize: 3.399MB
  Number pixels: 7.99M
  Pixels per second: 15.55MB
  User time: 0.499u
  Elapsed time: 0:01.514
  Version: ImageMagick 6.7.6-3 2012-04-28 Q16 http://www.imagemagick.org
Link to large image (didn't want to inline a 3.4 MB image in the thread):

http://www.pggweb.com/bbimg/input.jpg
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: File size / resolution limits for MagickReadImage?

Post by magick »

iPhone's resources are constained. Add
  • setenv("MAGICK_DEBUG","cache",1);
just before MagickWandGenesis() and post the output to this forum. If you have a shell on your iPhone run this command and post the results:
  • convert -list resource
168gr
Posts: 47
Joined: 2013-01-09T22:46:19-07:00
Authentication code: 6789

Re: File size / resolution limits for MagickReadImage?

Post by 168gr »

magick wrote:iPhone's resources are constained. Add
  • setenv("MAGICK_DEBUG","cache",1);
just before MagickWandGenesis() and post the output to this forum.

Code: Select all

2013-02-23T21:48:31+04:00 0:00.312 0.062u 6.8.1 Cache test3[5060]: cache.c/DestroyPixelCache/1440/Cache
  destroy
2013-02-23T21:48:31+04:00 0:00.327 0.078u 6.8.1 Cache test3[5060]: cache.c/DestroyPixelCache/1440/Cache
  destroy


-----
test3.c main 23 Empty input file `input.jpg' @ error/jpeg.c/JPEGErrorHandler/318
magick wrote:If you have a shell on your iPhone run this command and post the results:
  • convert -list resource
Don't have a shell on the phone. If the above debug info isn't enough, and you think that would be helpful information, I could try to figure out how to get one.
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: File size / resolution limits for MagickReadImage?

Post by magick »

ImageMagick depends on external delegate libraries for certain image formats such as JPEG. It is likely that the JPEG delegate library is failing and reporting an exception to ImageMagick. We did not write, nor do we support the JPEG library. You can reduce the memory requirements with scaling. Its not useful if you want the final image size to match the original but for now, let's see if its a memory resource issue. Add
  • MagickSetOption(mw1,"jpeg:size","512x512");
right after NewMagickWand(). Does the conversion work now?
168gr
Posts: 47
Joined: 2013-01-09T22:46:19-07:00
Authentication code: 6789

Re: File size / resolution limits for MagickReadImage?

Post by 168gr »

magick wrote:ImageMagick depends on external delegate libraries for certain image formats such as JPEG. It is likely that the JPEG delegate library is failing and reporting an exception to ImageMagick. We did not write, nor do we support the JPEG library. You can reduce the memory requirements with scaling. Its not useful if you want the final image size to match the original but for now, let's see if its a memory resource issue. Add
  • MagickSetOption(mw1,"jpeg:size","512x512");
right after NewMagickWand(). Does the conversion work now?
No, same output as above.
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: File size / resolution limits for MagickReadImage?

Post by magick »

Ok, let's see if the problem is the JPEG delegate library. On your computer, convert the JPEG to PNG:
  • convert input.jpg input.png
Now modify your MagickWand program to read input.png. Does the program successfully read the PNG image?
168gr
Posts: 47
Joined: 2013-01-09T22:46:19-07:00
Authentication code: 6789

Re: File size / resolution limits for MagickReadImage?

Post by 168gr »

Convert works. Program won't read the png file though.

Code: Select all

pgg@g550 ~/test
$ cat test3.c
#include <stdio.h>
#include <wand/magick_wand.h>

#define ThrowWandException(wand) \
{ \
        char *description; \
        ExceptionType severity; \
        description=MagickGetException(wand,&severity); \
        printf("\n\n-----\n%s %s %lu %s\n",GetMagickModule(),description); \
        description=(char *) MagickRelinquishMemory(description); \
        exit(-1); \
}

int main() {

  MagickWand *mw1 = NULL;

  setenv("MAGICK_DEBUG","cache",1);
  MagickWandGenesis();
  mw1 = NewMagickWand();
//  MagickSetOption(mw1,"jpeg:size","512x512");

  if( !MagickReadImage( mw1, "input.png" ) )
    ThrowWandException( mw1 );

  MagickWriteImage( mw1, "output.png" );

  mw1 = DestroyMagickWand(mw1);

  MagickWandTerminus();

  return 0;
}


pgg@g550 ~/test
$ gcc test3.c -o test3 -lMagickWand-Q16 -lgdi32 -L/home/pgg/ImageMagick-6.8.1/lib -I/home/pgg/ImageMagick-6.8.1/include/ImageMagick -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16

pgg@g550 ~/test
$ convert input.jpg input.png

pgg@g550 ~/test
$ ls -l
total 14340
-rw-r--r-- 1 pgg None  3398782 Feb 23 12:06 input.jpg
-rw-r--r-- 1 pgg None 11226154 Feb 24 21:03 input.png
-rwxr-xr-x 1 pgg None      974 Feb 24 21:02 test3.c
-rwxr-xr-x 1 pgg None    52978 Feb 24 21:02 test3.exe

pgg@g550 ~/test
$ ./test3.exe
2013-02-24T21:03:46+05:00 0:00.040 0.031u 6.8.1 Cache test3[3132]: cache.c/DestroyPixelCache/1440/Cache
  destroy
2013-02-24T21:03:46+05:00 0:00.065 0.062u 6.8.1 Cache test3[3132]: cache.c/DestroyPixelCache/1440/Cache
  destroy


-----
test3.c main 24 ImproperImageHeader `input.png' @ error/png.c/ReadPNGImage/3699
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: File size / resolution limits for MagickReadImage?

Post by magick »

Curious, ok once more. Let's try a simple image format and see if we can get a baseline effort that works. This time, try PNM:
  • convert input.jpg input.pnm
Now modify your code to read input.pnm. If that fails, again let's see if its a resource problem. Try
  • convert input.jpg -resize 512x512 input.pnm
168gr
Posts: 47
Joined: 2013-01-09T22:46:19-07:00
Authentication code: 6789

Re: File size / resolution limits for MagickReadImage?

Post by 168gr »

Odd stuff

Code: Select all

pgg@g550 ~/test
$ ls -l
total 3376
-rw-r--r-- 1 pgg None 3398782 Feb 23 12:06 input.jpg
-rwxr-xr-x 1 pgg None     974 Feb 25 07:04 test3.c
-rwxr-xr-x 1 pgg None   52978 Feb 24 21:02 test3.exe

pgg@g550 ~/test
$ cat test3.c
#include <stdio.h>
#include <wand/magick_wand.h>

#define ThrowWandException(wand) \
{ \
        char *description; \
        ExceptionType severity; \
        description=MagickGetException(wand,&severity); \
        printf("\n\n-----\n%s %s %lu %s\n",GetMagickModule(),description); \
        description=(char *) MagickRelinquishMemory(description); \
        exit(-1); \
}

int main() {

  MagickWand *mw1 = NULL;

  setenv("MAGICK_DEBUG","cache",1);
  MagickWandGenesis();
  mw1 = NewMagickWand();
//  MagickSetOption(mw1,"jpeg:size","512x512");

  if( !MagickReadImage( mw1, "input.pnm" ) )
    ThrowWandException( mw1 );

  MagickWriteImage( mw1, "output.pnm" );

  mw1 = DestroyMagickWand(mw1);

  MagickWandTerminus();

  return 0;
}

/*
gcc test3.c -o test3 -lMagickWand-Q16 -lgdi32 -L/home/pgg/ImageMagick-6.8.1/lib -I/home/pgg/ImageMagick-6.8.1/include/ImageMagick -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16
*/

pgg@g550 ~/test
$ gcc test3.c -o test3 -lMagickWand-Q16 -lgdi32 -L/home/pgg/ImageMagick-6.8.1/lib -I/home/pgg/ImageMagick-6.8.1/include/ImageMagick -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16

pgg@g550 ~/test
$ convert input.jpg input.pnm

pgg@g550 ~/test
$ ./test3.exe
2013-02-25T07:05:57+05:00 0:00.047 0.046u 6.8.1 Cache test3[4044]: cache.c/DestroyPixelCache/1440/Cache
  destroy
2013-02-25T07:05:57+05:00 0:00.063 0.061u 6.8.1 Cache test3[4044]: cache.c/DestroyPixelCache/1440/Cache
  destroy


-----
test3.c main 24 ImproperImageHeader `input.pnm' @ error/pnm.c/ReadPNMImage/294

pgg@g550 ~/test
$ convert input.jpg -resize 512x512 input.pnm

pgg@g550 ~/test
$ ./test3.exe
2013-02-25T07:06:15+05:00 0:00.031 0.031u 6.8.1 Cache test3[2520]: cache.c/DestroyPixelCache/1440/Cache
  destroy
2013-02-25T07:06:15+05:00 0:00.062 0.062u 6.8.1 Cache test3[2520]: cache.c/DestroyPixelCache/1440/Cache
  destroy


-----
test3.c main 24 ImproperImageHeader `input.pnm' @ error/pnm.c/ReadPNMImage/294

pgg@g550 ~/test
$ ls -l
total 3956
-rw-r--r-- 1 pgg None 3398782 Feb 23 12:06 input.jpg
-rw-r--r-- 1 pgg None  589839 Feb 25 07:06 input.pnm
-rwxr-xr-x 1 pgg None     974 Feb 25 07:04 test3.c
-rwxr-xr-x 1 pgg None   52978 Feb 25 07:05 test3.exe
Thanks for your help with this.
168gr
Posts: 47
Joined: 2013-01-09T22:46:19-07:00
Authentication code: 6789

Re: File size / resolution limits for MagickReadImage?

Post by 168gr »

OK, maybe I was barking up the wrong tree here with blaming the iPhone 'simage format ...


After a few days away, I went back to work on this, and fabricated some more test images, this time with Gimp. And it appears what makes MagickReadImage() fail is the image file size, not resolution.

Code: Select all

pgg@g550 ~/test
$ ls -l *.jpg
-rwxr-xr-x 1 pgg None 261815 Mar  7 21:58 test-input1.jpg
-rwxr-xr-x 1 pgg None 262253 Mar  7 21:58 test-input2.jpg

pgg@g550 ~/test
$ ls -lh *.jpg
-rwxr-xr-x 1 pgg None 256K Mar  7 21:58 test-input1.jpg
-rwxr-xr-x 1 pgg None 257K Mar  7 21:58 test-input2.jpg
test-input1.jpg and all other images under 256K will load just fine.
test-input2.jpg and all other images over 256K fail to load.

Also tested with bmp files - the 256K limit applies there too:

Code: Select all

pgg@g550 ~/test
$ ls -lh *.bmp
-rwxr-xr-x 1 pgg None 256K Mar  7 22:13 test-input1.bmp
-rwxr-xr-x 1 pgg None 258K Mar  7 22:13 test-input2.bmp
test-input1.bmp succeeds, test-input2.bmp fails.



Where I should go from here? How do I tell ImageMagick that big files are OK? Thank you.
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: File size / resolution limits for MagickReadImage?

Post by magick »

We assume it fails on the iPhone? We tried your code on a 275K JPEG image with ImageMagick 6.8.3-7 on a Linux machine and it ran without complaint. Its must be a resource problem or perhaps an I/O problem. You can change 'cache' to 'trace' in the setenv() method. It will display copious amounts of logging information that may give us a clue.

Can you compile ImageMagick from source. If so, change MagickMaxBufferExtent in magick/blob.h to (64*8192-2). Does that make a difference?
168gr
Posts: 47
Joined: 2013-01-09T22:46:19-07:00
Authentication code: 6789

Re: File size / resolution limits for MagickReadImage?

Post by 168gr »

I'll see about compiling from source with the modification you suggested. Thanks.

In the meantime, here's the test program output with 'trace' set: lots of stuff
168gr
Posts: 47
Joined: 2013-01-09T22:46:19-07:00
Authentication code: 6789

Re: File size / resolution limits for MagickReadImage?

Post by 168gr »

magick wrote:We assume it fails on the iPhone? We tried your code on a 275K JPEG image with ImageMagick 6.8.3-7 on a Linux machine and it ran without complaint.
To be clear, the only involvement of the iPhone was that is was the source of the first test photos. The iPhone camera is totally out of the equation now.

I'm compiling and running the program on a Windows 7 laptop, using the cygwin environment. The target is a win32 application.
Post Reply