magick translate PAM to PNG ignores compression option

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Locked
PatrickPhotog
Posts: 7
Joined: 2020-02-04T12:06:49-07:00
Authentication code: 1152

magick translate PAM to PNG ignores compression option

Post by PatrickPhotog »

magick appears to apply compression to .PNG files even when told not to:

https://imagemagick.org/script/command- ... p#compress
Use pixel compression specified by type when writing the image.
Specify +compress to store the binary image in an uncompressed format.
-compress type ; Choices are: None, ...
Tried both.

Code: Select all

pat@nova:~/exp/pam> magick  pf-269362.red.sin0.5_1.03.pam  +compress        pf-269362.red.sin0.5_1.03.im.png;  
pat@nova:~/exp/pam> magick  pf-269362.red.sin0.5_1.03.pam  -compress None   pf-269362.red.sin0.5_1.03.im2.png;  
pat@nova:~/exp/pam> lsr -s  pf-269362.red.sin0.5_1.03.pam  pf-269362.red.sin0.5_1.03.im*.png
pf-269362.red.sin0.5_1.03.pam         145317167
pf-269362.red.sin0.5_1.03.im.png       98943404   << Way too small to hold uncompressed bitmap
pf-269362.red.sin0.5_1.03.im2.png      98943404

Code: Select all

pat@nova:~/exp/pam> head -n7  pf-269362.red.sin0.5_1.03.pam
P7
WIDTH 7378
HEIGHT 4924
DEPTH 2
MAXVAL 65535
TUPLTYPE GRAYSCALE_ALPHA
ENDHDR
pat@nova:~/exp/pam> head -n7  pf-269362.red.sin0.5_1.03.pam | wc
      7      12      79
With a 79 byte header, the PAM uncompressed bitmap size is 145317088 Bytes
Checking: m 145317088 /7378 /4924 -> 4 bytes per pixel. No compression.

I noticed a similar thread 5 years ago in which the Linux always compressed but the windoz version worked.

I can't find a Linux viewer for PAM files so I need to convert them for viewing.
For temporary local viewing and a 10TB drive, compression is unnecessary.


pat@nova:~/bin> magick -version
Version: ImageMagick 7.0.8-20 Q16 x86_64 2018-12-25 https://imagemagick.org
Copyright: © 1999-2019 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenMP
Delegates (built-in): bzlib fontconfig freetype jng jpeg lcms ltdl png tiff webp x zlib

uname -a (OpenSuse Leap 15)
Linux nova 4.12.14-lp150.12.82-default #1 SMP Tue Nov 12 16:32:38 UTC 2019 (c939e24) x86_64 x86_64 x86_64 GNU/Linux

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: magick translate PAM to PNG ignores compression option

Post by fmw42 »

PNG compression is lossless. So why does it matter if it is compressed or not?

See

https://imagemagick.org/script/command- ... hp#quality
https://imagemagick.org/Usage/formats/#png_write

PatrickPhotog
Posts: 7
Joined: 2020-02-04T12:06:49-07:00
Authentication code: 1152

Re: magick translate PAM to PNG ignores compression option

Post by PatrickPhotog »

Time and data quality are both important.
If you don't lose any data, time does not matter? I don't follow your logic.

It takes 13 seconds to compress a 145MB PAM to PNG. And, I have hundreds to do per job, thousands per day some days.

Writing a PAM file takes 122ms:
WBHB: Wrote 145.317167MB (79 + 145317088) in 121.728ms, 1193.786 MB/s to file pf-269369.red.sp0.5.m1.16.pam

timeit magick pf-269362.red.sin0.5_1.03.pam -compress None pf-269362.red.sin0.5_1.03.im2.png;
TI: Elapsed time = 13.222 sec -> 7.42 MB/sec

mult 13.333 /0.121728 -> 109.5 times as long to write. A fraction of a second is not bad. Over a dozen seconds adds up over a day / week.

You are saving 46 MB of disk space at a cost of 12 seconds of your time. A 2TB drive costs $84. Do the math...
$84 / 2E12B * 46 E6 B = $0.001932 worth of disk space saved if you never use it again.

Let's calculate your hourly value:
0.001921$ / 12 sec * 60 sec/min * 60 min/hr = 57.96 CENTS per hour. And, you can rewrite that disk space a million times.

My time is worth more than that.

Compression is a useful option. I was hoping that somebody could spot my command line error.

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: magick translate PAM to PNG ignores compression option

Post by fmw42 »

OK. If time is an issue, I see your point. Have you tried the commands for setting the quality and compression via the defines in the second link I sent? I believe they give better control and the first link suggests that.

What about lossless TIFF?

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: magick translate PAM to PNG ignores compression option

Post by magick »

Try this command:

Code: Select all

magick pf-269362.red.sin0.5_1.03.pam  -define png:compression-level=0 -define png:compression-filter=0 -define png:color-type=2 pf-269362.red.sin0.5_1.03.im.png
This command took less than 2 seconds to complete for us for a mocked up 145MB PAM image file.

User avatar
fmw42
Posts: 26383
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: magick translate PAM to PNG ignores compression option

Post by fmw42 »

I would look into saving as TIFF rather than PNG. The file size is smaller and it might run faster. Test:

Code: Select all

convert logo: logo.png
convert logo.png  -define png:compression-level=0 -define png:compression-filter=0 -define png:color-type=2 x1.png
convert logo.png -compress none x1.tif
identify logo.png x1.png x1.tif
logo.png PNG 640x480 640x480+0+0 8-bit sRGB 256c 27398B 0.000u 0:00.000
x1.png PNG 640x480 640x480+0+0 8-bit sRGB 922814B 0.000u 0:00.000
x1.tif TIFF 640x480 640x480+0+0 8-bit sRGB 256c 309006B 0.000u 0:00.000


It is strange (BUG?) that using -quality 00 does not achieve the same results.

Code: Select all

convert logo.png -quality 00 PNG24:x2.png
identify x2.png
x2.png PNG 640x480 640x480+0+0 8-bit sRGB 67178B 0.000u 0:00.000

This produces a much smaller (more compressed) result.

PatrickPhotog
Posts: 7
Joined: 2020-02-04T12:06:49-07:00
Authentication code: 1152

Re: magick translate PAM to PNG ignores compression option

Post by PatrickPhotog »

I had to change the PNG color type to 4 for Grayscale - Alpha:

Code: Select all

timeit magick pf-269362.red.sin0.5_1.03.pam  -define png:compression-level=0 -define png:compression-filter=0 -define png:color-type=4 pf-269362.red.sin0.5_1.03.im.png
TI: Elapsed time = 4.183 sec
The size is right and the channels are right and it is 3x faster. Much better.

I have a Samsung 2TB 970 EVO+ PCIe 3, a 4 GHz Skylake and 64GB of 3200 MHz CL16 DDR4. There is no hardware I know of anywhere near twice this fast but you are seeing < 2seconds and I am getting > 4 sec. Impressive!

Are you using a custom magick build fully optimized for your chipset with -march $CPU, -Ofast, etc.? I may be running on Suse code de-optimized for least common denominator hardware like NAHALEM or core2. Pthreads?

You are seeing great performance! I may attempt a source build! I get a 6x speed improvement compiling dcraw from source vs binary!

PatrickPhotog
Posts: 7
Joined: 2020-02-04T12:06:49-07:00
Authentication code: 1152

Re: magick translate PAM to PNG ignores compression option

Post by PatrickPhotog »

Code: Select all

timeit magick pf-269362.red.sin0.5_1.03.pam  +compress pf-269362.red.sin0.5_1.03.im.tif
TI: Elapsed time = 443.617 ms
Darn fast! The file size is right at ~145MB

Code: Select all

Channel depth:
    Gray: 16-bit
    Alpha: 16-bit
   Gray:
      min: 0  (0)
      max: 65535 (1)
      mean: 6948.25 (0.106023)
      standard deviation: 12953.1 (0.197652)
      kurtosis: 7.53538
      skewness: 2.75987
      entropy: 0.634877
    Alpha:
      min: 0  (0)
      max: 42693 (0.651453)
      mean: 4807.5 (0.0733578)
      standard deviation: 9931.49 (0.151545)
      kurtosis: 5.08598
      skewness: 2.46357
      entropy: 0.640839
The Alpha looks right in `identify -verbose` but is displays with opacity === 100%, zero transparency anywhere in multiple viewers.
It is a PGM, not a transparent file!

Gwenview, Okular 100% opaque. Crashes in XV. Black in ShowFoto. Xnview shows it solid.

TIFF is such a byzantine format (IRS invention?). That's why I like the 79 byte, fully readable PAM header and simple bitmap.

XNview actually decodes the PAM file, but it shows Acid Trip colors instead of boring grayscale. Gruesome color mapping.

How hard could it be to write a viewer for a PAM file? Essentially a PGM with Opacity?!

Locked