Performance of _repr_png_ with larger images

[smason]

I've just been using PIL to work with some larger images in IPython/Jupyter (~3k by 2k pixel images) and noticed that it can take a long time (nearly 10 seconds) for them to appear in the output. I had a poke in the code and found that it's due to Image._repr_png_ using default compression levels which is causing it to take a significant amount of time.

I tried adding compress_level=1 to the save call which reduces the wait to a couple of seconds–much nicer for interactive use.

I see it's had a recent refactor (in #7135) to also include JPEG support which is nice, but I could put a quick patch together if making this sort of change is considered sensible.

[radarhere]

I've created PR #7242 to resolve this.

[smason]

That was quick, thanks for that!

It occurred to me later that it might be considered reasonable to resample very large images down as well. This would obviously be a change in semantics, but given this ostensibly is for "display purposes" it might be a reasonable thing to do.

I can't find much in the way of guidance on this in either:

• https://ipython.readthedocs.io/en/stable/config/integrating.html or
• https://ipython.readthedocs.io/en/stable/api/generated/IPython.display.html

What to use when resampling a "large" image seems to bring policy awkwardly into the mix, but maybe something like:

THRESHOLD_SIZE = 1200
factor = max(image.size) // THRESHOLD_SIZE
if factor > 1:
    image = image.resize((image.width // factor, image.height // factor), Resampling.BOX)

in an attempt to reduce artifacts by keeping to integer scaling. Having to pick a threshold feels nasty and makes me think this shouldn't be done. Exposing this at a module level would let users control this, but I can predict confused users submitting bug reports if the image is resampled.

A similar argument could be made for reducing higher bit-depth images down to 8BPC.

Thought it worth writing this up in case anybody else has ideas / for future reference.

[homm]

A similar argument could be made for reducing higher bit-depth images down to 8BPC.

Pillow only supports 8BPC or single-channel 32-bit floats.

[smason]

Pillow only supports 8BPC or single-channel 32-bit floats.

what are you referring to here? I've personally used JPEG2000 images with unusual bit-depths which got exposed as I;16 pixels, and the PNG support doesn't mention float support.

• https://pillow.readthedocs.io/en/stable/handbook/concepts.html#modes
• https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html#png

[radarhere]

Pillow only stores multichannel image modes with up to 8 bits per channel, or single channel images with up to 32 bits.

[smason]

Pillow only stores multichannel image modes with up to 8 bits per channel, or single channel images with up to 32 bits.

Okay, that makes more sense.

It still makes sense (to me and my use cases anyway) to reduce higher bit depth images down (to L) in the response from _repr_*_ as I've done in #7259. Probably best to continue discussion there.