Enhancement: make compression property for heif output mandatory

[markg85]

Feature request

What are you trying to achieve?

Creating an HEIF thumbnail. It's surprisingly difficult!

When you searched for similar feature requests, what did you find that might be related?

There's a couple related issues like libvips (i have it), x265 (i have it) and aom ( i have it too). Still creating HEIF (or strictly HEIC) images isn't working at all.

What would you expect the API to look like?

...heif({ compression: 'hevc' }).toFile(...)
Should produce an image.
Without the compression flag, it creates an AVIF image (which is super confusing!).
With the compression flag it creates an empty file.

What alternatives have you considered?

So, i'm using these images in a mobile app.
I actually want JXL, but that support is very limited thanx to google blocking it.. so that's out of the question.
AVIF is fine too, but that support too is quite limited on mobile still.
Then HEIF is next. And that doesn't work in this library. So.. yay...

Please provide sample image(s) that help explain this feature

N/A, above should be clear.

With regard to HEIF support, could that please be hardened.
The mere fact that HEIF - by default, if you don't provide the compression flag - generates an AVIF image is extremely confusion. I'm sure there's a very good reason for doing this, but i don't see it. To me it seems like that feature shouldn't exist. Or spam the user with warnings that setup is chosen that will be confusion.

But other then that, clearly something isn't working yet there isn't a single error anywhere. It might nice to have a look at that and be a little error happy. :)

I'm using ArchLinux here and have no problem with HEIC images, that all works just fine.
Any clue on how to get this working?

[lovell]

Without the compression flag, it creates an AVIF image (which is super confusing!).

https://sharp.pixelplumbing.com/api-output#heif

HEIF is a container, which supports HEVC and AV1 codecs used to create HEIC and AVIF images respectively. The documented default compression for the HEIF container is AV1, which was chosen as it is not patent encumbered and the prebuilt binaries support it.

With the compression flag it creates an empty file.

Please can you ensure you are using the latest version of libvips, libheif and x265 and that they are configured correctly e.g. libheif might be setup to use a plugin system for its codecs.

If you are still having problems please provide complete details about the environment, dependencies, code and sample images you are using that allows someone else to reproduce. A standalone repo with a Dockerfile might be suitable for this.

[markg85]

Hi @lovell,

To me it sounds super strange when i explicitly call heif to get an heic file but it turning out to be an avif one instead. Technically you are correct, but if you go by common sense then this makes no sense. I even had to use a binary diff tool to figure out that the heif and avif command both make avif files (granted, i skimmed over the docs else i would've know it earlier). That is a bug in my eyes even though you describe it as a "feature". Technically you can even put jpeg files inside and heif container https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format#Variants. The fact that this idiocy exists in heif (which very well assumes heic, not any of the others) might well be a reason for it's complete failure in user adoption besides the casual patent crap.

My proposal would be to remove heif - and any reference to it - from your library and proceed with heic instead. The f suffixed version is a container format that causes all this confusion. The c sufficed version is explicitly an image - a frame - that uses the HEVC encoding. It would allow you to not be bothered by different compressions for an heic image. It's simpler imho.

On my part however, I already was very reluctantly trying that because of the reasons mentioned in my first post. I wanted AVIF or very preferably JXL. The later isn't possible because there is no package for it in flutter that implements is. For avif however, there is: https://pub.dev/packages/flutter_avif (i didn't know, but i do now and am using it). So i have no more need for heic images. As my development work is already cram-packed, i'm not going to debug this library for a feature i don't need anymore. Sorry for that, you'll hopefully understand. I do happily keep using your awesome library for AVIF files though. And soon probably jxl too.:)

[lovell]

The HEIF container supports a growing number of codecs, and conversely a growing number of new video codecs are selecting HEIF as their container for still images. Being based on ISOBMFF (née Quicktime) it's perhaps not the most efficient container but it's here to stay. Conceptually it's rather like TIFF.

https://github.com/strukturag/libheif/blob/41bf6a16d7716bdf1a3609ab8ceea1444fd47b50/libheif/heif.h#L588-L608

The specs for multi-frame HEIF and TIFF both allow different codecs per frame within the same file, although many encoders don't yet support this.

i explicitly call heif to get an heic file

Would making the .heif({ compression }) property mandatory in the API help reduce confusion, i.e. remove the default? Whilst HEVC is patent-encumbered and therefore unsupported in prebuilt binaries there will not be an explicit .heic().

granted, i skimmed over the docs else i would've know it earlier

How could the documentation be made clearer?

[markg85]

Let's go with some mutually agreeable assumptions :)

When using your library do you agree that:

1. When using your library and looking at heif, you assume it does create an heic. Not an avif file, right?
2. You do not expect .avif(..) and .heif(...) in their default forms to output the exact same avif file, right?

If you agree with those assumptions then your defaults for heif are wrong.
If you insist on keeping heif then at least default it to heic and spam the user with - helpful descriptive - errors if it doesn't work.

[lovell]

I guess the answers depend on the point in time. A reframed question might be: should an image using the HEIF container have a default compression? For example, if this were 2017 then the answer would be yes, hevc.

A brief historical interlude:

heif() was added 4 years ago as an experimental API in v0.23.0, its use requiring a globally installed libvips compiled with support for libheif and lib(de|x)265, and compression defaulted to hevc.

avif() was added in v0.27.0 at the same time as the prebuilt binaries added support for AVIF, and the compression default of heif() switched to av1 as part of marking the API as no-longer experimental. The idea here was that in 2021 HEIF had moved beyond being an HEVC-only container and that most people using sharp to create HEIF images would be using av1 because browsers were starting to support it and sharp's prebuilt binaries now made it easy to do so.

Returning to now, in the world outside of sharp, the HEIF container continues to be adopted by further codecs. A good example that I think has potential is the patent-free MPEG5 EVC "baseline" profile. This is why I'm suggesting the removal of a default compression from heif(), which should hopefully make things a bit more future-proof, the API more explicit, and with (as you suggest) helpful error messages.

[markg85]

Returning to now, in the world outside of sharp, the HEIF container continues to be adopted by further codecs. A good example that I think has potential is the patent-free MPEG5 EVC "baseline" profile.

Sorry to be blunt here, but i couldn't care less about patents. I think they are destructive to software and should be nuked from existence. In my view if there is an open source library just publicly available then patents can go to.. well, you get my drift ;)

I do get this view is harder to maintain if you're in the US, the patent system - and rights - is just wildly different there compared to here in the EU.

This is why I'm suggesting the removal of a default compression from heif(), which should hopefully make things a bit more future-proof, the API more explicit, and with (as you suggest) helpful error messages.

That would be a good change!

For what it's worth. heif smells like HEVC/H.265 and it's name clearly is borrowed from that era. If they aim to go for a container format then they should rebrand to a container format that isn't so - on the surface - tied to HEVC.

To make the joke complete, and effectively giving me a huge distaste for ever using that format, is that they seem to add in all the crappy garbage formats from the past in that "high efficiency" container. Like look at the spec: https://nokiatech.github.io/heif/technical.html they added JPEG, JPEG XR and even GIF is in there! If there's anything low efficiency it's GIF and they added it. Moreover a true - proven - superior image format would be JPEG XL (not XR), yet they thus far haven't added that one in. What a joke.

[lovell]

v0.33.0 is now available with the mandatory .heif({ compression: ... }) property, thanks for the idea.