Add ability to read PNG cICP chunk from PNG 3rd Edition via libpng

[digitaltvguy]

Is your feature request related to a problem? Please describe.
PNG is not parsing cICP for image format metadata added in PNG 3rd Edition and libpng 1.6.47

Describe the solution you'd like
The available metadata should be read and written so proper conversions or pass thru of metadata occurs

Describe alternatives you've considered
None

Additional context
cICP is metadata defined for video imagery defined by ITU-R H.273

[zachlewis]

I, too, have been thinking about submitting an issue re: exploring how to best support CICP metadata in image formats that support it (PNG, AVIF, HEIF, JXL off the top of my head), and to what capacity.

CICP (Coding Independent Code Points), codified in ITU-T H.273, is a means for using a tuple of integers to communicate characteristics of a variety of, traditionally, video encodings (think: NCLX atom data). The tuple is a series of three or four values depending on who you ask, that map to enumerations representing specific sets of ColourPrimaries, TransferCharacteristics, MatrixCoefficients, and a VideoFullRangeFlag.

One particularly useful and increasingly popular use of CICP is describing HDR encodings to browsers and image viewers capable of displaying the image as intended. As you might imagine, "HDR support" is kind of a scattershot, across image formats, browsers, platforms, desktop environments, and... well, display capabilities, of course. Point being, CICP tuples are an easy way to inform downstream consumers of the intended handling of image data. It's not unlike ICC in that way; it's just intended as a simpler way to convey a constrained set of allegedly commonly used video encodings.

(One thing about CICP that I would not describe as particularly "cash money" as it's known in the trade, is that there's no enumeration for Gamma 2.4; we can talk about the specifics of how to handle that particular use case for different formats, as I don't think the same rules apply everywhere).

As far as support in OIIO goes, I think, it would be cool if we had ways of manually specifying (or overwriting) cICP metadata chunks in formats that support it. Seems like some pretty low-hanging fruit to provide a means for conveying CICP data in a Spec, and/or adding specific functionality in oiiotool for fetching / setting (and maybe somehow conveying to downstream FFMPEG encoders?) CICP tuple metadata. (See, for example, the avifenc tool built with libavif)

Beyond this, I think OCIO, in a parallel effort, should provide methods OIIO can use to inform OIIO's own decoding behavior when CICP metadata is detected; as well as methods for divining CICP-tuple data from ultimate OCIO display-view IBA operations (and certain color space conversion operations), if possible. There's a whole lot to discuss on the OCIO side re: how exactly this would work; but as far as OIIO is concerned, I think it's more a matter of applying given CICP tuple data as-correctly-as-possible, as demanded by each format.

How do we feel about handling non-RGB encodings?

[digitaltvguy]

(One thing about CICP that I would not describe as particularly "cash money" as it's known in the trade, is that there's no enumeration for Gamma 2.4; we can talk about the specifics of how to handle that particular use case for different formats, as I don't think the same rules apply everywhere).

For BT.709 the reference refers to the OETF. BT.1886 is the SDR EOTF mentioned in “Note 1” on page 10 of H.273 which is used in combination with BT.709. BT.1886 establishes the use of Gamma 2.4 (to perceptually reverse the OETF).

[lgritz]

It would be great for somebody to write up a detailed plan for exactly what this means for OIIO. It's not JUST dealing with the the 4 bytes of cICP -- I think that just within PNG, that probably immediately implicates several other corners of PNG that we probably do not correctly read, write, or propagate. And then there's the matter of what we should do when converting to and from other file formats (some of which support it, some don't) and how exactly it interacts with the OCIO-based color management we already have.

This is way out of my expertise. Hopefully, people with more color management knowledge than me will step up and take ownership of this.

[JGoldstone]

Strictly speaking BT.1886 only matches a 2.4 gamma (for a 100 nit display) when Lb == 0 and Lw == 100:

Ideally CICP support would pervade not just OIIO but OCIO and OpenEXR as well; I've been asked to think about this.

I do note that one thing not covered by CICP right now are the four encodings covered in SMPTE ST 2115, where Sony's S-Gamut3 and S-Gamut3.cine, Panasonic's Vatican V-Gamut, the Canon "cinema gamut", and the S-Log3, V-Log and two varieties of Canon log encoding are described. (ST 2115 needs revision but there are some deep issues that come not from ST 2115 but from the four other SMPTE docs that depend on it, but I digress.)

@lgritz , two people have asked me to look into CICP, and ideally one of them might not be pro bono. Given that this is going to (or at least I would think it should) involve more than one ASWF project, should discussion of the issues take place here, or is there some more natural place? Maybe once I have thought about it even a little, I should open up an issue with a more general title. Would that be OK, and point this to that, when it exists?

This probably "isn't that kind of crowd" but if anyone wants to discuss this over coffee at NAB on the Monday or Tuesday, I'll be there.

[digitaltvguy]

Please take a look at ISO_TS_22028-5_2023 which defines a diffuse white metadata element (this is a paid document). On top of CICP, this specifies a method to identify reference white that has changed from the "nominal reference white". There is some work to standardize this. I will post more info when available for SMPTE members. This will allow graphics (SDR or HDR to specify their reference white in cd/m2

[zachlewis]

I've been giving this a lot of thought, too, and will do my best to contribute however I can...!

It would be great for somebody to write up a detailed plan for exactly what this means for OIIO. It's not JUST dealing with the the 4 bytes of cICP -- I think that just within PNG, that probably immediately implicates several other corners of PNG that we probably do not correctly read, write, or propagate. And then there's the matter of what we should do when converting to and from other file formats (some of which support it, some don't) and how exactly it interacts with the OCIO-based color management we already have.

Your intuition is totally correct -- supporting cICP chunk reading and writing is such a little thing, but it's such a big thing... There are all sorts of implications for how OIIO and OCIO ought to interact, w.r.t. the header metadata we care about, whether to persist the metadata , whether it informs an ad-hoc "autocc" OCIO processor "on read"; and whether an ultimate OCIO transform informs new header metadata "on write"... that kind of sexy stuff.

So, there's a lot to iron out; and the exact plan for OIIO will become more clear in tandem with the exact plan for OCIO. We need to develop a meta-metadata schema, among other things.

All this said -- as far as this particular issue goes, I think it would be extremely useful and intuitive if we could convey and manipulate CICP data with a reserved "oiio:cicp" as a vector of 4 integers, and implement support for reading inspecting and writing of the data on a per-plugin basis... that alone would be pretty huge.

@JGoldstone, I caught the tail end of you mentioning something about ST 2115 at Monday's color interop meeting -- funny, I'd just learned about ST 2115 while investigating the differences between CLF 3.0 and CLF 4.0, and I went down a tiny rabbit hole ascertaining exactly what you've confirmed for me above, about the enums representing those particular transfer functions and chromaticities. (It's unfortunate that they went with "LogC2" and "LogC3" for the Canon transforms...). Totally agree, though, as long as we're tacking CICP metadata to OCIO color spaces, it makes perfect sense to me to use OCIO as a vector for identifying and conveying the relevant ST 2115 identities, especially since that information is apparently relevant to authoring ST2136-1 "profiles", whatever those are.

@digitaltvguy TS 22028-5 is exactly where my research last night led me -- these slides in particular. It seems like exactly what we're after.

[JGoldstone]

I've been reluctant to fork over the $150 for that document but given that you're saying one should understand it, and that another group is saying I should read it along with ISO 21496-1 to understand how gain maps work in practice, I guess I will just call it a deductible business expense.

For those following along at home:

ISO/TS 22028-5:2023, "Photography and graphic technology — Extended colour encodings for digital image storage, manipulation and interchange: Part 5: High dynamic range and wide colour gamut encoding for still images (HDR//WCG)"

and

ISO/DIS 21496-1, "Digital Photography — Gain map metadata for image conversion: Part 1: Dynamic Range Conversion"

[digitaltvguy]

BTW - BT.2100-3 was released with an update that changes floating point "1" to represent "Reference White".

Nominal Reference White Luminance is obtained from an HDR camera and a 100% reflectance white card resulting in a nominal luminance of 203 cd/m2 on a PQ display or on an HLG display that has a nominal peak luminance capability of 1 000 cd/m2.

Reference White Luminance (vs Nominal Reference White Luminance) could potentially be other luminance levels from my understanding.

[JGoldstone]

@zachlewis (and perhaps @digitaltvguy ), maybe you can look into a rabbit hole in which I've trapped myself and perhaps pull me out. If you pull down SMPTE ST 2067-60:2023, "Interoperable Master Format — Application #6 UHDTV Program Workflow (AVC)", you will find this table:

This ties the S-Log3, V-Log, and two Canon log encoding functions to the Rec. 2020 primaries. But the problem is, each of those log encoding functions is usually tied to an associated gamut that is not the one given in Rec. 2020, AND those other gamuts all are larger than that of Rec. 2020:

So what's up with that? What do you do with an image where the content was captured in S-Log3/S-Gamut3.cine (say) and where parts of the scene-referred image would, if converted to Rec. 2020 primaries in the use of this IMF application, fall outside Rec. 2020?

I see at least two possibilities, the first much more probable the second:

1. The authors of IMF Application 6 didn't have much experience with scene-referred content and assumed it would all fall within the Rec. 2020 gamut when the camera-vendor-specific encodings were matrixed to Rec. 2020
2. The authors did realize this could happen but believed such cases could be handled using negative RGB values, cf. this illustration from Giorgianni & Madden:

It is possible to get negative RGB values if you are evil and feed full-range data into a pipeline that expects legal range. So the second possibility is just ... barely ... possible. Maybe. Maybe only partially and for a subset of the camera scene-referred encoding spaces. I've been trying to figure out a legit way to chart chromaticities that arise from doing that, which means I have to re-activate certain matplotlib-related parts of my brain and start using Thomas' colour library again.

@digitaltvguy , maybe you can help me out here, if I am correctly guessing who you are IRL.

If I'm right and it's more the first alternative above that would describe the ST 2067-60 situation, then would we want to somehow dictate (via API typing for static checking or via runtime validation for dynamic checking) that only some combinations of an ICIP color primaries code point and an ICIP transfer characteristics code point were valid?

[digitaltvguy]

I think IMF was meant for content repositories (fulfillment libraries). By fixing the gamut to BT.2020, they've simplified the process where only transfer function mapping will be necessary.