Reliably record colorspace between ACES conversions

[maxpareschi]

Hi, we're trying to migrate our internal transcoding system (which is relying on nuke commandline) to oiio (either using oiio tool or directly with python bindings).

Our workflow is to get whatever the client sends, which is either one of the linearized main camera formats (the show I'm on is RED) or AP0 exrs, and convert it to ACEScg for the whole pipeline up until client delivery, which gets converted back to the source colorspace.

In particular I'm now trying to have the oiio:ColorSpace set as lin_ap1 on the acescg converted output from an AP0 file to no avail. Tried setting it with --sattrib oiio:ColorSpace ACEScg and even with python bindings using new_buf.specmod().set_colorspace("ACEScg") or new_buf.specmod().attribute("oiio:ColorSpace", "ACEScg").

The actual writing can only happen if I do an erase_attribute("acesImageContainerFlag") since it I leave it as it is it complains about strict mode (which should really not happen if I try to write an ACEScg file btw) but anything I do to set a colorspace value in the attribs gets nothing.

Is there a way? I'm using oiio 3.1.12, python 3.13 and aces 1.3 ocio 2.1 studio config v1.0.0.

Maybe I missed something on the docs.

Thanks

[lgritz]

Maybe @zachlewis or other color experts can have a more authoritative answer? But I'll do my best with what I know.

As I understand it, ACES "container files" MUST be AP0, and cannot be ACEScg. So that's why you need to erase the flag that marks it as an ACES container, if you want to save with a different color space.

I can't quite tell from your description everything that's going on. In particular, I am not sure if you're saying you are having trouble transforming the pixels, or if you are transforming the pixels but not saving the file with the right metadata?

Are you expecting the setting of the attribute to do a color space transformation?

[zachlewis]

I can't quite tell from your description everything that's going on. In particular, I am not sure if you're saying you are having trouble transforming the pixels, or if you are transforming the pixels but not saving the file with the right metadata?

Are you expecting the setting of the attribute to do a color space transformation?

I have the same questions.

Are you applying a colorconvert IBA? Is that producing the correct results when you remove the acesImageContainerFlag attribute?

If so, If I'm understanding correctly, this is happening with ACES2065-1 plates that you're trying to convert to ACEScg as part of your ingestion pipeline. You're able to apply the colorconvert IBA properly, but OIIO complains because the acesImageContainerFlag metadata is persisting from the source input, instead of getting removed from the spec when the colorconvert IBA updates oiio:ColorSpace. In other words, you have to manually do something (remove the acesImageContainerFlag attribute) whenever you convert from ACES2065-4 plates, just because that attribute is set upstream, all because, by default, OIIO is strict about disallowing the acesImageContainerFlag metadata for any non-ACES2065-1 encoding.s

If that's the case... yeah, I agree, the mechanics are too complex, and the acesImageContainerFlag business should behave better out-of-the-box. Fortunately, it's a pretty easy fix -- when we resolve the color metadata in the spec at the end of each colorspace-aware IBA (e.g., anywhere we're setting oiio:ColorSpace), we can reconcile / remove any conflicting metadata, such as acesImageContainerFlag for color spaces that aren't equivalent to lin_ap0_scene. And then, if folks want to keep that flag enabled for whatever reason, they can explicitly set the attribute before writing.

We may need to revisit the acesImageContainerFlag stuff in general, if it's not behaving intuitively or as expected.
See also this https://academysoftwarefdn.slack.com/archives/C05782U3806/p1768326165271729

So, to answer your question:

• The colorconvert IBA should be correctly updating the oiio:ColorSpace attribute on its own. You shouldn't need to adjust it manually.
• The acesImageContainerFlag / "strict" mode warning is sounding because the oiio:ColorSpace is being properly updated to a non-ACES2065-1 color space, which conflicts with the presence of the acesImageContainerFlag metadata inherited from the source plates.
• So, the correct workflow for now is to manually erase the acesImageContainerFlag, since you're not writing ACES2065-4 plates, as you've been doing.

And we'll improve the mechanism so that the acesImageContainerFlag does not persist when converting to a non-ACES color space. (We will not do the opposite -- i.e., we will not enable the acesImageContainerFlag when converting to ACES2065-1; the only way acesImageContainerFlag gets written is if it's set manually, or if an ACES2065-4 buffer never encounters a color-space-aware IBA.

Does that make sense?

[maxpareschi]

well, aside from not being too documented i don't really have any problem pixel wise, nor to set and delete metadata to make it work. so at this point everything is working, but i wanted to remove our internal metadata for colorspace and use oiio:ColorSpace for better inferring and uniformity. Problem is that oiio:ColorSpace is being set by oiio without any possibility of control: if i --sattrib or any other way of setting that metadata i always end up with nothing. of course this is only for exrs, for other formats we will need to dump that meta in some other container or make assumptions.
It was just metadata. The whole aces container business was just surpriring since it's not documented, but it was easy to solve once understood which data to remove/add. I just think that if you don't recognize a file as ap0 when encoding you could probably just remove that automatically to avoid surprises for other people like me.

[zachlewis]

ah -- okay, i think a few things might be conspiring against you.

1. oiio:ColorSpace is an internal OIIO attribute -- OIIO never writes it to container metadata, and I don't believe OIIO reads it (although i could be mistaken).
2. (Instead, OIIO will support reading and writing the colorInteropID EXR metadata attribute very soon! That's the attribute you should use to communicate the color encoding downstream. See An ID for Color Interop DRAFT as well as Identifying the Color Space of OpenEXR files DRAFT -- long story short, OIIO will automatically read and write this metadata by default in the near future)
3. That being said, per the above specs, the presence of the acesImageContainerFlag key always supersedes other forms of container-specific color metadata (including the colorInteropID attribute).

I believe that applying the --colorconvert IBA to ACEScg is actually updating the value of oiio:ColorSpace correctly; we're failing to unset the acesImageContainerFlag metadata when we update oiio:ColorSpace to reflect the output color space of the color convert IBA; thus, OIIO complains when it realizes the stale acesImageContainerFlag metadata persisted from the source is inconsistent with OIIO's internal accounting of the color space encoding, because "strict mode" is supposed to guard against inadvertently persisting untruthful or contradictory acesImageContainerFlag metadata. (whereas "relaxed mode" is supposed to permit lossless compression schemes and, I suppose, allow folks to lie about whether they're actually writing linear AP0 RGB data).

In any case, this is frustrating and surprising behavior -- of course folks should be able to ingest ACES2065-4 plates without hassle. I think two things need to happen:

1. We need to ensure that "acesImageContainerFlag" gets unset if oiio:ColorSpace is set to something other than ACES2065-1. This means, we probably need some fancier behavior around the ImageSpec::set_colorspace method, and maybe even some kind of validation / reconciliation step after the "oiio:ColorSpace" attribute is manually adjusted through other means. (We could stand to have a validity sweep for other attributes as well -- chromaticities, cicp, and timecode come to mind -- would be nice to convert string representations of those attributes to the representations expected by ImageSpec).

2. We should maybe consider not defaulting to strict mode, because if all you're trying to do is convert uncompressed ACES2065-4 plates to losslessly compressed EXR sequences, OIIO is still gonna complain about it, because, strictly speaking, ST 2065-4 EXRs are uncompressed half-float encodings with only a single mono or stereo RGB/A channel layer, and must have chromaticities metadata set as well. In practice, folks specify / turn over piz-compressed ACES2065-4 masters quite often; but some QC tools are stricter than others. Bottom line, OIIO's default behavior should be a little more lax. The "strict" mode intends to guard against people deliberately setting acesImageContainerFlag under contradictory conditions; but I don't think we'd intended to force people to manually remove persistent acesImageContainerFlag metadata if they're not even trying to set that attribute in the first place. Instead, maybe, by default, we should simply unset contradictory acesImageContainerFlag metadata just before writing, if we can ascertain that the attribute is persisting from the native spec (i.e., if the attribute was set in the source image sequence)? And "strict" versus "relaxed" elicits different behavior, depending on whether the attribute is present because it was set in the source image sequence, as opposed to being explicitly set by the user. If the user is explicitly setting the attribute, but also trying to write a srgb-rec709 32f dwaa-encoded EXR... they deserve to get yelled at.

So: "strict" versus "relaxed" should govern how eager OIIO is to yell at you for breaking the rules, IFF you're explicitly setting the flag in the first place; otherwise, "strict" versus "relaxed" dictates whether or not to drop the acesImageContainerFlag attribute at write time (if it was set in the native spec, on read), depending on how far we've strayed from the ST 2065-4 chapter and verse. (e.g., in strict mode, we drop the attribute if we're not actually writing uncompressed 16f RGBA lin_ap0_scene data; and in "relaxed" mode, we'll allow lossless compression schemes, as long as we're actually writing lin_ap0_scene data. And in either mode, we'll set the aces container attribute + AP0 chromaticities automatically if we happen to be writing a single RGB/A layer of uncompressed half-float lin_ap0_scene data and we haven't deliberately set colorInteropID or contradictory chromaticities metadata; and in "off" mode, we don't automatically enable the acesImageContainerFlag metadata at all; we'll unset it if applying a color-space-aware IBA, and we'll unset it if using lossy compression when writing, but otherwise, "off" mode means the acesImageContainerFlag will persist from the source container unless subsequent actions invalidate it; otherwise, users have to manually set the metadata if they want it, and our heuristics will not prevent them from lying about what they're doing.

Does this make sense? It seems like this arrangement would cause the least amount of surprise, and would seemingly align with the intent of "strict" / "relaxed" / "off" strategies for reliably writing or not writing valid aces container metadata.