Add bounded BMFF foreign-meta merge, docs, tests

Implement bounded BMFF foreign top-level `meta` merge/replace/strip and ICC property merge support. Adds new BMFF parsing/merge logic and rewrite policy in metadata_transfer.cc, updates Nikon MakerNote flash decoding, and extends tests to cover BMFF XMP/ICC transfer paths. Add CMake cache options (OPENMETA_FFMPEG_EXECUTABLE, OPENMETA_BMFF_HEIF/AVIF/CR3_TEST_TARGET) and wire them into the external image-usability and transfer release gates (with optional ffmpeg fallback). Update documentation (README, multiple docs/sphinx/md files) to describe the new bounded contract and test behavior, add CHANGES.md, and bump VERSION to 0.4.7.

Author: ssh4net

CHANGES.md

@@ -0,0 +1,59 @@
+# OpenMeta Changes
+
+## 0.4.7 - 2026-04-27
+
+Changes compared with `0.4.6`.
+
+### Added
+
+- Added bounded foreign top-level BMFF `meta` merge support for parseable
+  HEIF/AVIF/CR3-style item graphs. OpenMeta can now merge prepared
+  Exif/XMP/JUMBF/C2PA item metadata into an existing foreign `meta` graph
+  instead of appending a second competing `meta` box.
+- Added BMFF XMP replacement and strip support for foreign item graphs that
+  satisfy the bounded primary-item contract: a single parseable `iinf`,
+  `iloc` version 0/1/2, `pitm`, and at most one `idat`.
+- Added bounded BMFF ICC property merge support for foreign `meta` graphs.
+  Existing ICC `colr/prof` and `colr/rICC` properties are removed from
+  `iprp/ipco`, existing `ipma` associations are compacted/remapped, and the
+  transferred `colr/prof` property is associated with the primary item.
+- Added public CMake cache options for external BMFF usability checks:
+  `OPENMETA_BMFF_HEIF_TEST_TARGET`, `OPENMETA_BMFF_AVIF_TEST_TARGET`,
+  `OPENMETA_BMFF_CR3_TEST_TARGET`, and `OPENMETA_FFMPEG_EXECUTABLE`.
+- Added external image-usability gate coverage for BMFF ICC and XMP transfer
+  on configured HEIF/AVIF/CR3 targets when local tools can validate them.
+
+### Changed
+
+- BMFF edit/apply now preserves non-`meta` top-level boxes while rebuilding
+  supported foreign `meta` item/property graphs in place.
+- BMFF embedded-XMP strip mode no longer requires an OpenMeta-authored
+  metadata `meta` box when the foreign graph is parseable and has a valid
+  primary-item relationship.
+- BMFF summary output and gate checks now cover both `bmff_item mime/xmp` and
+  `bmff_property colr/prof` transfer results.
+- Public transfer docs now describe the updated BMFF preserve/replace/strip
+  contract and the configured-target validation limits.
+
+### Fixed
+
+- Fixed duplicate or stale BMFF ICC associations by replacing prior ICC
+  properties and remapping `ipma` instead of adding competing property
+  entries.
+- Fixed supported foreign BMFF XMP strip/replacement paths that previously
+  failed even when the target graph had enough structure for a safe bounded
+  rewrite.
+- Fixed Nikon MakerNote FlashInfo decoding for `0107`/`0108` and
+  `0300`/`0301` layouts by emitting Flash Group A/B/C control-mode fields
+  with ExifTool-compatible contextual names.
+
+### Tests And Validation
+
+- Added focused BMFF API tests for foreign Exif/XMP item replacement, XMP
+  strip, ICC property replacement, existing `ipma` merge/remap, `iloc`
+  rebasing, and fail-safe rejection of unsupported foreign graphs.
+- Extended Nikon MakerNote tests for Flash Group A/B/C control-mode decoding.
+- Extended the public transfer release and image-usability gates to accept
+  configured BMFF target files and optional ffmpeg decode fallback.
+- Verified the public release with the unit test suite, transfer release gate,
+  external image-usability gate, documentation build, and whitespace checks.

CMakeLists.txt

@@ -86,6 +86,26 @@ if(NOT OPENMETA_EXIFTOOL_EXECUTABLE)
   endif()
 endif()
 
+set(OPENMETA_FFMPEG_EXECUTABLE "" CACHE FILEPATH "Optional ffmpeg executable for external image decode checks")
+if(NOT OPENMETA_FFMPEG_EXECUTABLE)
+  find_program(_openmeta_ffmpeg_executable
+    NAMES ffmpeg
+    HINTS ${CMAKE_PREFIX_PATH}
+    PATH_SUFFIXES bin)
+  if(_openmeta_ffmpeg_executable)
+    set(OPENMETA_FFMPEG_EXECUTABLE "${_openmeta_ffmpeg_executable}"
+      CACHE FILEPATH "Optional ffmpeg executable for external image decode checks" FORCE)
+  endif()
+endif()
+
+set(OPENMETA_BMFF_HEIF_TEST_TARGET "" CACHE FILEPATH "Optional existing HEIF/HEIC target image for external transfer usability checks")
+set(OPENMETA_BMFF_AVIF_TEST_TARGET "" CACHE FILEPATH "Optional existing AVIF target image for external transfer usability checks")
+set(OPENMETA_BMFF_CR3_TEST_TARGET "" CACHE FILEPATH "Optional existing CR3 target image for external transfer usability checks")
+set(_openmeta_bmff_test_target_args
+  "-DBMFF_HEIF_TEST_TARGET=${OPENMETA_BMFF_HEIF_TEST_TARGET}"
+  "-DBMFF_AVIF_TEST_TARGET=${OPENMETA_BMFF_AVIF_TEST_TARGET}"
+  "-DBMFF_CR3_TEST_TARGET=${OPENMETA_BMFF_CR3_TEST_TARGET}")
+
 # Optional path to local dependency repos (used for the FuzzTest wrapper).
 # Expected layout:
 #   <root>/fuzztest
@@ -431,7 +451,9 @@ if(OPENMETA_BUILD_TOOLS)
           "-DMETATRANSFER_BIN=$<TARGET_FILE:metatransfer>"
           "-DOIIOTOOL_BIN=${OPENMETA_OIIOTOOL_EXECUTABLE}"
           "-DEXIFTOOL_BIN=${OPENMETA_EXIFTOOL_EXECUTABLE}"
+          "-DFFMPEG_BIN=${OPENMETA_FFMPEG_EXECUTABLE}"
           "-DWORK_DIR=${CMAKE_CURRENT_BINARY_DIR}/_metatransfer_image_usability"
+          ${_openmeta_bmff_test_target_args}
           -P "${CMAKE_CURRENT_SOURCE_DIR}/tests/metatransfer_image_usability_test.cmake"
         COMMENT "Running metatransfer external image usability gate"
         VERBATIM
@@ -699,6 +721,8 @@ if(OPENMETA_BUILD_TESTS)
         "-DOPENMETA_PYTHONPATH=${CMAKE_CURRENT_BINARY_DIR}/python"
         "-DOIIOTOOL_BIN=${OPENMETA_OIIOTOOL_EXECUTABLE}"
         "-DEXIFTOOL_BIN=${OPENMETA_EXIFTOOL_EXECUTABLE}"
+        "-DFFMPEG_BIN=${OPENMETA_FFMPEG_EXECUTABLE}"
+        ${_openmeta_bmff_test_target_args}
         -P "${CMAKE_CURRENT_SOURCE_DIR}/tests/transfer_release_gate.cmake"
       COMMENT "Running transfer release gate"
       VERBATIM
@@ -713,6 +737,8 @@ if(OPENMETA_BUILD_TESTS)
         "-DOPENMETA_PYTHONPATH=${CMAKE_CURRENT_BINARY_DIR}/python"
         "-DOIIOTOOL_BIN=${OPENMETA_OIIOTOOL_EXECUTABLE}"
         "-DEXIFTOOL_BIN=${OPENMETA_EXIFTOOL_EXECUTABLE}"
+        "-DFFMPEG_BIN=${OPENMETA_FFMPEG_EXECUTABLE}"
+        ${_openmeta_bmff_test_target_args}
         -P "${CMAKE_CURRENT_SOURCE_DIR}/tests/transfer_release_gate.cmake"
     )
   elseif(TARGET metatransfer)
@@ -724,6 +750,8 @@ if(OPENMETA_BUILD_TESTS)
         "-DWORK_DIR=${CMAKE_CURRENT_BINARY_DIR}/_transfer_release_gate"
         "-DOIIOTOOL_BIN=${OPENMETA_OIIOTOOL_EXECUTABLE}"
         "-DEXIFTOOL_BIN=${OPENMETA_EXIFTOOL_EXECUTABLE}"
+        "-DFFMPEG_BIN=${OPENMETA_FFMPEG_EXECUTABLE}"
+        ${_openmeta_bmff_test_target_args}
         -P "${CMAKE_CURRENT_SOURCE_DIR}/tests/transfer_release_gate.cmake"
       COMMENT "Running transfer release gate"
       VERBATIM
@@ -736,6 +764,8 @@ if(OPENMETA_BUILD_TESTS)
         "-DWORK_DIR=${CMAKE_CURRENT_BINARY_DIR}/_transfer_release_gate"
         "-DOIIOTOOL_BIN=${OPENMETA_OIIOTOOL_EXECUTABLE}"
         "-DEXIFTOOL_BIN=${OPENMETA_EXIFTOOL_EXECUTABLE}"
+        "-DFFMPEG_BIN=${OPENMETA_FFMPEG_EXECUTABLE}"
+        ${_openmeta_bmff_test_target_args}
         -P "${CMAKE_CURRENT_SOURCE_DIR}/tests/transfer_release_gate.cmake"
     )
   endif()
@@ -780,7 +810,9 @@ if(OPENMETA_BUILD_TESTS)
           "-DMETATRANSFER_BIN=$<TARGET_FILE:metatransfer>"
           "-DOIIOTOOL_BIN=${OPENMETA_OIIOTOOL_EXECUTABLE}"
           "-DEXIFTOOL_BIN=${OPENMETA_EXIFTOOL_EXECUTABLE}"
+          "-DFFMPEG_BIN=${OPENMETA_FFMPEG_EXECUTABLE}"
           "-DWORK_DIR=${CMAKE_CURRENT_BINARY_DIR}/_metatransfer_image_usability"
+          ${_openmeta_bmff_test_target_args}
           -P "${CMAKE_CURRENT_SOURCE_DIR}/tests/metatransfer_image_usability_test.cmake"
       )
     endif()

README.md

@@ -142,9 +142,12 @@ In practice:
   terms are not part of the public CI dependency story; SDK-backed coverage is
   treated as maintainer or release validation.
 - PNG, WebP, JP2, JXL, bounded BMFF, and EXR all have real first-class
-  transfer entry points. BMFF file edits are currently limited to
-  OpenMeta-managed metadata-only `meta` boxes; targets with foreign top-level
-  `meta` boxes are rejected instead of risking an unusable HEIF/AVIF result.
+  transfer entry points. BMFF file edits can replace OpenMeta-authored
+  metadata-only `meta` boxes and can merge, replace, or strip bounded
+  Exif/XMP/JUMBF/C2PA metadata items plus bounded ICC `colr/prof` properties
+  in a parseable foreign top-level `meta` graph. Arbitrary BMFF
+  scene/property-graph rewrite remains out of scope for the bounded writer
+  path.
 - EXR is still narrower than the container-edit targets: it emits safe string
   header attributes through the transfer core, can materialize a prepared
   `ExrAdapterBatch` for host exporters, and Python can inspect that prepared

VERSION

@@ -1 +1 @@
-0.4.6
+0.4.7

docs/development.md

@@ -317,8 +317,15 @@ Current v1 behavior is:
       metadata-only `meta` box from the same bounded contract. It preserves
       non-`meta` top-level BMFF boxes and replaces the OpenMeta-authored
       metadata-only `meta` box with the prepared BMFF items/properties.
-      Targets with foreign top-level `meta` boxes are rejected until OpenMeta
-      can merge real BMFF item/property graphs safely.
+    - For targets with a parseable foreign top-level `meta` box, OpenMeta can
+      merge, replace, or strip bounded Exif/XMP/JUMBF/C2PA items by extending
+      `iinf`, `iloc`, `idat`, and `iref` with `cdsc` references to the primary
+      item. This path requires a single parseable `iinf`, `iloc` version
+      0/1/2, `pitm`, and at most one `idat`. Bounded ICC transfer removes
+      prior ICC `colr/prof` and `colr/rICC` properties from `iprp/ipco`,
+      compacts/remaps existing `ipma` associations, appends the transferred
+      `colr/prof` property, and associates it with the primary item.
+      Arbitrary BMFF scene/property graph rewrite remains unsupported.
     - CLI/Python `metatransfer` wrappers expose both BMFF summaries and this
       bounded edit path; `--target-heif`, `--target-avif`, and `--target-cr3`
       now accept `--source-meta PATH` plus `--output PATH` for metadata
@@ -816,6 +823,19 @@ cmake --build build-tests --target openmeta_gate_transfer_release
 ctest --test-dir build-tests -R openmeta_transfer_release_gate --output-on-failure
 ```
 
+The external image-usability gate can also use existing BMFF target files when
+local tools cannot create them:
+`OPENMETA_BMFF_HEIF_TEST_TARGET`, `OPENMETA_BMFF_AVIF_TEST_TARGET`, and
+`OPENMETA_BMFF_CR3_TEST_TARGET`. Arbitrary configured targets exercise the ICC
+property and XMP item transfer/read-back paths. The EXIF image-property
+transfer path only runs for configured targets that match the 64x32,
+3-channel fixture shape, so the gate does not intentionally write mismatched
+image geometry. The configured XMP assertion is based on OpenMeta's BMFF
+summary; ExifTool title validation is also applied for formats where ExifTool
+exposes the generic BMFF XMP item. If the local `oiiotool` build cannot decode
+a configured BMFF target after rewrite, `OPENMETA_FFMPEG_EXECUTABLE` can
+provide the decode fallback.
+
 The public GitHub Actions workflow `.github/workflows/ci.yml` runs two Linux
 variants of these public release gates:
 - self-contained non-Python, non-DNG-SDK

docs/metadata_backend_matrix.md

@@ -115,8 +115,7 @@ For the public per-target preserve/replace guarantees, see
 ### ISO-BMFF metadata items (HEIF / AVIF / CR3)
 
 - This bounded transfer path is metadata-item/property oriented. OpenMeta also
-  supports a bounded OpenMeta-managed BMFF metadata edit path over an existing
-  BMFF target file.
+  supports a bounded BMFF metadata edit path over an existing BMFF target file.
 - Current prepared item routes:
   - `bmff:item-exif`
   - `bmff:item-xmp`
@@ -145,16 +144,26 @@ For the public per-target preserve/replace guarantees, see
 - The shared package-batch persistence/replay layer can own and hand off
   stable BMFF item and property payload bytes.
 - `metatransfer` / `openmeta.transfer_probe(...)` expose BMFF summaries,
-  including `bmff_property colr/prof ...`.
+  including `bmff_item mime/xmp ...` and `bmff_property colr/prof ...`.
 - `metatransfer --target-heif|--target-avif|--target-cr3 --source-meta ... -o ...`
-  performs bounded metadata-only edit only for targets with no foreign
-  top-level `meta` box, or with a prior OpenMeta-authored metadata-only `meta`
-  box from the same bounded contract. Foreign top-level `meta` boxes are
-  rejected until OpenMeta can merge real BMFF item/property graphs safely.
-- Embedded-XMP strip mode removes only OpenMeta-authored metadata `meta` boxes.
-  Foreign BMFF metadata graphs are not modified; recognized foreign XMP `mime`
-  items, including `iinf` version 0/1/2 tables, make strip mode fail explicitly
-  instead of silently claiming removal.
+  performs bounded metadata edits for targets with no foreign top-level `meta`
+  box, with a prior OpenMeta-authored metadata-only `meta` box from the same
+  bounded contract, or with a parseable foreign top-level `meta` item graph.
+  The foreign-`meta` path merges, replaces, or strips bounded
+  Exif/XMP/JUMBF/C2PA metadata items by extending `iinf`, `iloc`, `idat`, and
+  `iref` with `cdsc` references to the primary item. It requires a single
+  parseable `iinf`, `iloc` version 0/1/2, `pitm`, and at most one `idat`.
+- Foreign-`meta` ICC property merge is bounded to
+  `bmff:property-colr-icc`. OpenMeta removes prior ICC `colr/prof` and
+  `colr/rICC` properties from `iprp/ipco`, compacts/remaps existing `ipma`
+  associations, appends the transferred `colr/prof` property, and associates
+  it with the primary item. Broader non-ICC property replacement and arbitrary
+  scene/property-graph rewrites remain out of scope for the current contract.
+- Embedded-XMP strip mode removes XMP from OpenMeta-authored metadata `meta`
+  boxes and from parseable foreign top-level `meta` item graphs that satisfy
+  the same bounded merge contract. Foreign graphs without a primary item
+  relationship (`pitm`) or outside the supported `iinf`/`iloc`/`idat`/`iref`
+  shape still fail explicitly instead of silently claiming removal.
 - The same bounded BMFF edit contract now also participates in the core /
   file-helper C2PA signer path:
   - sign-request derivation
@@ -163,7 +172,7 @@ For the public per-target preserve/replace guarantees, see
   - staged bmff:item-c2pa apply before bounded metadata-only edit
 - Out of scope for the current BMFF contract:
   - thin CLI/Python signer-input exposure for BMFF
-  - merging or rewriting foreign top-level BMFF meta item/property graphs
+  - arbitrary rewrite of foreign top-level BMFF scene/property graphs
   - full BMFF signed rewrite/re-sign beyond the bounded metadata-only edit path
 
 ### EXR (OpenEXR)

docs/metadata_transfer_plan.md

@@ -55,7 +55,7 @@ The first public write-side sync controls are also in place:
 | WebP | Bounded but real | Prepared bundle, compiled emit, bounded chunk rewrite/edit, file-helper roundtrip | Not a general WebP chunk editor |
 | JP2 | Bounded but real | Prepared bundle, compiled emit, bounded box rewrite/edit, file-helper roundtrip | `jp2h` synthesis is still out of scope |
 | JXL | Bounded but real | Prepared bundle, compiled emit, bounded box rewrite/edit, file-helper roundtrip | Still narrower than JPEG/TIFF |
-| HEIF / AVIF / CR3 | Bounded but real | Prepared bundle, compiled emit, bounded OpenMeta-managed BMFF item/property edit, file-helper roundtrip | Not broad BMFF writer parity; foreign top-level `meta` graphs are rejected until merge support exists |
+| HEIF / AVIF / CR3 | Bounded but real | Prepared bundle, compiled emit, OpenMeta-managed BMFF item/property edit, constrained foreign-`meta` item merge/replacement/strip plus bounded ICC property merge, file-helper roundtrip | Not broad BMFF writer parity; arbitrary foreign `meta` scene/property-graph rewrite is still unsupported |
 | EXR | Bounded but real | Prepared bundle, compiled emit, direct backend attribute emit, prepared-bundle to `ExrAdapterBatch` bridge, CLI/Python transfer surface | No file rewrite/edit path yet; current transfer payload is safe string attributes only |
 
 ## What Is Already Implemented
@@ -104,17 +104,23 @@ OpenMeta now has explicit end-to-end read-backed transfer tests for:
 - source JPEG -> WebP edit/apply -> read-back
 - source JPEG -> JP2 edit/apply -> read-back
 - source JPEG -> JXL edit/apply -> read-back
-- source JPEG -> OpenMeta-managed HEIF edit/apply -> read-back
-- source JPEG -> OpenMeta-managed AVIF edit/apply -> read-back
-- source JPEG -> OpenMeta-managed CR3 edit/apply -> read-back
+- source JPEG -> bounded HEIF edit/apply -> read-back
+- source JPEG -> bounded AVIF edit/apply -> read-back
+- source JPEG -> bounded CR3 edit/apply -> read-back
+- source ICC -> bounded BMFF ICC property edit/apply -> read-back and
+  external validation when local HEIF/AVIF/CR3 encoders can create usable
+  targets
+- source XMP -> bounded BMFF XMP item edit/apply -> read-back and external
+  validation on configured HEIF/AVIF/CR3 targets where local tools expose the
+  transferred XMP payload
 
 That does not make all targets equally mature, but it does mean the transfer
 core has real roundtrip regression gates across the primary supported export
 families.
 
 The primary writer family is also covered by a deterministic compatibility-dump
-gate for JPEG, TIFF, DNG, PNG, WebP, JP2, JXL, and OpenMeta-managed HEIF,
-AVIF, and CR3. That gate
+gate for JPEG, TIFF, DNG, PNG, WebP, JP2, JXL, and bounded HEIF, AVIF, and
+CR3 item metadata edits. That gate
 checks the prepared EXIF/XMP routes, edit/apply status, dual-write XMP
 writeback summary, and decoded metadata dump for the managed source fields.
 Additional compatibility-dump gates cover sidecar-only writeback with explicit
@@ -136,6 +142,13 @@ In a Python-enabled test tree it also runs:
 - `openmeta_python_transfer_probe_smoke`
 - `openmeta_python_metatransfer_edit_smoke`
 
+The external image-usability gate can use optional configured HEIF, AVIF, and
+CR3 target files via CMake cache paths when local tools cannot create those
+formats. Arbitrary configured BMFF targets exercise the ICC property and XMP
+item transfer/read-back routes; the EXIF image-property route remains limited
+to the 64x32, 3-channel fixture shape to avoid intentionally mismatched
+geometry.
+
 ## Per-Target Notes
 
 ### JPEG
@@ -288,10 +301,16 @@ Implemented as a bounded BMFF target family:
 - bounded `bmff:item-c2pa`
 - bounded `bmff:property-colr-icc`
 - bounded OpenMeta-managed metadata-only `meta` rewrite path
-- explicit strip-mode rejection for recognized foreign XMP item graphs,
-  including `iinf` version 0/1/2 `mime` entries
-- fail-safe rejection for foreign top-level `meta` boxes until real BMFF
-  item/property merge support exists
+- constrained foreign top-level `meta` item merge for parseable `iinf`,
+  `iloc` version 0/1/2, `pitm`, optional single `idat`, and primary-item `cdsc`
+  references
+- bounded foreign top-level `meta` ICC property merge by replacing prior ICC
+  `colr/prof` and `colr/rICC` properties, remapping `ipma`, and associating the
+  transferred `colr/prof` property with the primary item
+- bounded foreign top-level `meta` XMP replacement and strip support for
+  parseable item graphs that satisfy the same primary-item contract
+- fail-safe rejection for unsupported foreign top-level `meta` shapes and
+  broader BMFF item/property graph rewrite shapes outside the bounded contract
 
 ### EXR
 
@@ -787,7 +806,7 @@ embedded/sidecar writeback across the primary writer target family.
 #### 3. Compare-Backed Release Validation
 
 - [x] promote the current primary-target roundtrip checks into explicit release-facing compare gates
-- [x] add compare-backed validation for `TIFF`, `DNG`, `PNG`, `WebP`, `JP2`, `JXL`, and OpenMeta-managed bounded `BMFF` target outputs, plus fail-safe rejection for foreign top-level `meta` BMFF targets
+- [x] add compare-backed validation for `TIFF`, `DNG`, `PNG`, `WebP`, `JP2`, `JXL`, and bounded `BMFF` target outputs, including fail-safe rejection for unsupported foreign top-level `meta` BMFF shapes
 - [x] cover embedded-only, sidecar-only, and dual-write XMP flows in release-facing compare validation
 - [x] add compare-backed validation for explicit sidecar-base overrides and destination-sidecar cleanup behavior
 - [x] gate the primary writer family on deterministic read-back of managed metadata after edit/apply