Add RAW query, test runtime path, and EXIF recovery

Adds conservative RAW-processing queries and plumbing, a CTest runtime-library path option, improved EXIF/TIFF recovery, and C2PA trusted-chain enforcement.

Details:
- API: metadata_query.h: new MetadataQueryKind::RawProcessing, several Raw-related semantic kinds and match terms, and query_raw_processing_metadata().
- Validation: validate.h adds verify_require_trusted_chain to opt-in trusted certificate-chain enforcement for draft C2PA verification; CLI/docs updated to document the option.
- Build/tests: CMake adds OPENMETA_TEST_RUNTIME_LIBRARY_PATH and openmeta_apply_test_runtime_environment() to prepend LD_LIBRARY_PATH/DYLD_LIBRARY_PATH/PATH for CTest-launched external tools; README and docs updated.
- Container scan: implement scan_exif_preamble_tiff and use it in scan_auto to recover standalone EXIF/TIFF when a short non-TIFF or malformed-JPEG prefix precedes an Exif preamble.
- JUMBF/C2PA: jumbf_decode enhancements to collect detached-payload candidates from nested/numeric claim references and better handle malformed COSE/claim shapes.
- Cleanup: removed several unused Nikon MakerNote helper functions and minor formatting fixes across sources and many documentation updates to reflect the new behaviors and transfer-safety refinements.

Purpose: enable safer RAW-processing inspection, let tests run external tools with non-default C++ runtimes, improve robustness recovering EXIF payloads in weird inputs, and provide opt-in stricter C2PA verification semantics.

Author: ssh4net

CHANGES.md

@@ -47,12 +47,19 @@ Changes compared with `0.4.8`.
   reduction/forward matrix sets, DNG white-balance vector sets, and
   lens-correction table groups. Candidate value-shape labels now include
   `vector_set`, `matrix_set`, and `table`.
+- Added `query_raw_processing_metadata(...)` plus Python wrappers for
+  conservative RAW-processing queries covering black/white levels,
+  linearization tables, CFA/sensor layout, source geometry, and raw-storage
+  identifiers.
 - Python `Document` and `TransferSourceSnapshot` now expose thin wrappers for
   the experimental semantic metadata query API, including generic
   `metadata_query(kind)` and focused query helpers.
 
 ### Changed
 
+- Added `OPENMETA_TEST_RUNTIME_LIBRARY_PATH` so CTest-launched external
+  validation tools can run with a matching non-default C++ runtime lookup path,
+  and documented the `libc++` test-prefix workflow.
 - Phase One-family IIQ MakerNote detection now recognizes Leaf/Credo-style
   files as Phase One MakerNotes before the generic Kodak `IIII` fallback.
 - Rendered-image transfer safety now treats Phase One/Leaf RAW sensor geometry,
@@ -66,8 +73,16 @@ Changes compared with `0.4.8`.
   computational, thermal, color/WB, geometry/storage, raw-data, sensor,
   lens-correction, preview/face geometry, stitch/panorama geometry, or
   vendor-private table metadata.
+- Rendered-image transfer safety now uses the current DNG tag numbers for
+  source-bound profile/gain tables, raw digests/storage identifiers,
+  forward matrices, and opcode/correction lists.
 - `metaread` and `python -m openmeta.python.metaread` now print compact Phase
   One RAW geometry/processing summaries when those decoded fields are present.
+- Draft C2PA verification now exposes opt-in trusted certificate-chain
+  enforcement through `ValidateOptions`, `metavalidate`, `metadump`, and Python
+  `read()`/`validate()` wrappers. Without this option, signature status and
+  chain-trust detail remain separate signals; with it, untrusted or missing
+  chains fail verification instead of reporting a loose `verified` result.
 - `metaread` and `python -m openmeta.python.metaread` now print compact
   Sony/Canon/Nikon/Fujifilm/Pentax/Panasonic/Olympus/Kodak/Minolta/Sigma/
   Samsung/Ricoh/Apple/DJI/Google/FLIR/Casio/Sanyo/KyoceraRaw/Reconyx/HP/JVC/
@@ -92,6 +107,12 @@ Changes compared with `0.4.8`.
 
 ### Fixed
 
+- Fixed standalone EXIF/TIFF recovery when a file has a short non-TIFF prefix
+  or malformed JPEG prefix before the `Exif` preamble.
+- Fixed draft C2PA verification status handling for malformed COSE_Sign1 byte
+  arrays, unresolved explicit detached-payload references, and nested numeric
+  claim references with conflicting label/URI fields.
+
 ### Tests And Validation
 
 - Added public synthetic coverage for Leaf/Credo IIQ MakerNote detection and
@@ -105,6 +126,8 @@ Changes compared with `0.4.8`.
 - Added public synthetic coverage for grouped semantic-query candidates:
   DNG color matrix sets, DNG white-balance vector sets, and vendor
   lens-correction table groups.
+- Added public synthetic coverage for standalone EXIF/TIFF recovery after
+  unknown-prefix and malformed-JPEG-prefix inputs.
 - Added public synthetic coverage for
   Sony/Canon/Nikon/Fujifilm/Pentax/Panasonic/Olympus/Kodak/Minolta/Sigma/
   Samsung/Ricoh/Apple/DJI/Google/FLIR/Casio/Sanyo/KyoceraRaw/Reconyx/HP/JVC/
@@ -122,8 +145,9 @@ Changes compared with `0.4.8`.
   stitch/panorama table entries.
 - Added rendered-image writer-output coverage for JPEG, TIFF/DNG, PNG, WebP,
   JP2, JXL, HEIF, AVIF, and CR3 metadata rewrites to verify serialized outputs
-  omit source RAW calibration, vendor private RAW tables, MakerNotes, camera
-  raw settings XMP, and source JUMBF while preserving safe EXIF/XMP fields.
+  omit source RAW calibration, raw digests/gain metadata, vendor private RAW
+  tables, MakerNotes, camera raw settings XMP, and source JUMBF while
+  preserving safe EXIF/XMP fields.
 - Added BMFF transfer coverage for merging metadata into a foreign top-level
   `meta` item table that uses `iinf` version 2.
 - Added BMFF transfer coverage that verifies multiple foreign top-level `meta`

CMakeLists.txt

@@ -98,6 +98,9 @@ if(NOT OPENMETA_FFMPEG_EXECUTABLE)
   endif()
 endif()
 
+set(OPENMETA_TEST_RUNTIME_LIBRARY_PATH "" CACHE STRING
+  "Optional runtime library path prepended to CTest tests that launch external tools")
+
 set(OPENMETA_BMFF_HEIF_TEST_TARGET "" CACHE FILEPATH "Optional existing HEIF/HEIC target image for external transfer usability checks")
 set(OPENMETA_BMFF_AVIF_TEST_TARGET "" CACHE FILEPATH "Optional existing AVIF target image for external transfer usability checks")
 set(OPENMETA_BMFF_CR3_TEST_TARGET "" CACHE FILEPATH "Optional existing CR3 target image for external transfer usability checks")
@@ -106,6 +109,34 @@ set(_openmeta_bmff_test_target_args
   "-DBMFF_AVIF_TEST_TARGET=${OPENMETA_BMFF_AVIF_TEST_TARGET}"
   "-DBMFF_CR3_TEST_TARGET=${OPENMETA_BMFF_CR3_TEST_TARGET}")
 
+function(openmeta_apply_test_runtime_environment test_name)
+  if(NOT OPENMETA_TEST_RUNTIME_LIBRARY_PATH)
+    return()
+  endif()
+
+  if(WIN32)
+    set(_openmeta_runtime_var "PATH")
+    set(_openmeta_runtime_sep ";")
+    set(_openmeta_runtime_existing "$ENV{PATH}")
+  elseif(APPLE)
+    set(_openmeta_runtime_var "DYLD_LIBRARY_PATH")
+    set(_openmeta_runtime_sep ":")
+    set(_openmeta_runtime_existing "$ENV{DYLD_LIBRARY_PATH}")
+  else()
+    set(_openmeta_runtime_var "LD_LIBRARY_PATH")
+    set(_openmeta_runtime_sep ":")
+    set(_openmeta_runtime_existing "$ENV{LD_LIBRARY_PATH}")
+  endif()
+
+  set(_openmeta_runtime_value "${OPENMETA_TEST_RUNTIME_LIBRARY_PATH}")
+  if(_openmeta_runtime_existing)
+    set(_openmeta_runtime_value
+      "${_openmeta_runtime_value}${_openmeta_runtime_sep}${_openmeta_runtime_existing}")
+  endif()
+  set_tests_properties(${test_name} PROPERTIES
+    ENVIRONMENT "${_openmeta_runtime_var}=${_openmeta_runtime_value}")
+endfunction()
+
 # Optional path to local dependency repos (used for the FuzzTest wrapper).
 # Expected layout:
 #   <root>/fuzztest
@@ -887,6 +918,13 @@ if(OPENMETA_BUILD_TESTS)
         -P "${CMAKE_CURRENT_SOURCE_DIR}/tests/python_metatransfer_edit_smoke_test.cmake"
     )
   endif()
+
+  if(OPENMETA_TEST_RUNTIME_LIBRARY_PATH)
+    get_property(_openmeta_test_names DIRECTORY PROPERTY TESTS)
+    foreach(_openmeta_test_name IN LISTS _openmeta_test_names)
+      openmeta_apply_test_runtime_environment("${_openmeta_test_name}")
+    endforeach()
+  endif()
 endif()
 
 if(OPENMETA_BUILD_FUZZERS)

README.md

@@ -259,6 +259,8 @@ Useful options:
 - `-DOPENMETA_BUILD_FUZZERS=ON` for Clang + libFuzzer targets
 - `-DOPENMETA_USE_LIBCXX=ON` when linking against dependencies built with
   `libc++`
+- `-DOPENMETA_TEST_RUNTIME_LIBRARY_PATH=/path/to/runtime-libs` when CTest
+  launches external tools that need a non-default C++ runtime lookup path
 - `-DOPENMETA_WITH_DNG_SDK_ADAPTER=ON` to enable the optional Adobe DNG SDK
   bridge (requires a discoverable `dng_sdk` package; intentionally excluded
   from public GitHub Actions CI)

docs/api_stability.md

@@ -22,7 +22,7 @@ different status.
 | Generic metadata traversal: `visit_metadata(...)`, `MetadataSink`, `ExportOptions`, `ExportItem` | `openmeta/interop_export.h` | Stable | v1 traversal contract. Borrowed names are valid only during `MetadataSink::on_item(...)`. |
 | `ExportNameStyle::Canonical` and `ExportNameStyle::XmpPortable` | `openmeta/interop_export.h` | Stable | Stable naming modes for key-space-aware and portable exports. |
 | `ExportNameStyle::FlatHost` | `openmeta/interop_export.h` | Stable | Stable v1 flat host naming contract. See [flat_host_mapping.md](flat_host_mapping.md). |
-| Semantic metadata query: `query_metadata(...)`, `query_crop_metadata(...)`, and focused query helpers | `openmeta/metadata_query.h` | Experimental | Query contract for inspection matches plus normalized candidates. Current coverage includes crop/active-area, exposure/gain, white balance, color, lens correction, and orientation metadata across standard tags, selected DNG tags, fuzzy XMP paths, and vendor RAW-processing classification, including grouped `matrix_set`, `vector_set`, and `table` candidates for related non-crop metadata. Python `Document` and `TransferSourceSnapshot` mirror this as thin dictionary-returning wrappers. |
+| Semantic metadata query: `query_metadata(...)`, `query_crop_metadata(...)`, and focused query helpers | `openmeta/metadata_query.h` | Experimental | Query contract for inspection matches plus normalized candidates. Current coverage includes crop/active-area, exposure/gain, white balance, color, lens correction, orientation, and RAW-processing metadata across standard tags, selected DNG tags, fuzzy XMP paths, and vendor RAW-processing classification. Grouped candidates include `matrix_set`, `vector_set`, and `table` shapes for related non-crop metadata, including RAW black/white levels, linearization, CFA/sensor layout, source geometry, and raw-storage identifiers. Python `Document` and `TransferSourceSnapshot` mirror this as thin dictionary-returning wrappers. |
 | Vendor RAW-processing summaries: `vendor_raw_processing_from_store(...)`, `classify_vendor_raw_processing_field(...)` | `openmeta/vendor_raw_processing.h` | Experimental | Conservative grouped source-RAW/source-processing field summaries for decoded Sony, Canon, Nikon, Fujifilm, Pentax, Panasonic, Olympus, Kodak, Minolta, Sigma, Samsung, Ricoh, Apple, DJI, Google, FLIR, Casio, Sanyo, KyoceraRaw, Reconyx, HP, JVC, GE, Motorola, Nintendo, and Microsoft MakerNotes, including vendor-private, computational, thermal, preview, face-geometry, and stitch/panorama buckets. Intended for audit/UI and rendered-transfer safety decisions, not for writing vendor RAW/source-processing values into rendered targets. |
 | Transfer safety audit: `transfer_safety_audit_from_store(...)` | `openmeta/metadata_transfer.h` | Experimental | Preflight summary of source entries and entries filtered or invalidated by `TransferSafetyMode`, including Sony/Canon/Nikon/Fujifilm/Pentax/Panasonic/Olympus/Kodak/Minolta/Sigma/Samsung/Ricoh/Apple/DJI/Google/FLIR/Casio/Sanyo/KyoceraRaw/Reconyx/HP/JVC/GE/Motorola/Nintendo/Microsoft RAW/source-processing buckets. Intended for diagnostics and host UI before preparing rendered-image transfers. |
 | Source snapshot type and read helpers: `TransferSourceSnapshot`, `read_transfer_source_snapshot_file(...)`, `read_transfer_source_snapshot_bytes(...)`, `build_transfer_source_snapshot(...)` | `openmeta/metadata_transfer.h` | Experimental | Current snapshots are decoded-store-backed and do not preserve raw source packets for passthrough. Const reuse is safe when callers do not mutate the snapshot and do not share returned result objects across writers. |

docs/development.md

@@ -17,7 +17,7 @@ model should stay compact:
 | --- | --- | --- |
 | Decoding | Find metadata carriers and decode EXIF, XMP, IPTC, ICC, Photoshop IRB, JUMBF/C2PA, EXR, and related blocks into `MetaStore` entries. | High, about 90-95% for the current target scope. |
 | Interpretation | Normalize names and values, group entries by meaning, and classify source-bound data such as RAW crop, color, lens-correction, sensor, and vendor-private fields. | Medium-high, about 75-85%. |
-| Query | Find entries by name, fuzzy term, or semantic group, for example crop/border/active-area fields or exposure/gain fields across standard and vendor metadata. | Low, about 15-20%. |
+| Query | Find entries by name, fuzzy term, or semantic group, for example crop/border/active-area, exposure/gain, color/WB, orientation, lens-correction, and RAW-processing fields across standard and vendor metadata. | Low, about 20-25%. |
 | Creation | Build fresh metadata entries from host-provided values. | Medium, about 55-65%. |
 | Editing | Modify existing logical metadata entries while preserving valid surrounding structure. | Medium, about 60-70%. |
 | Transfer | Move metadata between files using explicit compatible-file or rendered-image safety policies. | Medium-high, about 80-85%. |
@@ -35,15 +35,18 @@ rather than hiding ambiguity behind a single value.
 
 The first experimental C++ query surface is `openmeta/metadata_query.h`.
 It returns both raw matches and normalized candidates for crop/active-area,
-exposure/gain, white balance, color, lens correction, and orientation queries.
+exposure/gain, white balance, color, lens correction, orientation, and
+RAW-processing queries.
 Crop queries include DNG crop tags, `ActiveArea`, Phase One/Leaf raw geometry,
 and fuzzy crop/border-style XMP property paths. The non-crop queries expose
 per-entry value candidates and reuse standard tag names, selected DNG tags,
 fuzzy XMP paths, and vendor RAW-processing classification where applicable.
 They also append grouped candidates for related DNG color matrix/calibration/
 reduction/forward matrix tags, DNG white-balance vector tags, and
-lens-correction table groups, using `matrix_set`, `vector_set`, and `table`
-value shapes.
+lens-correction table groups. RAW-processing queries add conservative groups
+for black/white levels, linearization tables, CFA/sensor layout, source
+geometry, and raw-storage identifiers. Grouped candidates use `matrix_set`,
+`vector_set`, and `table` value shapes.
 Python `Document` and `TransferSourceSnapshot` mirror this as thin wrappers
 returning the same match/candidate dictionary shape.
 
@@ -75,7 +78,10 @@ dependencies let OpenMeta decode more content:
   `OPENMETA_C2PA_VERIFY_BACKEND`): enables backend selection/reporting fields
   (`none|auto|native|openssl`) and draft verification flow. Native backend
   availability is platform-based (Windows/macOS), while OpenSSL availability
-  is discovered via `find_package(OpenSSL)` when needed.
+  is discovered via `find_package(OpenSSL)` when needed. By default this
+  reports cryptographic signature status and trust-chain detail separately;
+  use `--c2pa-verify-require-trusted-chain` when validation must fail for an
+  untrusted or missing certificate chain.
 
 If you link against dependencies that were built with `libc++` (common when
 using Clang), configure OpenMeta with:
@@ -84,6 +90,15 @@ using Clang), configure OpenMeta with:
 -DOPENMETA_USE_LIBCXX=ON
 ```
 
+When CTest launches external validation tools from the same dependency prefix,
+those tools must also be able to find their matching C++ runtime libraries. If
+the runtime is not discoverable through the system loader, pass an explicit
+runtime directory:
+
+```bash
+-DOPENMETA_TEST_RUNTIME_LIBRARY_PATH=/path/to/runtime-libs
+```
+
 ## Versioning
 
 `VERSION` is the single source of truth for the project version:
@@ -106,6 +121,9 @@ using Clang), configure OpenMeta with:
 
 #Validate with sidecar + MakerNotes + C2PA verify status
 ./build/metavalidate --xmp-sidecar --makernotes --c2pa-verify input.jpg
+
+#Require C2PA signature verification and a trusted certificate chain
+./build/metavalidate --c2pa-verify --c2pa-verify-require-trusted-chain input.jpg
 ```
 `metavalidate` CLI is a thin wrapper over `openmeta::validate_file(...)`.
 Machine-readable JSON output includes issue codes suitable for gating, for
@@ -126,6 +144,9 @@ example `xmp/output_truncated` and `xmp/invalid_or_malformed_xml_text`.
 #Portable sidecar + draft C2PA verify scaffold status reporting
 ./build/metadump --format portable --c2pa-verify --c2pa-verify-backend auto input.jpg output.xmp
 
+#Portable sidecar with trusted-chain C2PA verification required
+./build/metadump --format portable --c2pa-verify --c2pa-verify-require-trusted-chain input.jpg output.xmp
+
 #Explicit input / output form
 ./build/metadump -i input.jpg -o output.xmp
 
@@ -723,6 +744,11 @@ This policy surface is intentionally marked draft and may be refined.
       `verification_failed`) when a signature entry provides algorithm +
       signing input + public key material (`public_key_der`/`public_key_pem` or
       `certificate_der`).
+    - opt-in trusted-chain enforcement via
+      `verify_require_trusted_chain` /
+      `--c2pa-verify-require-trusted-chain`; without this option, signature
+      verification and certificate-chain trust are reported as separate
+      signals.
     - COSE_Sign1 support (array or embedded CBOR byte-string forms): extracts
       `alg` from protected headers, reconstructs Sig_structure signing bytes
       when payload is present, extracts `x5chain` from unprotected headers, and
@@ -806,6 +832,11 @@ Note: if your GoogleTest was built against `libc++` (common with Clang),
 build OpenMeta against the same C++ standard library. Otherwise you may see
 link errors involving `std::__1` vs `std::__cxx11`.
 
+If external test tools were built with `libc++` and need a custom runtime
+lookup path, set `OPENMETA_TEST_RUNTIME_LIBRARY_PATH` at configure time. This
+is applied to CTest-launched tests through `LD_LIBRARY_PATH` on Linux and
+`DYLD_LIBRARY_PATH` on macOS.
+
 Build + run:
 ```bash
 cmake -S . -B build-tests -G Ninja -DCMAKE_BUILD_TYPE=Debug \

docs/host_integration.md

@@ -30,11 +30,13 @@ Use the narrowest public API that matches your host:
 | Existing target file or template | `execute_prepared_transfer_file(...)` + `persist_prepared_transfer_file_result(...)` |
 | EXR writer | `build_exr_attribute_batch_from_file(...)` |
 | Host-owned metadata object model | `visit_metadata(...)` |
+| Host metadata inspection/search UI | `openmeta/metadata_query.h` focused query helpers |
 | JPEG/JXL/WebP/PNG/JP2/BMFF encoder path | `prepare_metadata_for_target_file(...)` + adapter view or backend emitter |
 | Adobe DNG SDK objects/files | `dng_sdk_adapter.h` |
 
-There is no public fuzzy-search API yet. Query through exact keys and build
-your own display or search layer on top.
+For inspection/search UI, prefer the experimental semantic query helpers before
+building a separate fuzzy layer. They report source entries, confidence, value
+shape, and normalized candidates while preserving ambiguity.
 
 ## Adapter Classes
 
@@ -430,7 +432,7 @@ Use `TransferProfile::safety` for the broad source/destination relationship:
 | Mode | Use when | Transfer policy |
 | --- | --- | --- |
 | `CompatibleFile` | Metadata is repackaged or recompressed into a compatible file/pixel representation | Preserve source camera, color, ICC, and camera-specific data except target-owned image-layout fields |
-| `RenderedImage` | Pixels may have changed, especially RAW-to-JPEG/PNG/WebP/JXL/HEIF/AVIF export | Keep general/time/GPS/IPTC/portable XMP; drop source raw color calibration, linearization/crop/correction metadata, vendor RAW/source-processing geometry/color/correction/thermal/computational/private/stitch fields, camera raw settings XMP, source ICC, opaque MakerNotes, and non-C2PA JUMBF |
+| `RenderedImage` | Pixels may have changed, especially RAW-to-JPEG/PNG/WebP/JXL/HEIF/AVIF export | Keep general/time/GPS/IPTC/portable XMP; drop source raw color calibration, profile/gain tables, raw digests/storage identifiers, linearization/crop/correction metadata, vendor RAW/source-processing geometry/color/correction/thermal/computational/private/stitch fields, camera raw settings XMP, source ICC, opaque MakerNotes, and non-C2PA JUMBF |
 
 See [writer_target_contract.md](writer_target_contract.md#transfer-safety-matrix)
 for the detailed per-group transfer matrix.