Skip to content

Commit ace0842

Browse files
ssh4netssh4net
committed
Add compatibility dump API, docs and contracts
Introduce a deterministic compatibility dump feature and related public contracts. Adds a new compatibility_dump header and implementation, options for metadata/transfer dumps, and a unit test; wires the new source and test into CMake. Publish stability and contract docs (API stability, compatibility dump, FlatHost mapping, XMP sync/writeback) including Sphinx rst pages and update cross-references in existing docs. Surface stable/experimental contract annotations and constants in public headers (interop_export, metadata_capabilities, metadata_transfer) and add the FlatHost contract version constant. These changes provide a stable v1 compatibility dump and naming contract so downstream hosts can produce deterministic baselines and understand API stability.
1 parent 82cdc5d commit ace0842

27 files changed

Lines changed: 2092 additions & 12 deletions

‎CMakeLists.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,7 @@ set(OPENMETA_SOURCES
218218
src/openmeta/byte_arena.cc
219219
src/openmeta/bmff_fields_decode.cc
220220
src/openmeta/ccm_query.cc
221+
src/openmeta/compatibility_dump.cc
221222
src/openmeta/console_format.cc
222223
src/openmeta/container_payload.cc
223224
src/openmeta/container_scan.cc
@@ -590,6 +591,7 @@ if(OPENMETA_BUILD_TESTS)
590591

591592
add_executable(openmeta_tests
592593
tests/ccm_query_test.cc
594+
tests/compatibility_dump_test.cc
593595
tests/container_payload_test.cc
594596
tests/container_scan_test.cc
595597
tests/bmff_fields_decode_test.cc

‎docs/api_stability.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# API Stability
2+
3+
This page defines the adoption status for public OpenMeta APIs.
4+
Python bindings mirror these labels unless a Python wrapper documents a
5+
different status.
6+
7+
## Stability Levels
8+
9+
| Level | Meaning |
10+
| --- | --- |
11+
| Stable | Intended for downstream use. Breaking changes require a new contract version, a compatibility path, or a documented migration. |
12+
| Experimental | Public and tested, but the exact shape or semantics may still evolve while the surrounding workflow is being hardened. |
13+
| Internal | Publicly visible only because it is part of a lower-level implementation surface. Do not build new downstream integrations on it unless another doc names it as supported. |
14+
15+
## Host-Facing API Map
16+
17+
| API surface | Header | Stability | Notes |
18+
| --- | --- | --- | --- |
19+
| Runtime capability query: `metadata_capability(...)` | `openmeta/metadata_capabilities.h` | Stable | v1 query contract for read, structured decode, transfer preparation, target edit, and raw-preservation status by format/family. |
20+
| Compatibility dumps: `dump_metadata_compatibility(...)`, `dump_transfer_compatibility(...)` | `openmeta/compatibility_dump.h` | Stable | Stable v1 line-oriented compatibility dump contract. See [compatibility_dump.md](compatibility_dump.md). |
21+
| XMP sync and writeback policy enums: `XmpConflictPolicy`, existing-carrier precedence enums, `XmpWritebackMode`, destination carrier modes | `openmeta/xmp_dump.h`, `openmeta/metadata_transfer.h` | Stable | Stable bounded writer policy for generated portable XMP. See [xmp_sync_policy.md](xmp_sync_policy.md). |
22+
| Generic metadata traversal: `visit_metadata(...)`, `MetadataSink`, `ExportOptions`, `ExportItem` | `openmeta/interop_export.h` | Stable | v1 traversal contract. Borrowed names are valid only during `MetadataSink::on_item(...)`. |
23+
| `ExportNameStyle::Canonical` and `ExportNameStyle::XmpPortable` | `openmeta/interop_export.h` | Stable | Stable naming modes for key-space-aware and portable exports. |
24+
| `ExportNameStyle::FlatHost` | `openmeta/interop_export.h` | Stable | Stable v1 flat host naming contract. See [flat_host_mapping.md](flat_host_mapping.md). |
25+
| Source snapshot type and read helpers: `TransferSourceSnapshot`, `read_transfer_source_snapshot_file(...)`, `read_transfer_source_snapshot_bytes(...)`, `build_transfer_source_snapshot(...)` | `openmeta/metadata_transfer.h` | Experimental | Current snapshots are decoded-store-backed and do not preserve raw source packets for passthrough. Const reuse is safe when callers do not mutate the snapshot and do not share returned result objects across writers. |
26+
| Fileless preparation: `prepare_metadata_for_target_snapshot(...)` | `openmeta/metadata_transfer.h` | Experimental | Intended for hosts that already decoded metadata and want to prepare transfer artifacts without reopening the source file. |
27+
| Snapshot execution: `execute_prepared_transfer_snapshot(...)` | `openmeta/metadata_transfer.h` | Experimental | Intended for deferred save/writeback from a reusable decoded source snapshot. |
28+
| Bundle execution: `execute_prepared_transfer_bundle(...)` | `openmeta/metadata_transfer.h` | Experimental | Intended for hosts that already own a prepared bundle and destination bytes. Treat bundles as immutable except through documented patch helpers. |
29+
| Adapter-view execution: `build_prepared_transfer_adapter_view(...)`, `emit_prepared_transfer_adapter_view(...)` | `openmeta/metadata_transfer.h` | Experimental | Target-neutral operation view for host-owned encoders and writers. Route and dispatch details may still evolve. |
30+
| Generated transfer payload internals, route strings, low-level package chunks, and diagnostic counters not documented by a stable API page | `openmeta/metadata_transfer.h` | Internal | These fields may be useful for tests and diagnostics, but they are not a compatibility contract for downstream integrations. |
31+
32+
## Practical Guidance
33+
34+
Use stable APIs for normal application integrations. Use experimental APIs when
35+
they match a real workflow and the integration can track OpenMeta releases.
36+
Avoid internal surfaces unless you are contributing to OpenMeta itself or
37+
writing a test that is intentionally tied to implementation details.

‎docs/compatibility_dump.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# Compatibility Dump Contract
2+
3+
OpenMeta exposes a deterministic text dump for downstream compatibility tests.
4+
The goal is to let host projects compare names, value shapes, origins, and
5+
transfer decisions without keeping binary metadata packet baselines.
6+
7+
Contract constant:
8+
9+
```cpp
10+
openmeta::kCompatibilityDumpContractVersion == 1
11+
```
12+
13+
Python exposes the same version as:
14+
15+
```python
16+
openmeta.COMPATIBILITY_DUMP_CONTRACT_VERSION == 1
17+
```
18+
19+
## Metadata Dump
20+
21+
Use `dump_metadata_compatibility(...)` for C++:
22+
23+
```cpp
24+
#include "openmeta/compatibility_dump.h"
25+
26+
openmeta::MetadataCompatibilityDumpOptions options;
27+
options.style = openmeta::ExportNameStyle::FlatHost;
28+
29+
std::string out;
30+
openmeta::dump_metadata_compatibility(store, options, &out);
31+
```
32+
33+
Python `Document` and `TransferSourceSnapshot` expose the same metadata dump:
34+
35+
```python
36+
doc = openmeta.read("source.jpg")
37+
text = doc.compatibility_dump()
38+
```
39+
40+
The v1 metadata dump is line-oriented ASCII text:
41+
42+
```text
43+
openmeta.compat.metadata version=1 ...
44+
entry index=0 name="Make" key_kind="exif_tag" value_kind="text" ...
45+
```
46+
47+
Each emitted entry is based on `visit_metadata(...)`, so ordering, duplicate
48+
handling, and naming follow the selected `ExportNameStyle`. By default the
49+
dump uses the stable `FlatHost` contract.
50+
51+
Metadata entry lines include:
52+
53+
- exported name
54+
- key family
55+
- value kind
56+
- scalar or array element type
57+
- text encoding
58+
- count
59+
- optional safe value text
60+
- optional origin block/order/wire fields
61+
- optional per-entry flags
62+
63+
Text and byte values are escaped with the same terminal-safe ASCII rules used
64+
by other OpenMeta diagnostic output. Large values are bounded by
65+
`max_value_bytes`.
66+
67+
## Transfer Dump
68+
69+
Use `dump_transfer_compatibility(...)` for C++ transfer/writeback decisions:
70+
71+
```cpp
72+
std::string out;
73+
openmeta::TransferCompatibilityDumpOptions options;
74+
openmeta::dump_transfer_compatibility(result, persisted_or_null, options, &out);
75+
```
76+
77+
The v1 transfer dump is line-oriented ASCII text:
78+
79+
```text
80+
openmeta.compat.transfer version=1 target_format="png" ...
81+
policy index=0 subject="xmp_iptc_projection" ...
82+
block index=0 kind="xmp" route="png:itxt-xmp" ...
83+
execute edit_requested=true ...
84+
writeback xmp_sidecar_requested=true ...
85+
persist status="ok" ...
86+
```
87+
88+
Transfer dump lines include:
89+
90+
- target format and high-level read/prepare status
91+
- prepared policy decisions
92+
- prepared block kind, route, order, and payload size
93+
- execute/edit status and output sizes
94+
- XMP sidecar request, output, and cleanup decisions
95+
- optional persisted file/sidecar/cleanup result
96+
97+
The dump intentionally records decisions and sizes, not full binary payloads.
98+
This keeps downstream tests stable across equivalent packet serialization
99+
changes.
100+
101+
## Stability
102+
103+
This is a stable v1 contract. Future releases may add fields or lines, but
104+
existing v1 field names and meanings should not change without a new contract
105+
version or a documented compatibility path.

‎docs/development.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1105,7 +1105,8 @@ Current C++ interop entry points:
11051105

11061106
Python interop behavior:
11071107
- `Document.export_names(style=ExportNameStyle.FlatHost, ...)` exposes the
1108-
shared flat-host naming contract used by host-side metadata mapping layers.
1108+
stable v1 flat-host naming contract used by host-side metadata mapping
1109+
layers. See [flat_host_mapping.md](flat_host_mapping.md).
11091110
- `Document.ocio_metadata_tree(...)` is safe-by-default and raises on unsafe
11101111
raw byte payloads; use `Document.unsafe_ocio_metadata_tree(...)` for
11111112
legacy/raw fallback output.

‎docs/flat_host_mapping.md‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# FlatHost Mapping Contract
2+
3+
`ExportNameStyle::FlatHost` is the stable v1 naming contract for host-owned
4+
metadata object models that prefer flat attribute names over OpenMeta's native
5+
key shapes.
6+
7+
Contract constant:
8+
9+
```cpp
10+
openmeta::kFlatHostExportContractVersion == 1
11+
```
12+
13+
## Scope
14+
15+
The contract covers names emitted by:
16+
17+
```cpp
18+
openmeta::visit_metadata(store, options, sink)
19+
```
20+
21+
with:
22+
23+
```cpp
24+
options.style = openmeta::ExportNameStyle::FlatHost;
25+
```
26+
27+
Python mirrors this name contract through:
28+
29+
```python
30+
doc.export_names(style=openmeta.ExportNameStyle.FlatHost)
31+
```
32+
33+
The C++ traversal exposes `ExportItem::entry`, so C++ hosts can project values
34+
from the original `Entry::value`. Python `export_names(...)` is name-only.
35+
36+
## Ordering And Duplicates
37+
38+
- Entries are emitted in `MetaStore::entries()` order.
39+
- Deleted entries are skipped.
40+
- Duplicate names are preserved and emitted separately.
41+
- OpenMeta does not merge, reconcile, or suffix duplicate `FlatHost` names.
42+
- If a host requires unique keys, it must apply its own deterministic collision
43+
policy after traversal.
44+
45+
## Value Projection
46+
47+
`FlatHost` is a naming contract, not a value-conversion contract.
48+
49+
- C++ receives the original `Entry` through `ExportItem::entry`.
50+
- `Entry::value.kind`, `elem_type`, `count`, and storage are unchanged.
51+
- Text encoding remains the original decoded `TextEncoding`.
52+
- Byte payloads remain byte payloads; unsafe text conversion is not performed
53+
by `visit_metadata(...)`.
54+
- Hosts that need a text-only export should use a safe formatting layer on top
55+
of `Entry::value`.
56+
57+
## Namespace Behavior
58+
59+
`FlatHost` intentionally keeps common camera-style names short while preserving
60+
namespace prefixes where ambiguity matters.
61+
62+
| Source family | FlatHost rule |
63+
| --- | --- |
64+
| TIFF root and preview IFD tags | Use the known tag alias without a prefix, for example `Make`, `ModifyDate`, `ImageWidth`. |
65+
| EXIF and interoperability IFD tags | Prefix with `Exif:`, for example `Exif:ExposureTime`, `Exif:ISO`, `Exif:CreateDate`. |
66+
| GPS IFD tags | Prefix with `GPS:`, for example `GPS:GPSLatitude`. |
67+
| MakerNote IFDs | Emit only when `include_makernotes` is true, using `MakerNote:<ifd>:<tag-name-or-hex>`. |
68+
| XMP simple properties | Emit selected known namespaces as `XMP:`, `TIFF:`, `Exif:`, or `DC:` plus the simple property name. |
69+
| ICC header fields and tags | Emit `ICC:<field>` or `ICC:tag:<signature>`. |
70+
| EXR part 0 known aliases | Map selected standard attributes to common host names, for example `owner -> Copyright`, `capDate -> DateTime`, `expTime -> ExposureTime`. |
71+
| EXR part 0 unknown attributes | Emit `openexr:<name>`. |
72+
| EXR non-zero parts | Emit `openexr:part:<index>:<name>`. |
73+
| Unsupported key families | Fall back to the canonical OpenMeta name when one exists. |
74+
75+
Complex XMP paths are not flattened. XMP properties are emitted only when the
76+
property path is a simple token containing letters, digits, `_`, or `-`; paths
77+
with `/`, `[`, `]`, or other punctuation are skipped by `FlatHost`.
78+
79+
## Name Policy
80+
81+
`ExportOptions::name_policy` controls tag aliasing:
82+
83+
- `ExportNamePolicy::ExifToolAlias` is the default and uses OpenMeta's
84+
ExifTool-compatible aliases where OpenMeta has a clean-room mapping.
85+
- `ExportNamePolicy::Spec` preserves native/spec-style tag names where
86+
available and uses `Tag_0x....` for unknown EXIF tags.
87+
88+
The policy does not change ordering, duplicate preservation, or value
89+
projection.
90+
91+
## Filtering
92+
93+
`FlatHost` skips:
94+
95+
- deleted entries
96+
- EXIF pointer tags under the default alias policy
97+
- embedded metadata blob tags such as EXIF-stored XMP, ICC, IPTC, Photoshop
98+
IRB, and MakerNote blob tags under the default alias policy
99+
- MakerNote IFDs when `include_makernotes` is false
100+
- complex XMP paths and unknown XMP namespaces
101+
- selected structural EXR header attributes that are not useful as host
102+
metadata attributes, such as `channels`, `compression`, and data windows
103+
104+
## Stability
105+
106+
This is a stable v1 naming contract. Future OpenMeta releases may add mappings
107+
for newly decoded metadata families, but existing v1 names should not change
108+
without a new contract version or a documented compatibility path.

‎docs/host_integration.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,13 @@ OpenMeta is not an image encoder. The usual pattern is:
1111

1212
If you want the shortest end-to-end examples first, start with
1313
[quick_start.md](quick_start.md).
14+
For public API adoption status, see [api_stability.md](api_stability.md).
15+
For the stable flat host naming contract, see
16+
[flat_host_mapping.md](flat_host_mapping.md).
17+
For deterministic host compatibility baselines, see
18+
[compatibility_dump.md](compatibility_dump.md).
19+
For generated XMP merge and writeback precedence, see
20+
[xmp_sync_policy.md](xmp_sync_policy.md).
1421

1522
## Pick The Integration Path
1623

‎docs/metadata_transfer_plan.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -779,15 +779,15 @@ writer-confidence slice above; it should be sequenced around it.
779779

780780
- [x] add a small runtime capability query API for read, structured decode,
781781
transfer preparation, and target edit support by format and metadata family
782-
- [ ] mark public host-facing APIs with stability levels such as stable,
782+
- [x] mark public host-facing APIs with stability levels such as stable,
783783
experimental, or internal; start with `visit_metadata(...)`, snapshot
784784
read/build, fileless execution, and bundle execution
785-
- [ ] publish the generic `FlatHost` mapping contract: name style, duplicate
785+
- [x] publish the generic `FlatHost` mapping contract: name style, duplicate
786786
handling, type projection, deterministic ordering, and namespace behavior
787-
- [ ] add a deterministic compatibility dump for names, values, scalar types,
787+
- [x] add a deterministic compatibility dump for names, values, scalar types,
788788
origins, and transfer/writeback decisions so downstream tests can avoid
789789
binary-packet baselines
790-
- [ ] document final conflict and precedence decisions for generated EXIF/XMP,
790+
- [x] document final conflict and precedence decisions for generated EXIF/XMP,
791791
IPTC/XMP, source embedded XMP, destination embedded XMP, and destination
792792
sidecar XMP
793793

‎docs/quick_start.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,13 @@ contract, see [metadata_transfer_plan.md](metadata_transfer_plan.md).
1818

1919
If you already own the encoder or host API, see
2020
[host_integration.md](host_integration.md).
21+
For public API adoption status, see [api_stability.md](api_stability.md).
22+
For the stable flat host naming contract, see
23+
[flat_host_mapping.md](flat_host_mapping.md).
24+
For deterministic host compatibility baselines, see
25+
[compatibility_dump.md](compatibility_dump.md).
26+
For generated XMP merge and writeback precedence, see
27+
[xmp_sync_policy.md](xmp_sync_policy.md).
2128

2229
## 1. Add OpenMeta To Your CMake Project
2330

0 commit comments

Comments
 (0)