Skip to content

Commit 4b4bc56

Browse files
committed
Support prepared EXR batches and TIFF/IFD1
Bump VERSION and add docs/workflow for GitHub Pages; update README and docs to describe EXR and TIFF progress. Add CI yamllint rule and .github workflow to build and publish Sphinx docs. Introduce EXR host-integration helpers in src/include/openmeta/exr_adapter.h and src/openmeta/exr_adapter.cc: new types and functions to materialize ExrAdapterBatch from a prepared transfer bundle (build_prepared_exr_attribute_batch) and a file-helper wrapper (build_exr_attribute_batch_from_file). Add parsing/encoding helpers for the prepared EXR string-attribute payload format and error handling. Significant enhancements to TIFF/EXIF handling in src/openmeta/metadata_transfer.cc: add IFD1 support and explicit subIFD indexing, new TransferExifIfdRef classification, support for preserving downstream page/aux tails, and plumbing to emit the IFD1 pointer when present. Introduce big/64-bit TIFF (BigTIFF) support via a TiffLayout abstraction: widen count/offset fields, add read/write helpers for 64-bit values, adjust parsing/serialization for classic and BigTIFF variants, handle endian conversion for new types, and extend parsed/serialized structures to track next-ifd offsets and subIFDs. Tests and Python bindings were updated accordingly.
1 parent ebc7fc3 commit 4b4bc56

18 files changed

+4507
-229
lines changed

.github/workflows/docs-pages.yml

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
---
2+
name: docs-pages
3+
4+
on:
5+
pull_request:
6+
paths:
7+
- ".github/workflows/docs-pages.yml"
8+
- "CMakeLists.txt"
9+
- "README.md"
10+
- "SECURITY.md"
11+
- "NOTICE.md"
12+
- "cmake/Doxyfile.in"
13+
- "docs/**"
14+
- "src/include/**"
15+
push:
16+
branches:
17+
- main
18+
tags:
19+
- "*"
20+
paths:
21+
- ".github/workflows/docs-pages.yml"
22+
- "CMakeLists.txt"
23+
- "README.md"
24+
- "SECURITY.md"
25+
- "NOTICE.md"
26+
- "cmake/Doxyfile.in"
27+
- "docs/**"
28+
- "src/include/**"
29+
workflow_dispatch:
30+
31+
permissions:
32+
contents: read
33+
34+
concurrency:
35+
group: docs-pages-${{ github.ref }}
36+
cancel-in-progress: true
37+
38+
jobs:
39+
build:
40+
runs-on: ubuntu-latest
41+
steps:
42+
- name: Checkout
43+
uses: actions/checkout@v6
44+
45+
- name: Setup Pages
46+
uses: actions/configure-pages@v5
47+
48+
- name: Setup Python
49+
uses: actions/setup-python@v6
50+
with:
51+
python-version: "3.12"
52+
cache: "pip"
53+
cache-dependency-path: docs/requirements.txt
54+
55+
- name: Install docs dependencies
56+
run: |
57+
sudo apt-get update
58+
sudo apt-get install -y doxygen graphviz ninja-build
59+
python -m pip install --upgrade pip
60+
python -m pip install -r docs/requirements.txt
61+
62+
- name: Configure docs build
63+
run: >
64+
cmake -S . -B build -G Ninja
65+
-DOPENMETA_BUILD_SPHINX_DOCS=ON
66+
-DOPENMETA_BUILD_SHARED=OFF
67+
-DOPENMETA_BUILD_TOOLS=OFF
68+
69+
- name: Build docs
70+
run: cmake --build build --target openmeta_docs_sphinx --parallel
71+
72+
- name: Upload Pages artifact
73+
uses: actions/upload-pages-artifact@v4
74+
with:
75+
path: build/docs/html
76+
77+
deploy:
78+
if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/')
79+
needs: build
80+
runs-on: ubuntu-latest
81+
permissions:
82+
pages: write
83+
id-token: write
84+
environment:
85+
name: github-pages
86+
url: ${{ steps.deployment.outputs.page_url }}
87+
steps:
88+
- name: Deploy to GitHub Pages
89+
id: deployment
90+
uses: actions/deploy-pages@v4

.yamllint

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
extends: default
3+
4+
rules:
5+
truthy:
6+
# GitHub Actions uses the unquoted top-level key `on`.
7+
check-keys: false

README.md

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,14 @@ Read-path coverage is broad and regression-gated. Write/edit support is real
1212
for the main transfer targets, but parts of that API surface are still draft
1313
and may change as the transfer contract stabilizes.
1414

15+
Current planning estimate:
16+
17+
| Milestone | Status |
18+
| --- | --- |
19+
| Read parity on tracked still-image corpora | About `99-100%` |
20+
| Transfer / export milestone | About `80-85%` |
21+
| Overall product milestone | About `97-98%` |
22+
1523
Current baseline-gated snapshot on tracked corpora:
1624
- HEIC/HEIF, CR3, and mixed RAW EXIF tag-id compare gates are passing.
1725
- EXR header metadata compare is passing for name/type/value-class checks.
@@ -84,11 +92,22 @@ Current target status:
8492

8593
In practice:
8694
- JPEG and TIFF are the strongest transfer targets today.
95+
- TIFF edit support now covers classic TIFF, BigTIFF, bounded `ifd1`
96+
chain rewrite with preserved downstream page tails, and bounded
97+
TIFF/DNG-style SubIFD rewrite with preserved downstream auxiliary tails
98+
and preserved trailing existing children when only the front subset is
99+
replaced.
100+
- For DNG-like TIFF sources, the current bounded merge policy is:
101+
replace the source-supplied front preview/aux structures, preserve
102+
existing target page tails and trailing auxiliary children.
87103
- PNG, WebP, JP2, JXL, bounded BMFF, and EXR all have real first-class
88104
transfer entry points.
89105
- EXR is still narrower than the container-edit targets: it emits safe string
90-
header attributes through the transfer core, but it does not rewrite full
91-
EXR files yet.
106+
header attributes through the transfer core, can materialize a prepared
107+
`ExrAdapterBatch` for host exporters, and Python can inspect that prepared
108+
EXR attribute batch through the direct `build_exr_attribute_batch_from_file`
109+
binding or the helper-layer `openmeta.python.get_exr_attribute_batch(...)`,
110+
but OpenMeta does not rewrite full EXR files yet.
92111
- Writer-side sync behavior is now partially explicit instead of implicit:
93112
generated XMP can independently keep or suppress EXIF-derived and
94113
IPTC-derived projection during transfer preparation.

VERSION

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
0.4.5
1+
0.4.6

docs/doxygen.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,23 @@ cmake --build build --target openmeta_docs_sphinx
3737
Output goes to `build/docs/html/index.html`. Doxygen XML goes to
3838
`build/docs/doxygen/xml/index.xml`.
3939

40+
## Publish on GitHub Pages
41+
42+
The repository includes a GitHub Actions workflow at
43+
`.github/workflows/docs-pages.yml` that builds the Sphinx site and publishes it
44+
to GitHub Pages.
45+
46+
Recommended setup:
47+
48+
- In the repository settings, set **Pages** to **GitHub Actions** as the
49+
source.
50+
- Pull requests build the docs but do not deploy.
51+
- Pushes to `main` and tag pushes build and deploy the site.
52+
53+
The workflow only runs when docs inputs change: `docs/**`, `src/include/**`,
54+
`README.md`, `SECURITY.md`, `NOTICE.md`, `CMakeLists.txt`, or the workflow
55+
itself.
56+
4057
## Generate API docs (manual)
4158

4259
From the `OpenMeta/` repo root:

docs/metadata_transfer_plan.md

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,8 @@ The core rule is:
2626

2727
## Current Status
2828

29+
Current planning estimate for this lane: about `80-85%`.
30+
2931
Source-side readiness is already strong:
3032
- tracked EXIF read gates are green on `HEIC/HEIF`, `CR3`, and mixed RAW corpora
3133
- tracked MakerNote gates are green
@@ -45,13 +47,13 @@ The first public write-side sync controls are also in place:
4547
| Target | Status | Current shape | Main limits |
4648
| --- | --- | --- | --- |
4749
| JPEG | First-class | Prepared bundle, compiled emit, byte-writer emit, edit planning/apply, file helper, bounded JUMBF/C2PA staging | Not a full arbitrary metadata editor yet |
48-
| TIFF | First-class | Prepared bundle, compiled emit, edit planning/apply, file helper, streaming edit path | Broader TIFF/DNG rewrite coverage is still narrower than JPEG |
50+
| TIFF | First-class | Prepared bundle, compiled emit, classic-TIFF and BigTIFF edit planning/apply, bounded `ifd1` chain rewrite with preserved downstream page tails, bounded TIFF/DNG-style SubIFD rewrite with preserved downstream auxiliary tails and preserved trailing existing children when only the front subset is replaced, file helper, streaming edit path | Broader TIFF/DNG rewrite coverage is still narrower than JPEG |
4951
| PNG | Bounded but real | Prepared bundle, compiled emit, bounded chunk rewrite/edit, file-helper roundtrip | Not a general PNG chunk editor |
5052
| WebP | Bounded but real | Prepared bundle, compiled emit, bounded chunk rewrite/edit, file-helper roundtrip | Not a general WebP chunk editor |
5153
| JP2 | Bounded but real | Prepared bundle, compiled emit, bounded box rewrite/edit, file-helper roundtrip | `jp2h` synthesis is still out of scope |
5254
| JXL | Bounded but real | Prepared bundle, compiled emit, bounded box rewrite/edit, file-helper roundtrip | Still narrower than JPEG/TIFF |
5355
| HEIF / AVIF / CR3 | Bounded but real | Prepared bundle, compiled emit, bounded BMFF item/property edit, file-helper roundtrip | Not broad BMFF writer parity |
54-
| EXR | Bounded but real | Prepared bundle, compiled emit, direct backend attribute emit, CLI/Python transfer surface | No file rewrite/edit path yet; current transfer payload is safe string attributes only |
56+
| EXR | Bounded but real | Prepared bundle, compiled emit, direct backend attribute emit, prepared-bundle to `ExrAdapterBatch` bridge, CLI/Python transfer surface | No file rewrite/edit path yet; current transfer payload is safe string attributes only |
5557

5658
## What Is Already Implemented
5759

@@ -85,6 +87,13 @@ These support the public transfer flow:
8587
OpenMeta now has explicit end-to-end read-backed transfer tests for:
8688
- source JPEG -> JPEG edit/apply -> read-back
8789
- source JPEG -> TIFF edit/apply -> read-back
90+
- source JPEG -> TIFF edit/apply with `ifd1` -> read-back
91+
- source TIFF/BigTIFF with existing multi-page `ifd1 -> next` chain ->
92+
replace `ifd1` -> preserve downstream tail
93+
- source DNG-like TIFF with `subifd0` + `ifd1` -> TIFF edit/apply -> read-back
94+
- source DNG-like TIFF with `subifd0` + `ifd1` -> BigTIFF edit/apply -> read-back
95+
- source TIFF/BigTIFF with existing `subifd0 -> next` auxiliary chain ->
96+
replace `subifd0` -> preserve downstream auxiliary tail
8897
- source JPEG -> PNG edit/apply -> read-back
8998
- source JPEG -> WebP edit/apply -> read-back
9099
- source JPEG -> JP2 edit/apply -> read-back
@@ -119,6 +128,17 @@ Also a first-class target.
119128
Implemented:
120129
- EXIF, XMP, ICC, and IPTC transfer
121130
- edit planning and apply
131+
- classic-TIFF and BigTIFF rewrite support
132+
- bounded `ifd1` chain rewrite support, including preserving an existing
133+
downstream page tail when `ifd1` is replaced
134+
- bounded TIFF/DNG-style SubIFD rewrite support, including preserving an
135+
existing downstream auxiliary tail when `subifdN` is replaced
136+
- bounded front-subset `SubIFD` replacement that preserves trailing existing
137+
children from the target file
138+
- bounded DNG-style merge policy in the file-helper path:
139+
source-supplied preview/aux front structures replace the target front
140+
structures, while existing target page tails and trailing auxiliary
141+
children are preserved
122142
- file-based helper flow
123143
- streaming edit output
124144

@@ -184,6 +204,19 @@ It still keeps the older integration bridge:
184204
- `build_exr_attribute_part_views(...)`
185205
- `replay_exr_attribute_batch(...)`
186206

207+
The transfer lane now also exposes:
208+
- `build_prepared_exr_attribute_batch(...)`
209+
- `build_exr_attribute_batch_from_file(...)`
210+
- Python `build_exr_attribute_batch_from_file(...)` for direct file-to-batch
211+
host-side inspection without going through the generic transfer probe
212+
- Python helper wrappers:
213+
`openmeta.python.probe_exr_attribute_batch(...)` and
214+
`openmeta.python.get_exr_attribute_batch(...)`
215+
216+
That keeps EXR host integrations on the transfer path: callers can prepare one
217+
`TransferTargetFormat::Exr` bundle, then materialize a native
218+
`ExrAdapterBatch` without re-projecting from the source `MetaStore`.
219+
187220
Current EXR transfer scope is intentionally conservative:
188221
- safe flattened `string` header attributes
189222
- backend emission through `ExrTransferEmitter`

src/include/openmeta/exr_adapter.h

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
#include "openmeta/exr_decode.h"
44
#include "openmeta/meta_store.h"
5+
#include "openmeta/metadata_transfer.h"
56

67
#include <cstddef>
78
#include <cstdint>
@@ -94,6 +95,18 @@ struct ExrAdapterReplayResult final {
9495
std::string message;
9596
};
9697

98+
/// File-helper options for \ref build_exr_attribute_batch_from_file.
99+
struct BuildExrAttributeBatchFileOptions final {
100+
PrepareTransferFileOptions prepare;
101+
ExrAdapterOptions adapter;
102+
};
103+
104+
/// File-helper result for \ref build_exr_attribute_batch_from_file.
105+
struct BuildExrAttributeBatchFileResult final {
106+
PrepareTransferFileResult prepared;
107+
ExrAdapterResult adapter;
108+
};
109+
97110
/**
98111
* \brief Builds one owned EXR-native attribute batch from a \ref MetaStore.
99112
*
@@ -107,6 +120,39 @@ build_exr_attribute_batch(const MetaStore& store, ExrAdapterBatch* out,
107120
const ExrAdapterOptions& options
108121
= ExrAdapterOptions {}) noexcept;
109122

123+
/**
124+
* \brief Builds one EXR-native attribute batch from a prepared EXR transfer
125+
* bundle.
126+
*
127+
* This bridges the transfer pipeline with EXR host integrations: callers can
128+
* prepare metadata once with \ref prepare_metadata_for_target using
129+
* \ref TransferTargetFormat::Exr, then materialize a host-facing
130+
* \ref ExrAdapterBatch without re-projecting from the source \ref MetaStore.
131+
*
132+
* Current EXR prepared transfer blocks serialize one logical `string`
133+
* attribute per block and target part `0`.
134+
*/
135+
ExrAdapterResult
136+
build_prepared_exr_attribute_batch(const PreparedTransferBundle& bundle,
137+
ExrAdapterBatch* out,
138+
const ExrAdapterOptions& options
139+
= ExrAdapterOptions {}) noexcept;
140+
141+
/**
142+
* \brief Reads one file, prepares EXR transfer metadata, then materializes a
143+
* host-facing \ref ExrAdapterBatch.
144+
*
145+
* This is the direct public file-helper for thin bindings and host tools.
146+
* Internally it wraps \ref prepare_metadata_for_target_file, forces the
147+
* prepared target format to \ref TransferTargetFormat::Exr, then calls
148+
* \ref build_prepared_exr_attribute_batch.
149+
*/
150+
BuildExrAttributeBatchFileResult
151+
build_exr_attribute_batch_from_file(
152+
const char* path, ExrAdapterBatch* out,
153+
const BuildExrAttributeBatchFileOptions& options
154+
= BuildExrAttributeBatchFileOptions {}) noexcept;
155+
110156
/**
111157
* \brief Builds contiguous per-part spans over one \ref ExrAdapterBatch.
112158
*

0 commit comments

Comments
 (0)