Add HDF5 metadata sidecar support (hdf5-meta extension)

Implements the optional hdf5-meta extension: a columnar HDF5 sidecar
alongside the .sigmf-meta JSON for faster, smaller column-oriented
metadata access on Recordings with large captures/annotations arrays.

- sigmf/hdf5.py: writer (write_hdf5_sidecar), full-dict reader
  (read_hdf5_sidecar), and SigMFFileHDF5 — a lazy, columnar reader that
  serves captures/annotations as numpy columns/arrays without building
  per-row dicts (the actual speedup). Entry points hdf5.open (zero JSON)
  and hdf5.fromfile (discover via one JSON read, prefer fresh sidecar).
- SigMFFile.tofile(write_hdf5=True): writes the sidecar and declares the
  extension. sigmf.fromfile is unchanged and always reads pure JSON, so
  existing behavior and the authoritative-JSON contract are preserved.
- Stale-sidecar guard via a source-metadata digest; h5py is an optional
  dependency (pip install sigmf[hdf5]), lazily imported.
- tests/test_hdf5.py: round-trip, columnar reads, discovery, stale/
  corrupt/missing fallback, edge cases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: Cass Dalton

README.md

@@ -61,6 +61,33 @@ meta.tofile("recording")
 meta.tofile("recording.sigmf.gz")
 ```
 
+### HDF5 metadata sidecar (optional)
+
+For recordings with very large `captures`/`annotations`, the optional
+`hdf5-meta` extension can write a columnar HDF5 sidecar next to the
+`.sigmf-meta` file. The JSON metadata stays complete and authoritative; the
+sidecar is a smaller, faster cache for column-oriented access. Requires the
+optional `h5py` dependency: `pip install sigmf[hdf5]`.
+
+```python
+import sigmf
+from sigmf import hdf5
+
+# write the sidecar alongside the JSON (declares the hdf5-meta extension)
+meta.tofile("recording", write_hdf5=True)  # also writes recording.sigmf-meta.h5
+
+# fast columnar read: open ONLY the sidecar, no JSON parsing, no per-row dicts
+with hdf5.open("recording.sigmf-meta.h5") as fast:
+    starts = fast.annotations_column("core:sample_start")  # numpy column
+    table = fast.annotations_array()                       # structured array
+
+# or discover via the JSON once, then prefer the sidecar when present & fresh
+fast = hdf5.fromfile("recording.sigmf-meta")  # SigMFFileHDF5 if usable, else SigMFFile
+
+# the standard reader is unchanged and always reads pure JSON
+meta = sigmf.fromfile("recording.sigmf-meta")
+```
+
 ### Docs
 
 **[Please visit our documentation for full API reference and more info.](https://sigmf.readthedocs.io/en/latest/)**

pyproject.toml

@@ -38,11 +38,15 @@ dependencies = [
     sigmf_validate = "sigmf.validate:main"
     sigmf_convert = "sigmf.convert.__main__:main"
     [project.optional-dependencies]
+    hdf5 = [
+        "h5py",         # for the optional hdf5-meta metadata sidecar
+    ]
     test = [
         "ruff",
         "pytest",
         "pytest-cov",
         "hypothesis",   # next-gen testing framework
+        "h5py",         # exercise the optional hdf5-meta sidecar in tests
     ]
 
 [tool.setuptools]

sigmf/__init__.py

@@ -13,6 +13,7 @@
     archive,
     archivereader,
     error,
+    hdf5,
     keys,
     schema,
     siggen,