Skip to content

Commit 90bf3f5

Browse files
authored
chore(bios): vendor the registry from emu-atlas and drop the in-tree generator (#1543)
The BIOS registry's home is now [emu-atlas](https://github.com/danielcopper/emu-atlas) — the data there is byte-identical and the generator was ported at parity — so this repo stops carrying both halves of the ownership. - `scripts/generate_bios_registry.py` is deleted; regeneration happens upstream (emu-atlas `scripts/` + its data README document the procedure). - `defaults/bios_registry.json` stays exactly where runtime code and packaging expect it, but is now a vendored artifact: `defaults/README.md` records provenance (emu-atlas release `v0.1.0`) and the update procedure (fetch at a tag via the raw URL — atlas releases carry no binary assets, the tagged tree is the artifact), and `defaults/bios_registry.json.sha256` pins the bytes. CI and the mise lint task verify the checksum alongside the existing gavel-native gate, so a hand-edit or silent swap fails the pipeline. - No runtime changes: `FirmwareService`, `domain/bios.py`, the load path, and the release-zip smoke test are untouched. Closes #1541.
1 parent bdf99fa commit 90bf3f5

7 files changed

Lines changed: 85 additions & 460 deletions

File tree

.github/workflows/ci.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,12 @@ jobs:
169169
# update when pulling a newer romm-gavel release).
170170
- name: Check vendored gavel native core checksum
171171
run: cd py_modules/native && sha256sum -c libgavel-x86_64-linux.so.sha256
172+
# The BIOS registry ships in the plugin zip and is vendored verbatim from an
173+
# emu-atlas release — there is no in-tree generator. A hand-edited snapshot
174+
# must fail CI. Verify the vendored data against its checksum (bumped only by
175+
# a deliberate defaults/README.md update when pulling a newer emu-atlas release).
176+
- name: Check vendored BIOS registry checksum
177+
run: cd defaults && sha256sum -c bios_registry.json.sha256
172178

173179
# Skipped when SONAR_TOKEN is structurally unavailable: Dependabot PRs and
174180
# PRs from forks (GitHub withholds secrets from `pull_request` runs whose head

CLAUDE.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -244,7 +244,12 @@ in [`_vendor/README.md`](py_modules/_vendor/README.md). Compiled binaries (no so
244244
`py_modules/native/` instead (inside one of the fixed directories the Decky CLI packs into the plugin zip) — downloaded
245245
verbatim from an upstream release with a pinned SHA-256 (CI re-verifies it; the release smoke test asserts the artifact
246246
ships in the zip), loaded by an adapter via `ctypes` with no Python fallback; provenance and the update procedure live
247-
in [`native/README.md`](py_modules/native/README.md).
247+
in [`native/README.md`](py_modules/native/README.md). Vendored **data** (no source in this repo) follows the same
248+
discipline: `defaults/bios_registry.json` is copied verbatim from an
249+
[emu-atlas](https://github.com/danielcopper/emu-atlas) release with a pinned SHA-256
250+
(`defaults/bios_registry.json.sha256`, CI-verified via `mise run gate`; the release smoke test asserts it ships in the
251+
zip) — never hand-edit it, regeneration happens upstream; provenance and the update procedure live in
252+
[`defaults/README.md`](defaults/README.md).
248253

249254
**Process boundaries — `main.py` vs `bootstrap.py`**: `[ours]` `main.py` owns the Decky lifecycle (`_main`, `_unload`)
250255
and the callable surface (one `async def` per `@callable`). `bootstrap.py` owns adapter instantiation and service

defaults/README.md

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Packaged defaults
2+
3+
Reference data that ships inside the plugin. The Decky CLI flattens this directory into the plugin root at package time,
4+
so the runtime reads these files by their bare name (no `defaults/` prefix); do not move or rename them.
5+
6+
## `bios_registry.json` — vendored from emu-atlas
7+
8+
The BIOS registry: which firmware files each platform and libretro core want, with the hashes and sizes that identify
9+
them. Read at runtime by `FirmwareService` (via `domain/bios.py`) to classify what a platform needs and whether a local
10+
file is the right one.
11+
12+
It is **vendored verbatim** from [emu-atlas](https://github.com/danielcopper/emu-atlas), where the registry and its
13+
generator now live:
14+
15+
- **Upstream:** <https://github.com/danielcopper/emu-atlas>
16+
- **Release:** `v0.1.0`
17+
- **Upstream path:** `atlas/data/bios_registry.json`
18+
- **Checksum:** pinned in `bios_registry.json.sha256` (SHA-256)
19+
20+
This repo carries only the data snapshot — there is no in-tree generator. Generation is a dev-time, offline step that
21+
lives upstream (emu-atlas `scripts/generate_bios_registry.py`, documented in emu-atlas `atlas/data/README.md`); it
22+
derives the registry from the libretro `libretro-core-info` and `libretro-database` checkouts. **Never hand-edit the
23+
data here** — a manual edit would silently diverge from the released snapshot and break the checksum gate.
24+
25+
### How to update
26+
27+
1. Fetch the registry at the release tag (emu-atlas releases carry no binary assets — the tagged source tree is the
28+
artifact):
29+
30+
```sh
31+
curl -fL -o defaults/bios_registry.json \
32+
"https://raw.githubusercontent.com/danielcopper/emu-atlas/<tag>/atlas/data/bios_registry.json"
33+
```
34+
35+
2. Regenerate the pinned checksum and verify it (the bare filename keeps `sha256sum -c` working from within this
36+
directory):
37+
38+
```sh
39+
cd defaults && sha256sum bios_registry.json > bios_registry.json.sha256 && sha256sum -c bios_registry.json.sha256
40+
```
41+
42+
3. Bump the **Release** tag above.
43+
4. Re-run the firmware tests (`tests/services/test_firmware.py`, `tests/domain/test_bios.py`) — a `required` flag flip,
44+
a removed entry, or a changed hash is a behavior change for consumers, so call it out in the PR description.
45+
46+
The checksum is re-verified by CI (`.github/workflows/ci.yml`, mirrored in `mise run gate` / `mise run lint`) and the
47+
release smoke test asserts the registry ships in the plugin zip, so both a hand-edited snapshot and a dropped file fail
48+
the pipeline.
49+
50+
## `config.json` — in-tree default
51+
52+
The platform-slug map and other default configuration. Unlike `bios_registry.json`, this is maintained in this repo (not
53+
vendored) and carries no checksum gate.

defaults/bios_registry.json.sha256

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
26b0ba5f9578b80542f780342e4d99e24aa7b0479a11186bf21dc32a9a5bdbfe bios_registry.json

docs/architecture/backend-architecture.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1157,6 +1157,24 @@ construction and is itself the `ResolveUploadConflictFn` seam (`services/protoco
11571157
`domain.sync_action.resolve_upload_conflict` kernel is **not** a runtime fallback; it is retained as the differential
11581158
oracle and the gavel-ladder-vector target in tests (`tests/adapters/test_gavel_native.py`), so the shipped binary is
11591159
proven to match the same contract the Python kernel is held to.
1160+
1161+
#### FirmwareService notes — the vendored BIOS registry
1162+
1163+
The BIOS registry — which firmware files each platform and libretro core want, with the hashes and sizes that identify
1164+
them — is the data `FirmwareService` reads (via `domain/bios.py`) to classify what a platform needs and whether a local
1165+
file is the right one. It ships as `defaults/bios_registry.json` and is **vendored verbatim** from an
1166+
[emu-atlas](https://github.com/danielcopper/emu-atlas) release, where the registry and its generator now live — there is
1167+
no in-tree generator, and regeneration is a dev-time, offline step upstream (from the libretro `libretro-core-info` and
1168+
`libretro-database` sources).
1169+
1170+
- **What ships**: `defaults/bios_registry.json` (emu-atlas `v0.1.0`, upstream path `atlas/data/bios_registry.json`),
1171+
copied verbatim with a pinned SHA-256 checksum. Provenance and the update procedure live in
1172+
[`defaults/README.md`](https://github.com/danielcopper/decky-romm-sync/blob/main/defaults/README.md). The checksum is
1173+
re-verified by CI (`.github/workflows/ci.yml`, mirrored in `mise run gate`) and the release smoke test asserts the
1174+
registry is present in the plugin zip, so both a hand-edited snapshot and a dropped file fail the pipeline.
1175+
- **Never hand-edit it**: a manual edit would silently diverge from the released snapshot and break the checksum gate. A
1176+
registry change lands by pulling a newer emu-atlas release, re-pinning the checksum, and bumping the tag in
1177+
`defaults/README.md`.
11601178
- **Path resolution**: the `.so` is resolved relative to the adapter module (`__file__`), mirroring
11611179
`sqlite_migrations.MIGRATIONS_DIR` — a fatal-load bundled resource must resolve identically in the installed plugin
11621180
(`<plugin>/py_modules/native/…`) and in the repo checkout the real-`bootstrap()` test tiers run from, where the plugin

mise.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ run = [
6262
"python scripts/check_lock_sync.py",
6363
"python scripts/check_markdown_links.py",
6464
"cd py_modules/native && sha256sum -c libgavel-x86_64-linux.so.sha256",
65+
"cd defaults && sha256sum -c bios_registry.json.sha256",
6566
]
6667

6768
[tasks.gate]

0 commit comments

Comments
 (0)