Skip to content

Commit bdf99fa

Browse files
authored
test: prove the kernel against the emu-atlas machine vectors (#1542)
Adds the [emu-atlas](https://github.com/danielcopper/emu-atlas) conformance tier, mirroring the gavel pattern: the 16 machine vectors from release v0.1.0 are vendored byte-identical under `tests/atlas_vectors/`, and every vector is driven through this plugin's own kernel — the fixture file tree is materialized into `tmp_path` and `RetroDeckPathsAdapter` + `RetroArchConfigAdapter` + `domain/save_path` must reproduce the placements the published contract expects. The overlap between the two codebases is partial, so an explicit per-vector allowlist declares the check level: 8 vectors run end-to-end (RetroDECK flavors — the saves root is this plugin's own source), 7 check the shared retroarch.cfg interpretation (standalone/native saves roots the plugin does not model; `<content_dir>` ⇔ sort-by-content, `<core>` ⇔ sort-by-core, ContentDir ⇔ next-to-ROMs), 1 has no plugin equivalent (nothing-installed — the plugin always resolves fallback paths). A collection-time guard fails the run if an upstream vector appears or disappears without an allowlist decision — no silent skips. Docs: sibling subsections next to the gavel tier in `docs/contributing/development.md` and `CLAUDE.md`.
1 parent 236514e commit bdf99fa

5 files changed

Lines changed: 851 additions & 0 deletions

File tree

CLAUDE.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -377,6 +377,20 @@ kernels: **ladder** against `domain/sync_action.resolve_upload_conflict`, **deci
377377
kernel pass** — updating means deliberately re-copying the JSON and bumping the commit reference in that folder's
378378
`README.md`.
379379

380+
### emu-atlas conformance vectors — vendored contract tier
381+
382+
The same self-conformance pattern proves the plugin's save-path kernel agrees with
383+
[emu-atlas](https://github.com/danielcopper/emu-atlas), the config-aware emulator-knowledge library extracted from this
384+
plugin, where the two overlap. Its `machines` vector family (16 fixture machines in, detected installations + save
385+
placements out) runs against the real adapters (`RetroDeckPathsAdapter` + `RetroArchConfigAdapter`) and domain functions
386+
(`resolve_save_dir` / `compute_local_save_target`) in `tests/test_atlas_machine_vectors.py`. The overlap is partial by
387+
design — the plugin has no standalone-RetroArch saves-root concept (its saves base always comes from RetroDECK paths) —
388+
so every vector carries an explicit `_CHECK_LEVELS` entry: `full` (end-to-end dir + filename), `layout-only` (only the
389+
`retroarch.cfg` interpretation — sort flags or the `ContentDir` classification), or `n/a` (no overlap). No vector is
390+
silently skipped: a new upstream vector without an allowlist entry fails at collection. Vectors are vendored verbatim
391+
under `tests/atlas_vectors/machines/` at a pinned upstream release tag; **never edit a vector to make the kernel pass**
392+
— updating means re-copying the JSON and bumping the tag in that folder's `README.md`.
393+
380394
### Frontend component tests — `@decky/api` event harness
381395

382396
`src/test-utils/decky-api-mock.ts` exposes an in-memory event bus that `addEventListener` / `removeEventListener` route

docs/contributing/development.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,39 @@ python -m pytest tests/domain/test_sync_action_gavel_vectors.py tests/domain/tes
124124
Updating the vectors means deliberately re-copying the JSON from the matching upstream `vectors/<family>/` directory and
125125
bumping the commit reference in `tests/domain/gavel_vectors/README.md`; never edit a vector to match the kernel.
126126

127+
### emu-atlas conformance vectors
128+
129+
The config-aware emulator knowledge — where a RetroArch / RetroDECK install keeps its saves — is likewise published as a
130+
standalone library, [emu-atlas](https://github.com/danielcopper/emu-atlas), extracted from this plugin. Its `machines`
131+
vector family (16 fixture machines in, detected installations + save placements out) runs against the plugin's own
132+
save-path kernel in `tests/test_atlas_machine_vectors.py`, so the two can't silently drift. Each vector materializes a
133+
`{path: content}` file tree under a `tmp_path` fake home, then drives the real adapters (`RetroDeckPathsAdapter` +
134+
`RetroArchConfigAdapter`) and the domain save-path functions (`resolve_save_dir` / `compute_local_save_target`).
135+
136+
The overlap is partial, so every vector carries an explicit check level (an `_CHECK_LEVELS` allowlist entry that also
137+
records _why_):
138+
139+
- **`full`** — end-to-end placement. The plugin derives the saves root the same way atlas does (from `retrodeck.json`,
140+
or the `~/retrodeck` fallback), so the final directory + filename strings are compared. Covers the RetroDECK-flavor
141+
`InSaveDir` cases and the RetroDECK-first coexistence case.
142+
- **`layout-only`** — only the `retroarch.cfg` interpretation overlaps. The plugin has no standalone-RetroArch
143+
saves-root concept (its saves base always comes from RetroDECK paths), so a vector whose placement hangs off a
144+
standalone `savefile_directory` is checked on the `SaveLayout` the plugin derives from the same cfg text — the sort
145+
flags for an `InSaveDir` placement, or the `ContentDir` (next-to-ROM) classification.
146+
- **`n/a`** — no overlap (the plugin has no installation-enumeration surface, so atlas's "nothing detected" outcome has
147+
no plugin equivalent). The check only guards that the vector stays in its non-checkable shape.
148+
149+
No vector is silently skipped: a new upstream vector without an allowlist entry (or a stale entry for a removed one)
150+
fails at collection. The vectors are vendored verbatim under `tests/atlas_vectors/machines/` at a pinned upstream
151+
release tag — no submodule, no network in CI. Run it like any other test:
152+
153+
```bash
154+
python -m pytest tests/test_atlas_machine_vectors.py -q
155+
```
156+
157+
Updating means deliberately re-copying the JSON from upstream `vectors/machines/` and bumping the release tag in
158+
`tests/atlas_vectors/README.md`; never edit a vector to match the kernel.
159+
127160
Every backend feature or callable where testing makes sense should have unit tests covering:
128161

129162
- **Happy path** — normal successful operation

tests/atlas_vectors/README.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# emu-atlas conformance vectors — vendored
2+
3+
These JSON files are vendored verbatim from [danielcopper/emu-atlas](https://github.com/danielcopper/emu-atlas)
4+
`vectors/machines/`, at release tag `v0.1.0`. The layout mirrors upstream — one subdirectory per vector family:
5+
6+
- `machines/` — fixture machines in, detected installations + save placements out, run by
7+
`tests/test_atlas_machine_vectors.py`.
8+
- `named-cases.json` — the 16 curated machine vectors (each carries a `rationale`).
9+
10+
emu-atlas is the config-aware emulator-knowledge library extracted from this plugin; its machine vectors are the
11+
normative contract for where RetroArch/RetroDECK installs keep saves, and the save-placement expectations are
12+
oracle-derived from this repo's `domain/save_path.resolve_save_dir` / `compute_local_save_target`. The test proves the
13+
plugin's own kernel still agrees with the published contract where the two overlap.
14+
15+
## Updating
16+
17+
There is no submodule and no network access in CI — a vector change must surface as a reviewable diff in this repo.
18+
Updating means deliberately re-copying the files from the matching upstream `vectors/<family>/` directory and bumping
19+
the release tag above. Do not reformat the copied files; they must stay byte-for-byte identical to upstream. Never edit
20+
a vector to make the kernel pass — a genuine divergence is a finding to triage, not a vector to rewrite.

0 commit comments

Comments
 (0)