Skip to content

feat(store): iaf migrate-store + dual-store contract suite (epic #540 phase 3d)#547

Merged
MDUYN merged 5 commits into
feature/iaf-ohlcv-chunk-storefrom
feature/iaf-migrate-store
May 12, 2026
Merged

feat(store): iaf migrate-store + dual-store contract suite (epic #540 phase 3d)#547
MDUYN merged 5 commits into
feature/iaf-ohlcv-chunk-storefrom
feature/iaf-migrate-store

Conversation

@MDUYN
Copy link
Copy Markdown
Collaborator

@MDUYN MDUYN commented May 11, 2026

Phase 3d — iaf migrate-store + dual-store contract suite

Stacked on #546 (Phase 3c). Closes the open Phase 3 deliverables of epic #540.

What's in the box

1. iaf migrate-store CLI

iaf migrate-store --from local-dir   --src ./bt-old \
                  --to   local-tiered --dst ./bt-new

Delegates to dst.copy_from(src), which means the migration is:

  • Incremental — bundle by bundle.
  • Restartable — re-running over an existing destination just continues.
  • Tier-aware — when copying into a local-tiered store, identical OHLCV chunks are written exactly once across the destination regardless of how many bundles reference them (the Phase 3c invariant).
  • Selectable--handles a,b,c for partial migrations.

A migrate_store(...) programmatic helper is also exposed for in-process pipelines.

2. BacktestStoreContractTest — parameterised dual-store suite

A single unittest.TestCase runs an identical scenario against every concrete BacktestStore implementation (LocalDirStore, LocalTieredStore) using subTest(store=label). Adding a future store (e.g. a remote S3BacktestStore) is one entry in _STORES = [...] away from full conformance coverage.

Coverage:

  • Protocol + SupportsCopyFrom runtime conformance.
  • writeopen round-trip preserves algorithm_id.
  • summary_only honoured.
  • exists reflects writes; idempotent delete removes the handle.
  • Missing handle raises StoreHandleNotFoundError.
  • iter_handles and __len__ agree with the written set.
  • iter_index_rows yields one row per bundle.
  • copy_from works for both full migration and a handle subset.

3. Bug fix: LazyOhlcvDict.items() / .values()

The previous implementation inherited the empty backing dict's iteration, so any code path doing

for k, v in bt.ohlcv.items():
    ...

silently dropped every blob after a tiered round-trip — including the migration code we're adding here. Caught by test_ohlcv_dedup_during_migration and fixed by walking the manifest with lazy materialisation, matching the existing __iter__ / __getitem__ semantics.

What is not in this PR

Byte-identical Tier-2 → Backtest reassembly (so .iafbt could become export-only) is intentionally deferred. The current model where the bundle is canonical and Tier-1/2/3 are derived is simpler, preserves the existing round-trip contract bit-for-bit, and is what every test in the contract suite already exercises against both stores. A future slice can promote Tier-2 to authoritative once we have a stronger reassembly story for the envelope's metadata fields.

Tests

  • tests/cli/test_migrate_store_command.py — 7 tests (programmatic API + CLI: round-trip, handle subset, OHLCV dedup across migration, unknown-kind validation).
  • tests/services/backtest_store/test_store_contract.py — 8 contract tests × 2 stores = 16 subTests.
  • Targeted suite (tests/services/backtest_store/ + tests/services/backtest_index/ + tests/cli/): 128 / 128 passing + 26 subTests.
  • Full non-scenario suite (tests/ minus tests/scenarios): 1705 / 1705 passing with no regressions from the LazyOhlcvDict fix.

Stack

dev
 └─ #537  feature/bundle-format-v2
     └─ #543  feature/iaf-index-cli
         └─ #544  feature/iaf-backtest-store           (3a)
             └─ #545  feature/iaf-local-tiered-store    (3b)
                 └─ #546  feature/iaf-ohlcv-chunk-store (3c)
                     └─ #???  feature/iaf-migrate-store (3d) ★

Phase 3 of epic #540 is now feature-complete.

MDUYN added 5 commits May 11, 2026 16:38
…phase 3d)

Closes the open Phase 3 deliverables that turn the new store
abstraction into something users can actually move data through:

- iaf migrate-store --from <kind> --src <path> --to <kind> --dst <path>
  delegates to dst.copy_from(src), so it is incremental, restartable,
  and tier-aware: when the destination is a local-tiered store,
  identical OHLCV chunks are written exactly once across the
  destination regardless of how many bundles reference them
  (Phase 3c invariant). Optional --handles subset selector for
  partial migrations.
- migrate_store() programmatic helper for in-process pipelines.
- BacktestStoreContractTest: a parameterised conformance suite
  that runs identical scenarios against every concrete store
  implementation (LocalDirStore, LocalTieredStore today, future
  remote stores tomorrow). Catches divergence as a failing subTest
  with the store class name in the label. Covers Protocol +
  SupportsCopyFrom conformance, write/open round-trip, summary_only,
  exists, idempotent delete, missing-handle errors, listing,
  iter_index_rows, and copy_from with both full and subset handle
  selection.
- bug fix in LazyOhlcvDict: items() and values() were inheriting
  the empty backing dict's iteration, so any code path that did
  'for k, v in bt.ohlcv.items()' silently dropped every blob after
  a tiered round-trip. Now both methods walk the manifest and
  materialise lazily on access. Caught by the migration dedup
  test.

Note on what is *not* in this PR: byte-identical Tier-2 -> Backtest
reassembly (so .iafbt could become export-only) is intentionally
deferred. The current model where the bundle is canonical and
Tier-1/2/3 are derived is simpler, preserves the existing
round-trip contract bit-for-bit, and is what every test in the
contract suite already exercises against both stores.

Targeted suite (backtest_store + backtest_index + cli): 128 / 128
passing + 26 subTests. Full non-scenario suite: 1705 / 1705 passing
with no regressions from the LazyOhlcvDict fix.
…L dashboard

Two new sections in examples/storage_layer_demo/demo.py:

- 6b. _print_backtest_full_report(): per-run breakdown (window /
  days / orders / trades / positions / final_value), end-of-backtest
  positions snapshot, first few trades, and a richer slice of
  per-run BacktestMetrics (cagr, annual_volatility,
  max_drawdown_absolute, gross_profit/loss, best_trade, max
  consecutive wins/losses) with safe n/a fallbacks. Built on top
  of the existing compact _print_backtest_report().

- 9. Storage layer -> HTML dashboard: wires the Tier-1 SQLite
  index, the Tier-2 LocalDirStore and the BacktestReport HTML
  dashboard end-to-end. rank_index() picks the top-N bundles from
  SQLite alone, store.open(handle) materialises just those via the
  BacktestStore protocol, BacktestReport(backtests=[...]).save()
  renders a self-contained interactive HTML dashboard. Demonstrates
  that the new storage layer plugs straight into the existing
  reporting stack with no glue code.

README updated to describe both new sections.
- New feature bullet linking the storage_layer_demo
- New '<details>' section explaining Tier-1 SQLite index, Tier-2
  BacktestStore adapters (LocalDirStore / LocalTieredStore) and
  Tier-3 content-addressed OHLCV chunks
- Python + CLI workflow showing the canonical pattern:
  build_index -> rank_index -> store.open(handle) ->
  BacktestReport(backtests=[...]).save(...)
- Links to examples/storage_layer_demo/ for the runnable end-to-end
Add a 'From backtest results to a report' subsection under
'Backtest Analysis & Dashboard' demonstrating the canonical paths
from a Backtest (or list of Backtests) to a BacktestReport:

- single event-driven app.run_backtest(...)
- a sweep via app.run_vector_backtests(..., backtest_storage_directory=...)
- loading a persisted folder back via BacktestReport.open(directory_path=..., workers=-1)

Cross-links to the Backtest Storage Layer section for sweeps that
scale into the thousands.
New 'Getting Started/Backtest Storage Layer' page covering:

- mental model (Tier-1 SQLite / Tier-2 Parquet / canonical .iafbt /
  Tier-3 content-addressed OHLCV)
- the BacktestStore protocol and when to pick LocalDirStore vs
  LocalTieredStore
- the canonical 5-step developer workflow:
    run sweep -> build index -> filter/rank in SQLite ->
    materialise winners -> render report
- 'Avoid overloading your report.html': size-vs-bundle table,
  the BacktestReport.open(directory_path=...) anti-pattern,
  rules of thumb for narrow vs mega-reports
- pointers to examples/storage_layer_demo and the migrate-store CLI

Wired into the Getting Started sidebar between backtest-reports and
deployment, and added a tip block on backtest-reports.md pointing to
it for users with thousands of backtests.
@MDUYN MDUYN force-pushed the feature/iaf-migrate-store branch from a0751e9 to adfeaa0 Compare May 12, 2026 09:52
@MDUYN MDUYN merged commit 63fbb5e into feature/iaf-ohlcv-chunk-store May 12, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant