v8.8.0

v8.8.0 — TWR metrics, broker balance tracker, stop orders

This release adds three major capabilities plus a number of supporting
improvements.

✨ Highlights

Time-Weighted Return (TWR) metrics

TWR variants for CAGR, Sharpe, volatility, returns, drawdown, and
equity curve. All TWR metrics consume snapshot.cash_flow to scrub
external deposits/withdrawals from performance, giving a true
alpha-only view of the strategy.

New BacktestMetrics fields:

• twr_equity_curve
• twr_drawdown_series
• twr_max_drawdown
• twr_max_drawdown_duration

The HTML backtest dashboard surfaces these as a dashed alpha-only
overlay on the equity and drawdown charts.

Broker balance tracker + Context.sync_portfolio()

• New BrokerBalanceTracker for managing scheduled deposits/withdrawals
• Context.sync_portfolio() reconciles local portfolio state against
  the broker and records cash flows on every sync event
• Cash flows are now first-class so they can be subtracted from
  performance metrics (TWR)
• New docs page: Advanced Concepts → Portfolio sync

Stop and stop-limit orders

• StopOrderType enum + validation
• Order model gains stop_price / limit_price fields
• CCXT executor maps to exchange-native stop orders
• Backtest order service simulates trigger conditions per bar

Vector backtest scheduled deposits

The vector engine now honours BrokerBalanceTracker schedules so
DCA-style accumulation strategies can be backtested with realistic
external cash flows.

Authenticated external data (#529)

Optional headers parameter on Context.fetch_csv / fetch_json /
fetch_parquet and on DataSource.from_csv / from_json /
from_parquet. Cache keys are separated per credential and header
values are redacted ("***") in DataSource.to_dict() so secrets
don't leak into diagnostic payloads.

📚 Examples

• examples/strategies_showcase/ — 26 reference strategies
• examples/advanced_tutorials/cross-sectional-pipelines/
• examples/rust_vs_python_benchmark/

🔄 Compatibility

• BacktestMetrics adds new fields with defaults — from_dict() is
  forward/backward compatible (unknown keys are dropped, missing keys
  default).
• Backtest fixtures under
  tests/resources/backtest_reports_for_testing/ were regenerated to
  include the new twr_* fields.

✅ Test status

1668 passed, 42 skipped.

Full Changelog: v8.7.3...v8.8.0

v8.7.3

Highlights

Documentation, deployment hardening, security and CLI scaffolding follow-up to v8.7.2.

Docs

• New Getting Started/metrics.md reference page covering both per-run BacktestMetrics and cross-window BacktestSummaryMetrics, grouped by domain (returns, risk-adjusted, drawdown, per-trade, win/loss, exposure, multi-window robustness)
• Backtest.get_backtest_summary() getter added for symmetry with the other run/metrics getters
• README feature bullets now deep-link to the relevant docs pages
• application-setup.md: formatted folder tree, fixed notebook table, rewrote the example with the declarative generate_buy_signals / generate_sell_signals API
• backtest-reports.md rewritten to recommend recalculate_backtests_in_directory (memory-safe streaming) as the canonical entry point

Deprecations

• recalculate_backtests(List[Backtest]) is now deprecated (DeprecationWarning) and will be removed in a future major release. Holding many backtests in the parent process is memory-unsafe (each Backtest carries portfolio snapshots, trades and timeseries — thousands easily consume tens of GB before any work starts). Use recalculate_backtests_in_directory(src_dir, ...) instead, which streams from disk inside worker processes and keeps parent memory flat.

CLI

• init now scaffolds the recommended project layout (data/, backtest_results/, reports/, resources/, notebooks, .gitkeep markers)
• All variant templates (default, default-web, aws-lambda, azure-function) rewritten for the current declarative TradingStrategy API
• Azure variant correctly emits function_app.py (Azure Functions Python v2 model)
• Test suite aligned with the current generated layout (strategies/my_strategy.py, root data_providers.py)

Azure Function deployment

• deploy_to_azure_function polls for app readiness instead of time.sleep(60)
• Pushes app settings (SCM_DO_BUILD_DURING_DEPLOYMENT, ENABLE_ORYX_BUILD) before publish
• Adds --build remote --python to func azure functionapp publish (mandatory for native wheels on Linux Consumption)
• Bumped runtime 3.10 → 3.11
• Fixed CORS process await bug; CORS errors downgraded to warnings
• host.json no longer hides Request traces; explicit Information log level for the framework logger
• Prints az webapp log tail command at end

Security

• npm overrides block in docusaurus/package.json pinning serialize-javascript ^7.0.5 and postcss ^8.5.10
• Closes 3 open Dependabot alerts (1 high, 2 moderate)
• npm audit reports 0 vulnerabilities

Full changelog: v8.7.2...v8.7.3

v8.7.2

Bug fixes — backtest pipeline memory

v8.7.1 bounded in-flight tasks but two underlying causes of OOM
remained for users with thousands of large backtests:

1. recalculate_backtests(List[Backtest]) requires the entire batch
   to be resident in the parent process before the call. With
   thousands of backtests holding portfolio snapshots, trades and
   timeseries, that alone is tens of GB.
2. ProcessPoolExecutor reuses worker processes across tasks. Heavy
   metric calculations leave behind cached pandas / polars / numpy
   buffers the worker's allocator never returns to the OS, so RSS
   grows over time even with bounded in-flight tasks.

New streaming APIs

from investing_algorithm_framework import (
    recalculate_backtests_in_directory,
    iter_backtests_from_directory,
)

# Stream-recalculate every bundle on disk; the parent never holds a
# Backtest in memory.
recalculate_backtests_in_directory("./backtests", workers=4)

# Process backtests one at a time without materialising a list.
for bt in iter_backtests_from_directory("./backtests"):
    do_something(bt)
    del bt

recalculate_backtests_in_directory loads, recalculates and saves
each backtest inside a worker process. The Backtest never crosses
the process boundary — workers return only the destination path and a
small index row, used to rewrite index.parquet incrementally.

Worker recycling

max_tasks_per_child=16 was added to:

• recalculate_backtests
• recalculate_backtests_in_directory
• load_backtests_from_directory
• migrate_backtests

On Python 3.11+ this uses the native ProcessPoolExecutor parameter;
on 3.10 it is emulated by closing and re-opening the executor every
max_tasks_per_child × workers completions. This keeps long-running
batches' RSS bounded.

v8.7.1

Bug fixes

Memory usage in recalculate_backtests and migrate_backtests

Both helpers could blow up RAM (multiple GB) when run with multiple workers on large batches. Workers were returning entire Backtest objects (full snapshots, trades, timeseries) through the multiprocessing result queue, and tasks were submitted up-front so the executor feeder buffered every backtest at once.

• recalculate_backtests: workers now return only the freshly computed per-run metrics and summary; the parent merges them back into the existing Backtest objects. In-flight tasks are bounded to workers, so peak memory scales with workers, not len(backtests).
• migrate_backtests: index.parquet is now built from small row dicts returned by the workers, so the parent no longer re-loads every freshly written bundle into memory after migration. ex.map was replaced with a bounded submit + wait(FIRST_COMPLETED) loop.

migrate_backtests not exported from the package root

migrate_backtests is now exported from investing_algorithm_framework:

from investing_algorithm_framework import migrate_backtests

Previously it was only reachable via investing_algorithm_framework.domain.

v8.7.0

v8.7.0

Highlights

This release introduces the cross-sectional Pipeline API and a brand-new optimized backtest persistence format.

Cross-sectional Pipeline API

• Phase 1 — API foundation (#501): Cross-sectional Pipeline primitives, factor algebra, screen/rank semantics
• Phase 2 — Engine (#502): VectorPipelineEngine with factor arithmetic, cross-sectional transforms, lazy executor; wired into VectorBacktestService
• Phase 3 — Live envelope (#503): Universe-refresh cache, per-pipeline resilience, warmup-window validation at strategy construction
• Risk-neutrality primitives (#504): Group-relative demeaning / standardisation factors with groups= support
• New examples under examples/cross-sectional-pipelines/
• Pipeline tests converted from pytest to unittest for consistency with the rest of the suite

.iafbt Backtest Bundle Format (#516, closes #487)

• Custom binary container: IAFB magic header + uint32 version + zstd-compressed msgpack body — explicit, versioned, compressed, language-portable
• Content-addressed parquet OHLCV side-store with shared deduplication across runs
• BacktestIndex (parquet) for filter-without-load and fast discovery
• ~21× smaller, ~27× fewer files than the legacy directory layout
• 3× faster Backtest.open (1025 ms → 334 ms) via datetime.fromisoformat fast-path in hot from_dict paths
• Dashboard auto-discovers .iafbt bundles alongside legacy directories
• migrate_backtests: streaming, parallel, resume-safe (skip_existing=True), with optional delete_source and include_ohlcv flags; CLI: iaf migrate-backtests
• recalculate_backtests: ~1.9× faster after replacing an O(N²) snapshot lookup in get_rolling_sharpe_ratio with a dict; format-agnostic — works on bundle- and legacy-loaded backtests
• README + Docusaurus docs updated

PRs included

• #501 — Pipeline API Phase 1
• #502 — VectorPipelineEngine (Phase 2a–2d)
• #503 — Pipeline live envelope (Phase 3a–3d)
• #504 — Pipeline risk-neutrality primitives
• #515 — Pipeline live envelope merge
• #516 — .iafbt bundle format + migration + perf

Tests

• 1642 unittests passing (42 skipped)

v8.6.1

v8.6.1

Patch release with a critical fix for walk-forward backtest summaries.

Bug fixes

• fix(backtesting): aggregate summary.json across all walk-forward runs (#511, #512) — Backtest.save() and Backtest.merge() now always rebuild backtest_summary from the current backtest_runs. Previously, multi-window walk-forward backtests silently wrote a single-window summary to disk. Also fixes B1–B5 from the issue:
  • total_loss / total_loss_percentage now use gross-loss magnitude semantics (sum(abs(net_gain)) over losing trades) and divide by initial_unallocated.
  • _compound_percentage_returns now expects/returns decimals, consistent with the rest of the framework (the earlier whole-percentage assumption silently produced results off by ~100× in multi-window aggregations).
  • create_backtest_metrics now populates trading_symbol, initial_unallocated, and backtest_date_range_name on per-run metrics so downstream aggregation has the context it needs.
  • Documented total_growth* (legacy alias of total_net_gain*) and average_* (time-weighted means across windows) field semantics.

Recalculating existing on-disk results

Backtest folders produced by older versions of the framework can be rebuilt in-place using the public recalculate_backtests() API:

from pathlib import Path
from investing_algorithm_framework import Backtest, recalculate_backtests

ROOT = Path("./backtest_results")
backtest_dirs = [p.parent for p in ROOT.rglob("runs") if p.is_dir()]
backtests = [Backtest.open(d) for d in backtest_dirs]
recalculate_backtests(backtests)
for bt, d in zip(backtests, backtest_dirs):
    bt.save(d)

Full diff: v8.6.0...v8.6.1

v8.6.0

Changes

• feat: extract consistency and stability score functions into reusable metrics module
• perf(backtest): speed up parallel backtests on Windows/WSL
• chore: bump version to v8.6.0

v8.5.0

What's Changed

• feat: Jupyter Notebook / IPython Magic Integration (#454) by @MDUYN in #481

Full Changelog: v8.4.0...v8.5.0

v8.4.0

What's Changed

Bug Fixes

• Windows unittest database locking (#291): Fixed SQLite file locking issues on Windows by force-closing StaticPool DBAPI connections and properly releasing file locks before database deletion
• Unicode encoding: Fixed UnicodeEncodeError in validate_backtest_checkpoints on Windows (cp1252 → utf-8)
• Test isolation: Added teardown_sqlalchemy() to all test teardowns to prevent stale database state between tests

New Features

• Record function (#455): Added ability to record custom variables during strategy execution and backtesting

CI/CD

• Added Windows to CI test matrix (ubuntu, macos, windows × Python 3.10, 3.11)
• Per-OS test status badges in README (Linux, macOS, Windows for main and dev)
• Increased CI timeout to 40 minutes for Windows coverage runs

Chores

• Removed scripts/ directory
• Renamed test data files with Windows-incompatible characters (colons → dashes)

v8.3.0

What's Changed

• feat: add Yahoo Finance OHLCV data provider by @MDUYN in #470
• chore(deps): bump picomatch from 2.3.1 to 2.3.2 in /docusaurus by @dependabot[bot] in #404
• chore(deps): bump path-to-regexp from 0.1.12 to 0.1.13 in /docusaurus by @dependabot[bot] in #409
• chore(deps): bump brace-expansion from 1.1.12 to 1.1.13 in /docusaurus by @dependabot[bot] in #415
• chore(deps): bump lodash from 4.17.23 to 4.18.1 in /docusaurus by @dependabot[bot] in #416
• chore(deps): bump follow-redirects from 1.15.11 to 1.16.0 in /docusaurus by @dependabot[bot] in #423
• chore(deps): bump requests from 2.32.5 to 2.33.0 by @dependabot[bot] in #405
• chore(deps): bump pygments from 2.19.2 to 2.20.0 by @dependabot[bot] in #410
• chore(deps): bump aiohttp from 3.13.2 to 3.13.4 by @dependabot[bot] in #411
• chore(deps): bump cryptography from 46.0.5 to 46.0.7 by @dependabot[bot] in #421
• chore(deps): bump mako from 1.3.10 to 1.3.11 by @dependabot[bot] in #427
• chore(deps): bump nbconvert from 7.17.0 to 7.17.1 by @dependabot[bot] in #461
• chore(deps): bump python-dotenv from 1.2.1 to 1.2.2 by @dependabot[bot] in #464
• feat: add CSV, JSON, and Parquet URL data providers by @MDUYN in #472
• feat: Blotter System for Trade Documentation and Order Book Management by @MDUYN in #474
• Release: merge dev into main by @MDUYN in #475

Full Changelog: v8.2.0...v8.3.0