Merge pull request #477 from coding-kitties/feature/record-function

feat: add context.record() for tracking custom variables during backtests

Author: MDUYN

README.md

@@ -77,7 +77,8 @@ This framework is built around the full loop: **create strategies → backtest t
 - 📉 **Benchmark Comparison** — Beat-rate analysis vs Buy & Hold, DCA, risk-free & custom benchmarks
 - 📄 **One-Click HTML Report** — Self-contained file, no server, dark & light theme, shareable
 - 🌐 **Load External Data** — Fetch CSV, JSON, or Parquet from any URL with caching and auto-refresh
-- 🚀 **Build → Backtest → Deploy** — Local dev, cloud deploy (AWS / Azure), or monetize on Finterion
+- � **[Record Custom Variables](https://coding-kitties.github.io/investing-algorithm-framework/Advanced%20Concepts/recording-variables)** — Track any indicator or metric during backtests with `context.record()`
+- �🚀 **Build → Backtest → Deploy** — Local dev, cloud deploy (AWS / Azure), or monetize on Finterion
 
 </details>
 
@@ -272,6 +273,7 @@ report.save("my_report.html")
 | **[Cloud Deployment](https://coding-kitties.github.io/investing-algorithm-framework/Getting%20Started/deployment)** | Deploy to AWS Lambda, Azure Functions, or run as a web service |
 | **[Market Data Providers](https://coding-kitties.github.io/investing-algorithm-framework/Advanced%20Concepts/custom-data-providers)** | Built-in providers for CCXT, Yahoo Finance, Alpha Vantage, and Polygon — or build your own |
 | **[Load External Data](https://coding-kitties.github.io/investing-algorithm-framework/Data/external-data)** | Fetch CSV, JSON, or Parquet from any URL with caching, date parsing, and pre/post-processing |
+| **[Record Custom Variables](https://coding-kitties.github.io/investing-algorithm-framework/Advanced%20Concepts/recording-variables)** | Track any indicator or metric during backtests with `context.record()` |
 | **[Strategies](https://coding-kitties.github.io/investing-algorithm-framework/Getting%20Started/strategies)** | OHLCV, tickers, custom data — Polars and Pandas native |
 | **[Extensible](https://coding-kitties.github.io/investing-algorithm-framework/Advanced%20Concepts/custom-data-providers)** | Custom data providers, order executors, and strategy classes |
 
@@ -306,7 +308,7 @@ python -m unittest discover -s tests
 
 - [Open an issue](https://github.com/coding-kitties/investing-algorithm-framework/issues/new) for bugs or ideas
 - Read the [Contributing Guide](https://coding-kitties.github.io/investing-algorithm-framework/Contributing%20Guide/contributing)
-- PRs go against the `develop` branch
+- PRs go against the `dev` branch
 
 ## Risk Disclaimer
 

docusaurus/docs/Advanced Concepts/recording-variables.md

@@ -0,0 +1,142 @@
+---
+sidebar_position: 3
+---
+
+# Recording Custom Variables
+
+During backtesting you often want to track custom indicators, metrics, or signals alongside your trades — for example an RSI value, a moving average, or a custom score. The `record()` API lets you store **any** key-value pair at each backtest iteration so you can analyse it after the run completes.
+
+Recorded values are stored on the `BacktestRun` object and fully support serialization (save & load).
+
+## Event-Driven Backtests
+
+In an event-driven backtest your strategy's `on_run` method receives a `context` object. Call `context.record()` with arbitrary keyword arguments:
+
+```python
+from investing_algorithm_framework import TradingStrategy, TimeUnit
+
+class MyStrategy(TradingStrategy):
+    time_unit = TimeUnit.DAY
+    interval = 1
+    symbols = ["BTC"]
+
+    def on_run(self, context, data):
+        ohlcv = data["BTC/EUR_1d"]
+        close = ohlcv["Close"]
+
+        # Compute any indicator you like
+        sma_20 = close.rolling(20).mean().iloc[-1]
+        rsi = compute_rsi(close).iloc[-1]
+
+        # Record them — keys can be anything
+        context.record(
+            sma_20=sma_20,
+            rsi=rsi,
+            signal_strength=0.85,
+        )
+```
+
+Each call stores the values together with the current backtest timestamp. You can call `record()` multiple times per iteration — values are appended.
+
+:::tip
+`context.record()` is a **no-op** in live mode, so you can leave the calls in your production strategy without any overhead.
+:::
+
+## Vectorized Backtests
+
+Vectorized backtests don't use a `context` object. Instead, override `generate_recorded_values()` on your strategy and return a dictionary of `pandas.Series`:
+
+```python
+import pandas as pd
+from investing_algorithm_framework import TradingStrategy
+
+class MyVectorStrategy(TradingStrategy):
+    symbols = ["BTC"]
+
+    def generate_buy_signals(self, data):
+        # ... your buy logic ...
+        pass
+
+    def generate_sell_signals(self, data):
+        # ... your sell logic ...
+        pass
+
+    def generate_recorded_values(self, data):
+        ohlcv = data["BTC/EUR_1d"]
+        close = ohlcv["Close"]
+
+        return {
+            "sma_20": close.rolling(20).mean(),
+            "rsi": compute_rsi(close),
+        }
+```
+
+Each key becomes a recorded variable with the Series index as timestamps.
+
+## Accessing Recorded Values
+
+After a backtest completes the recorded values are available on the `BacktestRun`:
+
+```python
+from investing_algorithm_framework import create_app
+
+app = create_app()
+# ... configure app, add strategy, data sources ...
+
+backtest = app.run_backtest()
+run = backtest.backtest_runs[0]
+
+# Dict[str, List[Tuple[datetime, Any]]]
+print(run.recorded_values)
+
+# Example: extract RSI time series
+for dt, value in run.recorded_values["rsi"]:
+    print(f"{dt}: RSI = {value}")
+```
+
+### Converting to a DataFrame
+
+You can easily convert recorded values into a pandas DataFrame for plotting or further analysis:
+
+```python
+import pandas as pd
+
+rsi_series = pd.Series(
+    {dt: val for dt, val in run.recorded_values["rsi"]}
+)
+sma_series = pd.Series(
+    {dt: val for dt, val in run.recorded_values["sma_20"]}
+)
+
+df = pd.DataFrame({"rsi": rsi_series, "sma_20": sma_series})
+print(df)
+```
+
+## Serialization
+
+Recorded values are included when you save and load a `BacktestRun`:
+
+```python
+# Save
+run.save("/path/to/output")
+
+# Load
+from investing_algorithm_framework import BacktestRun
+loaded = BacktestRun.open("/path/to/output")
+print(loaded.recorded_values)
+```
+
+The values are stored in the `run.json` file under the `recorded_values` key.
+
+## Supported Value Types
+
+You can record any JSON-serializable value:
+
+| Type | Example |
+|------|---------|
+| `float` | `context.record(rsi=70.5)` |
+| `int` | `context.record(signal=1)` |
+| `str` | `context.record(regime="bullish")` |
+| `dict` | `context.record(meta={"score": 0.9})` |
+| `list` | `context.record(weights=[0.3, 0.7])` |
+| `bool` | `context.record(is_trending=True)` |

docusaurus/sidebar.js

@@ -121,6 +121,10 @@ const sidebars = {
                     type: 'doc',
                     id: 'Advanced Concepts/PARALLEL_PROCESSING_GUIDE',
                 },
+                {
+                    type: 'doc',
+                    id: 'Advanced Concepts/recording-variables',
+                },
             ],
         },
         {

docusaurus/sidebars.js

@@ -121,6 +121,10 @@ const sidebars = {
                     type: 'doc',
                     id: 'Advanced Concepts/PARALLEL_PROCESSING_GUIDE',
                 },
+                {
+                    type: 'doc',
+                    id: 'Advanced Concepts/recording-variables',
+                },
             ],
         },
         {

investing_algorithm_framework/app/context.py

@@ -53,6 +53,7 @@ def __init__(
         self._blotter = None
         self._fx_rate_provider = None
         self._base_currency = None
+        self._recorded_values = {}  # key -> list of (datetime, value)
 
     def _validate_target_symbol(self, target_symbol, market=None):
         """
@@ -2334,3 +2335,64 @@ def get_transactions(self):
             list[Transaction]: Recorded transactions.
         """
         return self._blotter.get_transactions()
+
+    def record(self, **kwargs):
+        """
+        Record arbitrary key-value pairs at the current backtest timestamp.
+
+        This method allows you to store any custom indicator, metric, or
+        variable during a backtest. Each key creates a time series of
+        values that can be retrieved after the backtest completes via
+        ``BacktestRun.recorded_values``.
+
+        The values are stored as a list of ``(datetime, value)`` tuples
+        per key, allowing you to track any indicator over time.
+
+        This method only records during backtesting. In live mode it is
+        a no-op.
+
+        Args:
+            **kwargs: Arbitrary key-value pairs to record. Keys are
+                strings, values can be any type (float, int, str,
+                dict, list, etc.).
+
+        Example::
+
+            def on_run(self, context, data):
+                context.record(
+                    rsi=compute_rsi(data),
+                    sma_20=compute_sma(data, 20),
+                    signal_strength=0.85,
+                )
+        """
+        is_backtest = self.configuration_service.config.get(
+            BACKTESTING_FLAG, False
+        )
+
+        if not is_backtest:
+            return
+
+        current_datetime = self.configuration_service.config.get(
+            INDEX_DATETIME
+        )
+
+        for key, value in kwargs.items():
+            if key not in self._recorded_values:
+                self._recorded_values[key] = []
+            self._recorded_values[key].append((current_datetime, value))
+
+    def get_recorded_values(self):
+        """
+        Get all recorded values from the context.
+
+        Returns:
+            dict: A dictionary mapping keys to lists of
+                ``(datetime, value)`` tuples.
+        """
+        return self._recorded_values
+
+    def clear_recorded_values(self):
+        """
+        Clear all recorded values from the context.
+        """
+        self._recorded_values = {}

investing_algorithm_framework/app/strategy.py

@@ -850,6 +850,32 @@ def generate_scale_out_signals(
         """
         return None
 
+    def generate_recorded_values(
+        self, data: Dict[str, Any]
+    ) -> Union[Dict[str, pd.Series], None]:
+        """
+        Optional method to generate recorded values for vectorized
+        backtesting. Override this to record arbitrary indicators,
+        metrics, or variables as time series during a vectorized
+        backtest.
+
+        Each key in the returned dict becomes a recorded variable
+        with the Series index as timestamps and the Series values
+        as the recorded data.
+
+        This is the vectorized equivalent of calling
+        ``context.record()`` in event-driven backtests.
+
+        Args:
+            data (Dict[str, Any]): The market data for the strategy.
+
+        Returns:
+            Dict[str, Series] | None: A dictionary where keys are
+                variable names and values are pandas Series with the
+                recorded values. Return None to not record anything.
+        """
+        return None
+
     def on_trade_closed(self, context: Context, trade: Trade):
         pass
 

investing_algorithm_framework/domain/backtesting/backtest_run.py

@@ -112,6 +112,7 @@ class BacktestRun:
     metadata: Dict[str, str] = field(default_factory=dict)
     signals: Dict[str, Dict[str, Any]] = field(default_factory=dict)
     signal_events: List[Dict[str, Any]] = field(default_factory=list)
+    recorded_values: Dict[str, List] = field(default_factory=dict)
 
     def to_dict(self) -> dict:
         """
@@ -167,6 +168,16 @@ def ensure_iso(value):
                     "date": ensure_iso(evt["date"])
                 } for evt in self.signal_events
             ],
+            "recorded_values": {
+                key: [
+                    {
+                        "datetime": ensure_iso(entry[0]),
+                        "value": entry[1]
+                    }
+                    for entry in entries
+                ]
+                for key, entries in self.recorded_values.items()
+            },
         }
 
     @staticmethod
@@ -330,10 +341,26 @@ def open(directory_path: Union[str, Path]) -> 'BacktestRun':
                     pass
             signal_events.append(parsed)
 
+        # Parse recorded_values
+        raw_recorded = data.pop("recorded_values", {})
+        recorded_values = {}
+        for key, entries in raw_recorded.items():
+            parsed_entries = []
+            for entry in entries:
+                dt = entry.get("datetime")
+                if isinstance(dt, str):
+                    try:
+                        dt = datetime.fromisoformat(dt)
+                    except (ValueError, TypeError):
+                        pass
+                parsed_entries.append((dt, entry.get("value")))
+            recorded_values[key] = parsed_entries
+
         return BacktestRun(
             backtest_metrics=backtest_metrics,
             signals=signals,
             signal_events=signal_events,
+            recorded_values=recorded_values,
             **data
         )
 

investing_algorithm_framework/infrastructure/services/backtesting/event_backtest_service.py

@@ -112,6 +112,8 @@ def run(
             backtest_date_range=backtest_date_range,
             number_of_runs=event_loop_service.total_number_of_runs,
             risk_free_rate=risk_free_rate,
+            recorded_values=event_loop_service.context
+            .get_recorded_values(),
         )
 
     def generate_schedule(
@@ -175,6 +177,7 @@ def _create_backtest_run(
         backtest_date_range: BacktestDateRange,
         number_of_runs: int,
         risk_free_rate: float,
+        recorded_values: dict = None,
     ) -> BacktestRun:
         """
         Create a BacktestRun from the current state after event loop execution.
@@ -184,6 +187,7 @@ def _create_backtest_run(
             backtest_date_range: The date range of the backtest.
             number_of_runs: Total number of strategy executions.
             risk_free_rate: Risk-free rate for metrics calculation.
+            recorded_values: Optional dict of recorded values from context.
 
         Returns:
             BacktestRun: The completed backtest run with metrics.
@@ -215,6 +219,7 @@ def _create_backtest_run(
             positions=self._position_repository.get_all(
                 {"portfolio": portfolio.id}
             ),
+            recorded_values=recorded_values or {},
         )
 
         # Calculate and add metrics