Merge pull request #3 from quantbai/dev

release: v0.3.0

Author: quantbai

CHANGELOG.md

@@ -7,6 +7,19 @@ Numerical changes are marked with [NUMERICAL].
 
 ## [Unreleased]
 
+## [0.3.0] - 2026-03-24
+
+### Fixed
+- [NUMERICAL] `ts_covariance`: unified to ddof=0 (population), consistent with all other
+  variance/std operators. Fixes the broken identity `cov/(std_x*std_y) == corr` which
+  previously had ~5-20% error due to mixed ddof. Cross-validated against numpy (diff < 1e-15).
+
+### Changed
+- OPERATORS.md rewritten as pure operator reference manual (signatures, behavior, edge cases)
+- Design rationale moved to CLAUDE.md Section 4.1 (developer-facing)
+- Fixed incorrect signatures in docs: `trade_when`, `scale`, `bucket`
+- Fixed README example code to use only columns present in sample data
+
 ## [0.2.0] - 2026-03-23
 
 ### Added
@@ -24,7 +37,7 @@ Numerical changes are marked with [NUMERICAL].
 
 ### Fixed
 - [NUMERICAL] `ts_product`: silently returned null for negative inputs; now correctly handles negative values via sign-magnitude decomposition
-- [NUMERICAL] `ts_covariance`: used ddof=0 (population) inconsistent with `ts_corr` (ddof=1); aligned to ddof=1 (sample)
+- [NUMERICAL] `ts_covariance`: added explicit ddof parameter (was using Polars default)
 - [NUMERICAL] `divide()`: no zero-denominator protection; now returns null where abs(divisor) < 1e-10
 - [NUMERICAL] `inverse()`: no zero protection; now returns null where abs(x) < 1e-10
 - `ts_regression` rettype=7 (MSE): implicit Inf-to-null on window=2; now has explicit guard

CLAUDE.md

@@ -36,6 +36,8 @@ elvers/
 tests/
   conftest.py              Test fixtures (make_ts, make_factor)
   test_*.py                One test file per operator module
+OPERATORS.md               Operator specification: numerical conventions, per-operator behavior, design rationale
+CLAUDE.md                  Development standards (this file)
 ```
 
 ---
@@ -92,13 +94,34 @@ git push -u origin fix/bug-name
 
 ### 4.1 Numerical Correctness (Highest Priority)
 
+Operator behavior reference: [OPERATORS.md](OPERATORS.md).
+The rules below are for writing new code:
+
 - All divisions MUST have explicit zero guards:
   `pl.when(denom.abs() < 1e-10).then(None).otherwise(num / denom)`
 - NEVER rely on the Factor constructor's implicit Inf-to-null conversion as normal logic flow
-- Statistical convention: ddof=0 (population) for std/variance, ddof=1 (sample) for corr/cov.
-  This is consistent across the entire library.
 - Null semantics: null propagates naturally through Polars expressions. Boundary cases
-  (zero denominator, constant window, insufficient data) must be handled explicitly.
+  (zero denominator, constant window, insufficient data) must be handled explicitly
+
+#### Design Decisions (rationale for current conventions)
+
+- **NaN/Inf unified to null**: eliminates the NaN-infection problem (`NaN + 1 = NaN`)
+  that silently corrupts downstream computations. The Factor constructor converts on
+  creation so the entire library operates on a single missing-value type.
+- **ddof=0 everywhere**: rolling windows and cross-sections operate on the full observed
+  population, not a sample from a larger one. ddof=0 is semantically correct and avoids
+  n=1 division-by-zero (ddof=1 divides by n-1=0).
+- **ts_corr/ts_autocorr use ddof=1 internally**: Polars `rolling_corr(ddof=0)` has a bug
+  where ddof only applies to the covariance numerator, not the variance denominator,
+  producing values outside [-1, 1]. Reported: https://github.com/pola-rs/polars/issues/16161.
+  Correlation is ddof-invariant (cancels in ratio), so ddof=1 output is correct.
+- **rank range (0, 1] not [0, 1]**: a rank of 0 is ambiguous (could mean "missing" or
+  "lowest"). Strictly positive range ensures every ranked value is distinguishable from null.
+- **Zero guard threshold 1e-10**: conservative enough to catch near-zero denominators,
+  small enough not to interfere with legitimate small values in financial data.
+- **ts_product sign-magnitude decomposition**: naive `exp(sum(log(x)))` fails for negative
+  inputs because `log(x)` is undefined for x < 0. Separating sign and magnitude handles
+  this correctly.
 
 ### 4.2 Operator Writing Rules
 
@@ -178,7 +201,8 @@ negative values. Previously returned null, now returns correct product.
 
 ## 7. Code Review Rules
 
-- All PRs require at least one reviewer approval before merge
+- When the team has multiple developers, enable "Require approvals" in branch protection
+- Currently (single-developer mode): CI status checks are required, review approval is optional
 - Reviewer must verify:
   1. Tests pass and cover the change
   2. Numerical correctness (manually verify at least one expected value)
@@ -220,26 +244,26 @@ pytest tests/ -v
 ruff check elvers/
 
 # 2. Update version number (single source of truth)
-#    Edit elvers/__init__.py: __version__ = "0.2.0"
+#    Edit elvers/__init__.py: __version__ = "X.Y.Z"
 
 # 3. Update CHANGELOG.md
-#    Move items from [Unreleased] to [0.2.0] - YYYY-MM-DD
+#    Move items from [Unreleased] to [X.Y.Z] - YYYY-MM-DD
 
 # 4. Commit the release
 git add elvers/__init__.py CHANGELOG.md
-git commit -m "release: v0.2.0"
+git commit -m "release: vX.Y.Z"
 git push origin dev
 
 # 5. Create PR: dev -> main on GitHub
-#    Title: "release: v0.2.0"
+#    Title: "release: vX.Y.Z"
 #    Wait for CI to pass and review approval
 #    Squash merge on GitHub
 
 # 6. Tag on main (after PR merged)
 git checkout main
 git pull origin main
-git tag v0.2.0
-git push origin v0.2.0
+git tag vX.Y.Z
+git push origin vX.Y.Z
 
 # 7. Automated (triggered by tag push):
 #    - CI runs full test suite again
@@ -251,14 +275,14 @@ git push origin v0.2.0
 
 ### What Happens Automatically
 
-When you push a tag like `v0.2.0`:
+When you push a tag like `vX.Y.Z`:
 
 1. `.github/workflows/publish.yml` triggers
 2. Runs full test suite on Python 3.10-3.13 (safety net)
 3. If tests pass: builds package, publishes to PyPI
 4. Creates a GitHub Release page at github.com/quantbai/elvers/releases
    with auto-generated release notes from commit messages
-5. Users can now `pip install elvers==0.2.0`
+5. Users can now `pip install elvers==X.Y.Z  # specific version`
 
 ### What You See on GitHub After Release
 
@@ -268,29 +292,15 @@ When you push a tag like `v0.2.0`:
 
 ---
 
-## 10. One-Time Setup (for repository admin)
-
-### PyPI Trusted Publisher (required for automated publishing)
-
-1. Go to https://pypi.org -> Your projects -> elvers -> Publishing
-2. Add a new publisher:
-   - Owner: quantbai
-   - Repository: elvers
-   - Workflow name: publish.yml
-   - Environment: (leave blank)
+## 10. Setup
 
-### GitHub Branch Protection (strongly recommended)
+### Infrastructure (already configured)
 
-1. GitHub repo -> Settings -> Branches -> Add rule
-2. Branch name pattern: `main`
-3. Enable:
-   - "Require a pull request before merging"
-   - "Require approvals" (1 minimum)
-   - "Require status checks to pass before merging"
-   - Select required status check: "test"
-4. Save changes
+- PyPI Trusted Publisher: configured for quantbai/elvers -> publish.yml
+- GitHub Branch Protection on main: require PR, require CI status checks
+- GitHub Actions: ci.yml (push/PR) + publish.yml (tag-triggered release)
 
-### Local Development Setup (every developer)
+### Local Development Setup (every new developer)
 
 ```bash
 git clone https://github.com/quantbai/elvers.git
@@ -302,33 +312,15 @@ pre-commit install
 
 ---
 
-## 11. Commands Reference
+## 11. Quick Reference
 
 ```bash
-# === Setup ===
-pip install -e ".[dev]"          # Install with dev dependencies
-pre-commit install               # Install git hooks
-
-# === Daily Development ===
+pip install -e ".[dev]"          # Setup
+pre-commit install               # Git hooks
 pytest tests/ -v                 # Run all tests
-pytest tests/test_timeseries.py -v                # Single file
-pytest tests/test_timeseries.py::TestTsProduct -v # Single class
-ruff check elvers/               # Lint check
-ruff check elvers/ --fix         # Auto-fix lint issues
+pytest tests/test_timeseries.py::TestTsProduct -v  # Single test class
+ruff check elvers/ --fix         # Lint + auto-fix
 ruff format elvers/              # Format code
-
-# === Git ===
-git status                       # See what changed
-git diff                         # See actual changes
-git add <files>                  # Stage specific files (never git add -A)
-git commit -m "type(scope): msg" # Commit with convention
-git push origin <branch>         # Push to remote
-git log --oneline -10            # Recent history
-
-# === Release ===
-python -m build                  # Build package locally (for testing)
-git tag v0.2.0                   # Create version tag
-git push origin v0.2.0           # Push tag (triggers publish)
 ```
 
 ---