revamped ops bench for better cost tuning

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Author: pranav-walimbe

bench/ops/README.md

@@ -1,49 +1,64 @@
 # Ops Calibration Benchmark
 
-Measures warm probe time per `(index_kind, query_kind)` at multiple N values and derives
-the `per_ns` constants used in `CostFactors` (`src/planner/calibration.rs`).
+Measures the 9 `CostFactors` ns/op constants used by the query planner's cost model
+(`src/planner/cost.rs`, `src/planner/calibration.rs`).
 
-## What it measures
-
-| Constant | Index | Query kind | Dataset |
-|----------|-------|------------|---------|
-| `kdtree_knn_ns` | KDTree | kNN | clustered points |
-| `kdtree_range_ns` | KDTree | range | clustered points |
-| `rtree_knn_ns` | RTree | kNN | polygons |
-| `rtree_range_ns` | RTree | range | polygons |
-| `grid_range_ns` | Grid | range | uniform points |
-| `scan_ns_per_item` | BruteForce | kNN | uniform points |
-| `build_ns_per_item` | KDTree | — | clustered points |
-
-Brute-force scan is only measured up to `--brute-max-n` (default 100,000) since it grows
-as `Q×N` and becomes impractical at large N.
+## Method
 
-All data is generated over `[0, 1]²`. Range queries use a `0.1 × 0.1` bbox (selectivity ≈ 0.01).
-kNN uses k=5. Each timing is the median of `--runs` repetitions.
+- Each dataset size generates synthetic data with numpy (uniform or clustered points) or
+  `shapely.box` (polygons), then wraps it in a `SpatialFrame`/`Engine`.
+- Build cost times only `engine.build_index()` on a fresh engine. Probe cost times the repeated queries against a pre-built index.
+- The constant is `time / workload_term`, where the term is read directly off the cost
+  model formula in `cost.rs` (e.g. `Q * N` for `scan_ns_per_item`).
+- This ratio is computed at multiple dataset sizes and we return the median as the suggested value. 
 
-## How constants are derived
 
-Each measured warm time `T` (ms) is inverted through the cost model formula:
+## What it measures
 
-| Constant | Formula |
-|----------|---------|
-| `kd_knn_ns`, `rt_knn_ns` | `T × 1e6 / (Q × (log₂N + k))` |
-| `kd_range_ns`, `rt_range_ns` | `T × 1e6 / (Q × (log₂N + sel×N))` |
-| `grid_range_ns` | `T × 1e6 / (Q × sel×N)` |
-| `scan_ns_per_item` | `T × 1e6 / (Q × N)` |
-| `build_ns_per_item` | `T × 1e6 / (N × log₂N)` |
+| Constant | Op timed | Dataset | Term |
+|---|---|---|---|
+| `scan_ns_per_item` | brute-force kNN, no index | uniform points | `Q * N` |
+| `grid_build_ns_per_item` | build | uniform points | `N` |
+| `kdtree_build_ns_per_item` | build | clustered points | `N * log2(N)` |
+| `rtree_build_ns_per_item` | build | polygons | `N * log2(N)` |
+| `grid_range_ns` | range probe | uniform points | true hit total |
+| `kdtree_range_ns` | range probe | clustered points | `Q * log2(N)` + true hit total |
+| `rtree_range_ns` | range probe | polygons | `Q * log2(N)` + true hit total |
+| `kdtree_knn_ns` | kNN probe | clustered points | `Q * (log2(N) + k)` |
+| `rtree_knn_ns` | kNN probe | polygons | `Q * (log2(N) + k)` |
 
 ## Running
 
 ```
 uv run python -m bench.ops
 ```
 
-Optional flags:
+Flags:
 
 ```
---sizes N [N ...]     dataset sizes to sweep (default: 10000 100000 500000 1000000)
---queries Q           queries per timing call (default: 500)
---runs R              repetitions per measurement, median taken (default: 3)
---brute-max-n N       skip brute-force above this N (default: 100000)
+--runs R   timing repetitions per measurement, the minimum is taken (default: 3)
+--seed S   RNG seed for data and query generation (default: 42)
 ```
+
+## Example Output
+
+```
+Suggested CostFactors (copy into src/planner/calibration.rs):
+
+Brute Force
+    scan_ns_per_item:            100.80,
+
+Points
+    grid_build_ns_per_item:      84.74,
+    kdtree_build_ns_per_item:    4.36,
+    grid_range_ns:               211.38,
+    kdtree_range_ns:             81.19,
+    kdtree_knn_ns:               148.71,
+
+Polygons
+    rtree_build_ns_per_item:     74.17,
+    rtree_range_ns:              176.16,
+    rtree_knn_ns:                1299.75,
+
+elapsed: 20.3 s   peak RSS: 268.9 MiB
+```