Skip to content

Commit 800244a

Browse files
fix(benchmarks): address PR #76 review feedback from @timon0305
Drop unnecessary actions: write on benchmarks CI job; assert explicit search hit count on list response; document 10x memory ceiling and v1 template limitations in README; note NoopSink export rounds are stateless.
1 parent ea3b1ca commit 800244a

5 files changed

Lines changed: 10 additions & 2 deletions

File tree

.github/workflows/ci.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -210,7 +210,6 @@ jobs:
210210
runs-on: ubuntu-latest
211211
permissions:
212212
contents: read
213-
actions: write
214213
steps:
215214
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
216215
with:

benchmarks/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,10 @@ The memory test also runs as part of the normal `pytest` suite (timing benchmark
2929

3030
Large JSONL files (5000+ lines) are generated at test session scope under pytest's temp directory — not committed to git.
3131

32+
Corpora repeat one row from `tests/fixtures/session_with_tools.jsonl`, so parse/export numbers measure steady-state throughput on a narrow schema slice — not full parser branch coverage. Treat as v1 baselines, not exhaustive perf proof.
33+
34+
The memory test (`test_parse_memory.py`) is intentionally **not** skipped by `--benchmark-skip`; it runs in the main `pytest` job and builds the session-scoped 5000-line fixture once per session.
35+
3236
## CI
3337

3438
The `benchmarks` workflow job uploads `benchmark-results.json` as a downloadable artifact. There is no regression gate yet.

tests/benchmarks/test_export_bench.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ def test_bulk_export_session_count(
2323
projects = [{"name": "bench-project", "path": str(export_corpus), "display_name": "Bench"}]
2424

2525
def _run() -> object:
26+
# NoopSink + since="all" + empty last_export_sessions: no disk/state writes per round.
2627
return run_bulk_export(
2728
projects=projects,
2829
since="all",

tests/benchmarks/test_parse_memory.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,9 @@
1111
def test_large_parse_peak_memory_under_ceiling(parse_large_file: Path) -> None:
1212
path = parse_large_file
1313
file_bytes = path.stat().st_size
14+
# Issue #7 ceiling: Python heap peak (tracemalloc) vs on-disk JSONL size. Parsed
15+
# dict/str objects often exceed raw bytes; 10x is a generous v1 guard — relax with
16+
# a comment here if the parser legitimately grows.
1417
ceiling = file_bytes * 10
1518

1619
tracemalloc.start()

tests/benchmarks/test_search_bench.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,5 @@ def _run() -> object:
1616

1717
resp = benchmark(_run)
1818
assert resp.status_code == 200
19-
assert resp.get_json(), "expected search hits from synthetic searchable tokens"
19+
hits = resp.get_json()
20+
assert isinstance(hits, list) and len(hits) > 0, "expected search hits from synthetic corpus"

0 commit comments

Comments
 (0)