Fix stale benchmark imports and add benchmarks typecheck to CI

[jonathanMLDev]

Summary

Fixes benchmarks/latency.ts import paths broken by the src/core/ restructure and adds CI coverage so benchmark import/type drift is caught before merge.

• Correct 4 stale imports to src/core/... (pinecone-client, client-context, namespaces-cache, guided-query-tool)
• Add benchmarks/tsconfig.json (noEmit, extends root) and npm run typecheck:benchmarks
• Add offline npm run benchmark:smoke (--smoke: WARMUP=1, ITERATIONS=3, no baseline.json, skips query_with_rerank)
• Wire both checks into npm run ci and .github/workflows/ci.yml

Related issue

• Close Fix stale benchmark imports + add benchmarks typecheck to CI #208

Why

Root tsconfig.json only includes src/**/*, so npm run typecheck never compiled benchmarks/. The harness still pointed at pre-restructure paths and failed at module resolution; CI had no step to catch it.

CI changes

Step	Command
Type check benchmarks	npm run typecheck:benchmarks (after main typecheck)
Benchmark smoke run	npm run benchmark:smoke (after CLI smoke test)

Emit output is unchanged: benchmark typecheck uses a separate noEmit project; dist/ remains scoped to src/.

Smoke mode behavior

--smoke is network-free for CI:

• Skips query_with_rerank — the bench double’s rerankResults override is dead post-restructure; rerank now goes through rerankResultsImpl(this.indexSession.ensureClient(), …) and would hit real Pinecone HTTP
• Bootstraps dummy PINECONE_API_KEY / PINECONE_INDEX_NAME so legacy ServerContext config resolution works offline

Full npm run benchmark (non-smoke) still runs all scenarios including query_with_rerank.

Follow-up (out of scope)

• Fix the rerank bench double so the full harness stays offline and query_with_rerank latency is meaningful again (mock indexSession.ensureClient() like indexes.test.ts)
• scripts/test-search.ts has the same class of stale imports and is not typechecked

Test plan

• npm run typecheck:benchmarks passes
• npm run benchmark:smoke exits 0 (4 scenarios, no query_with_rerank, no baseline.json)
• Broken import fails typecheck:benchmarks (TS2307)
• npm run build — no dist/benchmarks/
• npm run ci green locally

Summary by CodeRabbit

• Tests

  • Extended CI validation with benchmark type checking.
  • Added a benchmark smoke run in CI to quickly detect benchmark regressions.

• Chores

  • Added dedicated npm scripts for benchmark smoke and benchmark type checking.
  • Improved smoke-mode benchmark behavior to run a reduced workload and avoid unnecessary baseline output.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 091ba1c9-ea88-4676-858a-b5a37eafdd0b

📥 Commits

Reviewing files that changed from the base of the PR and between dc671df and 120a5f3.

📒 Files selected for processing (2)

• benchmarks/latency.ts
• benchmarks/tsconfig.json

🚧 Files skipped from review as they are similar to previous changes (1)

• benchmarks/latency.ts

---

📝 Walkthrough

Walkthrough

The benchmark harness now supports --smoke execution, corrected core imports, dedicated query-client typing, and safer mock output handling. Benchmark typechecking and smoke execution are added to npm scripts and CI through a separate no-emit TypeScript configuration.

Changes

Benchmark validation

Layer / File(s)	Summary
Smoke-mode benchmark execution benchmarks/latency.ts	Adds reduced warmup and iteration counts, corrected core imports, smoke-only environment defaults, rerank skipping, and conditional baseline generation.
Query benchmark client and output handling benchmarks/latency.ts	Introduces a dedicated query client shape, updates mock method wiring, normalizes result fields, and handles missing column widths.
Benchmark typecheck and CI wiring benchmarks/tsconfig.json, package.json, .github/workflows/ci.yml	Adds no-emit benchmark typechecking and runs benchmark typecheck and smoke commands in CI.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant GitHubActions
  participant npm
  participant latency.ts
  GitHubActions->>npm: run benchmark:smoke
  npm->>latency.ts: execute with --smoke
  latency.ts->>latency.ts: use reduced iterations and skip reranking

Loading

Possibly related PRs

• cppalliance/pinecone-read-only-mcp-typescript#79: The benchmark harness changes build on the benchmark runner introduced in that PR.

Suggested reviewers: wpak-ai

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: fixing stale benchmark imports and adding benchmark typechecking to CI.
Linked Issues check	✅ Passed	The PR corrects the stale benchmark imports, adds a separate benchmark typecheck project, and wires benchmark execution/typecheck into CI.
Out of Scope Changes check	✅ Passed	The changes stay focused on benchmark path fixes and CI coverage, with no clear unrelated code changes introduced.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@02014e0). Learn more about missing BASE report.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #219   +/-   ##
=======================================
  Coverage        ?   85.07%           
=======================================
  Files           ?       46           
  Lines           ?     2399           
  Branches        ?      826           
=======================================
  Hits            ?     2041           
  Misses          ?      357           
  Partials        ?        1           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.