docs: dogfood report for v3.10.1-dev.80#1180
Conversation
|
Claude finished @carlos-alm's task —— View job PR Review: Dogfood Report v3.10.1-dev.80Review Progress
Root Cause AnalysisProblem Being Solved: This PR adds comprehensive dogfooding documentation for codegraph v3.10.1-dev.80, providing validation that 12 newly-ported native extractors achieve byte-identical parity with WASM engines and documenting 4 critical bugs discovered during testing. Why This Approach: Documentation-only PR that follows the established pattern in Risk Assessment: Zero risk - pure documentation with no code changes. Backlog Compliance
Critical ConcernsNone identified. This is exemplary documentation: Strengths
Quality Indicators
Content ValidationThe report demonstrates the exact kind of quality validation codegraph needs:
Value to ProjectThis documentation:
Final Recommendation
|
Greptile SummaryThis docs-only PR applies two targeted fixes to the v3.10.1-dev.80 dogfood report in response to prior review comments: correcting extractor count references from 12 → 11, and replacing the incomplete jina-base benchmark placeholder with an explicit "not completed" marker linked to follow-up issue #1181.
Confidence Score: 5/5Docs-only change with no code modifications — safe to merge. All three text changes are correct and internally consistent with the rest of the report. The extractor count now reads '11' everywhere (§5 table, §6 intro, §12 summary, PR description), and the jina-base row is clearly marked as incomplete with a follow-up link rather than an ambiguous empty placeholder. No files require special attention. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Prior Review Comments] --> B[Fix 1: Extractor count 12 → 11]
A --> C[Fix 2: jina-base placeholder → explicit incomplete marker]
B --> D[§6 intro paragraph]
B --> E[§12 overall assessment]
C --> F[§8 embedding benchmark table\njina-base row links to #1181]
D --> G[Report internally consistent\n11 everywhere: §5 table, §6, §12, PR desc]
E --> G
F --> H[Clear: no data collected\nnot redacted]
Reviews (6): Last reviewed commit: "docs: align §12 overall assessment to 11..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
|
||
| ## 6. Release-Specific Tests (changes since v3.10.0) | ||
|
|
||
| The v3.10.1-dev.80 series adds 12 native extractor ports (#1097–#1107), several language-specific fixes (#1109, #1122, #1123, #1124, #1127, #1128, #1158), a MCP enhancement (#1149), benchmark-stability fixes (#1119, #1120, #1131, #1133, #1134), and CI hardening (#1146, #1151, #1164). |
There was a problem hiding this comment.
The PR description states "12 newly-ported native extractors validated," but the report consistently counts 11 throughout — the §5 table lists 11 languages, the §5 conclusion reads "all 11 newly-ported extractors," and the §6 changelog entry enumerates 11 ports (#1097–#1107). One of these two numbers is wrong and will cause confusion when the report is referenced later. If the correct count is 11, the PR description should be updated to match; if it's 12, the missing language should be added to the §5 table and the §6 test row.
|
|
||
| - No regressions vs the v3.10.0 baseline in `generated/benchmarks/BUILD-BENCHMARKS.md`. The corpus shrank (745 → 612 files) due to PR #1134's fixture exclusion, but per-file metrics improved on every engine. | ||
| - Native fast-skip preflight (#1054) is firing as expected: 16 ms no-op rebuild matches WASM's, validating the `detectNoChanges` short-circuit. | ||
| - The 1-file rebuild gap (WASM 45ms vs Native 67ms) is the inverse of full-build performance — WASM's lighter orchestrator setup wins on tiny incremental work. |
There was a problem hiding this comment.
jina-base embedding benchmark published as incomplete
The embedding benchmark table has jina-base (768d) with the value _benchmark still running at report cut_. Publishing a report with a known-pending data point makes the §8 table misleading — reviewers cannot assess whether jina-base regressed, improved, or is even viable for the release. Either wait for the run to finish and fill in the numbers, or explicitly mark this row as "not completed, see follow-up" and omit the empty columns so it's clear no data was collected rather than data being redacted.
There was a problem hiding this comment.
Fixed in ca4146c. Replaced the "benchmark still running at report cut" placeholder with an explicit "not completed in this session" marker linked to follow-up issue #1181, and zeroed out the empty Hit@k columns with em-dashes so it's clear no data was collected (rather than redacted). The follow-up issue tracks finishing the jina-base run and backfilling the numbers.
Fix two Greptile P1/P2 findings in the v3.10.1-dev.80 dogfood report: - §6 changelog said "12 native extractor ports (#1097-#1107)" but the PR range and the §5 parity table both enumerate 11. Correct to 11 so the body is internally consistent (the §5 table and conclusion already read 11/11). - §8 embedding-benchmark table left the jina-base (768d) row as "benchmark still running at report cut". Replace the placeholder with an explicit "not completed in this session" note and link to the follow-up issue #1181 so the missing data is clearly tracked rather than reading as redacted.
|
Fixed in 6d9e1bb. The §12 Overall Assessment opening sentence at line 435 now reads "11 new native extractors" — matching the §5 conclusion, §6 intro, §11 testing plan, and the §12 bullet list. The report is now fully self-consistent. |
1 participant
Summary
Adds the dogfooding report for
@optave/codegraph@3.10.1-dev.80(run on 2026-05-20).Test plan
fix/1175-embed-install-cwdontoorigin/main(separates docs from the code fix in fix(embed): install @huggingface/transformers into codegraph's host node_modules #1178 per one-PR-one-concern)