|
1 | | -# Knowledge flush — 1 insight |
| 1 | +# Consolidated review — knowledge PRs #6–#13 |
2 | 2 |
|
3 | | -Source: RNR-3440 (사내 잠재매물 주간 추출 스크립트 메모리 피크 저감). Candidate: |
4 | | -"QueryPie 프록시 경유로 대용량 결과를 스트리밍할 때 server-side named cursor 대신 |
5 | | -일반 커서 + `fetchmany` + openpyxl `write_only`." |
| 3 | +Eight fork PRs (`dch0202-rsquare`, 2026-07-28 → 2026-08-02) were reviewed together |
| 4 | +against `AGENTS.md`. Each PR was audited by an independent reviewer (format rules, |
| 5 | +sources, vague-qualifier ban, ≤120 body lines, index/log invariants), then |
| 6 | +cross-compared to catch duplication the per-PR flushes could not see — they branched |
| 7 | +independently off the same main and rewrote the same shared index/log files. Fork |
| 8 | +branches can't be edited from here and several PRs needed content changes (drop a |
| 9 | +duplicate, merge a colliding page), so this branch carries the reconciled end-state |
| 10 | +rather than merging each PR as-is (which would import the duplicates). |
6 | 11 |
|
7 | 12 | ## Verified best-practice |
8 | 13 |
|
9 | | -**Claim 1 — psycopg2 server-side (named) cursor requires a transaction; fails under autocommit.** |
10 | | -- Source: psycopg2 usage docs — `https://github.com/psycopg/psycopg2/blob/master/doc/src/usage.rst` (via Context7). Quote: "Named cursors are typically created 'WITHOUT HOLD', meaning they exist only within the current transaction. Attempting to fetch from them after a commit or in autocommit mode raises an exception." |
11 | | -- Matches my live repro (`can't use a named cursor outside of transactions`). → **verified** |
| 14 | +Sources are per-page and were live-verified in each originating PR's flush; the |
| 15 | +independent re-reviews re-checked them. Landed pages and their evidence base: |
12 | 16 |
|
13 | | -**Claim 2 — a client-side (default) cursor pulls the whole result set to the client on execute; `fetchmany` only caps the Python-list explosion.** |
14 | | -- Source: psycopg2 cursor/usage docs + FAQ (named-cursor advantage = "data is fetched in chunks … minimal client memory"). By contrast the default cursor buffers the full result in libpq. → **verified** |
| 17 | +| Page | Confidence | Source basis | |
| 18 | +|------|-----------|--------------| |
| 19 | +| backend/common/llm/completion-response-validation | verified | OpenAI reasoning guide + chat `object` spec (5 `finish_reason` values), vLLM/LiteLLM reasoning fields; field incident (200/`length`/empty content/8,173-char reasoning) | |
| 20 | +| backend/common/llm/context-window-budget | verified | Claude context-window docs, LiteLLM exception mapping, vLLM/Claude Code env-var docs | |
| 21 | +| backend/common/integrations/externally-owned-defaults | verified | OpenAI deprecations (notice windows) + models `list`, LiteLLM model_discovery; field incident (alias removed between PR verify and review → 400) | |
| 22 | +| backend/common/storage/object-key-persistence | verified | AWS S3 CompleteMultipartUpload + managed-upload API/source, aws-sdk-js issues #1158/#5656 | |
| 23 | +| infrastructure/containers/host-cgroup-visibility | field-tested | cgroup_namespaces(7), Docker `--cgroupns=host`, nsenter, k8s #103363; OrbStack repro | |
| 24 | +| infrastructure/observability/missing-container-metrics | verified/field-tested | k8s resource-metrics-pipeline docs, kube-prometheus-stack values, kubernetes-mixin; OrbStack #2217 repro | |
| 25 | +| platforms/environment/unicode-text-matching | verified | UAX #15, Unicode core §3.12, APFS FAQ, POSIX grep; local repro (macOS 15/APFS, grep 2.6.0-FreeBSD, Python 3.13) | |
| 26 | +| platforms/shells/command-text-inspected-before-execution | verified | Claude Code hooks docs, POSIX shell §2.6; local reproduction | |
| 27 | +| platforms/processes/non-interactive-cli-invocation | verified | GNU nohup, OpenBSD ssh/ssh_config, git, timeout man pages; no-request-in-gateway-log field incident | |
| 28 | +| qa/document-verification/spec-document-gates | field-tested | ESLint, Google mutation testing, RFC 2119, Vale, markdownlint; 32/32 mutant / 62/62 intact RFC sessions | |
| 29 | +| qa/document-verification/editing-a-gated-document | field-tested | pgrep, Vale, markdownlint; in-house editing methodology | |
| 30 | +| testing/quality/checks-that-cannot-pass | verified | James Shore AoAD2, POSIX grep exit status, Semgrep rule-testing, pytest exit codes; BSD/ugrep measurement | |
| 31 | +| testing/quality/spec-artifact-checks | verified | JSON Schema, ESLint RuleTester, pitest, GFM table spec; local cell-count repro + GitHub renderer cross-check | |
| 32 | +| testing/quality/harness-reverse-controls | verified | mutation-testing + CI-control sources; field repro (re-fetched all cited URLs, PASS) | |
15 | 33 |
|
16 | | -**Claim 3 — openpyxl `write_only` gives near-constant memory (<10 MB); one save only; lxml is for serialization speed, not the memory saving.** |
17 | | -- Source: openpyxl Optimised Modes — `https://openpyxl.readthedocs.io/en/stable/optimized.html` (via WebSearch). "keeping memory usage under 10Mb"; "A write-only workbook can only be saved once"; "make sure you have lxml installed" for large dumps (speed). |
18 | | -- This **corrects** the raw candidate's "lxml unnecessary" → precise form: unnecessary *for the memory win*, recommended *for large-dump speed*. Confirmed by my server test (write-only worked with lxml absent). → **verified** |
19 | | - |
20 | | -**Claim 4 — QueryPie blocks `BEGIN`, so server-side cursor is impossible there.** |
21 | | -- Environment-specific, no external source. Live repro in gui context: `autocommit=False` + named cursor → `[ENGINE] No permission to execute BEGIN statement`. → **field-tested**. Generalized in the page to "a read-only access-control proxy that blocks transaction control", with QueryPie as the concrete example (not a product-specific page). |
22 | | -- Memory figure 838 MB → 38 MB (300k synthetic rows) is my RNR-3440 measurement (`ru_maxrss`, separate processes). |
| 34 | +Three pages were reconciled from two overlapping PR versions each, keeping the more |
| 35 | +complete/better-sourced body and folding in the other's unique cases: |
| 36 | +- **completion-response-validation** — #12 body (all five `finish_reason` values, |
| 37 | + `tool_calls`/`function_call` carve-out, streaming, Responses API, "reasoning is |
| 38 | + scratch, not deliverable") kept in `llm/` (coherent with #6/#13); folded in #6's |
| 39 | + DeepSeek first-party edge + the field incident. |
| 40 | +- **externally-owned-defaults** — #12 generalized body (any repo-external resource) |
| 41 | + in `integrations/`; folded in #6's alias-removed field incident + the |
| 42 | + gateway-config-vs-live-upstream nuance. |
| 43 | +- **non-interactive-cli-invocation** — #12 body (GNU-nohup extension precision, |
| 44 | + ssh -n stdin-detach vs BatchMode, pre-log DNS/TLS/proxy + `curl -v`) kept; folded |
| 45 | + in #11's DEBIAN_FRONTEND, pager/color TTY case, wrapper-CLI case, field incident. |
23 | 46 |
|
24 | 47 | ## Existing-layer check |
25 | 48 |
|
26 | | -- Pages read: `databases/index.md`, `databases/query-optimization/keyset-pagination.md`, `backend/python/index.md`. |
27 | | -- Overlap: keyset-pagination is the nearest neighbor (both handle large result sets) but a **distinct** topic — pagination splits the read into many bounded queries; this page streams a *single* query's result in chunks. Not a duplicate → new page + **bidirectional `related` link** added to both. |
28 | | -- backend/python has no DB-cursor page; the psycopg2/openpyxl specifics live as concrete examples inside the databases page rather than a separate python page (no duplication). |
29 | | -- Conflicts: none found. |
| 49 | +Cross-PR and against-main duplication was the focus. Findings and resolutions: |
| 50 | + |
| 51 | +- **spec-artifact-checks (#8) ≡ document-conformance-checks (#9)** — same case |
| 52 | + (coverage-vs-validity split, per-check negative controls, GFM pipe parsing, |
| 53 | + ESLint/Semgrep/mutation examples). #9's report predated awareness of #8. → |
| 54 | + **#8 kept canonical; #9's page dropped, `testing/docs-as-spec` category not created.** |
| 55 | +- **completion-response-validation (#6) ≈ llm-response-completeness (#12)** — ~95% |
| 56 | + same case (HTTP 200 ≠ usable output; `length`/blank/reasoning-budget). → |
| 57 | + **merged into one `llm/` page; #12's `integrations/` copy dropped.** |
| 58 | +- **gateway-model-alias-defaults (#6) ≈ externally-owned-defaults (#12)** — ~80%; |
| 59 | + #12 generalizes the model-alias case to any external resource. → |
| 60 | + **kept the general `integrations/` page; #6's LLM-only page dropped.** |
| 61 | +- **non-interactive-cli-invocation** — created by BOTH #11 and #12 (file collision). |
| 62 | + → **single reconciled page.** |
| 63 | +- Distinct (no overlap, all landed): checks-that-cannot-pass, harness-reverse-controls, |
| 64 | + spec-document-gates, editing-a-gated-document, unicode-text-matching, |
| 65 | + command-text-inspected-before-execution, object-key-persistence, context-window-budget, |
| 66 | + host-cgroup-visibility, missing-container-metrics. |
| 67 | +- Reciprocal `related:` links added on existing pages (tests-that-cannot-fail, |
| 68 | + timeouts-and-retries, environment-config, release-gates, background-services, |
| 69 | + portable-shell-scripts, timezone-and-locale, paths-case-and-line-endings, |
| 70 | + acceptance-criteria, resource-limits-and-probes, logs-metrics-signals, |
| 71 | + minimum-case-set). A dropped-page backlink (#6 → gateway-model-alias-defaults on |
| 72 | + environment-config and release-gates) was retargeted to externally-owned-defaults. |
| 73 | +- Invariants verified programmatically: all `related:`/inline `[id]` references |
| 74 | + resolve, every page listed in its domain index, no duplicate ids, no page >120 |
| 75 | + body lines. |
30 | 76 |
|
31 | 77 | ## Routing decision |
32 | 78 |
|
33 | | -- Target: **`databases/query-optimization/streaming-large-result-sets.md`** (new page). |
34 | | -- Category `query-optimization` fits (memory-bounding how a query's result is pulled into the app is query-execution optimization); no new category needed. |
35 | | -- Registered in `databases/index.md` (query-optimization section) and appended to `log.md`. |
| 79 | +- `backend/common/llm/` (new) — LLM-specific server concerns: completion-response-validation, |
| 80 | + context-window-budget. Coherent home shared by #6 and #13. |
| 81 | +- `backend/common/integrations/` (new) — general repo-external-dependency concern: |
| 82 | + externally-owned-defaults. Kept separate from `llm/` because its scope is any |
| 83 | + external resource (bucket/queue/index), not LLM-only. |
| 84 | +- `backend/common/storage/` (new) — object-key-persistence. |
| 85 | +- `qa/document-verification/` (new) — spec-document-gates, editing-a-gated-document. |
| 86 | + Introduced by both #10 and #11; unified into one index section. |
| 87 | +- `testing/quality/` (existing) — checks-that-cannot-pass, spec-artifact-checks, |
| 88 | + harness-reverse-controls (test/check-authoring discipline, distinct from |
| 89 | + qa/document-verification which is release-process gate design). |
| 90 | +- `platforms/{environment,shells,processes}/` (existing) — unicode-text-matching, |
| 91 | + command-text-inspected-before-execution, non-interactive-cli-invocation. |
| 92 | +- `infrastructure/{containers,observability}/` (existing) — host-cgroup-visibility, |
| 93 | + missing-container-metrics. |
| 94 | + |
| 95 | +Source PRs #6–#13 are closed with a disposition comment crediting the author. |
0 commit comments