|
1 | | -# Knowledge flush — 1 insight |
| 1 | +# Knowledge flush — 4 insight(s) |
2 | 2 |
|
3 | | -Source: RNR-3440 (사내 잠재매물 주간 추출 스크립트 메모리 피크 저감). Candidate: |
4 | | -"QueryPie 프록시 경유로 대용량 결과를 스트리밍할 때 server-side named cursor 대신 |
5 | | -일반 커서 + `fetchmany` + openpyxl `write_only`." |
| 3 | +4 queued candidates → **3 pages** (2 new platforms pages, 1 new qa page in a new |
| 4 | +category). Two qa candidates were merged into a single page because they are the same |
| 5 | +case seen twice; nothing was dropped. |
| 6 | + |
| 7 | +Cross-Check: primary-source verification of every directive (man pages, official tool |
| 8 | +docs) plus local reproduction of the two mechanism claims against this repo's own hook |
| 9 | +source. No independent adversarial agent review was run for this flush — the |
| 10 | +verification is documentary and reproducible, not a second opinion. |
6 | 11 |
|
7 | 12 | ## Verified best-practice |
8 | 13 |
|
9 | | -**Claim 1 — psycopg2 server-side (named) cursor requires a transaction; fails under autocommit.** |
10 | | -- Source: psycopg2 usage docs — `https://github.com/psycopg/psycopg2/blob/master/doc/src/usage.rst` (via Context7). Quote: "Named cursors are typically created 'WITHOUT HOLD', meaning they exist only within the current transaction. Attempting to fetch from them after a commit or in autocommit mode raises an exception." |
11 | | -- Matches my live repro (`can't use a named cursor outside of transactions`). → **verified** |
| 14 | +### 1. A non-interactive flag is not a closed stdin → `confidence: verified` |
| 15 | +**Claim.** When invoking a prompt-capable CLI unattended, pass the tool's |
| 16 | +non-interactive switch *and* redirect stdin from `/dev/null`; a hang with zero output |
| 17 | +is evidence about the client until the far side's log shows the request arrived. |
| 18 | + |
| 19 | +| Source | What it establishes | |
| 20 | +|---|---| |
| 21 | +| [nohup(1)](https://man7.org/linux/man-pages/man1/nohup.1.html) | "If standard input is a terminal, redirect it from an unreadable file" — detaching includes taking terminal stdin away | |
| 22 | +| [ssh(1)](https://man.openbsd.org/ssh.1) | `-n` "Redirects stdin from /dev/null (actually, prevents reading from stdin). This must be used when ssh is run in the background" | |
| 23 | +| [ssh_config(5)](https://man.openbsd.org/ssh_config.5) | `BatchMode=yes` disables "password prompts and host key confirmation requests", "useful in scripts and other batch jobs where no user is present" | |
| 24 | +| [timeout(1)](https://man7.org/linux/man-pages/man1/timeout.1.html) | "Start COMMAND, and kill it if still running after DURATION"; exit 124 on timeout — a distinguishable "blocked" signal | |
| 25 | + |
| 26 | +Two established tools implement *both* halves (close stdin **and** a fail-fast |
| 27 | +interaction switch) as separate mechanisms, which is exactly the candidate's directive. |
| 28 | +The client/server split step is field-derived (a gateway access log showed zero |
| 29 | +requests from the host during two hangs; `</dev/null` fixed it immediately) and is |
| 30 | +labelled as such in the page's Field context rather than presented as documented. |
| 31 | + |
| 32 | +### 2. Command-inspecting gates read pre-expansion text → `confidence: verified` |
| 33 | +**Claim.** Write values literally in an argument a gate inspects; create files a gate |
| 34 | +reads in a *prior* command. |
| 35 | + |
| 36 | +- [Claude Code hooks docs](https://code.claude.com/docs/en/hooks) — `PreToolUse` runs |
| 37 | + "Before a tool call executes. Can block it"; the hook's stdin JSON carries |
| 38 | + `tool_input.command`, the unexecuted command string. Exit 2 blocks and "stderr text |
| 39 | + is fed back to Claude as an error message." |
| 40 | +- [POSIX shell 2.6](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html) |
| 41 | + — word expansion is performed by the shell as it processes the line, so an external |
| 42 | + reader of the command text sees none of it applied. |
| 43 | + |
| 44 | +**Reproduced locally** against this repo's own extraction pattern |
| 45 | +(``--body-file[= ]+[^ '"`]+``, `hooks/pre-flush-pr-gate.sh:56`), 2026-07-30: |
| 46 | + |
| 47 | +| Command text | Extracted | Refusal | |
| 48 | +|---|---|---| |
| 49 | +| `--body-file "$REPO/…"` | *(empty)* | "no `--body-file` found" | |
| 50 | +| `--body-file $REPO/…` | literal `$REPO/…` | "body file does not exist yet" | |
| 51 | +| `--body-file "/abs/…"` | *(empty)* | "no `--body-file` found" | |
| 52 | +| `--body-file /abs/…` | `/abs/…` | passes | |
| 53 | +| `--body-file=/abs/…` | `/abs/…` | passes | |
| 54 | + |
| 55 | +This **corrects the candidate**, which attributed the failure to quoting alone. The run |
| 56 | +shows two distinct failure modes with two different error messages — quotes defeat |
| 57 | +*extraction*, an unexpanded variable defeats *resolution* — and that a quoted |
| 58 | +**literal** path fails too. The page states both; the sharper form is what makes the |
| 59 | +error message diagnostic. |
| 60 | + |
| 61 | +### 3+4. Editing a document that text gates check → `confidence: field-tested` |
| 62 | +**Claim.** Inventory a file's gate anchors before editing; describe an upstream |
| 63 | +contract as an observed shape rather than with definition verbs; scope a check outside |
| 64 | +the region that quotes it; record scoped conditions, not global counts. |
12 | 65 |
|
13 | | -**Claim 2 — a client-side (default) cursor pulls the whole result set to the client on execute; `fetchmany` only caps the Python-list explosion.** |
14 | | -- Source: psycopg2 cursor/usage docs + FAQ (named-cursor advantage = "data is fetched in chunks … minimal client memory"). By contrast the default cursor buffers the full result in libpq. → **verified** |
| 66 | +The *hazards* are documented; the *directives* are field-derived, hence `field-tested` |
| 67 | +rather than `verified`: |
15 | 68 |
|
16 | | -**Claim 3 — openpyxl `write_only` gives near-constant memory (<10 MB); one save only; lxml is for serialization speed, not the memory saving.** |
17 | | -- Source: openpyxl Optimised Modes — `https://openpyxl.readthedocs.io/en/stable/optimized.html` (via WebSearch). "keeping memory usage under 10Mb"; "A write-only workbook can only be saved once"; "make sure you have lxml installed" for large dumps (speed). |
18 | | -- This **corrects** the raw candidate's "lxml unnecessary" → precise form: unnecessary *for the memory win*, recommended *for large-dump speed*. Confirmed by my server test (write-only worked with lxml absent). → **verified** |
| 69 | +| Source | What it establishes | |
| 70 | +|---|---| |
| 71 | +| [Vale `existence`](https://docs.vale.sh/checks/existence) | The check "looks for the 'existence' of particular tokens" as a word-bounded non-capturing group — a lexical gate matches patterns, not intent. This is why a purely descriptive sentence trips a "do not redefine" gate | |
| 72 | +| [Vale scopes](https://docs.vale.sh/topics/scopes.md) | Scopes restrict where a rule applies; "Any scope prefaced with `~` is negated" and scopes chain — keeping checks off regions such as code examples is first-class | |
| 73 | +| [markdownlint MD013](https://github.com/DavidAnson/markdownlint/blob/main/doc/md013.md) | Rules expose `code_blocks`/`tables`/`headings` booleans (default `true`) so quoted code can be excluded from a prose rule | |
| 74 | +| [pgrep(1)](https://man7.org/linux/man-pages/man1/pgrep.1.html) | "The running pgrep, pkill, or pidwait process will never report itself as a match" — self-exclusion is designed in because self-matching is the expected failure | |
19 | 75 |
|
20 | | -**Claim 4 — QueryPie blocks `BEGIN`, so server-side cursor is impossible there.** |
21 | | -- Environment-specific, no external source. Live repro in gui context: `autocommit=False` + named cursor → `[ENGINE] No permission to execute BEGIN statement`. → **field-tested**. Generalized in the page to "a read-only access-control proxy that blocks transaction control", with QueryPie as the concrete example (not a product-specific page). |
22 | | -- Memory figure 838 MB → 38 MB (300k synthetic rows) is my RNR-3440 measurement (`ru_maxrss`, separate processes). |
| 76 | +Field evidence retained in the page: a vague-word audit that matched its own quoted |
| 77 | +pattern (1 global hit → 3 after quoting the fix, 0 under an `awk`-scoped run; the same |
| 78 | +self-reference observed 3× across two documents), and a vocabulary gate |
| 79 | +(`(arena|pool)…(재정의|정의한다|규정한다)`) that failed a true descriptive sentence |
| 80 | +until it was rewritten as an observation, restoring the suite 60 → 61. |
23 | 81 |
|
24 | 82 | ## Existing-layer check |
25 | 83 |
|
26 | | -- Pages read: `databases/index.md`, `databases/query-optimization/keyset-pagination.md`, `backend/python/index.md`. |
27 | | -- Overlap: keyset-pagination is the nearest neighbor (both handle large result sets) but a **distinct** topic — pagination splits the read into many bounded queries; this page streams a *single* query's result in chunks. Not a duplicate → new page + **bidirectional `related` link** added to both. |
28 | | -- backend/python has no DB-cursor page; the psycopg2/openpyxl specifics live as concrete examples inside the databases page rather than a separate python page (no duplication). |
29 | | -- Conflicts: none found. |
| 84 | +**Read in full:** `INDEX.md`, `AGENTS.md`, `templates/page.md`, |
| 85 | +`wiki/platforms/index.md`, `wiki/qa/index.md`, |
| 86 | +`wiki/platforms/processes/background-services.md`, |
| 87 | +`wiki/platforms/shells/portable-shell-scripts.md`, `log.md`, plus the two in-flight |
| 88 | +pages `wiki/qa/document-verification/spec-document-gates.md` (PR #10) and |
| 89 | +`wiki/testing/docs-as-spec/document-conformance-checks.md` (PR #9), fetched from the |
| 90 | +fork. **Repo-wide greps** for `grep|self-referen|lexical|doc-as-spec`, |
| 91 | +`stdin|/dev/null|non-interactive|tty`, and `PreToolUse|pre-commit|hook`. |
| 92 | +**Queue dedup:** all 4 candidate hashes absent from `.processed.jsonl`. |
| 93 | + |
| 94 | +**Open-PR dedup was decisive here.** PRs #6–#10 are open and unmerged, and |
| 95 | +#7/#8/#9/#10 all sit in the doc-gate theme, so `main` alone understates coverage. |
| 96 | +Overlap verdicts: |
| 97 | + |
| 98 | +| Overlap candidate | Verdict | |
| 99 | +|---|---| |
| 100 | +| `qa-document-verification-spec-document-gates` (**PR #10, open**) | **Complement, not duplicate.** It owns the *gate author's* side (four axes, controls, fail-closed anchors). The new page owns the *document author's* side: what to do when your prose must survive gates that already exist, including gates whose pattern your text quotes. Neither self-reference scoping nor anchor inventory appears in #10; its nearest row ("Examples section satisfies the check") is a different cause | |
| 101 | +| `testing-docs-as-spec-document-conformance-checks` (**PR #9, open**) | No overlap. Positive/negative controls and GFM pipe parsing for checks under construction; says nothing about editing an already-gated document | |
| 102 | +| `testing-quality-tests-that-cannot-fail` | Adjacent, linked. Owns proving a *test* can fail. The new qa page routes gate-construction questions to it rather than restating them | |
| 103 | +| `qa-process-regression-scope` | Adjacent, linked. Supplies the "re-run the full set, compare the baseline" principle the new page applies to gate suites | |
| 104 | +| `platforms-processes-background-services` | Closest neighbour to insight 1 and **already covers `nohup … & disown`** for *lifetime*. It does not cover stdin as a blocking input or the client/server split. Kept separate (its "load when" is persistence), linked **both ways** | |
| 105 | +| `platforms-shells-portable-shell-scripts` | Closest neighbour to insight 2, owns quoting *for the shell* — and its rule is "quote every expansion". The new page narrows one argument read by an external gate, so an unqualified reader could see a contradiction; the new page's edge-case table states explicitly that it "narrows one argument, it does not license unquoted expansions elsewhere". Linked both ways | |
| 106 | +| `platforms-environment-path-resolution` | Linked only (literal-vs-resolved paths in non-interactive contexts) | |
| 107 | + |
| 108 | +**Conflicts flagged:** one *soft* directive tension (quote-everything vs. write-this-one- |
| 109 | +argument-literally), resolved inside the new page rather than by editing the old one. No |
| 110 | +factual contradiction found. |
| 111 | + |
| 112 | +**Reciprocal `related:` links added** to `background-services.md` and |
| 113 | +`portable-shell-scripts.md` (frontmatter only; `last_verified` deliberately not bumped, |
| 114 | +since nothing on those pages was re-verified). |
| 115 | + |
| 116 | +### Merge conflict to expect (please read before merging) |
| 117 | +This branch and **PR #10** both introduce the `## document-verification` section in |
| 118 | +`wiki/qa/index.md` at the same insertion point, each with its own page row. Whichever |
| 119 | +merges second will conflict there. **Resolution: keep one heading and both rows.** To |
| 120 | +keep the conflict to that single hunk, this branch deliberately does **not** touch |
| 121 | +`INDEX.md` — PR #10's qa route line ("automated verification of document deliverables |
| 122 | +(spec/RFC gates)") already covers this new page. If #10 is rejected instead, the qa |
| 123 | +route line in `wiki/qa/index.md` should gain a document-verification clause in a |
| 124 | +follow-up. No `related:` id in this branch points at #10's page, so nothing here breaks |
| 125 | +under either outcome; once both are merged the two pages are worth cross-linking. |
30 | 126 |
|
31 | 127 | ## Routing decision |
32 | 128 |
|
33 | | -- Target: **`databases/query-optimization/streaming-large-result-sets.md`** (new page). |
34 | | -- Category `query-optimization` fits (memory-bounding how a query's result is pulled into the app is query-execution optimization); no new category needed. |
35 | | -- Registered in `databases/index.md` (query-optimization section) and appended to `log.md`. |
| 129 | +| Insight | Target | New category? | |
| 130 | +|---|---|---| |
| 131 | +| 1 — non-interactive CLI hang | `platforms/processes/non-interactive-cli-invocation.md` (new page) | No — `processes` already owns process/session lifetime | |
| 132 | +| 2 — gate reads command text | `platforms/shells/command-text-inspected-before-execution.md` (new page) | No — `shells`, justified below | |
| 133 | +| 3 + 4 — self-reference & lexical gates | `qa/document-verification/editing-a-gated-document.md` (new page) | **Yes** — `document-verification`, the same category PR #10 introduces | |
| 134 | + |
| 135 | +**Insight 1 → platforms, not debugging.** Half the insight is diagnostic, but the |
| 136 | +artifact the reader changes is the invocation command, and `AGENTS.md` routes by owned |
| 137 | +artifact. `debugging-methodology-reproduce-first` is linked for the isolation half. |
| 138 | +Merging into `background-services` was rejected: that page's case is *persistence*, and |
| 139 | +adding a blocking-stdin case would drift its "load when". |
| 140 | + |
| 141 | +**Insight 2 → `shells`, not a new category and not `tools`.** The mechanism is expansion |
| 142 | +timing — *when* the shell rewrites the line relative to other readers of it — which is |
| 143 | +shell semantics, so `shells` holds two coherent pages (portability; expansion timing vs. |
| 144 | +external inspectors). `tools` is BSD-vs-GNU userland differences, which this is not. A |
| 145 | +dedicated category (e.g. `policy-gates`) was considered and rejected as a one-page |
| 146 | +category with no second member in sight. |
| 147 | + |
| 148 | +**Insights 3+4 → one page, not two.** Both are the document author's side of the |
| 149 | +doc-gate loop: 3 is "the gate matched the pattern I quoted", 4 is "the gate matched the |
| 150 | +verb I used / the anchor I moved". `AGENTS.md` requires one case per page, and the shared |
| 151 | +case is "writing prose inside a document that lexical gates run over" — the directives |
| 152 | +interleave (scope the check, phrase as observation, record a scoped condition, re-run the |
| 153 | +suite), so splitting would have produced two pages that each need the other. **New |
| 154 | +category justified:** the existing qa categories are `process` (human release process), |
| 155 | +`environments`, `bug-reports`, `exploratory` — none covers automated checks over a |
| 156 | +written deliverable. PR #10 reached the same conclusion independently, which is |
| 157 | +corroboration rather than duplication. |
| 158 | + |
| 159 | +### Invariants checked on this branch |
| 160 | +Body lines 83 / 82 / 93 (limit 120) · all four required sections present on each page · |
| 161 | +every `related:` id and inline `[page-id]` resolves (13 checks, 0 misses) · each new page |
| 162 | +listed in its domain index with a multi-use-case "load when" · every index relative link |
| 163 | +resolves · no banned vague qualifiers · `log.md` entry appended. |
0 commit comments