Skip to content

Latest commit

 

History

History
706 lines (578 loc) · 37.3 KB

File metadata and controls

706 lines (578 loc) · 37.3 KB

Query Doctor Code Audit

Last updated: 2026-06-14

This public audit tracks current engineering and safety risk areas at a level that is useful to contributors without publishing local calibration history. Detailed branch notes, real-batch measurements, private fixture identifiers, generated paths, and per-run investigation logs belong in local exclude-only notes.

Summary

Query Doctor has a deterministic diagnostic core: collection is bounded, Python extracts facts, browser presenters sanitize display data, and LLM output is hidden unless validation accepts it. The latest trust-boundary audit round has implemented guards for shared outbound HTTP egress, inline SQL-like trusted report rejection, marker schema-version binding, browser internal-fingerprint redaction, parent-side subprocess output caps, generated staging artifacts, artifact-route containment, committed fixture provenance, README screenshot provenance, and single-source packaging metadata. The largest remaining public risks are now release-readiness and maintenance risks: product usefulness gaps, review complexity, rendered-Markdown internal coupling, fixture depth, lower-priority redaction bypass variants, regex resource-bound regression drift, and merge-heavy local history that must be rewritten into reviewable semantic commits before public sharing.

Current Strengths

  • Recent queries, Running now, and Known Query ID workflows do not auto-run LLM reports or optimizer jobs.
  • Cloudera Manager and direct Impala collection paths are bounded and read-only for the supported workflow.
  • Production Impala workflows remain Impala-only. Direct Impala collection, normalized engine-fact projections, Trino fixtures, and Spark compact intake must not be combined into a fake multi-engine support claim.
  • The normalized engine-fact projection is not the product engine registry or a replacement for current Impala analyzer facts; existing Impala recommendations still come from the production analyzer, presenter, and report contracts until an explicit migration proves parity.
  • Metadata collection is read-only, allowlisted, bounded, explicit, and redacted.
  • Redaction is layered: explicit metadata keys, host fields, host assignments, URLs, IP addresses, auth headers, and configured secret values are scrubbed before local artifacts, logs, or browser fallback text can reuse them.
  • Browser display sanitization is centralized and covered by safety tests.
  • Web subprocess helpers bound captured child stdout/stderr per stream on the parent side, including real subprocess calls and defensive custom-runner returns; browser failure messages still do not display captured output.
  • Trusted report and optimizer outputs require deterministic validation before browser rendering.
  • Optimizer fallback can produce trusted non-SQL outcomes instead of exposing partial or untrusted drafts.
  • Query Optimizer recommendations prompts keep raw source SQL out of the prompt and rely on Python-owned candidates, SQL-shape digests, and fact digests; rewrite drafts are accepted only after deterministic SQL validation.
  • User-controlled web and optimizer text inputs are byte-bounded before regex-heavy parsing or validation paths.
  • Synthetic demo data and public README screenshots are kept raw-free.
  • Runtime packaging has no third-party dependencies; optional developer and E2E tooling stays outside the default install path.

Open Risk Areas

1. Optimizer usefulness remains narrow

Severity: medium for product value, low for trust.

The optimizer intentionally accepts only Python-owned, strictly validated SQL drafts plus trusted no-rewrite or recommendations-only outcomes. That keeps the workflow safe, but many real expensive queries still need manual guidance because the supported recipe set is narrow.

Keep future optimizer work focused:

  • add recipes only when detection, deterministic draft generation, validation, and regression fixtures are all available;
  • keep no-recipe guidance raw-free and explicit that it is manual review, not a trusted SQL draft;
  • use raw-free funnel and shape audits to choose recipe families, but do not publish local batch identifiers or private result tables.

2. Large modules make safety review expensive

Severity: medium.

Several web, batch, optimizer, and analyzer modules carry broad responsibilities because they grew around safety-sensitive flows. Large files are not a bug by themselves, but they make trust-boundary review harder.

When touching these areas:

  • keep behavior slices small;
  • extract presenters, parsers, validators, command builders, or render helpers only when a real boundary becomes clearer;
  • avoid formatting-only churn mixed with behavior changes;
  • add focused tests around the boundary being changed.

Implemented guard for second-engine architecture drift: Trino/Spark adapter flags, CLI roles, isolated compact web routes, and dev-only script taxonomy are now pinned by the machine-checkable engine capability manifest in query_doctor/engines/capabilities.py. Isolated compact browser routes are owned by query_doctor/web/preview_surfaces.py and tested against that manifest. Keep future Trino/Spark support-surface changes aligned through those registries instead of expanding independent docs, adapter, command-spec, web-router, and audit-script lists.

Implemented guard for Trino preview source-contract drift: accepted Trino preview source_type values, raw policy, required bounds, network-access class, auth-reference policy, source-schema gate, retry policy, fail-closed policy, and promotion gate now live in query_doctor/trino/source_contract_registry.py. The support-gap audit checks that implemented preview source types are registered, that network-capable source entries require safe auth references and bounded retry behavior, and that registry entries do not enable product surfaces, Details/trusted reports, Recent scans, optimizer behavior, SQL execution, raw storage, browser/report output, or metadata identifier output. Future Trino intake surfaces must update this registry and focused tests before support wording or routing changes.

Implemented guard for Trino bounded-reader drift: the production collector contract audit now pins reader status, bounded scope, implementation module, CLI role, and capability surface for existing local lanes, preview readers, local imports, contract-only checks, and aggregate metadata CLI summary. It also rejects broad query-history, Running, broad QueryInfo, statement, or EXPLAIN ANALYZE reader roles/capabilities before their closure gates exist. Future Trino reader additions must update the production collector contract audit, command/capability manifests, focused tests, and support matrix before support wording or product-surface changes.

Implemented guard for Trino query-linked production-review drift: the query-linked fact coverage audit now records the production_review_query_linked_v1 profile over raw-free bounded core fact families, linkage scopes, one-query source granularities, and retained operator/split-detail/telemetry blockers. The broader production closure gate rejects missing or drifted profile/status/requirement tracking before support wording can claim broader Trino production readiness.

Implemented guard for Trino operator/connector/telemetry decision drift: the query-linked fact coverage audit records operator_connector_telemetry_decision_v1, with connector metric signal as the only current bounded-supported decision and operator-level, split-detail, and JMX/OpenMetrics/OpenTelemetry linkage as deliberate unsupported gaps until raw-free source contracts exist. The broader production closure gate rejects missing or drifted decision-profile status, requirements, and counts.

Implemented guard for Trino product-metadata production-review drift: the product metadata collection audit records production_review_metadata_v1 over the allowlist/source lanes, aggregate-only metadata boundary, metadata fact namespace, bounded sources, redaction blocks, explicit metadata SQL policy, product-surface blocks, and retained product-metadata open blocker. The broader production closure gate rejects missing or drifted metadata profile status, requirements, and counts before product metadata support wording can change.

Implemented guard for Trino report/optimizer production-review drift: the report optimizer safety audit records production_review_report_optimizer_v1 over report/guidance families, materialized capabilities, raw-source policy fields, validator sentinel matrix, and blocked product-surface requirements. The broader production closure gate rejects missing or drifted report/optimizer profile status, requirements, and counts before LLM report, Query Optimizer job, generated SQL, SQL execution, or support wording can change.

Implemented guard for Trino shared-deployment production-review drift: the shared deployment boundary audit records production_review_shared_deployment_v1 over review families, deployment-config requirements, product-boundary requirements, capability requirements, release requirements, documentation requirements, and unsupported-surface blocks. The broader production closure gate rejects missing or drifted shared-deployment profile status, requirements, and counts before shared/non-local Trino deployment hardening or support wording can change.

Implemented guard for Trino browser/report production-review drift: the browser/report regression audit records production_review_browser_report_v1 over regression families, test files, materialized route capabilities, raw-output blocks, unsupported-surface blocks, download regressions, and public-claim regressions. The broader production closure gate rejects missing or drifted browser/report profile status, requirements, and counts before browser/report support wording or broad Trino release claims can change.

Implemented guard for Trino bounded production support-claim drift: the support-gap audit now pins bounded_production_claim_pinned plus bounded_production_claim_ready, product-surface summaries use local-production machine labels for retained-list Recent and One Query ID, and the production-closure audit reports the bounded claim ready only when every current raw-free tracking summary is accepted and collector evidence is linked to representative evidence. Broader/shared Trino expansion, Running, query-history crawling, product metadata collection, LLM reports, Query Optimizer jobs, generated SQL, and SQL execution remain separately blocked.

Implemented guard for Trino representative-evidence promotion-review drift: the retained representative evidence audit now requires the production_review_breadth_v1 profile to include raw-free handoff-suite, compact-readiness, product-surface, and support-gap summary kinds, with accepted ok input statuses. The audit also rejects product-surface or support-gap summary boundary drift that would imply broader closure or Trino SQL execution. Future retained evidence promotion work must keep those summary-kind, status, and boundary checks aligned with focused tests, collector-linkage checks, and the support matrix before support wording or release-claim changes.

Implemented guard for cross-engine fact-promotion drift: shared/distributed-SQL-family/source-boundary/support-boundary normalized fact promotion policy now lives in query_doctor/analyzer/engine_fact_promotion_policy.py. The support-gap audit checks policy coverage for Trino-visible non-engine-specific scopes, allowed-engine and scope alignment, raw-free-only policy, disabled product surfaces, and explicit promotion gates. Future shared, distributed, source, or support-boundary fact promotion must update this policy and focused consumer tests before support wording, routing, or product-surface changes.

Implemented first shared helper slice for dev-only readiness/handoff scripts: safe handoff artifact path comparison, output-overlap detection, and ASCII/sorted JSON artifact writing now live in query_doctor/safety/handoff_artifacts.py. Trino and Spark handoff scripts still own their engine-specific parsing, redaction guards, readiness gates, and safe error wording; future script changes should keep moving repeated orchestration helpers behind focused shared utilities instead of copying local path/output logic.

Remaining architecture backlog before broad parallel Trino/Spark feature work:

  • continue moving thick readiness/handoff script orchestration into focused dev-tool helpers when those scripts are next touched.

High-priority Impala architecture backlog:

  • Rendered analysis_facts.md still acts as a load-bearing interchange format for some analyzer, report, web, and optimization-candidate consumers. Label or heading changes can therefore degrade a consumer by parsing less evidence rather than by failing at a typed boundary.
  • Move those consumers toward analysis.json or a typed loader over the same deterministic facts, keeping Markdown as a render-only view.
  • The Recent batch scoring path now prefers typed analysis.json for core scoring components and score reasons, records markdown fallback source and reason in batch summaries, and keeps renderer-to-parser characterization coverage so the fallback remains deliberate. Query and stats optimization candidate scoring now also prefer typed analyzer evidence for impact, planning/opportunity, runtime counter-signal, and metadata-gap inputs when the full analyzer JSON contract is present; their candidate JSON records safe evidence-source and fallback-reason labels for legacy rendered-facts fallback. Report prompt assembly, browser Details/presenters, and any remaining analyzer consumers still need equivalent typed migration or contract guards before the Markdown coupling is closed.
  • Add characterization over representative fixtures before replacing any remaining Markdown parser or score source, so behavior changes are deliberate and reviewable.
  • Keep the full-pipeline leak-canary regression baseline passing while this migration proceeds. It exercises manual-profile intake, analysis, scoring, report prompt assembly, trusted report output, and browser rendering; any new generated sink in that path must be classified before marker checks run.

3. Optimizer prompt-injection guard coverage must stay explicit

Severity: low for current behavior, medium if new prompt slots regress.

The Query Optimizer rewrite prompt must include raw source SQL inside a delimited INPUT SQL block because the model needs the statement to rewrite. Current trust is correctly placed on deterministic output validation: only one read-only SELECT or WITH draft with preserved shape and recipe constraints can become trusted.

The rewrite prompts now explicitly frame INPUT SQL as untrusted data and tell the model to ignore instructions inside SQL comments, literals, identifiers, aliases, table names, and column names. Regression coverage includes a malicious source SQL comment that asks for unsafe SQL; when the model returns that unsafe draft, deterministic validation downgrades the result to a trusted no-rewrite recommendation outcome. Recommendations-only prompts also keep raw source SQL out of the prompt, mark source-derived digests and fact text as untrusted data, and omit instruction-like or unsafe digest values.

Maintenance guard:

  • keep the untrusted-data wording and downgrade tests with any future rewrite prompt changes;
  • add equivalent raw-free prompt-path tests whenever a new digest section or source-derived prompt slot is introduced.

4. Sanitized fixture depth is still the limiting factor

Severity: medium.

The project has useful synthetic and sanitized fixtures, but some analyzer and optimizer failure modes still need more representative raw-free coverage before the product can claim broader support.

Needed fixture work:

  • small golden synthetic/sanitized cases for stats, skew, data movement, metadata-missing, optimizer-candidate, and mixed-signal paths;
  • browser/report regression fixtures for raw-leak prevention;
  • installed-wheel CLI checks for every public console script;
  • explicit unsupported-shape fixtures for optimizer no-draft outcomes.

5. Impala normalized fact projection can drift from production facts

Severity: medium.

query_doctor/analyzer/impala_engine_facts.py projects existing Impala analyzer output into the normalized engine-fact contract for fixture calibration. It is not currently consumed by Impala scoring, Details, reports, optimizer guidance, or browser rendering. That keeps production behavior stable, but it also means the projection can drift from the analyzer facts that actually support user recommendations.

Keep future Impala projection work constrained:

  • expand representative Impala projection fixtures before adding new projected fact families;
  • do not wire product consumers to the projection until parity is proven against existing analyzer facts and representative sanitized batches;
  • keep the guard test that prevents accidental product imports unless a real migration slice updates this audit, docs, and consumer tests.

6. Outbound HTTP clients must keep one shared egress policy

Severity: low for current behavior, high if regressed.

Collection and LLM HTTP clients now share query_doctor.safety.http_egress for default outbound requests. The shared helper validates DNS-resolved destination classes, blocks metadata, link-local, reserved, documentation, multicast, and unspecified targets, and builds a no-redirect opener so a redirect cannot bypass the same target policy. CM, Prometheus, direct Impala daemon endpoints, and LLM clients use the configured-diagnostic policy because real deployments commonly use private or loopback endpoints. Spark History Server compact intake keeps the stricter public-target policy by default and switches to the configured policy only when local/private targets are explicitly opted in. Trino HTTP event archive, HTTP query-detail archive, and one-query pruned coordinator QueryInfo readers also use the configured-diagnostic policy and therefore share the same target validation and no-redirect behavior while remaining private-preview paths.

Response reads are parent-side bounded on these paths: Spark, Prometheus, direct Impala daemon endpoints, CM text/JSON, Trino HTTP archive/query-detail and coordinator QueryInfo reads, and LLM JSON/streaming responses. Guard tests cover unsafe literal destinations, DNS-resolved unsafe destinations, integer-IP loopback aliases, no-redirect handlers, safe default opener wiring, and default CM response caps.

Spark-specific coverage also keeps shared egress target violations fail-closed at the Spark collector boundary instead of downgrading them to optional endpoint warnings, with generic browser-safe wording for DNS failures.

Maintenance guard:

  • do not introduce a new outbound HTTP client that defaults to raw urllib.request.urlopen;
  • keep private-network and loopback destinations as explicit configured diagnostic targets, not arbitrary browser-form egress;
  • keep response byte caps mandatory for every fetched response path;
  • add a focused egress test whenever a new collector or LLM provider adds an HTTP surface.

Do not fix this by blindly blocking all private RFC1918 addresses: real CM, Impala, and Prometheus deployments commonly live on private networks. The policy must distinguish explicit configured diagnostic targets from arbitrary egress initiated through browser forms or optional URL overrides.

7. Report validators need adversarial wording coverage

Severity: high for trusted report semantics, medium for future language drift.

Current report validators include English and Russian overclaim detection and compare claims against deterministic facts, not against LLM wording. Adversarial coverage now rejects indirect unsupported causal wording, soft statistics-maintenance recommendation wording, English stats-maintenance fix/explanation overclaims, flexible row/cardinality estimate direction wording, and a compact EN/RU parity matrix for memory estimate direction, backend data skew, primary bottleneck, CM context-only metrics, and CM event context while paired safe wording remains allowed. Every supported report language has explicit overclaim coverage. Public report-language keys are normalized through the shared report-language registry, and unknown languages fail closed at config and CLI boundaries instead of silently falling back to another language. This closes the material semantic-claim gap where redaction could not help because the problem was an unsupported claim rather than raw operational text. Raw SQL-like text rejection also covers fenced snippets, line/item-level SQL, and inline prose that embeds SQL-like SELECT, WITH, DML/DDL, or metadata SHOW statements at the trust gate, not only browser/download display scrubbing.

No remaining guard work is tracked under this finding as of the current audit baseline. Keep the adversarial report-validator corpus in focused validation whenever report wording, trusted markers, or report safety gates change.

8. Fail-closed trusted-output paths must keep defensive coverage

Severity: low for current web exposure, medium if regressed.

Report finalization validates generated narrative text, appends deterministic facts, validates the final report again, and writes the final artifact only after both checks pass. The web command builders force strict report validation, and trusted marker loaders require the current strict validation mode and hash bindings before browser display. Direct trusted-marker tests reject non-strict report validation modes and non-strict_v2 optimizer validation modes, so a local/manual validation_mode=off artifact cannot become trusted browser output. Report and optimizer trusted markers also bind current marker schema versions before browser display, so artifacts validated by an older weaker marker contract do not silently survive an upgrade.

The remaining risk is coverage drift. Recent scan batch-job defensive fallback coverage now verifies that unexpected exceptions become a generic failed job message, leave the job flow terminal, and do not expose raw subprocess output, paths, SQL, model names, or artifact names in browser job status JSON.

Keep future changes guarded:

  • CLI validation_mode=off is an escape hatch for local/manual use and must not make an artifact trusted for browser display;
  • any new defensive web catch-all handler must have focused tests proving unexpected exceptions return redacted safe fallback messages, do not crash the job flow, and do not expose raw subprocess output, paths, SQL, model names, or artifact names.

9. Redaction adversarial coverage should include free-text host and secret variants

Severity: low for trusted browser/report surfaces, medium for local artifacts and logs.

The primary browser and trusted-report safety guarantee is raw-free fact construction from Python-owned facts. Redaction is still an important defense-in-depth layer for local profile/metadata artifacts, terminal logs, warning text, and defensive browser fallbacks.

Current redaction covers explicit host fields, host assignments, URLs, bare FQDNs, host-like single-label names, IP addresses, auth headers, cookies, common and less-common secret assignment names, configured secret values, metadata key parts, and stable host aliases. A compact adversarial corpus now covers local/log/browser fallback text for these cases, including credential, passphrase, private-key, and auth assignments. The same coverage also pins false-positive boundaries: SQL-style table identifiers, pool names, synthetic source-version labels, and safe local filenames must not be misclassified as hosts by local/log redaction. Browser display redaction separately hides model-name variants including gpt-4, gpt-4o, gpt_4_1, and gpt-lst.

The remaining gaps are lower-priority bypass variants:

  • IDN or Unicode host spellings are not normalized into ASCII before redaction;
  • non-dotted hostnames without host-like tokens remain intentionally keyword gated unless they appear in explicit host fields or assignments;
  • alternate IP encodings beyond dotted decimal and standard IPv6 remain low-priority bypass classes unless a future raw-ish local artifact surface depends on free-text redaction alone.

Required guard work:

  • keep the adversarial corpus updated whenever browser fallback, local log, or report display redaction accepts a new raw-ish text source;
  • keep paired false-positive tests for curated SQL/table/pool/file identifiers whenever host heuristics are expanded;
  • add Unicode/IDN-like host text coverage before depending on free-text redaction for such inputs;
  • keep intentionally keyword-gated host matching documented as a redaction limitation, and keep raw-free fact construction as the trusted-report and browser primary safety layer.

10. Regex resource-bound guards should stay explicit

Severity: low for current behavior, medium if regressed.

Current user-controlled inputs are capped before expensive processing: web POST bodies, pasted optimizer SQL, optimizer recommendations, Spark compact JSON, Trino fixture packages, fetched profile/metrics/event responses, and metadata outputs all have byte limits. The safety and validator regexes mostly use bounded repeats and simple scans rather than nested unbounded quantifiers.

The remaining risk is regression drift. A future validator or sanitizer regex could add catastrophic backtracking without violating the existing functional tests. Current focused guard coverage includes bounded pathological inputs for shared log redaction, browser error sanitization, trusted report SQL-like text validation, and optimizer recommendation validation; those tests assert safe redaction or safe rejection without depending on exact wall-clock timings.

Required guard work:

  • keep bounded pathological-input coverage for shared redaction/browser/report/optimizer validation paths whenever those regex surfaces expand, focused on completion and safe rejection rather than exact wall-clock performance;
  • keep input byte caps before regex-heavy parsing, validation, or sanitizer paths;
  • treat unclosed fenced-SQL extraction and similar lazy DOTALL scans as bounded by input caps, and add a focused guard before expanding those patterns.

Implemented guard: web subprocess child stdout/stderr capture is bounded as a parent-side safety net, even when child commands are expected to self-cap and write user-facing artifacts to files.

11. Pre-push history must be cleaned into reviewable semantic commits

Severity: high for public review readiness, low for content safety when public release scans pass.

The local integration workflow intentionally creates frequent merge commits as agents finish and merge worktrees into local main. That is useful for local coordination, but it is not a reviewable public branch shape. A release or review branch must not publish a large merge-heavy local history as-is.

Implemented guard: scripts/check_release_history_shape.py checks a proposed release or review branch against the configured public base ref, requires the base to be an ancestor of the release head, rejects excessive commit counts, rejects merge commits, and rejects WIP/fixup/draft commit subjects. The public release gate runs that guard when PUBLIC_RELEASE=1 is set, so a merge-heavy local integration branch fails before public handoff.

Before any requested push or public-sharing branch:

  • scan the full diff and history for secrets, raw production data, generated artifacts, local paths, hostnames, and real configuration;
  • rewrite local integration history into reviewable semantic commits that group behavior, tests, and docs by actual change;
  • squash WIP, fixup, repeated docs-audit, and mechanical local-merge commits into the semantic commits they support;
  • do not push local main directly to remote main; push a prepared review branch and keep protected-main behavior intact.

12. Packaging metadata should have one version source

Severity: low.

Runtime packaging is intentionally small: no third-party runtime dependencies, query_doctor* package discovery only, and package data limited to web static assets. The legacy setup.py shim reads console scripts from pyproject.toml, and now reads the package version from pyproject.toml as well, leaving the project table as the canonical version source.

Implemented guard:

  • tests assert legacy setup.py metadata uses the pyproject.toml version and does not carry a literal setup-version keyword;
  • console-script parity tests keep the legacy shim aligned with pyproject.toml;
  • keep the legacy editable-install shim only while it is useful for old tooling;
  • keep package metadata and installed-console-script tests in the release gate.

No remaining guard work is tracked under this finding as of the current audit baseline.

13. Demo fixture and screenshot provenance should be machine-checkable

Severity: low for current demo data, medium if the fixture corpus grows without guardrails.

The public demo pack is generated from synthetic case definitions and is tested for trusted artifacts, raw-free browser rendering, and local-only behavior. The public-release preflight and staged public-safety checks catch many private-data markers in changed files, the tracked tree, and history. Committed text fixtures under tests/fixtures/ are now scanned by a dedicated pytest guard using the same public-release scanner, so new fixture files must stay within the synthetic/example/redacted provenance boundary. README screenshots are now listed in docs/assets/readme-screenshot-provenance.json, which pins the synthetic demo-pack version, capture command, documented route, viewport, alt text, README usage, and actual PNG dimensions.

Implemented guard:

  • fixture public-safety provenance is checked for committed text fixtures;
  • README screenshot provenance is machine-readable and checked against the English/Russian READMEs, docs/demo-mode.md, and the PNG headers.

No remaining guard work is tracked under this finding as of the current audit baseline. Keep the manifest and test updated whenever README screenshots, demo-pack version, capture routes, or viewport dimensions change.

14. Case lifecycle guards must cover transient staging names

Severity: low if regressed.

Generated case artifacts are intentionally ignored, raw-free where public surfaces consume them, and kept out of git. Cleanup uses temporary staging and refresh directories during artifact replacement, selected-case refresh, and CM-timeseries refresh work.

Implemented guard:

  • .gitignore explicitly ignores .replace-*, .query-refresh-*, and .cm-timeseries-refresh-* staging directories at any tree depth, including non-default local corpus roots;
  • staged public-safety checks reject those staging directories even if someone force-adds them;
  • regression tests pin both git check-ignore behavior and staged-path rejection;
  • existing lifecycle tests keep cleanup paths in finally blocks and preserve existing final artifacts when replacement or refresh analysis fails.

No remaining guard work is tracked under this finding as of the current audit baseline. Keep any new generated staging directory family covered by both ignore rules and staged public-safety checks.

15. Browser and trusted-artifact boundaries must stay conservative

Severity: high if regressed.

Trusted browser/report surfaces must not expose raw SQL, raw profile text, raw metadata, local paths, case_dir, process logs, secrets, model names, runtime internals, or raw artifact filenames. The isolated owner-only selected-case source surface is the narrow raw-SQL browser exception and must keep the stricter rules in docs/safety-contract.md. Shared or non-local owner-raw deployments must resolve per-request viewer identity from an authenticated front door such as viewer_identity_header behind a trusted proxy; keytab or collection owner sets must not authorize raw reveal by themselves. The isolated owner-raw source surface must keep its global kill switch and emit only reason-coded raw-free audit lines.

Any change that renders analyzer facts, report content, optimizer content, collector errors, or generated artifacts must include focused tests proving the new trusted/report surface remains raw-free, or that an isolated owner-raw surface stays policy-gated and non-persisted.

Recent audit coverage found the current route and artifact path boundary sound: case identifiers are revalidated before path use, server-written case indexes choose batch cases, artifact readers use fixed filenames plus trusted markers, and resolved paths stay contained under the expected corpus.

Implemented guard:

  • route-level tests reject encoded and slash path-shaped IDs for both batch case report exports and Specific Query report exports;
  • route-level tests reject symlinked report files outside the case directory for both batch case and Specific Query report exports, even when marker-like metadata is present;
  • fixed-download filenames, markdown Content-Disposition, and browser-display model/runtime-name variants are pinned by focused tests.

No remaining guard work is tracked under this finding as of the current audit baseline. Any new browser artifact route must add equivalent traversal, symlink, fixed-download-name, and raw-free rendering coverage before it becomes trusted output.

16. Trino contract naming debt must stay closed during preview

Severity: medium.

The Trino preview path now uses neutral no_* limitation facts instead of borrowing Impala fact names, and the current query_list_* aggregate bucket namespace is guarded by a snapshot test. That closes the immediate audit issue where unsupported Trino coverage could squat another engine's namespace.

The previous bare Trino metric IDs have been renamed behind the trino_* prefix. planning_time_ms remains unprefixed because it is an explicit distributed-SQL-family fact with allowed_engines={"impala", "trino"}, not a Trino engine-specific fact. Any new Trino engine-specific fact must use a trino_*, query_detail_*, query_list_*, or neutral no_* prefix.

Before broader Trino support, trusted-report promotion, or any Details surface beyond the raw-free materialized local Trino case view:

  • keep Trino-only facts behind trino_*/source-shape prefixes unless a separate contract change promotes a specific fact to a family/shared scope with explicit allowed_engines;
  • keep no_* as the only limitation-fact idiom for unsupported Trino coverage;
  • keep query_list_* bucket growth behind an explicit contract/test update, or replace the bucket fan-out with a structured aggregate fact in a dedicated contract migration;
  • keep strict one-query readiness gates rejecting both query_list_* aggregate facts and local trino_metadata_* metadata-summary facts before any boundary can count as one-query Trino diagnosis evidence;
  • keep retained handoff-suite manifests confined to safe relative *.json artifact references and keep duplicate boundary/diagnosis references rejected so suite-width gates cannot count one artifact more than once;
  • keep local compact diagnosis rejecting local metadata-summary boundaries so aggregate metadata-coverage facts cannot be rendered as diagnosis;
  • run scripts/audit_trino_product_surface_boundary.py on retained compact boundary/diagnosis artifacts or a retained handoff-suite manifest before any product-surface promotion decision so live_known_query_diagnosis=one_query_pruned_query_info_local_production, live_recent_scan=retained_query_list_local_production, the local production support claim, and the compact-preview plus Recent/One Query ID web registry limits stay pinned; manifest mode must require diagnosis artifacts for every entry, and this audit must reject metadata-summary boundaries as aggregate coverage evidence, not product-surface diagnosis artifacts; keep its static source-import guard enabled so Details, trusted report, optimizer, and other unsupported product modules cannot import Trino preview diagnosis code outside the isolated compact-diagnosis route/page or explicit Recent/One Query ID beta paths;
  • run scripts/audit_trino_support_gap_matrix.py before broader support-surface decisions so registered Trino fact-family coverage, source-type registry coverage, neutral no_* gaps, blocked product adapter flags, and trino_support_gap_matrix_audit_v1 evidence stay aligned with the support-gap matrix;
  • keep the engine adapter and console-script registry language precise: Trino is registered only for bounded raw-free preview surfaces and the local One Query ID beta surface, including metadata source-contract checking, bounded local metadata CLI summary building, and bounded local metadata summary import, not unsupported production Recent surfaces, trusted report source reads, optimizer jobs, product metadata collection, user SQL execution, query-history crawling, or standalone production Query ID support.

17. Public documentation must not become a local run journal

Severity: medium.

Committed docs are public docs. The repository should keep durable contracts, runbooks, and sanitized summaries, while local continuation notes and raw validation evidence stay ignored.

Before committing documentation changes, run:

python3 scripts/check_staged_public_safety.py --changed
python3 scripts/audit_public_docs.py
python3 scripts/check_active_docs.py
python3 scripts/check_markdown_links.py
git diff --check

Update Rule

Update this audit when a public risk changes materially, a trust boundary moves, or a previously open risk is resolved by code and tests. Keep detailed local evidence in local exclude-only notes; commit only the durable public conclusion.