Skip to content

feat(triage): add Failure Triage Pack for test-time structured diagnostics #443

Description

@mk3008

Summary

Introduce Failure Triage Pack (Phase 2) to accelerate root-cause analysis for failures during test execution.

This feature is not "replay logging".
It produces structured diagnostics for fast triage and next-action decision.

  • Phase 1 (already done): Contract Guard = pre-execution prevention
  • Phase 2 (this issue): Failure Triage Pack = post-failure diagnosis
  • Phase 3 (later): Runtime Trace Assist = continuous observability

Scope (MVP)

In Scope

  • Target: test execution only (e.g. ztd test path)
  • Structured error normalization and artifact output
  • JSON artifact as source of truth, Markdown generated from JSON
  • Security-first defaults:
  • params masked
  • SQL body excluded
  • include only sqlHead (first N chars) + references (specId, paths)

Out of Scope (for MVP)

  • dev runtime aggregation (ztd run, local repository runtime path)
  • production/staging observability integration
  • full SQL body logging by default

Fixed stage enum (8)

  1. CATALOG_RESOLVE
  2. SQL_PARSE
  3. SQL_REWRITE
  4. CONTRACT_VALIDATE
  5. MAPPING_BUILD
  6. REPOSITORY_EXEC
  7. TEST_TRANSFORM
  8. TEST_RUN

Domain categories (MVP set)

  • CATALOG
  • PARSE_UNSUPPORTED
  • PARSE_SYNTAX
  • REWRITE_BUILD
  • MAPPING
  • TYPE_CONTRACT
  • STRUCTURE
  • REPOSITORY_RUNTIME
  • TEST_FIXTURE_DRIFT
  • TEST_CASE_INVALID
  • TEST_TRANSFORM_FAILURE

Error code policy

  • Format: ZTD-<DOMAIN>-NNN (e.g. ZTD-PARSE-002)
  • Once published, meaning must not change
  • Deprecated codes are not removed (deprecated: true), no hard deletion

nextActionId dictionary (MVP)

  • OPEN_SQL
  • OPEN_SPEC
  • RUN_LINT
  • RUN_FIXTURE_REGEN
  • CHECK_MAPPING
  • UPGRADE_RAWSQL
  • REPORT_BUG

Artifact output

  • artifacts/failure/<runId>.json
  • artifacts/failure/<runId>.md

runId format:
YYYYMMDDTHHMMSS.mmmZ_<env>_<pid>_<rand4>

Example:
20260215T034556.789Z_test_12345_a1b2

env enum (fixed): test | dev | ci
(MVP uses test first)


Minimum JSON shape (artifact)

{
"runId": "20260215T034556.789Z_test_12345_a1b2",
"occurredAt": "2026-02-15T03:45:56.789Z",
"specId": "user.findById",
"stage": "SQL_PARSE",
"domain": "PARSE_SYNTAX",
"errorCode": "ZTD-PARSE-002",
"isTestSide": true,
"causeSummary": "Syntax error near \"fromm\"",
"nextAction": {
"id": "OPEN_SQL",
"message": "Open SQL file and fix token near line 12."
},
"refs": {
"specPath": "src/catalog/specs/user.findById.ts",
"sqlPath": "src/sql/user/find_by_id.sql",
"sqlHead": "select * fromm user where user_id = :user_id"
},
"security": {
"paramsMasked": true,
"sqlBodyIncluded": false
}
}

Acceptance Criteria (DoD)

  1. On test-time failure, triage artifact JSON is always generated with:
    • specId, stage, domain, errorCode, causeSummary, nextAction.id
  2. Markdown report is generated from JSON (no duplicated logic)
  3. security block is always present and accurate
  4. At least these test-side failures are classified:
    • fixture drift
    • test case invalid
    • test transform failure
  5. Reviewer can identify first action within ~1 minute without rerun
  6. Existing Phase 1 guard behavior remains unchanged

Implementation Notes

• Error production responsibility: each package emits structured error hints
• Aggregation responsibility: ztd-cli normalizes and writes artifact
• Keep CLI thin; core value is structured error design + message quality

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions