Skip to content

feat(transactionlog): B.1c — system-transaction-log-writer (15/15 ACs, 100%)#420

Merged
remyluslosius merged 2 commits into
mainfrom
feat/slice-b-b1c-transaction-log
May 29, 2026
Merged

feat(transactionlog): B.1c — system-transaction-log-writer (15/15 ACs, 100%)#420
remyluslosius merged 2 commits into
mainfrom
feat/slice-b-b1c-transaction-log

Conversation

@remyluslosius
Copy link
Copy Markdown
Contributor

@remyluslosius remyluslosius commented May 29, 2026

Summary

Slice B.1c — system-transaction-log-writer implementation. Complete. All 15 spec ACs covered. Closes the Slice B.1 trunk (B.1a scheduler + B.1b kensa-executor + B.1c transaction log writer).

What landed

Component Files Purpose
Spec (draft → approved) app/specs/system/transaction-log-writer.spec.yaml 15 ACs
Migration 0011 0011_host_compliance_schedule.sql Same as B.1a's; safe duplicate
Migration 0012 0012_transaction_log.sql host_rule_state + transactions tables
Audit events 2 new codes: finding.persisted, writer.apply.failed Codegen'd
Package internal/transactionlog/ — types, doc, writer (~440 LOC) The write-on-change writer
Tests 16 sub-tests (2 source-inspection + 14 integration) 100% AC coverage

ACs satisfied (all 15)

AC Mechanism Test
AC-01 Single DB transaction per Apply, regardless of N pg_stat commit delta ≤ 10 after 50-rule Apply
AC-02 First scan: N first_seen rows row count checks
AC-03 Identical rescan: 0 new transactions, check_count++ DB state assertions
AC-04 One pass→fail flip: exactly 1 state_changed row DB state + change_kind check
AC-05 Same scan_id replay = no-op (idempotent) row count preserved
AC-06 FK violation rolls back the whole batch post-error: 0 rows persist
AC-07 DELETE hosts with extant transactions fails FK ON DELETE RESTRICT
AC-08 Non-JSON-object evidence rejected before INSERT 4-case table
AC-09 finding.persisted count = transactions row count emission counter
AC-10 1000-rule Apply ≤ 2s wall-clock timing budget
AC-11 50 concurrent Applys against distinct hosts — no deadlock -race goroutine fan-out
AC-12 No scan_baselines references anywhere in app/ filesystem walk
AC-13 No raw-SQL string-concat in package AST + regex inspection
AC-14 Oversize evidence rejected before INSERT + audit size cap + emission check
AC-15 DB error during Apply emits writer.apply.failed classified reason in detail

Architectural choices locked

  • Atomicity: validation phase runs BEFORE BEGIN. Oversize-evidence rejection is genuinely zero-INSERT (no rollback needed).
  • Pre-commit pending audit: finding.persisted fires only AFTER tx.Commit succeeds. Audit log reflects persisted state, not attempted.
  • Evidence schema check: minimal "must be JSON object" gate today; full KensaEvidence schema validation slots into validateResult once the OpenAPI components.schemas.KensaEvidence shape lands.
  • No scan_baselines: Python-era baseline table explicitly dropped. The prior transactions row IS the baseline. AC-12 source-inspection enforces project-wide.

Local validation

  • ✅ go build ./internal/transactionlog/ — clean
  • ✅ go test -race ./internal/transactionlog/ — 16 sub-tests pass against real Postgres + migrations 0001–0012
  • ✅ specter coverage — system-transaction-log-writer 15/15 = 100%

Slice B.1 trunk — DONE

PR Topic Status Coverage
#418 B.1a scheduler ready for review 15/15 (100%)
#419 B.1b kensa-executor ready for review 16/16 (100%)
this B.1c transaction-log-writer ready for review 15/15 (100%)

Total Slice B.1: 46 ACs across 3 specs, all at 100%.

Migration ordering note

Migration 0011_host_compliance_schedule.sql appears in PR #418, PR #419, and this PR. All three are byte-identical. When any of the three PRs merges first, the others' 0011 is a no-op for goose (already-applied). Migration 0012_transaction_log.sql is unique to this PR.

Relationship to other PRs

Next steps

With Slice B.1 trunk complete:

  • B.2 — liveness loop + drift detector (per boundary doc § 5.2)
  • B.3 — event bus + alert router
  • B.4 — fleet rollup queries

Each will follow the same per-component PR pattern.

This was referenced May 29, 2026
Merged
Merged
@remyluslosius remyluslosius enabled auto-merge (squash) May 29, 2026 05:31
@remyluslosius remyluslosius force-pushed the feat/slice-b-b1c-transaction-log branch 3 times, most recently from b9c185a to 125c1b4 Compare May 29, 2026 12:50
@remyluslosius
feat(transactionlog): B.1c — system-transaction-log-writer (15/15 ACs…
7cfc728 
…, 100%)

Closes Slice B.1 trunk. Compliance write-on-change persistence for the
Kensa-executor pipeline, complete with all 15 acceptance criteria.

Spec
  Promoted system-transaction-log-writer from draft → approved.
  15 ACs identical to PR #415's draft.

Migration 0011_host_compliance_schedule.sql
  Copy of B.1a's migration. Identical content; goose treats duplicate
  identical migrations as no-ops when both B.1a (#418) and this PR merge.

Migration 0012_transaction_log.sql
  - host_rule_state: ONE row per (host, rule). Current state, UPSERTed
    every Apply. Status CHECK constraint enforces the closed enum
    (pass/fail/skipped/error).
  - transactions: append-only state-change log. UNIQUE(scan_id, rule_id)
    enforces idempotency at the schema level (spec C-04).
  - Both tables FK to hosts(id) ON DELETE RESTRICT — historical
    findings outlive their host references (spec C-06).
  - Indexes: by (host_id, status) for current-fleet queries; by
    (host_id, rule_id, occurred_at DESC) for point-in-time temporal
    queries; by scan_id for idempotency check.

Audit events
  Added two new codes to events.yaml:
    finding.persisted    one per transactions row (spec AC-09)
    writer.apply.failed  per Apply-rollback (spec AC-15)
  Codegen produces audit.FindingPersisted and audit.WriterApplyFailed
  constants; events.gen.go grew from 96 → 98 events total.

internal/transactionlog package
  types.go   ApplyBatch, Result, Status / ChangeKind / FailureReason
             enums, sentinel errors, MaxEvidenceBytes (256 KiB cap).
  writer.go  Writer.Apply: single-tx-per-call. Steps:
               1. Validate every result (status, evidence size + shape).
                  Spec AC-08 / AC-14 reject BEFORE any INSERT — atomic.
               2. Idempotency: if any transactions row exists for the
                  scan_id, no-op (spec AC-05).
               3. BEGIN tx.
               4. Per result: read prior host_rule_state, decide
                  change_kind (first_seen / state_changed /
                  severity_changed / none), INSERT transactions only on
                  change, UPSERT host_rule_state with COALESCE-style
                  last_changed_at preservation.
               5. COMMIT.
               6. emit finding.persisted per state-change AFTER commit
                  (audit reflects what persisted, not what attempted).
             On any error: tx.Rollback + emit writer.apply.failed
             with classified reason (FK / deadlock / oversize / sqlc).

source_test.go (AC-12, AC-13)
  AC-12: walks every .go and .sql file under app/ asserting no
         scan_baselines / ScanBaseline references — the Python-era
         baselines table is explicitly dropped.
  AC-13: AST-parses every internal/transactionlog .go file asserting
         no database/sql import and no .Exec/.Query/.QueryRow whose
         SQL arg uses fmt.Sprintf or string concatenation.

writer_test.go (AC-01 through AC-11, AC-14, AC-15)
  16 sub-tests covering the writer behavior end-to-end against real
  Postgres:
    AC-01  pg_stat_database.xact_commit delta < 10 after 50-rule Apply
    AC-02  N first_seen rows on first scan
    AC-03  identical rescan = 0 new transactions, check_count++
    AC-04  one flip pass→fail = exactly 1 state_changed row
    AC-05  same scan_id replay = no-op
    AC-06  FK violation rolls back the whole batch (zero rows persist)
    AC-07  DELETE hosts with extant transactions fails (ON DELETE RESTRICT)
    AC-08  non-JSON-object evidence rejected (table-driven over 4 cases)
    AC-09  finding.persisted emission count = transactions row count
    AC-10  1000-rule Apply ≤ 2 seconds wall-clock
    AC-11  50 concurrent Applys against distinct hosts complete
    AC-14  oversize evidence rejected BEFORE INSERT; writer.apply.failed
           audit emitted with reason=evidence_oversize
    AC-15  FK violation emits writer.apply.failed with reason=fk_violation
           and detail.rule_count_attempted populated

Local validation
  go build ./internal/transactionlog/: clean
  go vet ./internal/transactionlog/: clean
  go test -race ./internal/transactionlog/ (unit + integration with
    real Postgres + migrations 0001-0012): 16 sub-tests pass
  specter coverage: system-transaction-log-writer 15/15 = 100%

Architectural choices worth flagging
  - Atomicity: validation phase runs BEFORE BEGIN, so oversize-evidence
    rejection is genuinely zero-INSERT (no rollback needed).
  - Pre-commit pending audit emissions: scan.completed / finding.persisted
    fire only AFTER tx.Commit succeeds, so the audit log truly reflects
    persisted state.
  - Evidence schema check: minimal "must be JSON object" gate today;
    full KensaEvidence-schema validation slots into validateResult when
    the OpenAPI components.schemas.KensaEvidence shape lands.

Slice B.1 trunk status
  B.1a scheduler             PR #418 — 15/15 ACs, ready for review
  B.1b kensa-executor        PR #419 — 16/16 ACs, ready for review
  B.1c transaction-log-writer this PR — 15/15 ACs

Total Slice B.1: 46 ACs covered across 3 specs. Ready to move on to
B.2 (liveness loop + drift detector) once these merge.
@remyluslosius
fix(transactionlog): make lint clean — remove dead helper + empty bra…
646f268 
…nch + gofmt
@remyluslosius remyluslosius force-pushed the feat/slice-b-b1c-transaction-log branch from 125c1b4 to 646f268 Compare May 29, 2026 12:58
@remyluslosius remyluslosius merged commit 78c9b84 into main May 29, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

size/XL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@remyluslosius