feat(cli): appmap sanitize — equality-preserving value tokenization

[kgilpin]

What

A new appmap sanitize command that makes committed AppMaps structurally incapable of carrying a secret, plus this repo adopting it end to end.

1. appmap sanitize <files..> (4b2cd63d2) — replaces every captured runtime value string with a per-file, equality-preserving token (<v1>, <v2>, …) assigned by first appearance. Equality within one AppMap still reads as data-flow; zero content characters are retained. Details:
   • Positional, not content-sniffing: only the format's captured-data fields are touched (parameter/receiver/return/message values, exception messages, header values with names kept, concrete paths with route templates kept, client URLs). Code-object names, classes, labels are schema and survive.
   • Kind-annotated tokens (<uuid:v3>, <int:v7>) — recognition informs the label, never exemption (session tokens are UUIDs).
   • SQL parameterized, fail-closed: literals stripped via normalizeSQL keeping statement shape; a statement it can't reliably strip (it returns the original — fail-open for its display use) is downgraded to a whole-statement token.
   • Allowlist: built-in enumerables (true/false/null…) plus curated vocabularies via --allow/--allow-file, exact whole-value match only.
   • Idempotent; atomic writes; malformed files skipped, not fatal. 9-test unit spec.
2. All 20 committed gold-trace baselines sanitized retroactively (234284a8c) — verified against the gold-traces engine's own pipeline: bless digests identical before/after for all 20 (values are outside the digest by design), so no false re-bless ever. 17% smaller as a side effect.
3. Review workflow → released review-action@v1 (b3484ec51), dropping the merged skills-ref override; v1's per-matrix sticky comments fix this repo's two-package matrix.
4. AppMap skills vendored as a submodule (491e32dcb) at .claude/appland-skills with committed relative symlinks, replacing machine-local links.

Why

Gold traces are committed, cloneable, and diff-invisible (*.appmap.json binary) — the one place a captured secret would live forever. Truncation caps leakage but retains whole short secrets (PINs, passwords ≤ cap); digests of low-entropy values are crackable offline. Tokenization retains zero bits while preserving the equality structure reviews actually use.

Review

A behavioral review of this branch ran against the sanitized baselines (asymmetric inputs: sanitized head vs trimmed base): compare clean 20/20, one 🟡 follow-up — unrecognized format fields currently pass through silently; a spec-driven audit + warn-on-unknown-keys mode is the fast follow.

Follow-ups (not this PR)

• Gold-traces skill switch trim → sanitize at bless (drafted, gated on this shipping in a release).
• Gold-trace manifest entry for the sanitize command itself.
• Format-version field audit (the 🟡 above).

Examples

[github-actions]

AppMap Behavioral Review — appmap sanitize value-redaction command

Revisions: b81492f4 vs main (33d2543e) · Date: 2026-07-11 ·
Commits: 4b2cd63d add appmap sanitize — equality-preserving value tokenization · 77ce035e sanitize exits non-zero when any file in the batch is skipped · 0cc5b94f sanitize eventUpdates too · 1157abc9 provenance markers for sanitize and trim · 5a414d98 normalize AppMaps as the first step of sanitize · e280c560 the masking walk sanitizes metadata itself · f76cd708 normalize HTTP routes in sanitize instead of tokenizing (plus 6b74f59b rename/PR-feedback, 45d73bf9 eslint style, and out-of-scope ci(appmap-review), chore(gold-traces), and .claude submodule commits)

Summary

Severity	Findings	Action required
🔴 High	0	—
🟡 Medium	0	—
🟢 Low	2	Optional: one documented residual to stay aware of, one small test-coverage gap.

Not merge-blocking. This is a clean, purely additive change: a new sanitize
command and its redaction library, plus provenance bookkeeping on trim. None of the
14 pre-existing behavioral baselines drifted — the only change the compare sees is the
two new traces the feature adds for its own coverage. The single most useful
follow-up is a unit test for the new trim "skip already-trimmed" branch (#2).

Findings

1 · 🟢 LOW — A template-less HTTP path keeps purely-alphabetic segments verbatim

File: packages/cli/src/lib/sanitizeAppMap.ts:237 · Context: normalizeSegment · Evidence: unit + gold test sanitizeAppMap sanitizes a composite AppMap end-to-end asserts /users/123/friends/alice → /users/:int/friends/alice

When an HTTP server request has no route template (normalized_path_info), its
concrete path_info is normalized to a route template rather than tokenized (so the
route stays stable across recordings, since it feeds the compare digest). A segment
that looks like an id becomes a :kind placeholder, but a purely-alphabetic segment
is kept as-is — so a username in /users/alice survives, indistinguishable from a
route word without the application's route table. This is explicitly documented in the
source and asserted by the test; it is a deliberate, accepted limitation, not a
regression.

Risk: A template-less path could retain a low-entropy identifier (e.g. a
username). In this repository the exposure is nil — the CLI gold traces contain no HTTP
server endpoints (OpenAPI is empty on both revisions), so the branch is never reached
here. Fix: No change recommended (the tradeoff is documented). If a future traced
surface serves real HTTP requests, prefer emitting normalized_path_info route
templates so the concrete path is fully tokenized instead.

2 · 🟢 LOW — New trim skip/provenance branch has no test

File: packages/cli/src/cmds/trim/trim.ts:120 · Context: trim handler (metadata.trimmed marker) · Evidence: source diff only — no gold trace and no unit test exercises this branch

trim now writes a metadata.trimmed provenance marker and skips a file already
trimmed by the same CLI version to the same cap (warning that a larger cap cannot
restore truncated values). The gold trace guards only trimAppMap — the value
truncation transform — which is why the trim baseline did not drift. The new
handler-level skip/marker logic is not covered by any test (trim.spec.ts was not
touched in this change).

Risk: Low — a bug in the skip guard would at worst re-trim (idempotent) or wrongly
skip a file, not leak data. Fix: Add a unit test that runs trim twice and asserts
the second run is skipped, and that a smaller cap on a re-run records
Math.min(prior, new):

// tests/unit/trim.spec.ts
it('skips a file already trimmed by this version to this cap', () => { /* run trim twice, assert "already trimmed, skipping" + unchanged output */ });

Checks performed

Check	Result	Note
Behavioral compare	✅ clean	2 new traces (the feature's own coverage); 0 of 14 existing traces changed or removed. Captured values, object ids, and timing are excluded from the digest, so every non-change is real.
Changes outside the PR's scope (Step 5)	✅ none	No trace in any untouched subsystem drifted; the change is purely additive (two new files + one command registration) plus provenance-only edits to trim.
Missing guards (Step 6)	✅	The security.sanitization boundary is fail-closed by construction and is exercised by both new traces, including the negative batch path.
Test/recording coverage (Step 2)	❌ 1 gap → #2	Redaction boundary is well covered; the new trim skip/marker branch is not.
SQL (Step 4b)	✅ clean	No SQL diff. sanitizeSql parameterizes literals (excluded from the digest) and masks the whole statement when the obfuscator leaves residue (fail-closed); the SQL-bearing traces did not drift.
HTTP (Step 4c)	✅ clean	No HTTP endpoints in the traced surface (OpenAPI empty on both sides). Path/URL normalization is covered by the unit suite and the composite trace, not by a live server request.
Intended changes verified	✅	sanitize command + sanitizeAppMap library added; trim/sanitize provenance markers added; the two new gold traces establish the sanitize baseline. No pre-existing behavior moved.

Review detail — features, coverage, labels, drift

Feature List

1. appmap sanitize command — replaces captured AppMap values with per-file, equality-preserving tokens (<v1>, <v2>, …); writes each output atomically; supports --allow/--allow-file verbatim allowlists and --output-dir.
2. sanitizeAppMap redaction library — the security boundary (security.sanitization): tokenizes parameters, return values, receivers, headers, and exception/test-failure messages; normalizes HTTP paths and client URLs to lossy route templates; parameterizes SQL and masks the whole statement when it can't be reliably obfuscated; strips credentials from metadata.git.repository.
3. Normalize-then-mask pipeline — sanitize runs buildAppMap().normalize() first, so masking walks the canonical form (eventUpdates merged, ids re-indexed, unreturned calls balanced).
4. Fail-closed batch semantics — exit code 0 means every file was sanitized; a batch with any unprocessable file still sanitizes the good files but exits non-zero so a skipped, possibly secret-bearing file never looks like success.
5. Provenance markers — metadata.sanitized (version + allow-values) and metadata.trimmed (version + max_length); both commands skip a file already processed by the same version/options and warn that masking/truncation is one-way.
6. Command registration — SanitizeCommand wired into the yargs CLI (cli.ts).

Only features 1–5 have runtime coverage, via the two new traces below; nothing here produced drift in a pre-existing trace.

Coverage Matrix

Feature	Covered by	Status
Redaction boundary (composite walk)	sanitizeAppMap sanitizes a composite AppMap end-to-end (gold) + rich sanitize.spec.ts unit suite	✅
Batch fail-closed (negative branch)	sanitize command handler fails when any file is skipped, while still sanitizing the rest (gold)	✅
SQL parameterize / fail-closed	composite gold + parameterizes SQL literals… / fails closed on SQL… unit tests	✅
HTTP path & client-URL normalization	composite gold + unit tests (no live HTTP-server trace exists in this surface)	✅ / —
Idempotency & token-reservation on re-runs	is idempotent, never assigns a token a marked document already contains unit tests	✅
trim skip-already-trimmed / marker branch	no test — the trim gold trace covers trimAppMap (the transform) only	❌ uncovered → #2

The redaction boundary is thoroughly guarded, including its fail-closed and negative
paths. The one gap is the new trim handler branch (#2); it is best closed by a unit
test rather than a gold trace (a warn-and-continue path does not warrant a committed
recording).

Suggested Labels

None required. The redaction boundary already carries @label security.sanitization
(ValueMasker.mask, sanitizeSql, normalizePath, normalizeUrl, sanitizeAppMap),
and no unlabeled function changed in the compare. The sanitize handler itself is a
CLI entrypoint, for which no application-code label in the taxonomy fits.

Behavioral Drift

Every one of the 14 pre-existing gold AppMaps changed on disk in this revision — they
were re-recorded and then blessed (sanitized and trimmed), gaining metadata.sanitized
and metadata.trimmed markers. Yet the behavioral digest — which ignores captured
values, object ids, and timing — reports zero changed traces. That is the expected
and reassuring outcome: the sanitize feature is additive (two new source files plus one
command registration), and the trim edit changed only provenance bookkeeping, not the
truncation transform the gold trace fingerprints. The only drift is the two new
traces (sanitizeAppMap … and sanitize command handler …), which are the feature's
own coverage and have no baseline to compare against. No pre-existing subsystem moved;
the SQL and OpenAPI surfaces are identical across revisions; the scanner found nothing
on either side.

---

Agent usage

As reported by Claude Code; the cost is computed by the agent from billed API usage.

Run	Model	Cost	Tokens read	Tokens written	Turns	Time
update	claude-haiku-4-5-20251001, claude-opus-4-8[1m]	$2.61	3M (98% from cache)	28.2k	54	9m 33s
review	claude-haiku-4-5-20251001, claude-opus-4-8[1m]	$2.04	1.7M (95% from cache)	27.9k	30	7m 6s
total		$4.65	4.8M (97% from cache)	56k	84	16m 39s

[Copilot]

Pull request overview

This PR adds a new appmap sanitize capability to the CLI to replace captured runtime value strings with equality-preserving tokens, and updates committed gold-trace AppMaps/workflows to use sanitized baselines and the released review action.

Changes:

• Added sanitizeAppMap + ValueTokenizer library to tokenize captured value slots (with SQL normalization + fail-closed behavior).
• Introduced a new appmap sanitize CLI command (with allowlist support and atomic writes) and wired it into the CLI entrypoint.
• Updated gold-trace baselines/manifests and the AppMap review workflow (review-action @v1, in-tree appmap on PATH), plus added an AppMap skills submodule.

Reviewed changes

Copilot reviewed 30 out of 34 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
packages/sequence-diagram/gold_traces/manifest.yaml	Updates gold-trace recording notes for sequence-diagram package.
packages/sequence-diagram/gold_traces/baseline/appmaps/jest/Specification/includes_all_relevant_actors.appmap.json	Sanitizes committed sequence-diagram baseline AppMap values.
packages/sequence-diagram/gold_traces/baseline/appmaps/jest/Sequence_diagram/HTTP_server_request/is_recorded.appmap.json	Sanitizes committed sequence-diagram HTTP baseline AppMap values.
packages/cli/tests/unit/cmds/sanitize.spec.ts	Adds unit coverage for sanitization behavior (tokenization, allowlist, SQL, idempotency, etc.).
packages/cli/src/lib/sanitizeAppMap.ts	Implements AppMap value tokenization + SQL normalization/fail-closed logic.
packages/cli/src/cmds/sanitize/sanitize.ts	Adds appmap sanitize command with allowlist + output-dir support and atomic writes.
packages/cli/src/cli.ts	Registers the new sanitize command in the CLI.
packages/cli/gold_traces/manifest.yaml	Adds gold-trace entries covering sanitize behavior (including SQL fail-closed branch).
packages/cli/gold_traces/baseline/appmaps/jest/trim_command/trimAppMap/truncates_values_across_every_captured_slot_while_preserving_structure.appmap.json	Sanitizes committed CLI trim baseline AppMap values.
packages/cli/gold_traces/baseline/appmaps/jest/tree_--filter/returns_only_sql_events_when_filter=sql.appmap.json	Sanitizes committed CLI query/tree baseline AppMap values.
packages/cli/gold_traces/baseline/appmaps/jest/stats_subcommand/analyzes_a_directory.appmap.json	Sanitizes committed CLI stats baseline AppMap values.
packages/cli/gold_traces/baseline/appmaps/jest/sequence_diagram_command/JSON_format/is_valid.appmap.json	Sanitizes committed CLI sequence-diagram baseline AppMap values.
packages/cli/gold_traces/baseline/appmaps/jest/sanitizeAppMap/shares_the_token_namespace_across_all_value_positions.appmap.json	Adds new sanitize gold-trace baseline (shared token namespace across slots).
packages/cli/gold_traces/baseline/appmaps/jest/sanitizeAppMap/parameterizes_SQL_literals_but_keeps_the_statement_shape.appmap.json	Adds new sanitize gold-trace baseline (SQL normalized branch).
packages/cli/gold_traces/baseline/appmaps/jest/sanitizeAppMap/fails_closed_on_SQL_it_cannot_reliably_parameterize.appmap.json	Adds new sanitize gold-trace baseline (SQL fail-closed branch).
packages/cli/gold_traces/baseline/appmaps/jest/prune_subcommand/correctly_reduces_the_size_of_an_appmap_from_a_base64url-encoded_filter.appmap.json	Sanitizes committed CLI prune baseline AppMap values.
packages/cli/gold_traces/baseline/appmaps/jest/index_command/fingerprinting_a_directory/fingerprints_AppMaps_and_writes_an_index.appmap.json	Sanitizes committed CLI index baseline AppMap values.
packages/cli/gold_traces/baseline/appmaps/jest/importAppmap/imports_an_end-to-end_recording_into_all_tables.appmap.json	Sanitizes committed CLI import baseline AppMap values.
packages/cli/gold_traces/baseline/appmaps/jest/functionHotspots/--class_filters_by_defined_class.appmap.json	Sanitizes committed CLI hotspots baseline AppMap values.
packages/cli/gold_traces/baseline/appmaps/jest/findCalls/--label_filters_via_the_labels_table.appmap.json	Sanitizes committed CLI findCalls baseline AppMap values.
packages/cli/gold_traces/baseline/appmaps/jest/endpoints/sorts_by_the_requested_key.appmap.json	Sanitizes committed CLI endpoints baseline AppMap values.
.gitmodules	Adds .claude/appland-skills as a git submodule.
.github/workflows/appmap-review.yml	Updates workflow to use review-action @v1 and expose in-tree appmap on PATH.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[dividedmind]

My biggest worry are the event updates; we need to also sanitize them, or (preferably) normalize the appmap before sanitization.

[github-actions]

AppMap Behavioral Review — sanitize baselines (@appland/sequence-diagram)

Revisions: bf932868 vs main (33d2543e) · Date: 2026-07-09 ·
Commits: 234284a8 chore(gold-traces): sanitize committed baselines — the only
commit touching this package (the appmap sanitize feature itself and its other
sanitize/gold-trace commits all live in packages/cli, out of scope here).

Summary

Severity	Findings	Action required
🔴 High	0	—
🟡 Medium	0	—
🟢 Low	0	—

Nothing blocks merge. The packages/sequence-diagram source is untouched, and the
behavioral compare of its six gold traces is clean: every recorded interaction ran
identically on both revisions. The baseline files changed on disk, but only because
their captured parameter values were replaced with sanitize tokens — a change the
comparison ignores by design.

Findings

No findings. No behavior changed between the two revisions.

Checks performed

Check	Result	Note
Behavioral compare	✅ clean	0 new / 0 removed / 0 changed traces across all 6 gold traces; elapsed time and captured values are excluded from the fingerprint, so a clean result means behavior really is identical.
Changes outside the PR's scope (Step 5)	✅ none	The baseline files were rewritten (values tokenized) but no interaction changed — mechanical propagation of sanitization, blast radius confirmed (see drift note).
Missing guards (Step 6)	✅	No security-labeled function changed; nothing to guard.
Test/recording coverage (Step 2)	✅	All 6 manifest features are exercised by a committed gold trace.
SQL (Step 4b)	✅ clean	No query added, removed, or changed; no new tables.
HTTP (Step 4c)	✅ clean	OpenAPI diff empty — no breaking or non-breaking request/response changes.
Scanner findings	✅	0 findings on both revisions.
Intended changes verified	✅	The committed baselines were sanitized (parameter values → <v1>/<v2> tokens); structure, ordering, and event count are unchanged.

Review detail — features, coverage, labels, drift

Feature List

No application-code changes in packages/sequence-diagram — src/ is byte-identical
between main and bf932868. The single in-package change is a maintenance edit:

1. Sanitized committed baselines — the six gold-trace recordings were rewritten so
   each captured parameter value becomes an equality-preserving token (<v1>, <v2>);
   identical values map to the same token. No recorded behavior changes.

Coverage Matrix

Feature	Covered by	Status
specification (actor/package selection)	Specification › includes all relevant actors	✅
labels (labels carried onto a diagram action)	Sequence diagram › labels › are reported on a labeled function action	✅
protocols-http (HTTP request as an action)	Sequence diagram › HTTP server request › is recorded	✅
protocols-sql (SQL query as an action)	Sequence diagram › SQL query › is recorded	✅
diff (two diagrams → PlantUML)	Sequence diagram diff › PlantUML › user found vs not found › UML matches expectation	✅
formatter-text (diff → Text output)	Sequence diagram diff › PlantUML › user found vs not found › Text matches expectation	✅

No coverage gaps — no new behavior was introduced in this package that would need a
new trace.

Suggested Labels

None. No function changed in the compare, so there is nothing newly worth labeling.

Behavioral Drift

The six baseline .appmap.json files differ from main at the file level, but the
comparison reports zero behavioral change. The only content that differs is the
parameters[*].value strings, now replaced with sanitize tokens (e.g. a node-3
parameter block → <v1>, an HTTP-callee block → <v2>), with equal originals mapping
to the same token. Event count (14 in the HTTP trace, unchanged), call ordering, the
class map, and every non-value field are identical. Because the change-detection
fingerprint deliberately excludes captured parameter and return values, this
value-only rewrite is invisible to it — exactly the outcome the sanitize feature is
meant to produce. This is acceptable mechanical propagation; the blast radius is the
committed baselines only, and it is confirmed intended.

---

Agent usage

As reported by Claude Code; the cost is computed by the agent from billed API usage.

Run	Model	Cost	Tokens read	Tokens written	Turns	Time
update	claude-haiku-4-5-20251001, claude-opus-4-8[1m]	$2.09	2.2M (97% from cache)	25.1k	46	9m 3s
review	claude-haiku-4-5-20251001, claude-opus-4-8[1m]	$1.26	1.2M (96% from cache)	16.7k	31	4m 33s
total		$3.35	3.4M (97% from cache)	41.8k	77	13m 36s