Skip to content

feat(calm-suite): improve calm-studio visualization — parity with calm-hub + threat/governance rendering#2731

Open
gjs-opsflo wants to merge 27 commits into
finos:mainfrom
gjs-opsflo:feat/calm-studio-viz-revamp
Open

feat(calm-suite): improve calm-studio visualization — parity with calm-hub + threat/governance rendering#2731
gjs-opsflo wants to merge 27 commits into
finos:mainfrom
gjs-opsflo:feat/calm-studio-viz-revamp

Conversation

@gjs-opsflo

@gjs-opsflo gjs-opsflo commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Part of #2686 (Part 1 of two; remaining parity items — bidirectional connects, position-persistence wiring, faceted grouping ELK integration, rich hover tooltip — queued for follow-up and called out in the Out of scope section below).

Use case

Architects authoring CALM documents in calm-studio currently can't visualize the governance richness that the CALM 1.2 schema already supports — threats, controls, mitigations, decorators. This PR brings calm-studio's rendering to parity with calm-hub-ui and adds editor-specific affordances that surface decorator-driven governance at the canvas level.

As a concrete demonstration, this PR ships a canonical fixture derived from a real-world loan approval solution architecture (CALM 1.2, 79 nodes, 70 relationships, 58 threats expressed as decorators[], 294 control references). The same document that renders cleanly in calm-hub now renders cleanly in calm-studio with composed-of nesting captured by parent-child containment (no redundant diamond arrows), severity-tinted borders in threat-overlay mode, and a detail drawer that surfaces controls + threats + decorators per node.

What's in this PR

Foundation — framework-agnostic pure modules

calm-suite/calm-studio/packages/calm-core/src/viz/:

  • badges/BadgeAPI.ts — adapter registry, indexes adapter output per node/edge ID with dedup + stable order.
  • badges/adapters/decoratorsAdapter.ts — reads canonical CALM 1.2 decorators[], infers severity from data.severity, legacy risk-level, or decorator type. Malformed entries logged + skipped.
  • badges/adapters/controlsAdapter.ts — counts node controls{}, emits count badge with severity scaled by total mitigations.
  • overlay/severityResolver.ts — aggregates highest severity across badges per node.
  • grouping/dimensions.ts + grouping/groupingEngine.ts — facet extractors (container / nodeType / aiDomain / owner / customKey) + virtual-group construction. Reserved for ELK virtual-grouping integration in follow-up.
  • positions/positionStore.ts — framework-agnostic localStorage wrapper for per-architecture node positions with quota-exceeded fallback.

All modules are pure TypeScript with zero Svelte runes — designed to relocate to a future @finos/calm-viz-core shared package (proposed in RFC #2690) as a rename + re-export rather than a rewrite.

Tests added: 36 unit tests in calm-core + 8 integration tests in studio app. Suites now report 118 passed | 1 todo (calm-core) and 440 passed (studio app).

Svelte 5 viz layer

calm-suite/calm-studio/apps/studio/src/lib/viz/:

  • StoresoverlayStore.svelte.ts, groupingStore.svelte.ts.
  • badges/BadgeChip.svelte, BadgeCluster.svelte.
  • nodes/NodeFrame.svelte — wrapper applying severity-driven gradient border tint + top-right badge cluster slot. All 11 existing node renderers route through it without changing their underlying SVGs or styling.
  • overlay/OverlayToggle.svelte — toolbar button toggling default ⇄ threat mode; disabled when arch has no threats.
  • overlay/ThreatPanel.svelte — right-side panel grouping threats by severity bucket; shown when overlay mode is threat AND no node selected.
  • drawer/DetailDrawer.svelte + four sections (ControlsSection, ThreatsSection, DecoratorsSection, ComposedOfSection) — selection-driven side panel.
  • integration/decorateFlowNodes.ts — bridges pure modules → Svelte Flow node data (badges, severity).

Canvas wiring

  • apps/studio/src/lib/stores/projection.ts — suppresses redundant composed-of and deployed-in edges when the target is already nested under the source via Svelte Flow's parentId (matches calm-hub-ui's relationshipParser.ts:170-172 pattern). Cross-hierarchy edges still render.
  • apps/studio/src/lib/canvas/CalmCanvas.svelte — adds <MiniMap> (bottom-right) with severity-tinted node dots.
  • apps/studio/src/routes/+page.svelte — calls decorateFromArch after every projection (4 sites), wires OverlayToggle into the canvas toolbar, mounts DetailDrawer (selection-driven) / ThreatPanel (overlay-driven) under a mutual-exclusion rule on the right anchor, and re-decorates nodes when overlay mode flips via a reactive effect.

Canonical fixture

apps/studio/static/fixtures/loan_approval_solution_arch_2026.canonical.calm.json — committed canonical-shape doc derived from a real reference architecture. Threats live in per-node decorators[] (canonical CALM 1.2 form), not legacy metadata.threat-model.*.

Walkthrough

apps/studio/docs/spike-walkthrough.md — feature tour, demo instructions, REPL verification, and out-of-scope follow-up list.

Out of scope / follow-up

  • Bidirectional connects edges (cosmetic).
  • GroupingDropdown + ELK virtual-grouping integration (pure-module side is in place; ELK integration is the missing piece).
  • Position-persist wiring through CalmCanvas drag commits (positionStore module is in place).
  • Perf-smoke test on synthetic 300-node graph.
  • Screenshots / GIF for this PR body (to capture manually before merge).
  • Shared semantics layer — tracked in RFC RFC: shared CALM semantics layer to align calm-hub-ui and calm-studio #2690.

Non-goals

Verification

# from repo root
npm run build --workspace calm-suite/calm-studio/packages/calm-core      # ESM/CJS/DTS clean
npm test --workspace calm-suite/calm-studio/packages/calm-core            # 118 + 1 todo
npm test --workspace @calmstudio/studio                                   # 440 passed

To exercise interactively:

npm run dev --workspace=@calmstudio/studio
# then open the canonical fixture and walk through the affordances described in
# apps/studio/docs/spike-walkthrough.md

Note on commit history

24 atomic commits, all DCO-signed. The branch was developed on a single feature branch (no merge commits); upstream main has moved a few days ahead since the branch base — happy to rebase if the merge bot can't fast-forward.

@rocketstack-matt rocketstack-matt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes — this breaks the CALM Studio E2E suite: 12 of 14 specs fail on this branch and all pass on main. The failures span all four spec files, so the app wedges on load rather than failing a single feature. Root cause is almost certainly the self-referential overlay $effect flagged inline.

Two non-code points:

  • Resolves #2686 overclaims — position-persistence wiring, rich hover tooltip, bidirectional connects edges, and faceted grouping are all listed as follow-up, so this reads better as Part of #2686 (merging with Resolves auto-closes the issue with half its parity items undone).
  • The unwired pure modules (positionStore, groupingEngine, dimensions, groupingStore) are dead code in this PR, but the issue scopes this as a spike and the body lists them as follow-up — noting, not blocking.

Remaining inline comments are nits.

Comment on lines +120 to +126
$effect(() => {
overlay.mode; // dependency
const model = getModel();
if (nodes.length > 0) {
nodes = decorateFromArch(nodes, model, { overlayMode: overlay.mode });
}
});

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This effect reads nodes (.length and as the decorateFromArch arg) and reassigns nodes with a fresh array each run. Because nodes is $state.raw, that reassignment re-triggers this same effect → effect_update_depth_exceeded, wedging reactivity on any document that has nodes. That's why the e2e suite fails here (12/14, across all four spec files) but passes on main. Scope the effect to overlay.mode only — the four projection sites already decorate:

Suggested change
$effect(() => {
overlay.mode; // dependency
const model = getModel();
if (nodes.length > 0) {
nodes = decorateFromArch(nodes, model, { overlayMode: overlay.mode });
}
});
$effect(() => {
const mode = overlay.mode; // the only dependency
untrack(() => {
if (nodes.length > 0) {
nodes = decorateFromArch(nodes, getModel(), { overlayMode: mode });
}
});
});

untrack needs adding to the import { tick, onMount } from 'svelte' on line 7.

id: `controls-${node['unique-id']}`,
source: 'controls',
kind: 'count',
severity: severityFromCount(controls),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity scales by control count here, but the PR body says “severity scaled by total mitigations”. mitigations is computed (line 21) yet only stored in data — it never feeds severity. Align the code or the wording.

<h3 class="dr-sec-h">Threats <span class="count">{threats.length}</span></h3>
{#each threats as t (t['unique-id'])}
<div class="dr-item">
<span class="dot" style:background-color={sevColor[(t.data?.severity as Severity) ?? 'unknown']}></span>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dot colour reads only t.data?.severity, diverging from severityFromDecorator (which also honours legacy risk-level and the type === 'threat' default). A threat the overlay tints orange can show a grey dot here — reusing severityFromDecorator keeps drawer and overlay consistent. (Line 10 also has a redundant double as CalmDecorator[] cast.)

gjs-opsflo added a commit to gjs-opsflo/architecture-as-code that referenced this pull request Jun 24, 2026
…t + spdx + severity

- +page.svelte: wrap overlay-mode re-decoration in untrack(); reading and reassigning nodes inside the effect tipped Svelte into effect_update_depth_exceeded and wedged the canvas on any document with nodes (E2E suite 12/14 fails).
- BadgeAPI.ts, decoratorsAdapter.ts: add missing SPDX headers (consistent with rest of calm-core/src/).
- controlsAdapter.ts: severity now scales by total mitigations (sum of mitigates[] lengths), aligning with PR body wording and badge intent.
- ThreatsSection.svelte: use severityFromDecorator() so the dot colour matches the overlay tint; drops redundant double cast.

Signed-off-by: Gourav Shah <gjs@opsflow.sh>
@gjs-opsflo

Copy link
Copy Markdown
Contributor Author

Thanks for the careful review @rocketstack-matt. Pushed aa767b32 addressing all four inline findings + the PR body framing.

Inline fixes:

  • +page.svelte:126 self-referential $effect (E2E wedge): applied your untrack suggestion verbatim — nodes.length/nodes read and reassignment now happen inside untrack, only overlay.mode is tracked. Added untrack to the svelte import on line 7 as noted, and a load-bearing comment on the effect explaining why the wrapper matters so a future edit doesn't re-introduce the cycle. Calm-core suite stays green; happy to re-run the studio E2E suite locally before next push if helpful.
  • Missing SPDX headers (BadgeAPI.ts, decoratorsAdapter.ts): added.
  • controlsAdapter severity scaling mismatch: fixed the code, not the wording — severity now scales by total mitigations count (≥1 low, ≥5 medium, ≥10 high), matching the PR description and the badge's intended "how much threat surface is being addressed" semantics. Updated the corresponding unit test cases to cover both control-count and per-control-mitigation axes.
  • ThreatsSection.svelte dot colour divergence + double cast: dot now flows through severityFromDecorator() so it honours legacy risk-level and the type === 'threat' default identically to the overlay tint; redundant as CalmDecorator[] cast removed.

PR body framing: changed Resolves #2686Part of #2686 (Part 1 of two; …) with the remaining parity items called out so merging won't auto-close the umbrella issue with half its scope undone.

On the unwired pure modules (positionStore, groupingEngine, dimensions, groupingStore): noted — these are scoped as Part 1 foundation for the consolidation RFC (#2690) and the queued follow-ups listed in the walkthrough. Happy to either (a) drop them from this PR and reintroduce when they get wiring callers, or (b) leave them as-is so RFC #2690 can validate the API surface against actual code. Either path works for me; flagging in case you have a preference before the next review pass.

@gjs-opsflo

Copy link
Copy Markdown
Contributor Author

Heads-up @rocketstack-matt — a UX pivot is incoming on this branch.

After local-running the spike on the canonical loan-approval doc and comparing side-by-side with calm-hub-ui, the user-facing experience clearly falls short of CalmHub's bar: dark-theme drawer competes with the existing Properties panel, default severity tints make the canvas noisy before user has any context, ELK spacing is too tight, no fit-on-import, and the detail surface is a persistent floating overlay rather than CalmHub's clean anchored-near-node popover.

CalmHub is the right reference for "what good looks like" here, and the spike currently doesn't meet that bar. The pure-modules foundation (already approved in your review of aa767b32) is unaffected — the pivot is purely in the Svelte UI layer.

Plan (Phase A — match CalmHub baseline, ~5 commits):

  • Light-default theme respect (read existing theme.svelte.ts; bias light when no preference).
  • Replace DetailDrawer with InlinePopover anchored to clicked node (CalmHub's pattern, image attached in user feedback). Eliminates the right-side panel collision with Properties.
  • Strip default severity tints from NodeFrame — node borders stay neutral by default; severity overlay becomes an opt-in toggle (not forced).
  • Single small controls-count badge per node (CalmHub's O n style) instead of the multi-badge cluster.
  • Edge style alignment (dashed, generous spacing).
  • ELK spacing increase (nodesep/layersep more generous) + auto-fit on import.

Plan (Phase B — studio-specific layers on top, ~3 commits):

  • Re-frame the threat overlay toggle as discoverable opt-in with a subtle hint when threats exist (already built; just changing default).
  • Inline edit affordances in the popover (Studio-unique, CalmHub is view-only).
  • Validation surfacing in the popover.

The PR description will be updated to reflect the pivot once Phase A lands. Sharing this now so a 5-commit force-stack doesn't appear without context. Happy to take any preference on whether you'd rather see it as separate force-pushes per commit (easier per-commit review) or a single squash on top.

@gjs-opsflo

gjs-opsflo commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Quick clarification on the previous comment — calling it a "pivot" overstated the commitment. The CalmHub-parity work is starting as an experiment in a parallel branch (experiment/calm-studio-viz-calmhub-ux off this branch), not as force-pushes onto this PR.

PR #2731 stands as-is: the pure-modules foundation + the Part-1 UI you've already reviewed. We'll keep this PR on its current shape and let the experiment prove itself separately before deciding how (or whether) it folds in.

If the experiment lands a coherent CalmHub-parity baseline, we'll surface it as either a follow-up PR or, with your call, a squashed update to this one. If it doesn't, the foundation here remains shippable on its own.

@rocketstack-matt

Copy link
Copy Markdown
Member

@gjs-opsflo several conflicts following merge of #2730 - can you take a look then I'll re-review.

gjs-opsflo added 19 commits July 2, 2026 08:31
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
Signed-off-by: Gourav Shah <gjs@opsflow.sh>
…-null

Signed-off-by: Gourav Shah <gjs@opsflow.sh>
…t + spdx + severity

- +page.svelte: wrap overlay-mode re-decoration in untrack(); reading and reassigning nodes inside the effect tipped Svelte into effect_update_depth_exceeded and wedged the canvas on any document with nodes (E2E suite 12/14 fails).
- BadgeAPI.ts, decoratorsAdapter.ts: add missing SPDX headers (consistent with rest of calm-core/src/).
- controlsAdapter.ts: severity now scales by total mitigations (sum of mitigates[] lengths), aligning with PR body wording and badge intent.
- ThreatsSection.svelte: use severityFromDecorator() so the dot colour matches the overlay tint; drops redundant double cast.

Signed-off-by: Gourav Shah <gjs@opsflow.sh>
A3 (finos#2730) promoted calm-suite/calm-studio/ to top-level calm-studio/
after this PR's original commits were authored. During rebase git
rename-detected existing files automatically, but the 31 new files
introduced by this PR (viz namespace + decorator/control/severity
adapters, overlay/grouping stores, badge/drawer/frame components,
canonical fixture, walkthrough doc, integration test scaffolding)
had no source to rename from and stayed at the old calm-suite/ path.

Pure move: 31 file renames, zero content change, empty calm-suite/
rtk tree removed. Import paths in already-relocated files already reference
the new locations via SvelteKit's $lib alias, so no code edits needed.

Signed-off-by: Gourav Shah <gjs@opsflow.sh>
@gjs-opsflo gjs-opsflo force-pushed the feat/calm-studio-viz-revamp branch from 5195c7f to f3089a9 Compare July 2, 2026 03:22
@initcron

initcron commented Jul 2, 2026

Copy link
Copy Markdown

@rocketstack-matt Rebased onto current main to resolve conflicts from #2730 (A3 workspace promotion).

What changed:

  • 26 original viz-revamp commits replayed onto post-A3 main via git rebase with 90% rename-threshold; git's rename detection handled 25/26 automatically.
  • One test-file location conflict resolved (git had staged the file at the new path — just needed git add).
  • 31 new-add files that git couldn't rename-detect (viz namespace, badge adapters, drawer sections, overlay/grouping stores, NodeFrame, canonical fixture, walkthrough doc, integration test) relocated from calm-suite/calm-studio/... to calm-studio/... in a single pure-move commit at the tip. Zero content change; empty calm-suite/ tree removed.

Verification (Node 26, fresh lockfile per AGENTS.md):

  • Studio tests: 30 files, 440 tests passing.
  • calm-core tests: 13 files, 118 tests passing (+ 1 todo).
  • Typecheck delta vs origin/main: zero new errors (46 files with pre-existing errors on both).

Branch now 27 commits ahead / 0 behind. Ready for re-review 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants