feat: LLM Runtime Integration modifier (L0-L4)

[raifdmueller]

Summary

Adds a cross-cutting LLM Runtime Integration modifier (L0–L4) that captures how the software uses LLMs at runtime, separate from the 5 code dimensions. This addresses a gap in the framework: the 5 dimensions describe the written code, but agentic systems (Claude Code, RAG assistants, chatbots) have qualitatively different runtime risks — prompt injection, unauthorized tool calls, code execution.

Closes #20. Implementation plan & decisions documented in the issue comments.

What's new

UI (between chart and tier badge)

• Pill-button row L0–L4 with level descriptions
• Callout box at L3+ pointing to OWASP LLM Top 10, Palo Alto SHIELD, Aikido VCAL, Google SAIF

Tier logic — hard multiplier

• L3 forces at least Tier 3, L4 forces at least Tier 4, regardless of the 5 code dimensions
• Rationale: An agent that can run `rm -rf` is safety-critical by definition

New presets

• Support Chatbot → L2 (Tier 2)
• RAG Knowledge Assistant → L3 (Tier 3, lifted by modifier)
• Coding Agent → L4 (Tier 4, lifted by modifier)

Existing presets reset the modifier to L0 on click.

Documentation sidebar

• New section LLM Runtime Integration (DE + EN) explaining build-time vs runtime, the escalation ladder, the tier multiplier, and framework references

Skills integration

• `shared/risk-model.md`: escalation ladder, tier calculation with modifier, grep patterns for LLM SDKs (anthropic, openai, langchain, @ai-sdk/*), agentic indicators, sandbox libraries
• `/risk-assess`: Step 2f (detection), Step 3d (always ask to confirm), Step 4a (tier with lift note)
• `/risk-mitigate`: Step 1 parses `llmRuntimeLevel`, new Step 3b with L3+ callout pointing to specialized frameworks; does not install runtime mitigations (architectural decisions belong with the user)

Commits (5 phases)

Phase	Commit	Scope
1+2	Data model + tier logic + UI	`utils.js`, `RiskRadar.jsx`, `RadarChart.jsx`, `i18n.js`, CSS
3	Agentic presets + reset	`i18n.js`, `RiskRadar.jsx`
4	Docs sidebar section	`i18n.js`
5	Skills integration	3 skill files

Design decisions

• Modifier, not 6th dimension — keeps MECE intact, avoids conflating "code written" with "what code does at runtime"
• Hard multiplier (not additive) — L4 is safety-critical regardless of blast radius, not proportional
• Fixed dark tooltip colors kept from prior PR
• Framework links, not own catalog — OWASP/SHIELD/VCAL/SAIF cover agentic risks more maturely; we classify and defer

Test plan

• Build passes (`npm run build`)
• Verified in browser (Edge headless): L4 lifts Tier to 4, L3 to 3, L0–L2 don't lift
• Verified: Coding Agent preset → Tier 4 + callout
• Verified: CSS Landing Page resets L0 after Coding Agent L4
• Verified: Support Chatbot → L2 (Generate)
• Verified: Docs sidebar renders new section in DE and EN with framework links
• Manual review of skills updates (no runtime behavior to test)

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

• New Features

  • Added LLM Runtime Integration dimension to assess risks from LLM SDK usage and agentic code patterns.
  • Introduced interactive runtime level selector (L0–L4) in the risk radar with real-time tier recalculation.
  • Added runtime-specific risk callouts and mitigation guidance for high-integration scenarios.
  • Expanded preset configurations to include Support-Chatbot, RAG Knowledge Assistant, and Coding-Agent archetypes.

• Documentation

  • Updated risk model and workflow documentation to reflect LLM runtime assessment integration.

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR implements LLM Runtime Integration as a cross-cutting modifier to the Risk Radar system, adding L0–L4 runtime risk detection and assessment independent of code dimensions. It updates tier calculations, skills documentation, UI components, and i18n content to support runtime risk selection and framework-based mitigation references.

Changes

Cohort / File(s)	Summary
LLM Runtime Integration Skills .claude/skills/risk-assess/SKILL.md, .claude/skills/risk-mitigate/SKILL.md, .claude/skills/shared/risk-model.md	Updated skill workflows to detect LLM SDK imports (Step 2f in risk-assess), extract runtime integration level per module, and emit runtime risk callouts for L3+. Modified tier calculation to use baseTier from 5 code dimensions plus a floor driven by llmRuntimeLevel, with final tier = max(baseTier, floor). Extended documentation with escalation levels and framework references.
Frontend Tier Calculation src/utils.js	Updated getTierIndex() to accept optional llmRuntimeLevel parameter (default 0). Tier now computed as max of base tier (from code dimensions) and floor (2 for L3+, 3 for L4+, else 0).
Radar Chart Components src/components/RadarChart.jsx, src/components/RiskRadar.jsx	Added llmRuntimeLevel state and ref to RiskRadar; passed as prop to RadarChart. Updated all getTierIndex() calls to include runtime level. RiskRadar now includes UI section with L0–L4 level buttons, conditionally displays callout box with framework links when L3+, and binds preset selection to corresponding runtime level.
LLM Runtime UI Styling src/components/RiskRadar.module.css	Added 8 new CSS classes (.llmRuntime*) for layout, button styling with hover effects (translateY, box-shadow), callout emphasis, and framework link styling with underline color transitions.
Internationalization & Presets src/i18n.js	Added cross-cutting llmRuntime content block (DE/EN) with L0–L4 labels, descriptions, callout text, and framework references. Extended presets array with three new entries—Support-Chatbot (L2), RAG Knowledge Assistant (L3), Coding Agent (L4)—each including llmRuntimeLevel field. Added documentation section docs.sections for runtime risk escalation.

Sequence Diagram

sequenceDiagram
    actor User
    participant RiskRadar as RiskRadar.jsx
    participant RadarChart as RadarChart.jsx
    participant Utils as getTierIndex()
    
    User->>RiskRadar: Click LLM Runtime Level (L0–L4)
    RiskRadar->>RiskRadar: Update llmRuntimeLevel state
    RiskRadar->>RiskRadar: Trigger tier recalculation
    RiskRadar->>Utils: getTierIndex(roundedValues, llmRuntimeLevel)
    Utils->>Utils: baseTier = max(code dimensions)
    Utils->>Utils: floor = (llmRuntimeLevel >= 3) ? 2 : (llmRuntimeLevel >= 4) ? 3 : 0
    Utils->>Utils: finalTier = max(baseTier, floor)
    Utils-->>RiskRadar: Return updated tier index
    RiskRadar->>RadarChart: Pass llmRuntimeLevel + new tier via props
    RadarChart->>RadarChart: Re-render with updated tier styling
    RiskRadar->>RiskRadar: Conditionally render callout (if L3+)
    RiskRadar-->>User: Display updated tier + framework links

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• PR #15: Modifies the same core files (src/components/RadarChart.jsx, src/components/RiskRadar.jsx, src/utils.js) with overlapping prop signatures and tier calculation updates—likely to have integration dependencies or conflicts.

Poem

🐰 A runtime twist beyond the code,
L-levels light the hidden road,
From passive sort to agent's dance,
The Radar's tier takes second glance!
Framework links where agentics dwell—
Hopping safer, code and shell! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main feature: adding an LLM Runtime Integration modifier with levels L0-L4. It directly reflects the primary change across the changeset.
Linked Issues check	✅ Passed	The PR implements all primary coding requirements from issue #20: LLM runtime levels L0-L4, tier multiplier logic, UI selection, i18n support, preset updates, and skills integration with auto-detection.
Out of Scope Changes check	✅ Passed	All changes are directly aligned with issue #20 objectives. Modifications span documentation, UI, data model, i18n, and skills integration as specified, with no unrelated changes detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}