Skip to content

Add Devin for Group Audit demo kit (RACM ontology, playbooks, reporting templates)#253

Open
devin-ai-integration[bot] wants to merge 1 commit into
DevOpsfrom
devin/1781883529-group-audit-demo-kit
Open

Add Devin for Group Audit demo kit (RACM ontology, playbooks, reporting templates)#253
devin-ai-integration[bot] wants to merge 1 commit into
DevOpsfrom
devin/1781883529-group-audit-demo-kit

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Jun 19, 2026

Copy link
Copy Markdown

Summary

Adds a self-contained "Devin for Group Audit" demo kit under audit/, using this banking app as the auditee. It demonstrates how Devin supports an internal-audit (third line of defence) function across three pillars the Lloyds Group Audit team cares about: an audit ontology (Risk & Control Matrix), control testing (ITGC + application controls + code review as a continuous control), and audit reporting. No application code is touched — this is documentation/config only.

The thesis encoded throughout: Devin is the execution & evidence layer of internal audit — under the auditor's sign-off it plans testing from a machine-readable control catalogue, executes tests, re-performs calculations, reviews code as a continuous control, and drafts cited workpapers/findings. It does not replace auditor judgement or independence.

The kit is deliberately grounded in real gaps that already exist in this repo, so the demo is repeatable and Devin discovers genuine findings (nothing is faked):

Control Result Gap (cited in racm.yaml)
APP-TXN-01 FAIL No amount > 0 validation → negative transfer steals from recipient (AccountService.java:51-135)
APP-TXN-02 FAIL transferAmount not @Transactional → partial failure destroys money
APP-ACC-04 FAIL Single hardcoded "USER" authority, no SoD (AccountService.java:99-101)
APP-SEC-05 FAIL csrf.disable() on state-changing endpoints (SecurityConfig.java:30)
ITGC-SEC-06 FAIL DB creds committed (application.properties:4-5)
ITGC-CM-07/08 FAIL No CODEOWNERS/branch protection; pipeline builds from a different upstream repo (Jenkinsfile:26)
ITGC-SDLC-09 PARTIAL Trivy/OWASP/Sonar present, but only test is contextLoads()
ITGC-DATA-10 FAIL ddl-auto=update auto-mutates schema (application.properties:9)

What's in audit/

  • racm.yaml — the ontology: 4 auditable entities, 8 risks, 6 regulations (ICFR / PRA-FCA / SMCR / Consumer Duty / Op-Resilience / GDPR), and 11 controls. Each control carries test_procedure, population, pass_criteria, evidence_required, and an expected_demo_result with file:line references.
  • ontology.md — the entity → risk → control → test → evidence → finding → action model.
  • playbooks/ — 4 modular (phase-gated) playbooks: Audit Planning & Coverage, ITGC Testing, Application-Control Re-performance, Findings & Reporting.
  • templates/ — workpaper, 5-Cs finding, audit-committee one-pager.
  • samples/ — a worked workpaper (APP-TXN-01) and a worked finding (FND-01) showing the output Devin produces.
  • ccm/ — a Continuous Controls Monitoring spec (scheduled re-testing, escalate-on-change).
  • README.md — the demo runbook with the 3-step flow (Ask Devin → Ask Devin → Devin session) and the expected-findings table.

Demo flow

  1. Ask Devin (discover): map each RACM control to where it lives in the code.
  2. Ask Devin (scope): produce the fieldwork test plan via the Planning playbook.
  3. Devin session (execute + report): run the ITGC + re-performance playbooks, then Findings & Reporting — workpapers, 5-Cs findings, and a committee one-pager, each result cited to file:line / commit / command output.

The corresponding Devin Playbooks and a RACM Knowledge note are created in the workspace separately so the @playbook: references resolve in a live demo.

Link to Devin session: https://app.devin.ai/sessions/fe3be37ee47e4bd99a23c86e30291623
Requested by: @Hammy-2025


Devin Review

Status Commit
⚪ Not started

Run Devin Review

Open in Devin Review (Staging)

…ng templates)

Co-Authored-By: Alex Hammett <alex.hammett@windsurf.com>
@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants