aperion-compass — local, offline AI governance self-assessment

Frameworks bundled:

⭐ Star this repo if "we have AI governance" should mean more than a slide

aperion-compass turns a governance framework into a runnable check: answer a guided questionnaire, point it at your exported logs, and get a scored conformance report — verifying your audit chain and agent credentials with real cryptography, not a checkbox. It runs entirely on your machine. No account, no telemetry, no upload.

If "prove it, don't claim it" is how you think about compliance, a ⭐ is the fastest way to help other engineers in regulated shops find it → Star aperion-compass on GitHub

aperion-compass is a single, offline Rust binary that runs a governance self-assessment against the EU AI Act and Singapore's IMDA Model AI Governance Framework for Agentic AI. You answer a plain-language questionnaire (or hand-edit / commit an answers file), and — this is the part that makes it more than a survey — you point it at files you already have:

• Audit-chain integrity — verifies a tamper-evident HMAC-SHA256 hash chain (seq / prev_hash / entry_hmac) with only the key.
• Human oversight — computes override rate, approval latency, and outlier reviewers from exported approval tickets (rubber-stamping and automation-bias signals).
• Action-risk coverage — deterministically tiers your tool calls T1/T2/T3 and reports how many irreversible actions had an approval.
• Agent identity — verifies Ed25519 agent credentials offline against a public key / JWKS (no shared secret, no running gateway).
• Logging completeness — field-presence stats over your request logs (identity, risk tier, decisions, perimeter).
• Signed attestation (v0.3) — seal the scored posture, evidence-check outcomes, and an anchor of the audit-chain tail into one Ed25519-signed bundle that an auditor or customer can verify offline, without trusting you or touching your gateway.

Objective evidence overrides a green self-attestation: claim an Article 12 tamper-evident log, hand Compass a chain that fails verification, and that control turns red — with a non-zero CI exit code. Every gap carries remediation guidance (a generic pattern first; Smartflow noted as one implementation).

It emits a self-contained HTML dashboard (opens from file://, attach it to a board deck), plus Markdown and JSON. A serve mode runs the same dashboard on localhost with a re-scan button.

---

Quickstart

# 1. Answer the questionnaire (writes compass-assessment.yaml)
compass assess --framework eu-ai-act,imda

# 2. Attach evidence you already export (all optional)
compass ingest \
  --vas         logs.jsonl \
  --chain       audit-chain.jsonl --chain-hmac-key file:hmac.key \
  --approvals   approvals.jsonl \
  --credentials agents.jsonl --jwks issuer-jwks.json

# 3. Score it and render the report
compass report --out report.html --format html,md,json

# ...or explore live in the browser
compass serve --port 8787

# 4. (v0.3) Seal the result into a signed, offline-verifiable attestation
compass attest generate --out compass-attestation.json
compass attest verify   --bundle compass-attestation.json   # anyone, offline

No evidence files? A questionnaire-only run still produces a full report — those controls are simply marked "self-attested".

Signed attestation bundles (v0.3)

A report is something you produce; an attestation is something an auditor or customer can verify without trusting you. compass attest generate assembles the scored posture, every evidence-check outcome, and a cryptographic anchor of your audit-chain tail into one JSON payload and Ed25519-signs it. compass attest verify recomputes the canonical payload and checks the signature offline against a published public key / JWKS — so any tampering (a nudged score, a removed integrity failure, a swapped framework) breaks verification with a non-zero exit code.

The signing key defaults to a stable per-user key at ~/.aperion-compass/attest-ed25519.key (created on first run), or pass your own with --signing-key file:…|base64:…|hex:…|env:… — e.g. reuse the same Ed25519 issuer key you already publish for AIDA agent credentials. generate also writes the public key as <out>.jwks.json so a verifier needs nothing but the bundle and that key. It reuses the exact Ed25519/JWKS primitive Compass already uses to verify agent credentials.

Don't have governance-grade logs? (most people don't)

That's the normal starting point, and it's the whole reason the tool exists. Two ways to get real evidence:

# Convert something you already have (provider export → evidence)
compass ingest --from openai   --input openai-export.json
compass ingest --from litellm  --input litellm-logs.jsonl
compass ingest --from bedrock  --input bedrock-invocations.jsonl
compass ingest --from csv-approvals --input approvals.csv   # Jira/ServiceNow export

# Or capture it from live traffic: run this in front of your model
# endpoint, point your SDK's base_url at it, come back in two weeks.
compass record --upstream http://localhost:4000 --out compass-record.jsonl

# Then see exactly what's still missing and how to fix each gap
compass doctor

Step-by-step guides for pulling logs from OpenAI, Azure OpenAI, LiteLLM, Bedrock, and LangSmith — plus the minimal field spec if you want to roll your own — live in docs/evidence/.

Try it on the bundled demo

cargo run --example gen_fixtures         # writes ./demo/*
compass report --assessment demo/compass-assessment.yaml --out report.html
compass verify --chain demo/chain.jsonl --chain-hmac-key file:demo/chain.key

Install

# Homebrew (macOS / Linux)
brew install AperionAI/tap/aperion-compass

# Cargo
cargo install aperion-compass

# Docker
docker run --rm -v "$PWD:/work" -w /work \
  ghcr.io/aperionai/compass:latest report --out report.html

# From source
git clone https://github.com/AperionAI/compass && cd compass
cargo build --release      # ./target/release/compass

Commands

Command	What it does
compass assess	Interactive questionnaire → compass-assessment.yaml (add --defaults to scaffold an editable file).
compass ingest	Register evidence files; or --from openai|litellm|bedrock|csv|csv-approvals --input <export> to convert a native export into evidence first.
compass doctor	Show which automated checks have evidence and, for every gap, the exact step to close it.
compass record	Localhost OpenAI-compatible proxy that captures a tamper-evident log from live traffic.
compass report	Score answers + evidence; write HTML / Markdown / JSON.
compass serve	Live localhost dashboard with a re-scan button.
compass verify	Standalone tamper-evident audit-chain verification.
compass frameworks	List the bundled frameworks and control counts.

Assessment as code

compass-assessment.yaml is designed to be committed. It is editable, diffable, and re-runnable, so a governance posture can be reviewed in a pull request and enforced in CI:

# .github/workflows/governance.yml (excerpt)
- run: compass report --out report.html --threshold 80
  # exit 0 = at/above threshold · 1 = below · 2 = evidence integrity failure

CI exit codes:

• 0 — overall score at or above --threshold (default 70).
• 1 — below threshold.
• 2 — an evidence-integrity check failed (tampered audit chain or an invalid agent credential). This takes precedence over the threshold.

Free tool vs. Smartflow

Compass is a point-in-time assessment from files. Smartflow is the continuous, runtime enforcement the reports point toward. Same concepts, same evidence formats — one measures, the other enforces.

	aperion-compass (this)	Smartflow
Cost	Free, open source (Apache-2.0)	Commercial
Runs	Locally, offline, from files	In your request path (gateway)
Mode	Point-in-time snapshot	Continuous, live
Identity	Verifies exported credentials	Issues + validates on the hot path
Audit	Verifies an exported chain	Produces the tamper-evident chain
Oversight	Scores exported tickets	Runs the approval queue
Action risk	Tiers logged calls	Blocks/holds T3 actions before they run
Reports	HTML / MD / JSON	Live console + regulator API
Data	Never leaves your machine	On-prem / in your cluster

Remediation text in reports links to the governance patterns at docs.aperion.ai.

Honest limitations

• It assesses; it does not enforce. A passing Compass report says your evidence and answers look good at a point in time. It cannot stop a bad action — that is a runtime concern.
• Verification is only as good as the export. Compass verifies the files you give it. It cannot know your export is complete, that the HMAC key is the real one, or that a log wasn't filtered before export.
• HMAC credentials aren't offline-verifiable. Only Ed25519 (public key) credentials can be checked without a secret; HMAC ones are reported as "unverifiable", not "valid".
• compass record forwards to http upstreams. For a hosted HTTPS API, put a local gateway (e.g. LiteLLM) in front — see docs/evidence/record.md. Direct-HTTPS upstreams and SSE streaming pass-through are on the roadmap.
• Not legal advice. This is a preparation aid, not a conformity assessment, and not a substitute for a notified body, counsel, or your regulator. The control catalogs are our reading of the frameworks; verdicts are conservative but opinionated.
• v1 scope. EU AI Act + IMDA agentic today. NIST AI RMF and document/PDF ingestion are fast-follows; PDF export is print-to-PDF from the HTML.

Development

cargo test                        # unit + integration tests
cargo run --example gen_fixtures  # regenerate ./demo/*
cargo clippy --all-targets

Layout: catalogs/ (bundled framework YAML), src/evidence/ (the offline checks), src/scoring.rs (verdicts + rollups), src/report/ (renderers), templates/dashboard.html (the embedded dashboard).

License

Apache-2.0. See LICENSE.