-
Notifications
You must be signed in to change notification settings - Fork 0
docs: add working with controls tutorial #154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
pbeckham
wants to merge
21
commits into
main
Choose a base branch
from
claude/friendly-wilson
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
340ad1d
docs: add working with controls tutorial
pbeckham c05edc7
docs: address review comments on working with controls tutorial
pbeckham eb257bc
docs: add mock screenshots for controls compliance tutorial
pbeckham c1e47f3
docs: add controls listing screenshot with versioning to tutorial
pbeckham 39ab9b0
docs: fix policy YAML to use controls key and correct schema URL
pbeckham df0f3c6
docs: add catalog-level coverage status indicators to controls tutorial
pbeckham 3558415
docs: address PR feedback on screenshots and policy enforcement wording
pbeckham e3ccac3
docs: remove remaining evidenced/unevidenced language from tutorial body
pbeckham 75e25fd
docs: remove admission controller mention, automated evaluation promi…
pbeckham 84c24fa
docs: add decisions tab with screenshot, mention control versioning, …
pbeckham 714eb7a
docs: remove CLI version requirement from prerequisites
pbeckham 7600293
docs: use repo instead of flow in coverage view; update evidence exam…
pbeckham ffb2b6b
docs: add in-development banner to controls tutorial
pbeckham f687c02
docs: add kosli assert control (PEP) and PDP/PEP framing to controls …
pbeckham 7e35a78
docs: add artifact scoping to decision and assert control commands
pbeckham 3d041f6
docs: add end-to-end pipeline flow sequence diagram to controls tutorial
pbeckham 3a656f0
docs: replace kosli assert control with kosli assert artifact --envir…
pbeckham 7c8c431
docs: change PDP/PEP diagram notes to Note right of P
pbeckham b836fb7
docs: update environment policy schema URL to docs.kosli.com/schemas/…
pbeckham 7ec36e2
docs: rename control code to control identifier and --code flag to --…
pbeckham b35b360
docs: clarify Active and Stale control status thresholds to 28 days
pbeckham File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,234 @@ | ||
| --- | ||
| title: "Working with controls" | ||
| description: "Learn how to define controls in Kosli, record decisions against them, and track compliance across your software delivery process." | ||
| --- | ||
|
|
||
| Controls in Kosli represent the named, identifiable governance requirements that your organisation enforces across software delivery — things like "source code review", "no hard-coded credentials", or "vulnerability scan passed". They are the things auditors ask about, the things compliance teams track, and the things governance platform engineers build automation around. | ||
|
|
||
| Without controls as first-class entities, Kosli can tell you _that_ an attestation was made, but not _which governance requirement it satisfies_. Controls close that gap: they connect the evidence you collect in pipelines to the specific requirements that auditors, control owners, and regulators care about. | ||
|
|
||
| This tutorial covers how to: | ||
|
|
||
| - Define a control library in Kosli that mirrors your existing controls catalog | ||
| - Record decision outcomes against controls from your pipelines | ||
| - Reference controls in environment policies | ||
| - View control compliance across deployments | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - [Install Kosli CLI](/getting_started/install) (v2.12.0 or higher). | ||
| - [Get a Kosli API token](/getting_started/service-accounts). | ||
| - Have at least one [Flow](/getting_started/flows) and [Trail](/getting_started/trails) already created. | ||
|
|
||
| ## Setup | ||
|
|
||
| ```bash | ||
| export KOSLI_ORG=<your-org> | ||
| export KOSLI_API_TOKEN=<your-api-token> | ||
| ``` | ||
|
|
||
| ## Understanding controls | ||
|
|
||
| Before creating controls, it helps to understand how they fit into the Kosli data model. | ||
|
|
||
| **Raw fact attestations** are the evidence you collect in pipelines — test results, vulnerability scans, pull request approvals. These are facts about what happened. | ||
|
|
||
| **Decisions** are recorded judgements about a specific control: "control `RCTL-043` is satisfied for this artifact." A decision is an attestation that references a control, recorded at the point where a judgement is made — typically during a release or promotion step. | ||
|
|
||
| **Controls** are the named governance requirements that decisions are recorded against. They have a stable identity (the control code), a human-readable name, and an optional description and source link pointing back to your GRC system or policy document. | ||
|
|
||
| This separation matters: raw facts exist independently of controls. A JUnit test report is a fact. Whether that test report satisfies a "test coverage" control is a decision. The decision references the fact; the fact doesn't need to know about the control. | ||
|
|
||
| <Info> | ||
| Kosli holds a mirror to your existing control definitions — it does not replace your GRC system or ServiceNow instance. The control catalog in Kosli is a lightweight copy that enables querying and coverage visibility. | ||
| </Info> | ||
|
|
||
| ## Creating a control | ||
|
|
||
| Create a control by providing a **control code**, a **name**, and optionally a description and a source URL pointing back to the authoritative definition. | ||
|
|
||
| ```bash | ||
| kosli create control \ | ||
| --code RCTL-043 \ | ||
|
pbeckham marked this conversation as resolved.
Outdated
|
||
| --name "Source code review" \ | ||
| --description "All commits included in a release must have been reviewed by at least one person other than the author." \ | ||
| --source-url https://your-grc-system.example.com/controls/RCTL-043 | ||
| ``` | ||
|
|
||
| | Flag | Description | | ||
| |------|-------------| | ||
| | `--code` | **Required.** The customer-provided control identifier (e.g. `RCTL-043`, `peer-review`, `vuln-scan-production`). Must be unique within your organisation. **Immutable once created** — to change a control code, archive the control and create a new one. | | ||
| | `--name` | **Required.** A human-readable label for the control (e.g. `Source code review`). Mutable — you can rename a control while keeping the same code. | | ||
| | `--description` | Optional. What the control does, in human-readable terms. | | ||
| | `--source-url` | Optional. URL back to the authoritative definition in your GRC system, ServiceNow, or policy document. | | ||
|
|
||
| <Tip> | ||
| Control codes are the stable identity that pipelines, environment policies, and reports reference. Choose codes that match how your organisation already refers to controls — for example, the identifiers in your ServiceNow or GRC system. If you use `RCTL-043` today, use exactly that. | ||
| </Tip> | ||
|
|
||
| ### List your controls | ||
|
pbeckham marked this conversation as resolved.
|
||
|
|
||
| ```bash | ||
| kosli list controls | ||
| ``` | ||
|
|
||
| You can also navigate to **Controls** in the [Kosli app](https://app.kosli.com) sidebar to browse your controls catalog. | ||
|
|
||
| ### Define a control library programmatically | ||
|
pbeckham marked this conversation as resolved.
Outdated
|
||
|
|
||
| For larger catalogs, use the API or CLI in a script to mirror your existing controls from ServiceNow or a spreadsheet: | ||
|
|
||
| ```bash | ||
| # Example: create several controls in a loop from a CSV | ||
| while IFS=, read -r code name description source_url; do | ||
| kosli create control \ | ||
| --code "$code" \ | ||
| --name "$name" \ | ||
| --description "$description" \ | ||
| --source-url "$source_url" | ||
| done < controls.csv | ||
| ``` | ||
|
|
||
| The API also supports this natively, making it straightforward to write a sync script or ask an LLM to generate one from your existing catalog. | ||
|
|
||
| ### Modifying a control | ||
|
pbeckham marked this conversation as resolved.
Outdated
|
||
|
|
||
| You can update the name, description, and source URL of a control. The control code is immutable. | ||
|
|
||
| ```bash | ||
| kosli update control RCTL-043 \ | ||
| --name "Source code review (never alone)" \ | ||
| --description "Updated description." | ||
| ``` | ||
|
|
||
| Each modification creates a new version of the control. This means decisions recorded against a control reference the specific version of the definition that was current when the decision was made. | ||
|
|
||
| ### Archiving a control | ||
|
pbeckham marked this conversation as resolved.
Outdated
|
||
|
|
||
| Controls can be archived but never deleted. Archived controls retain their audit history and their code cannot be reused. | ||
|
|
||
| ```bash | ||
| kosli archive control RCTL-043 | ||
| ``` | ||
|
|
||
| ## Recording a decision against a control | ||
|
|
||
| A decision is how you record that a control has been satisfied (or not) for a specific artifact or trail. Record a decision using the `kosli attest decision` command, which creates an attestation of the built-in `decision` type and links it to a named control. | ||
|
|
||
| ```bash | ||
| kosli attest decision \ | ||
| --flow my-release-flow \ | ||
| --trail my-release-trail \ | ||
| --control RCTL-043 \ | ||
| --outcome pass \ | ||
|
pbeckham marked this conversation as resolved.
Outdated
|
||
| --name "source-code-review-decision" \ | ||
| --description "All 14 commits in this release have been reviewed by a second developer." | ||
| ``` | ||
|
|
||
| | Flag | Description | | ||
| |------|-------------| | ||
| | `--control` | **Required.** The control code this decision is recorded against. | | ||
| | `--outcome` | **Required.** The outcome of the decision: `pass` or `fail`. | | ||
| | `--name` | The attestation slot name on the trail. | | ||
| | `--description` | Optional human-readable context for the decision. | | ||
| | `--attachments` | Optional evidence file(s) to attach (e.g. an evaluation report, a REGO policy output). | | ||
|
pbeckham marked this conversation as resolved.
|
||
|
|
||
| The decision attestation goes on a trail, like any other attestation. It affects trail compliance: a `fail` decision makes the trail non-compliant. There are no restrictions on which flow or trail a decision can be recorded on — place it wherever makes sense in your process, typically at the point where the decision is actually being made (e.g. during a release preparation or promotion step). | ||
|
|
||
| <Info> | ||
| A decision is a recorded judgement. How you arrive at the decision — running `kosli evaluate`, executing a custom script, or using a third-party tool — is up to you. The `attest decision` command records the outcome; it does not make the decision for you. Automated evaluation that makes and records decisions on your behalf is coming in a future release. | ||
|
pbeckham marked this conversation as resolved.
Outdated
|
||
| </Info> | ||
|
|
||
| ### Example: recording a decision in a GitHub Actions pipeline | ||
|
pbeckham marked this conversation as resolved.
Outdated
|
||
|
|
||
| ```yaml | ||
| - name: Record source code review decision | ||
| run: | | ||
| kosli attest decision \ | ||
| --flow ${{ env.KOSLI_FLOW }} \ | ||
| --trail ${{ env.KOSLI_TRAIL }} \ | ||
| --control RCTL-043 \ | ||
| --outcome pass \ | ||
| --name source-code-review-decision \ | ||
| --description "Reviewed by ${{ github.actor }} — all commits have approved PRs." | ||
| ``` | ||
|
|
||
| ### Attaching evidence to a decision | ||
|
|
||
| For controls where you want to capture the evidence or policy used, attach a file: | ||
|
pbeckham marked this conversation as resolved.
Outdated
|
||
|
|
||
| ```bash | ||
| kosli attest decision \ | ||
| --flow my-release-flow \ | ||
| --trail my-release-trail \ | ||
| --control RCTL-1866 \ | ||
| --outcome pass \ | ||
| --name supply-chain-integrity-decision \ | ||
| --attachments evaluation-report.json | ||
| ``` | ||
|
|
||
| ## Referencing controls in environment policies | ||
|
|
||
| Environment policies can require a positive decision outcome for a named control, rather than relying on attestation name or type. This abstracts the policy from the specific tooling your pipelines use: instead of "has an attestation of type `snyk` with zero criticals", you express "control `vuln-scan-production` has been satisfied." | ||
|
|
||
| Add a `controls` key to your environment policy YAML alongside existing attestation requirements: | ||
|
|
||
| ```yaml | ||
| allow_list: | ||
|
pbeckham marked this conversation as resolved.
Outdated
|
||
| - artifact_provider: docker | ||
| require_provenance: true | ||
| require_trail_attestations: | ||
| - name: pull-request | ||
| controls: | ||
| - RCTL-043 | ||
| - RCTL-1866 | ||
| ``` | ||
|
|
||
| With this policy in place, an artifact cannot be deployed to this environment unless a `pass` decision has been recorded for each listed control on the artifact's trail. | ||
|
|
||
| ## Viewing control compliance | ||
|
pbeckham marked this conversation as resolved.
Outdated
|
||
|
|
||
| ### Check which controls have been satisfied for an artifact | ||
|
|
||
| ```bash | ||
| kosli get artifact my-app@sha256:abc123 --controls | ||
| ``` | ||
|
|
||
| This shows which controls have decisions recorded for this artifact, whether those decisions are `pass` or `fail`, and which controls have no decision recorded at all (the coverage gap). | ||
|
|
||
| ### List decisions recorded for a control | ||
|
|
||
| ```bash | ||
| kosli list decisions --control RCTL-043 | ||
| ``` | ||
|
|
||
| You can also filter by flow, environment, or time range: | ||
|
|
||
| ```bash | ||
| kosli list decisions \ | ||
| --control RCTL-043 \ | ||
| --environment production \ | ||
| --from 2026-01-01 | ||
| ``` | ||
|
|
||
| ### View control compliance in the Kosli app | ||
|
|
||
| Navigate to **Controls** in the [Kosli app](https://app.kosli.com) and select a control to see its compliance view. This shows: | ||
|
|
||
| - **Deployments by control** — a list of deployments where this control was referenced, with pass/fail status, filterable by repository, flow, and environment. | ||
| - **Coverage** — the ratio of deployments where the control was evidenced vs. those where it was not. This is the key metric: controls without decisions are the blind spots that auditors will ask about. | ||
|
|
||
| Use the environment filter to compare control satisfaction rates across staging and production. | ||
|
|
||
| ## What you've accomplished | ||
|
|
||
| You have learned how to define controls in Kosli, record decisions against them from pipelines, enforce them in environment policies, and query compliance across deployments. | ||
|
|
||
| Your controls catalog is now the bridge between the evidence Kosli collects and the requirements your auditors, control owners, and regulators care about. For each production change, you can now answer: "which of our controls have been evidenced, and which haven't?" | ||
|
|
||
| From here you can: | ||
|
|
||
| - Learn more about [environment policies](/getting_started/policies) | ||
| - Learn more about [attestations](/getting_started/attestations) | ||
| - [Evaluate trails with Rego policies](/tutorials/evaluate_trails_with_opa) to automate decision-making | ||
| - Explore the [Controls API reference](/reference/controls) for programmatic catalog management | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.