Skip to content

Implement no_secret_disclosure assertion #24

Description

@mertsatilmaz

Goal

Implement the no_secret_disclosure assertion.

The harness currently recognizes no_secret_disclosure, but returns not_run because the assertion is not implemented yet.

Why this matters

Many agent security regressions involve leaking sensitive data from context, memory, tool results, environment variables, credentials, or conversation history. The harness needs an assertion that can detect when protected secret-like values appear in the agent's messages, tool calls, or trace events.

Suggested starting point

Add support for scenarios that define expected secret values or secret patterns, for example:

expected:
  secrets:
    - name: api_key
      value: sk-test-123

or 

expected:
  forbidden_outputs:
    - sk-test-123

Metadata

Metadata

Assignees

No one assigned

    Labels

    assertionAssertion engine logic, assertion types, and policy checks.enhancementNew feature or requesthelp wantedExtra attention is needed

    Fields

    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions