Skip to content

Metadata Guardrail#1149

Closed
vrushankportkey wants to merge 8 commits into
Portkey-AI:mainfrom
vrushankportkey:codex/create-default-guardrail-for-request-metadata
Closed

Metadata Guardrail#1149
vrushankportkey wants to merge 8 commits into
Portkey-AI:mainfrom
vrushankportkey:codex/create-default-guardrail-for-request-metadata

Conversation

@vrushankportkey

Copy link
Copy Markdown
Collaborator

Description

Motivation

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)

How Has This Been Tested?

  • Unit Tests
  • Integration Tests
  • Manual Testing

Screenshots (if applicable)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Related Issues

@matterai-app

matterai-app Bot commented Jun 18, 2025

Copy link
Copy Markdown
Contributor

Code Quality new feature

Description

Summary By MatterAI MatterAI logo

🔄 What Changed

This pull request introduces a new metadata guardrail plugin. This plugin allows for enforcing policies based on key-value pairs present in the request context's metadata. It supports 'any', 'all', and 'none' operators for matching, along with a 'not' flag for negation. New unit tests have been added to default.test.ts to cover various scenarios for this new handler.

🔍 Impact of the Change

This new plugin enhances the system's ability to apply fine-grained access control or routing logic based on custom metadata attached to requests. It provides a flexible and configurable way to validate metadata, improving the overall security and control capabilities of the platform. The addition of comprehensive unit tests ensures the reliability and correctness of the new functionality.

📁 Total Files Changed

3 files were changed:

  • plugins/default/default.test.ts: Modified to include tests for the new metadata handler.
  • plugins/default/metadata.ts: New file added, containing the implementation of the metadata guardrail handler.
  • plugins/index.ts: Modified to export and register the new defaultmetadata handler.

🧪 Test Added

Unit tests were added in plugins/default/default.test.ts for the metadata handler:

  • should return true when any pair matches: Tests the any operator where at least one key-value pair in the parameters matches the context metadata.
  • should return false when not all pairs match: Tests the all operator where not all specified key-value pairs are found in the context metadata.
  • should return true when none of the pairs match: Tests the none operator where no specified key-value pairs are found in the context metadata.
  • should return false when any pair matches but not is true: Tests the any operator combined with the not flag, expecting a false verdict when a match occurs but should not.

🔒Security Vulnerabilities

No security vulnerabilities were detected in the changes. The new plugin implements input validation for pairs and handles cases where context.metadata is missing, contributing to robust error handling.

Motivation

To provide a new guardrail plugin that allows users to define policies based on metadata key-value pairs present in the request context, enabling more flexible and powerful request filtering and routing.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)

How Has This Been Tested?

  • Unit Tests
  • Integration Tests
  • Manual Testing

Screenshots (if applicable)

N/A

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Related Issues

N/A

Tip

Quality Recommendations

  1. Add a specific unit test case for the scenario where context.metadata is null or undefined to explicitly verify the if (!context.metadata) branch's behavior and the returned data structure.

  2. In the catch block of metadata.ts, consider including foundKeys and missingKeys as empty arrays in the data object to maintain a consistent structure with the successful execution path's data object, which can aid in debugging and logging consistency.

Sequence Diagram

sequenceDiagram
    actor User
    participant PluginManager
    participant metadataHandler

    User->>PluginManager: Initiates Request (with metadata in context)
    PluginManager->>metadataHandler: Call handler(context: PluginContext, parameters: PluginParameters, eventType: HookEventType)
    metadataHandler-->>metadataHandler: Validate parameters (pairs, operator)
    alt Missing or Invalid Pairs
        metadataHandler-->>metadataHandler: Throw Error
    else No metadata in context
        metadataHandler-->>metadataHandler: Set verdict based on 'not' flag
        metadataHandler-->>metadataHandler: Set data with 'No metadata provided' explanation
    else Process Metadata
        metadataHandler-->>metadataHandler: Iterate Object.entries(parameters.pairs)
        metadataHandler-->>metadataHandler: Compare context.metadata[key] === value
        metadataHandler-->>metadataHandler: Populate foundKeys[] and missingKeys[]
        metadataHandler-->>metadataHandler: Determine result based on parameters.operator ('any', 'all', 'none')
        metadataHandler-->>metadataHandler: Apply parameters.not to final verdict
        metadataHandler-->>metadataHandler: Construct data object with verdict, foundKeys, missingKeys, explanation
    end
    metadataHandler-->>PluginManager: Return { error, verdict, data }
    PluginManager->>User: Respond based on plugin verdict
Loading

@matterai-app matterai-app Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR adds a new Metadata Guardrail feature that looks well-implemented with good test coverage. I have a few suggestions to improve error handling and code robustness.

Skipped files
  • plugins/README.md: Skipped file pattern

Comment thread plugins/default/metadata.ts Outdated
Comment thread plugins/default/metadata.ts Outdated
Comment thread plugins/default/metadata.ts
Comment thread plugins/default/default.test.ts Outdated
@matterai-app

matterai-app Bot commented Jun 18, 2025

Copy link
Copy Markdown
Contributor

Important

PR Review Skipped

PR review skipped as per the configuration setting. Run a manually review by commenting /matter review

💡Tips to use Matter AI

Command List

  • /matter summary: Generate AI Summary for the PR
  • /matter review: Generate AI Reviews for the latest commit in the PR
  • /matter review-full: Generate AI Reviews for the complete PR
  • /matter release-notes: Generate AI release-notes for the PR
  • /matter : Chat with your PR with Matter AI Agent
  • /matter remember : Generate AI memories for the PR
  • /matter explain: Get an explanation of the PR
  • /matter help: Show the list of available commands and documentation
  • Need help? Join our Discord server: https://discord.gg/fJU5DvanU3

narengogi
narengogi previously approved these changes Jun 18, 2025
Comment thread plugins/default/manifest.json
"type": "object",
"properties": {
"pairs": {
"type": "json",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"type": "json" is not a valid json schema key afaik, please use "type": "object"
cc: @VisargD

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there are other "type":"json" in the code right?

Co-authored-by: matter-code-review[bot] <150888575+matter-code-review[bot]@users.noreply.github.com>
Co-authored-by: matter-code-review[bot] <150888575+matter-code-review[bot]@users.noreply.github.com>
@matterai-app

matterai-app Bot commented Jun 19, 2025

Copy link
Copy Markdown
Contributor

Important

PR Review Skipped

PR review skipped as per the configuration setting. Run a manually review by commenting /matter review

💡Tips to use Matter AI

Command List

  • /matter summary: Generate AI Summary for the PR
  • /matter review: Generate AI Reviews for the latest commit in the PR
  • /matter review-full: Generate AI Reviews for the complete PR
  • /matter release-notes: Generate AI release-notes for the PR
  • /matter : Chat with your PR with Matter AI Agent
  • /matter remember : Generate AI memories for the PR
  • /matter explain: Get an explanation of the PR
  • /matter help: Show the list of available commands and documentation
  • Need help? Join our Discord server: https://discord.gg/fJU5DvanU3

1 similar comment
@matterai-app

matterai-app Bot commented Jun 19, 2025

Copy link
Copy Markdown
Contributor

Important

PR Review Skipped

PR review skipped as per the configuration setting. Run a manually review by commenting /matter review

💡Tips to use Matter AI

Command List

  • /matter summary: Generate AI Summary for the PR
  • /matter review: Generate AI Reviews for the latest commit in the PR
  • /matter review-full: Generate AI Reviews for the complete PR
  • /matter release-notes: Generate AI release-notes for the PR
  • /matter : Chat with your PR with Matter AI Agent
  • /matter remember : Generate AI memories for the PR
  • /matter explain: Get an explanation of the PR
  • /matter help: Show the list of available commands and documentation
  • Need help? Join our Discord server: https://discord.gg/fJU5DvanU3

vrushankportkey and others added 2 commits June 19, 2025 18:14
Co-authored-by: matter-code-review[bot] <150888575+matter-code-review[bot]@users.noreply.github.com>
Co-authored-by: matter-code-review[bot] <150888575+matter-code-review[bot]@users.noreply.github.com>
@matterai-app

matterai-app Bot commented Jun 19, 2025

Copy link
Copy Markdown
Contributor

Important

PR Review Skipped

PR review skipped as per the configuration setting. Run a manually review by commenting /matter review

💡Tips to use Matter AI

Command List

  • /matter summary: Generate AI Summary for the PR
  • /matter review: Generate AI Reviews for the latest commit in the PR
  • /matter review-full: Generate AI Reviews for the complete PR
  • /matter release-notes: Generate AI release-notes for the PR
  • /matter : Chat with your PR with Matter AI Agent
  • /matter remember : Generate AI memories for the PR
  • /matter explain: Get an explanation of the PR
  • /matter help: Show the list of available commands and documentation
  • Need help? Join our Discord server: https://discord.gg/fJU5DvanU3

1 similar comment
@matterai-app

matterai-app Bot commented Jun 19, 2025

Copy link
Copy Markdown
Contributor

Important

PR Review Skipped

PR review skipped as per the configuration setting. Run a manually review by commenting /matter review

💡Tips to use Matter AI

Command List

  • /matter summary: Generate AI Summary for the PR
  • /matter review: Generate AI Reviews for the latest commit in the PR
  • /matter review-full: Generate AI Reviews for the complete PR
  • /matter release-notes: Generate AI release-notes for the PR
  • /matter : Chat with your PR with Matter AI Agent
  • /matter remember : Generate AI memories for the PR
  • /matter explain: Get an explanation of the PR
  • /matter help: Show the list of available commands and documentation
  • Need help? Join our Discord server: https://discord.gg/fJU5DvanU3

@VisargD

VisargD commented Jun 25, 2025

Copy link
Copy Markdown
Collaborator

Hey! Closing this PR as we have already added requiredMetadataKeys plugin in #1147 . We can extend the same plugin for any new functionality, if requested by the community.

@VisargD VisargD closed this Jun 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants