Skip to content

feat: add suppressed_params support to BedrockGenerator#1842

Open
u7k4rs6 wants to merge 1 commit into
NVIDIA:mainfrom
u7k4rs6:feat/bedrock-suppressed-params
Open

feat: add suppressed_params support to BedrockGenerator#1842
u7k4rs6 wants to merge 1 commit into
NVIDIA:mainfrom
u7k4rs6:feat/bedrock-suppressed-params

Conversation

@u7k4rs6
Copy link
Copy Markdown

@u7k4rs6 u7k4rs6 commented Jun 5, 2026

Refs #1812

Why added

Issue #1812 surfaced that BedrockGenerator cannot send a Bedrock-compatible inferenceConfig for target models with parameter restrictions: Anthropic Claude 4.x on Bedrock rejects requests that set both temperature and topP. In comments on #1812, @jmartin-tech proposed a model-agnostic mechanism: support suppressed_params on BedrockGenerator the same way OpenAICompatible generators do, so operators can globally suppress any parameter known to be unsupported by their target.

This PR implements that suggestion.

Relationship to PR #1729 (merged)

PR #1729 added configurability at the probe layer (garak/probes/promptinject.py) for which generation_params promptinject's _generator_precall_hook overrides. This PR operates at the generator layer (garak/generators/bedrock.py), providing suppression for any Bedrock target regardless of which probe is running. The two mechanisms compose: #1729 addresses the promptinject-specific override behavior, this PR addresses the broader case where a Bedrock model rejects a parameter combination from any source.

Relationship to PR #1840 (open)

PR #1840 patches the immediate Anthropic case with a model-name-gated conditional in _call_model. This PR is the generic mechanism @jmartin-tech suggested in the #1812 thread. Once it lands, the same outcome is achievable via site config (suppressed_params: [topP]) or a per-model default suppression set, without model-name checks in _call_model. The two approaches aren't mutually exclusive, but the generic mechanism subsumes the special case.

How utilized

In garak.site.yaml:

plugins:
  generators:
    bedrock:
      BedrockGenerator:
        suppressed_params:
          - topP

Then run garak normally:

garak --target_type bedrock \
      --target_name us.anthropic.claude-sonnet-4-6 \
      --probes promptinject.HijackHateHumans \
      --generations 1

The topP field is absent from every inferenceConfig sent to bedrock-runtime.converse, even when downstream hooks (e.g., promptinject._generator_precall_hook) mutate generator.top_p mid-run.

Tests

  • test_inference_config_unchanged_with_empty_suppressed_params: refactored code is output-identical to the original for all four fields.
  • test_suppressed_params_blocks_top_p: topP absent when suppressed_params={"topP"} and top_p is non-None.
  • test_suppression_survives_attribute_mutation: suppression holds after generator.top_p is mutated mid-run, simulating promptinject._generator_precall_hook.
  • test_warn_on_pythonic_suppressed_key: WARNING logged at init when a Pythonic name (top_p) is used instead of the API name (topP).

All 15 tests in tests/generators/test_bedrock.py pass.

Signed-off-by: u7k4rs6 <utkarshbahuguna10@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant