OLS-3327 - Bedrock tests

[JoaoFula]

Description

add bedrock test files and adaptations to existing scripts

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up dependent library
• Bump-up library or tool used for development (does not change the final image)
• CI configuration change
• Konflux configuration change

Related Tickets & Documents

• Related Issue #
• Closes #

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

Summary by CodeRabbit

• Tests
  • Added new end-to-end coverage for multiple Bedrock-based configurations, including Anthropic, DeepSeek, and tool-calling scenarios.
  • Expanded cluster test runs to exercise both standard and role-based credential setups.
  • Added installation manifests for Bedrock-backed environments with common settings like authentication, logging, and data-collection controls.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign raptorsun for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Warning

Review limit reached

@JoaoFula, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 48 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 080dbc33-d3e1-42b5-8c16-59d65c8d69d4

📥 Commits

Reviewing files that changed from the base of the PR and between 0fe24bf and 13fdbd7.

📒 Files selected for processing (6)

• tests/config/operator_install/olsconfig.crd.bedrock_anthropic.yaml
• tests/config/operator_install/olsconfig.crd.bedrock_deepseek.yaml
• tests/config/operator_install/olsconfig.crd.bedrock_deepseek_tool_calling.yaml
• tests/e2e/utils/ols_installer.py
• tests/scripts/test-e2e-cluster-periodics.sh
• tests/scripts/test-e2e-cluster.sh

📝 Walkthrough

Walkthrough

Adds new Bedrock provider OLSConfig test fixtures for Anthropic, DeepSeek, and DeepSeek tool-calling scenarios. Introduces Bedrock IAM credential secret handling in the e2e installer module with a dedicated helper function. Wires new Bedrock test suites into both cluster e2e periodic and standard test scripts.

Changes

Bedrock e2e test support

Layer / File(s)	Summary
Bedrock IAM credential secret handling tests/e2e/utils/ols_installer.py	Adds _BEDROCK_IAM_ENV_KEYS/_BEDROCK_ROLE_ENV_KEYS constants and ensure_bedrock_iam_secret(creds) helper that reads AWS IAM/STS env vars based on a discriminator, skips creation if vars are missing, and creates the llmcreds secret; create_secrets(...) routes Bedrock providers to this helper and returns early.
Bedrock OLSConfig test fixtures tests/config/operator_install/olsconfig.crd.bedrock_anthropic.yaml, tests/config/operator_install/olsconfig.crd.bedrock_deepseek.yaml, tests/config/operator_install/olsconfig.crd.bedrock_deepseek_tool_calling.yaml	New OLSConfig YAML manifests configuring Bedrock provider with Anthropic/DeepSeek models, credentials reference, replicas, auth/logging, query filters, introspection, and data collection toggles.
Bedrock suite wiring in e2e scripts tests/scripts/test-e2e-cluster-periodics.sh, tests/scripts/test-e2e-cluster.sh	Adds run_suite invocations for Bedrock Anthropic/DeepSeek (iam and iam_role discriminators) plus a DeepSeek tool-calling suite, with comments clarifying PROVIDER_KEY_PATH is a discriminator, not a file path.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Script as e2e cluster script
  participant Installer as ols_installer.py
  participant Env as CI/Vault/Prow env vars
  participant Cluster as OpenShift cluster

  Script->>Installer: create_secrets(provider_name, creds)
  Installer->>Installer: check provider_name starts with "bedrock"
  Installer->>Installer: ensure_bedrock_iam_secret(creds)
  Installer->>Env: read IAM or STS role env keys
  alt required env vars missing
    Installer-->>Script: skip, log message
  else env vars present
    Installer->>Cluster: oc create secret generic llmcreds
    Cluster-->>Installer: secret created (or ignored if existing)
  end

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is concise and accurately reflects the main change: adding Bedrock-related tests and test support.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/e2e/utils/ols_installer.py`:
- Around line 288-291: The Bedrock credential selection in the helper that maps
`creds` to env keys currently treats every non-`iam_role` value as IAM, which
can hide bad discriminator values and break `create_secrets()` recovery. Update
the branching in the Bedrock env-key helper to accept only the supported
credential modes (`iam` and `iam_role`) and raise a clear failure for anything
else, so typos in `PROVIDER_KEY_PATH` do not silently choose the wrong key set.

In `@tests/scripts/test-e2e-cluster-periodics.sh`:
- Around line 75-76: The Bedrock tool-calling run is placed too early in the
periodic suite order, which can leave the tool-calling CR shape active for later
suites. Move the bedrock_deepseek_tool_calling run into the existing
tool_calling section in test-e2e-cluster-periodics.sh, alongside the other
tool-calling suites, so rhoai_vllm, rhelai_vllm, and certificates still run
after the non-tool-calling setup. Use the run_suite calls for
bedrock_deepseek_tool_calling and the surrounding tool_calling section as the
unique markers when relocating it.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d69150a1-6347-4793-b261-2298cffdd1f4

📥 Commits

Reviewing files that changed from the base of the PR and between 3442d5e and 0fe24bf.

📒 Files selected for processing (6)

• tests/config/operator_install/olsconfig.crd.bedrock_anthropic.yaml
• tests/config/operator_install/olsconfig.crd.bedrock_deepseek.yaml
• tests/config/operator_install/olsconfig.crd.bedrock_deepseek_tool_calling.yaml
• tests/e2e/utils/ols_installer.py
• tests/scripts/test-e2e-cluster-periodics.sh
• tests/scripts/test-e2e-cluster.sh

[JoaoFula]

/retest

[openshift-ci]

@JoaoFula: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-ols-cluster	13fdbd7	link	true	/test e2e-ols-cluster

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.