test: e2e integration tests for ExternalProvider/ExternalModel CRDs

[yossiovadia]

Summary

• New test/e2e-integration/ pytest suite — 32 passing, 7 xfail tests
• All tests send real HTTP traffic through the full gateway stack (Envoy → Kuadrant → BBR → llm-katan)
• Standalone helpers — no MaaS repo imports, portable to any cluster
• Comprehensive README with setup instructions for Kind and OpenShift

What's covered

Category	Tests	Status
Reconciler: resource creation (Service, ServiceEntry, DestinationRule, HTTPRoute)	10	Pass
Reconciler: negative cases (missing provider, missing Secret)	2	Pass
Reconciler: multiple provider types (OpenAI, Anthropic, Vertex)	5	Pass
Auth: negative (no auth, invalid token, fake key, random string → 401/403)	4	Pass
Auth: positive (valid API key → 200, response validation)	4	Pass
Auth: error paths (wrong model name, unsupported path, empty messages, non-existent route)	4	Pass
Lifecycle (delete model → HTTPRoute removed, delete/recreate provider → recovery)	3	Pass
Multi-provider weighted routing	3	xfail (PR #213)
Migration v1alpha1 → v1alpha2	4	xfail (not implemented)

xfail tests

Tests for unimplemented features are marked pytest.mark.xfail(reason="...") with the blocking PR/issue. They run but don't fail the suite. When a feature lands and a test starts passing, pytest reports XPASS — a signal to remove the marker.

Prerequisites

• Full MaaS + BBR stack deployed (local-deploy.sh or OpenShift)
• Kuadrant + Authorino for auth tests
• Gateway-level gateway-default-auth AuthPolicy for negative auth tests
• MaaSModelRef + MaaSAuthPolicy + MaaSSubscription for positive auth tests
• See test/e2e-integration/README.md for complete setup guide

How to run

# Kind (local port-forward)
kubectl port-forward -n istio-system svc/maas-default-gateway-istio 19080:80 &

GATEWAY_HOST="localhost:19080" \
INSECURE_HTTP="true" \
E2E_SKIP_TLS_VERIFY="true" \
E2E_SIMULATOR_ENDPOINT="3-13-21-181.sslip.io" \
E2E_MODEL_NAMESPACE="llm" \
  python -m pytest test/e2e-integration/ -v

# OpenShift
GATEWAY_HOST="$(oc get gateway maas-default-gateway -n openshift-ingress \
  -o jsonpath='{.status.addresses[0].value}')" \
E2E_SIMULATOR_ENDPOINT="3-13-21-181.sslip.io" \
E2E_MODEL_NAMESPACE="llm" \
  python -m pytest test/e2e-integration/ -v

Test plan

• All 32 tests pass on Kind cluster (maas-experimental) — verified locally
• 7 xfail tests correctly fail with expected reasons
• No impact on existing Go E2E tests (test/e2e/)
• README documents all env vars, prerequisites, and differences between Kind and OpenShift

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: yossiovadia

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [yossiovadia]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Warning

Rate limit exceeded

@yossiovadia has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 35 minutes and 38 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: b68a2e2c-710c-4fd1-a691-914d1f66e9f3

📥 Commits

Reviewing files that changed from the base of the PR and between 48b3a23 and 8c82d38.

📒 Files selected for processing (10)

• test/e2e-integration/.gitignore
• test/e2e-integration/README.md
• test/e2e-integration/conftest.py
• test/e2e-integration/helpers.py
• test/e2e-integration/requirements.txt
• test/e2e-integration/test_auth.py
• test/e2e-integration/test_lifecycle.py
• test/e2e-integration/test_migration.py
• test/e2e-integration/test_multiprovider.py
• test/e2e-integration/test_reconciler.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[rhods-ci-bot]

@yossiovadia: The following test has Succeeded:

OCI Artifact Browser URL

View in Artifact Browser

Inspecting Test Artifacts Manually

To inspect your test artifacts manually, follow these steps:

1. Install ORAS (see the ORAS installation guide).
2. Download artifacts with the following commands:

mkdir -p oras-artifacts
cd oras-artifacts
oras pull quay.io/opendatahub/odh-ci-artifacts:ai-gateway-group-test-s4rdf