[NA] [GHA] ci: trim genai/adk test matrix to reduce Vertex AI 429s#7403
Closed
alexkuzmik wants to merge 1 commit into
Closed
[NA] [GHA] ci: trim genai/adk test matrix to reduce Vertex AI 429s#7403alexkuzmik wants to merge 1 commit into
alexkuzmik wants to merge 1 commit into
Conversation
The GenAI and ADK integration suites make live Vertex AI calls against the shared opik-sdk-tests GCP project. Expanding the full 3.10-3.14 matrix (five jobs per suite, two suites running concurrently) re-multiplied those calls and tripped 429 RESOURCE_EXHAUSTED on the shared project's quota. Run only the oldest and newest supported Python versions (3.10, 3.14) for both suites. This still catches version-specific breakage at the compat endpoints while cutting the sustained live-request volume ~60%. The existing max-parallel concurrency cap is kept. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
⏱️ pre-commit per-hook timing
⏭️ 39 skipped (no matching files changed)
|
![baz-reviewer[bot]](https://avatars.githubusercontent.com/in/933528?s=60&v=4){kind=small}
Comment on lines
+38
to
+41
| # full 3.10-3.14 matrix re-multiplied those calls into 429 | ||
| # RESOURCE_EXHAUSTED, so we run only the oldest and newest supported | ||
| # Python versions — enough to catch version-specific breakage at a | ||
| # fraction of the request volume — and keep the concurrency cap. |
Contributor
There was a problem hiding this comment.
Duplicated Vertex AI matrix config
The strategy block for the Vertex AI suites is duplicated in lib-adk-tests.yml, so any matrix or max-parallel: 2 change has to be updated in two places and one suite can drift into lost coverage or 429s — should we extract it into a reusable workflow call or YAML anchor?
Want Baz to fix this for you? Activate Fixer
Collaborator
Author
|
Superseded by #7405, which folds this matrix trim together with the Sonnet model switch and the removal of the broken genai rate-limit retry code. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details
The GenAI and ADK integration suites make live Vertex AI calls against the shared
opik-sdk-testsGCP project. Expanding the full 3.10–3.14 matrix (five jobs per suite, and the two suites run concurrently) re-multiplied those live calls and tripped429 RESOURCE_EXHAUSTEDon the shared project's quota — surfacing as flaky failures unrelated to the code under test.lib-genai-tests.yml/lib-adk-tests.yml: run only the oldest and newest supported Python versions (3.10,3.14) instead of the fullvars.PYTHON_VERSIONSlist.max-parallel: 2concurrency cap is kept.Change checklist
Issues
AI-WATERMARK
AI-WATERMARK: yes
Testing
python -c "import yaml; ..."on both workflow files → they parse and resolvepython_versionto['3.10', '3.14'].genaiandadklanes now schedule 2 jobs each instead of 5.Not applicable: no library code changed, so no unit/integration tests were run locally.
Documentation
N/A — CI-only change, no user-facing or API surface.