Skip to content

Allow apt tests to fail#1115

Closed
mishushakov wants to merge 8 commits intomainfrom
mishushakov/optional-apt-tests
Closed

Allow apt tests to fail#1115
mishushakov wants to merge 8 commits intomainfrom
mishushakov/optional-apt-tests

Conversation

@mishushakov
Copy link
Copy Markdown
Member

@mishushakov mishushakov commented Feb 4, 2026

due to flakiness with ubuntu mirror to prevent breaking our CI when services are unavailable


Note

Low Risk
Test-only change that relaxes failure behavior for known-flaky apt install scenarios; main risk is reduced signal if real regressions slip through.

Overview
Marks aptInstall template tests as allowed to fail to avoid CI breakages from flaky Ubuntu mirrors.

In the JS SDK, introduces buildTemplateTestAllowFail in tests/setup.ts that catches build errors and logs [ALLOW_FAIL], and switches aptInstall.test.ts to use it. In the Python SDK, annotates the async and sync apt_install tests with pytest.mark.xfail so failures don’t fail the suite.

Written by Cursor Bugbot for commit d3f0b67. This will update automatically on new commits. Configure here.

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Feb 4, 2026

⚠️ No Changeset found

Latest commit: d3f0b67

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Comment thread packages/js-sdk/tests/template/methods/aptInstall.test.ts Outdated
Replace try-catch blocks that silently swallowed errors with Vitest's
.fails() modifier. This properly reports test results while still
allowing expected failures to pass CI, similar to pytest's xfail.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

Comment thread packages/js-sdk/tests/template/methods/aptInstall.test.ts Outdated
mishushakov and others added 6 commits February 4, 2026 18:15
Add xfail-style test helper similar to pytest's @pytest.mark.xfail:
- If test passes → test passes (ideal outcome)
- If test fails → logs XFAIL with reason, test still passes

This allows apt install tests to fail without breaking CI while still
clearly reporting failures in test output.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Vitest fixtures require object destructuring pattern in the callback.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a reusable allowFail() wrapper that works with any test fixture.
Tests wrapped with allowFail will log failures but not break CI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Vitest requires object destructuring pattern in fixture callbacks.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Vitest fixtures don't support generic wrappers with rest parameters.
Use a specific wrapper function instead.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@mishushakov
Copy link
Copy Markdown
Member Author

We should ideally fix it on the infra, not here

@ValentaTomas ValentaTomas deleted the mishushakov/optional-apt-tests branch March 20, 2026 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant