tests+ci: regress guard tranche 3 — tool rows P01/B01/B02/B03/F01/L04 - #437
Merged
Conversation
Adds RegressToolGuardTests, a manual-tagged sibling swift_test compiling the shared CLIIntegrationTests harness, for the regress-matrix rows that need tooling the required gate's deterministic target must not depend on: generator scripts (P01 cold-build heartbeat, B02/B03 XCFramework renders, F01 classified slice error), a Bazel fetch of the fixture's pinned rules (B01, via an explicit PATH pin — bazelisk is invisible in bazel's scrubbed test env), and an iOS simulator (L04's deterministic simctl-terminate crash/respawn trigger, dedicated device index 9). Tool preconditions skip locally but fail under PREVIEWSMCP_REQUIRE_DEDICATED_SIM so CI can never silently skip a row. The new non-required regress-tools job in ci.yml runs the target on the same labels (needs: ci keeps the serial runner working the required gate first) to build a standing flake record; promotion into the required gate is dropping the manual tag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…te accessor /simplify fold: assertRenders/assertFails/cleanSlate/fixture move to RegressRowAsserts (compiled into both guard targets via HARNESS_SRCS) so the two suites cannot drift; the three binary-framework rows share one per-process artifact generation; the fail-on-gate-vs-skip-locally policy gets its general name (RequiredGateEnforcement.enforced) instead of riding the sim-specific flag, which now delegates to it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adversarial review fold: provisionSimulator releases the host-wide sim flock on the RequiredDeviceUnavailable throw path (a leaked LOCK_EX self-deadlocks the remaining iOS rows for 10 minutes each); regress-tools timeout-minutes rises to 90 so a hung test hits bazel's 3600s timeout (step failure -> diagnostics step) before job cancellation would skip it; doc notes for the artifact task's .serialized dependency and CoreSimulatorHygiene's actual lock contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes out guard tranche 3 of the regress matrix: the rows needing tooling the required gate's deterministic target must not depend on, in a new
manual-tagged target plus a non-required CI signal job.The target
//previewsmcp/Tests/CLIIntegrationTests:RegressToolGuardTestscompiles the shared CLI harness (HARNESS_SRCS, including the newRegressRowAssertsrow helpers both guard suites now share) and guards:generate-sources.sh, fixture.buildcleaned per run).PATHincluding/opt/homebrew/bin: bazel's scrubbed test env otherwise hides bazelisk, which is why the existing bazel snapshot test tool-gate skips underbazel test.generate-artifacts.sh).has no iOS simulator slice (available: ios-arm64)error; daemon stays responsive.simctl terminate) is logged as crash Add CI, release workflow, and Homebrew distribution #1, respawns, the first successfulelementscarries the crash notice, and the notice clears on delivery.Tool preconditions skip locally but fail under the gate signal (
RequiredGateEnforcement.enforced, the general name the sim-specific flag now delegates to), so CI can never silently skip a row. iOS rows use dedicated device index 9.The CI job
Non-required
regress-toolsjob runs the target on the same labels (needs: cikeeps the serial runner working the required gate first; a red here never blocks auto-merge) to build a standing flake record. Promotion into the required gate = dropping themanualtag.Verification
CLIIntegrationTests230s).bazel test //...with both tier filters confirmed to exclude themanualtarget; lint exit 0.timeout-minutes: 90so bazel's timeout beats job cancellation and diagnostics still run).🤖 Generated with Claude Code