Skip to content

Commit dd2a1f5

Browse files
authored
fix: Update smart E2E selection fallback when job fails (MetaMask#24460)
<!-- Please submit this PR as a draft initially. Do not mark it as "Ready for review" until the template has been completely filled out, and PR status checks have passed at least once. --> ## **Description** <!-- Write a short description of the changes included in this pull request, also include relevant motivation and context. Have in mind the following questions: 1. What is the reason for the change? 2. What is the improvement/solution? --> Fix Smart E2E Selection job failure blocking PR builds and tests by adding a fallback ### Changes: - Add `final-outputs` step with `if: always()` to guarantee outputs are set even if AI analysis fails - Use `|| echo` to prevent script failures from stopping the action - Fix JSON quoting issue using `printf` to preserve `["ALL"]` format - Improve user-facing PR comment when AI is unavailable - Outputs `["ALL"]` with confidence `0` on failure, triggering all applicable tests ### How it works: 1. AI analysis runs and sets outputs 2. If AI fails, the `final-outputs` step catches it and sets fallback values 3. Downstream jobs receive guaranteed outputs and run tests accordingly ## **Related issues** - Failing job: https://github.com/MetaMask/metamask-mobile/actions/runs/20932157126/job/60146624935?pr=22076 ## **Changelog** <!-- If this PR is not End-User-Facing and should not show up in the CHANGELOG, you can choose to either: 1. Write `CHANGELOG entry: null` 2. Label with `no-changelog` If this PR is End-User-Facing, please write a short User-Facing description in the past tense like: `CHANGELOG entry: Added a new tab for users to see their NFTs` `CHANGELOG entry: Fixed a bug that was causing some NFTs to flicker` (This helps the Release Engineer do their job more quickly and accurately) --> CHANGELOG entry: ## **Related issues** Fixes: ## **Manual testing steps** ```gherkin Feature: my feature name Scenario: user [verb for user action] Given [describe expected initial app state] When user [verb for user action] Then [describe expected outcome] ``` ## **Screenshots/Recordings** <!-- If applicable, add screenshots and/or recordings to visualize the before and after of your change. --> ### **Before** <!-- [screenshots/recordings] --> ### **After** <!-- [screenshots/recordings] --> ## **Pre-merge author checklist** - [ ] I’ve followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Mobile Coding Standards](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/CODING_GUIDELINES.md). - [x] I've completed the PR template to the best of my ability - [ ] I’ve included tests if applicable - [ ] I’ve documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [x] I’ve applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. ## **Pre-merge reviewer checklist** - [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Strengthens the `smart-e2e-selection` action to avoid blocking builds when AI analysis fails and to guarantee consumable outputs. > > - New `final-outputs` step (with `if: always()`) sets `ai_e2e_test_tags` (fallback to `["ALL"]`) and passes through `ai_confidence` when analysis fails > - Wraps AI script with `|| echo` so failures don’t break the job; initializes default outputs in `ai-analysis` > - Updates outputs to read from `final-outputs` and always displays them > - PR comment logic: uses `pr_comment.md` if present; otherwise posts a clear fallback message; preserves skip messaging > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit c33e87b. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
1 parent 7e7fe22 commit dd2a1f5

1 file changed

Lines changed: 23 additions & 13 deletions

File tree

.github/actions/smart-e2e-selection/action.yml

Lines changed: 23 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,10 @@ inputs:
2828
outputs:
2929
ai_e2e_test_tags:
3030
description: 'E2E test tags to run (JSON array format)'
31-
value: ${{ steps.ai-analysis.outputs.ai_e2e_test_tags }}
31+
value: ${{ steps.final-outputs.outputs.ai_e2e_test_tags }}
3232
ai_confidence:
3333
description: 'AI confidence score (0-100)'
34-
value: ${{ steps.ai-analysis.outputs.ai_confidence }}
34+
value: ${{ steps.final-outputs.outputs.ai_confidence }}
3535

3636
runs:
3737
using: 'composite'
@@ -130,17 +130,31 @@ runs:
130130
echo "⏭️ Skipping AI analysis - $SKIP_REASON"
131131
else
132132
echo "✅ Running AI analysis for PR #$PR_NUMBER"
133-
# The script will generate the GH output variables
134-
node .github/scripts/e2e-smart-selection.mjs
133+
# The script will generate the GH output variables - don't fail if script errors
134+
node .github/scripts/e2e-smart-selection.mjs || echo "⚠️ AI analysis script failed, using fallback"
135135
fi
136136
137+
- name: Set final outputs with fallback
138+
id: final-outputs
139+
if: always()
140+
shell: bash
141+
run: |
142+
TAGS='${{ steps.ai-analysis.outputs.ai_e2e_test_tags }}'
143+
if [[ -n "$TAGS" ]]; then
144+
printf 'ai_e2e_test_tags=%s\n' "$TAGS" >> "$GITHUB_OUTPUT"
145+
else
146+
echo 'ai_e2e_test_tags=["ALL"]' >> "$GITHUB_OUTPUT"
147+
fi
148+
echo "ai_confidence=${{ steps.ai-analysis.outputs.ai_confidence }}" >> "$GITHUB_OUTPUT"
149+
137150
- name: Display AI Analysis Outputs
151+
if: always()
138152
shell: bash
139153
run: |
140154
echo "📊 Final GitHub Action Outputs:"
141155
echo "================================"
142-
echo "ai_e2e_test_tags: ${{ steps.ai-analysis.outputs.ai_e2e_test_tags }}"
143-
echo "ai_confidence: ${{ steps.ai-analysis.outputs.ai_confidence }}"
156+
echo "ai_e2e_test_tags: ${{ steps.final-outputs.outputs.ai_e2e_test_tags }}"
157+
echo "ai_confidence: ${{ steps.final-outputs.outputs.ai_confidence }}"
144158
echo "================================"
145159
146160
- name: Delete previous comments
@@ -191,14 +205,10 @@ runs:
191205
COMMENT_BODY="⏭️ **Smart E2E selection skipped** - ${SKIP_REASON}
192206
193207
All E2E tests pre-selected."
208+
elif [ -f "$COMMENT_FILE" ]; then
209+
COMMENT_BODY=$(cat "$COMMENT_FILE")
194210
else
195-
# Read analysis results from file
196-
if [ -f "$COMMENT_FILE" ]; then
197-
COMMENT_BODY=$(cat "$COMMENT_FILE")
198-
else
199-
echo "⚠️ PR comment file not found: $COMMENT_FILE - using default message"
200-
COMMENT_BODY="AI analysis completed but results file was not generated."
201-
fi
211+
COMMENT_BODY="⚠️ AI analysis was inconclusive. Selecting all E2E tests as a fallback, if applicable based on changed files."
202212
fi
203213
204214
# Build and post comment

0 commit comments

Comments
 (0)