refactor: replace custom qa runner with cali

[thymikee]

Summary

Replace the bespoke QA agent implementation with cali@0.4.0-0 and keep the EAS workflow integration slim.

• swap the custom AI SDK runner for cali qa
• generate a cali-context.json in the workflow wrapper and export PR comment outputs from the resulting report
• remove obsolete custom runner files and the extra iOS provisioning layer that cali now handles
• trim package dependencies and document the new flow

Validation

• npm install
• ./node_modules/.bin/cali qa --help
• npm run lint
• bash -n ./scripts/agent-qa/run-and-export.sh
• bash -n ./scripts/agent-qa/provision-android-emulator.sh
• node /Users/thymikee/.agents/skills/expo-cicd-workflows/scripts/validate.js .eas/workflows/agent-qa-mobile.yml

[expo]

Agent QA

Platform	Status	Issues
Android	✅ passed	N/A
iOS	✅ passed	N/A

Screenshots

Android	iOS
N/A	N/A

Full Android Report

Android

Status: passed

Lightweight Android QA pass completed on the app home screen. The build launched successfully, the default welcome/home UI rendered, and no visible crash, error, or warning overlays were present during inspection.

Acceptance Criteria

• Summary

Replace the bespoke QA agent implementation with cali@0.4.0-0 and keep the EAS workflow integration slim.

• swap the custom AI SDK runner for cali qa
• generate a cali-context.json in the workflow wrapper and export PR comment outputs from the resulting report
• remove obsolete custom runner files and the extra iOS provisioning layer that cali now handles
• trim package dependencies and document the new flow

Validation

• npm install
• ./node_modules/.bin/cali qa --help
• npm run lint
• bash -n ./scripts/agent-qa/run-and-export.sh
• bash -n ./scripts/agent-qa/provision-android-emulator.sh
• node /Users/thymikee/.agents/skills/expo-cicd-workflows/scripts/validate.js .eas/workflows/agent-qa-mobile.yml

Checked

• Launched the installed Android build and confirmed the app opened to the expected Home screen
• Verified visible starter content: Welcome, Step 1: Try it, Step 2: Explore, and tab bar entries Home/Explore
• Captured screenshot evidence of the initial app state

Issues

• No issues noted.

Screenshots

• App home screen
• App home screen (interactive snapshot)

Next Steps

• If desired, run a deeper interactive pass through the Explore tab and any PR-specific workflow paths not exposed on the landing screen
• Consider adding targeted acceptance criteria for workflow/documentation changes so future QA can validate those surfaces directly

Environment Notes

• Platform: Android
• Device: ci-android
• App id: dev.expo.easagentdevice
• Build ID: 96a1c291-6556-4b2f-a33a-aa713c3b99ed
• Screenshots saved under /home/expo/workingdir/build/artifacts/qa/screenshots

Publishers

• blob: ok (Uploaded 2 screenshots.)
• file: ok

Metadata

• Command: qa
• Workspace: .
• Repository: callstackincubator/eas-agent-device
• Pull Request: #3
• Build ID: 96a1c291-6556-4b2f-a33a-aa713c3b99ed
• Workflow: https://expo.dev/accounts/callstack.io/projects/eas-agent-device/workflows/019d6e8e-59e2-7771-8df1-f5012e86b4f6

Full iOS Report

iOS

Status: passed

Lightweight QA pass completed on the iOS build. The app launched successfully, Home and Explore tabs rendered, and the Explore tab navigation worked. No visible error or warning overlays were observed during inspection.

Acceptance Criteria

• Summary

Replace the bespoke QA agent implementation with cali@0.4.0-0 and keep the EAS workflow integration slim.

• swap the custom AI SDK runner for cali qa
• generate a cali-context.json in the workflow wrapper and export PR comment outputs from the resulting report
• remove obsolete custom runner files and the extra iOS provisioning layer that cali now handles
• trim package dependencies and document the new flow

Validation

• npm install
• ./node_modules/.bin/cali qa --help
• npm run lint
• bash -n ./scripts/agent-qa/run-and-export.sh
• bash -n ./scripts/agent-qa/provision-android-emulator.sh
• node /Users/thymikee/.agents/skills/expo-cicd-workflows/scripts/validate.js .eas/workflows/agent-qa-mobile.yml

Checked

• Confirmed app opens to the Home tab with expected starter content visible.
• Confirmed Explore tab is accessible and switches the screen content.
• Captured screenshots for the Home and Explore states.

Issues

• No issues noted.

Screenshots

• Explore tab state
• Home tab initial state

Next Steps

• No follow-up actions were suggested.

Environment Notes

• Platform: iOS
• Device: iPhone 17
• App id: dev.expo.easagentdevice
• Build ID: e31a1778-121c-4776-aa92-7fd3c82adfb1
• Build artifact path: /var/folders/51/y6phq4kx54bdjkgn5fc44z3m0000gn/T/download_build-extracted-XQGTbd/repacked-9335bb73-645e-4288-8d49-ba582f095373.app

Publishers

• blob: ok (Uploaded 2 screenshots.)
• file: ok

Metadata

• Command: qa
• Workspace: .
• Repository: callstackincubator/eas-agent-device
• Pull Request: #3
• Build ID: e31a1778-121c-4776-aa92-7fd3c82adfb1
• Workflow: https://expo.dev/accounts/callstack.io/projects/eas-agent-device/workflows/019d6e8e-59e2-7771-8df1-f5012e86b4f6

[FiberJW]

Hi! Just chiming in since GitHub notified me that GitHub app installation access tokens are showing up on this PR. I can see it here: JSON report > context > repository > provider/cloneUrl.

Can we strip out that info or redact that URL? It contains a short-lived installation access token that's primarily intended for secure execution environments (EAS servers) and shouldn't be published publicly (and, if so, will be immediately invalidated by GitHub).

Thanks!

[thymikee]

Thanks for flagging that @FiberJW! addressed