Skip to content

feat: rewrite cali as qa v2#70

Open
thymikee wants to merge 44 commits intomainfrom
v2
Open

feat: rewrite cali as qa v2#70
thymikee wants to merge 44 commits intomainfrom
v2

Conversation

@thymikee
Copy link
Copy Markdown

@thymikee thymikee commented Mar 30, 2026

Summary

Cali v2 turns the old one-off QA flow into a small CLI platform for mobile agent work.

Public commands in this PR:

  • cali qa
    • ship-ready mobile QA
  • cali review
    • experimental repository review
  • cali perf-review
    • experimental runtime performance investigation
  • cali dev
    • experimental repo-backed implementation flow
  • cali export-ci
    • shared CI export helper for single-platform and combined Android+iOS outputs

Core runtime changes:

  • shared command/runtime/tool-pack architecture
  • one normalized cali-context.json contract across commands
  • safer published reports with sanitized repository metadata
  • guaranteed blocked outputs for qa
  • .app.tar.gz support for iOS artifacts
  • limited agent-device stale-daemon recovery
  • screenshot normalization so captured screenshots land in report outputs reliably
  • explicit model auth only: AI_GATEWAY_API_KEY or Anthropic direct credentials
  • required role skills now auto-install via npx skills into ~/.cali/skills, with fallback to ./.cali/skills

Local and CI modes are now simpler:

  • local mobile runs use --local android|ios
  • GitHub Actions and EAS are detected automatically from the environment
  • --ci github-actions|eas remains only as an override
  • the old mobile-pr / eas-mobile-pr env model is removed

CI-facing entrypoints:

  • cali qa --platform ... --artifact ...
    • in GitHub Actions and EAS, provider context is inferred automatically
  • cali review --context ...
  • cali perf-review --context ... --platform ... --artifact ...
  • cali dev --context ...
  • cali export-ci --report ...
    • exports a single-platform CI contract from one report
  • cali export-ci --android ... --ios ...
    • exports one combined Android+iOS CI contract

export-ci writes:

  • ci-comment.md
  • ci-output.json

ci-output.json is the machine-readable CI summary and includes:

  • single-platform: kind, status, summary, topIssue, screenshots
  • multi-platform: kind, status, summary, topIssue, platforms.android, platforms.ios

Stable file-publisher outputs from qa:

  • report.json
  • section.md
  • status.txt
  • summary.txt
  • top-issue.txt
  • screenshots.md
  • screenshots.json
  • publisher-manifest.json

Why this helps integrators:

  • no separate context-generation step in CI
  • no custom blocked-report fallback generation
  • no custom single- or multi-platform comment renderer when using export-ci
  • local mode is one flag instead of named env presets
  • CI jobs can just build the artifact, boot the target, run cali, and optionally post ci-comment.md
  • GitHub posting stays outside Cali as a short workflow concern

# cali

Cali is an AI agent that helps you build React Native apps. It takes all the utilities and functions of a React Native CLI and exposes them as tools to an LLM.
Cali v2 is a QA-oriented CLI for mobile app review runs. The first shipped role is `cali qa`, which splits deterministic bootstrap from the agent phase and standardizes the resulting QA report.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That intro could be a bit more descriptive

cali qa \
--preset local-ios \
--artifact ./artifacts/MyApp.app \
--app-id com.example.myapp \
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is app-id required?
is artifact required, if we use EAS? (what about app.json)

--preset local-ios \
--artifact ./artifacts/MyApp.app \
--app-id com.example.myapp \
--device "iPhone 16" \
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could also be inferred I guess?


Special thanks to [@jedirandy](https://github.com/jedirandy) for donating the name `cali` on `npm`!
```bash
cali qa \
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be great to retain the interactive mode (so instead of qa, to have a conversation). This could open up opportunities to ask follow-up questions or clarify flows (we can add that in the future)

- presets: `eas-mobile-pr`, `local-android`, `local-ios`
- environment adapters: EAS env, local flags, JSON context
- tool packs: `skills`, `agent-device`
- publishers: `blob`, `file`
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to explain what is environment adapter, tool pack and publisher

const hasWrittenReport = hasToolActivity(steps, 'write_report')
const hasUsedDeviceTools = hasToolActivity(steps, 'agent_device')

if (hasWrittenReport || !hasUsedDeviceTools || stepNumber < 6) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why stepNumber < 6?

Copy link
Copy Markdown
Collaborator

@grabbou grabbou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants