Skip to content

ci(actions): integration tests#430

Open
MarkOSullivan94 wants to merge 2 commits into
mainfrom
392-update-github-actions-ci-to-support-running-integration-tests
Open

ci(actions): integration tests#430
MarkOSullivan94 wants to merge 2 commits into
mainfrom
392-update-github-actions-ci-to-support-running-integration-tests

Conversation

@MarkOSullivan94

@MarkOSullivan94 MarkOSullivan94 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Resolves #392

What

Runs the clerk_flutter integration tests in CI on an iPhone simulator (they were previously skipped because they need a device to run on), and posts a summary of all test results as a PR comment.

The workflow now has three jobs:

  • pr_check (ubuntu, unchanged apart from the reporting step) — formatting, analysis, unit and widget tests. Instead of rendering the test report itself, it now uploads its JSON test results as a reports-unit artifact.
  • integration_test (new, macos-latest) — bootstraps the workspace, boots the newest available iPhone simulator via xcrun simctl, then runs melos run test:integration -- -d <udid> --file-reporter=json:... and uploads the results as a reports-integration artifact.
  • test_report (new) — fans in after both jobs, downloads all report artifacts into one directory, runs dorny/test-reporter so the PR still gets a single combined Test Report check, and posts a test summary comment on the PR.

Test summary comment

.github/scripts/test_report_comment.dart (pure Dart, no dependencies) parses the JSON reports and generates the comment:

  • [!CAUTION] callout when at least one test failed, [!NOTE] when everything passed.
  • A collapsible per-package summary table (test count and success rate).
  • For packages with failures only: a collapsible breakdown listing just the files that contain a failing test, with failing test names highlighted in red — so the comment stays skimmable instead of tabling every test in the repo.

The comment is posted via marocchino/sticky-pull-request-comment, so each push updates the existing comment in place rather than creating a new one. A workflow-level concurrency group cancels superseded in-flight runs on PR branches (runs on main are never cancelled), so an older run can't finish late and overwrite the comment or check with stale results.

Sample of the failure-case rendering: #430 (comment)

Notes

  • Report uploads use if: always() and the report/comment steps run on !cancelled(), so the combined report and comment are still produced when tests fail — which is when they matter most. If a job dies before producing any reports, its upload just warns and the report covers whatever the other job produced.
  • melos 6.3.2 appends args passed after -- to the script command, so the -d/--file-reporter forwarding works without changes to melos.yaml (verified locally).
  • The macOS job compiles the example iOS app from scratch, so expect ~15–25 min on first runs. Caching pods/pub could be a follow-up if it becomes a bottleneck.

🤖 Generated with Claude Code

@github-actions github-actions Bot requested a review from slightfoot July 7, 2026 06:41
@MarkOSullivan94 MarkOSullivan94 added ci/cd Continuous Integration and Continuous Deployment automation tests labels Jul 7, 2026
@MarkOSullivan94

Copy link
Copy Markdown
Collaborator Author

Sample of the automated test summary comment (failure case) — for rendering verification, safe to delete.

Caution

At least one test failed. Please review the test results below.

View Test Results
Package Number of Tests Success Rate
clerk_auth 645 99%
clerk_flutter 3 100%
📂 clerk_auth

test/unit/clerk_auth/persistor_test.dart

Test Result
Persistor.none · initialize completes without error
Persistor.none · terminate completes without error
$\color{red}{\text{Persistor.none · read returns null for any key}}$
Persistor.none · read returns null for different types
Persistor.none · write completes without error
Persistor.none · write does not persist values
Persistor.none · delete completes without error
Persistor.none · is a const instance

test/unit/utils/extensions_test.dart

Test Result
$\color{red}{\text{StringExtension · initials · returns initials from space-delimited words}}$
StringExtension · initials · returns uppercase initials
StringExtension · initials · handles single word
StringExtension · initials · handles empty string
StringExtension · initials · filters out non-alphanumeric starting characters
StringExtension · initials · includes numeric initials
StringExtension · b64decoded · decodes base64 encoded string
StringExtension · b64decoded · decodes base64 with padding
StringExtension · orNullIfEmpty · returns string when not empty
StringExtension · orNullIfEmpty · returns null when empty
MapExtension · toStringMap · converts map to string map
MapExtension · toStringMap · handles empty map
ListExtension · addOrReplaceAll · adds new items to list
ListExtension · addOrReplaceAll · replaces existing items by identifier
ListExtension · removeFirstOrNull · removes and returns first matching item
ListExtension · removeFirstOrNull · returns null when no match found
ListExtension · doesNotContain · returns true when item not in list
ListExtension · doesNotContain · returns false when item in list
DurationExt · isZero returns true for zero duration
DurationExt · isZero returns false for non-zero duration
DurationExt · isNotZero returns true for non-zero duration
DurationExt · isNotZero returns false for zero duration
DateTimeExt · epoch is a const DateTime at epoch
DateTimeExt · utcEpochSeconds creates DateTime from seconds
DateTimeExt · utcEpochMillis creates DateTime from milliseconds
Epoch · add returns DateTime with duration added
Epoch · subtract returns DateTime with duration subtracted
Epoch · compareTo returns negative for non-epoch DateTime
Epoch · difference returns zero for two epochs
Epoch · difference returns negative duration for future DateTime
Epoch · isAfter returns false for future DateTime
Epoch · isBefore returns true for future DateTime
Epoch · isAtSameMomentAs returns true for epoch
Epoch · isAtSameMomentAs returns false for non-epoch
Epoch · date components are correct
Epoch · isUtc returns true
Epoch · timeZoneName returns UTC
Epoch · timeZoneOffset returns zero
Epoch · toIso8601String returns correct format
Epoch · toLocal returns local DateTime at epoch
Epoch · toUtc returns itself
Epoch · weekday returns 0
ObjectIdentity · shortHash returns 5 character hex string
ObjectIdentity · describeIdentity includes hash

@MarkOSullivan94 MarkOSullivan94 marked this pull request as ready for review July 7, 2026 11:43
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

Note

All tests passed.

View Test Results
Package Number of Tests Success Rate
clerk_auth 673 100%
clerk_backend_api 2394 100%
clerk_flutter 788 100%
clerk_flutter_integration 2 100%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd Continuous Integration and Continuous Deployment automation tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update GitHub Actions CI to support running integration tests

1 participant