Re-enable integration test trigger and route cross-org dispatch through emu-access (#5034)

## Summary

- Reverts the intent of #4899 (temporary stub) and restores automatic
integration test triggering. The original PR could not simply be
reverted (as initially intended) because of the new distinction between
runners with cross-org access and ones with same-org access.
- Both upstream blockers are now resolved:
- **eng-dev-ecosystem side:**
[databricks-eng/eng-dev-ecosystem#1252](https://github.com/databricks-eng/eng-dev-ecosystem/pull/1252?timeline_per_page=5)
moved `mark-as-pending` / `mark-as-success` / `mark-as-failure` onto
`linux-ubuntu-latest-ghec-access` runners so cross-org check-run writes
no longer 403 on the `databricks` org allowlist.
- **CLI side (this PR):** the cross-org `gh workflow run` dispatch is
moved onto `linux-ubuntu-latest-emu-access`, following the pattern from
databricks/databricks-sdk-go#1638, so it no longer 403s on the
`databricks-eng` org allowlist.
- The job is split in two:
- `integration-trigger` (deco runners) handles same-org `Integration
Tests` check writes for the PR-skip and merge-group-auto-approve paths,
using the `DECO_TEST_APPROVAL` app token. Testmask-based gating and the
pre-#4899 summaries (`Skipped (changes do not require integration
tests)` / `Auto-approved for merge queue (tests already passed on PR)`)
are restored.
- `trigger-tests` (emu-access runners) mints the `DECO_WORKFLOW_TRIGGER`
token and issues the cross-org `gh workflow run cli-isolated-pr.yml` /
`cli-isolated-nightly.yml` dispatches.
- `integration-trigger-dependabot` is unchanged.

NO_CHANGELOG=true

## Test plan

- [x] On this PR, confirm `integration-trigger` runs on
`databricks-deco-testing-runner-group` and succeeds.
- [x] Confirm `trigger-tests` runs on
`databricks-release-runner-group-emu-access`; `Generate GitHub App
Token` step succeeds (no 403); `Trigger integration tests (pull
request)` dispatches `cli-isolated-pr.yml` on
`databricks-eng/eng-dev-ecosystem` with `pull_request_number` and
`commit_sha` inputs.
- [x] On `databricks-eng/eng-dev-ecosystem`, confirm the dispatched
`cli-isolated-pr` run appears (event: `workflow_dispatch`) and its
`checkout` job uploads the `update-check-action` and `gh-report-action`
artifacts.
- [x] Confirm `mark-as-pending` runs on
`linux-ubuntu-latest-ghec-access` and updates the `Integration Tests`
check on the PR commit to `in_progress`.
- [x] When the `integration-tests-prod` matrix finishes, confirm
`mark-as-success` / `mark-as-failure` updates the check to `success` /
`failure`. (Known separate issue: `integration-tests-prod` on `main` has
been failing due to a Go 1.25.9 toolchain fetch against
`proxy.golang.org`; that is out of scope here.)
- [ ] Merge-queue path: after ready-for-merge, confirm
`integration-trigger` writes the `Auto-approved for merge queue (tests
already passed on PR)` check.
- [ ] Push-to-main path: after merge, confirm a `workflow_dispatch` run
of `cli-isolated-nightly.yml` appears on eng-dev-ecosystem keyed to the
merge commit SHA.

This pull request and its description were written by Isaac.

Author: mihaimitrea-db

.github/workflows/push.yml

@@ -376,25 +376,50 @@ jobs:
             exit 1
           fi
 
-  # Skip integration tests (temporarily disabled).
-  # Creates a passing check for PRs and auto-approves for merge groups.
+  # Trigger integration tests in a separate repository.
+  # Writes the same-org "Integration Tests" check run for skip/auto-approve
+  # paths on deco runners. The cross-org `gh workflow run` dispatch is split
+  # into the sibling `trigger-tests` job so it can run on emu-access runners
+  # that are allowlisted in the databricks-eng org.
   integration-trigger:
+    needs:
+      - testmask
+
     if: >-
       (github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork && github.actor != 'dependabot[bot]') ||
       (github.event_name == 'merge_group')
 
     runs-on:
-      group: databricks-protected-runner-group-large
-      labels: linux-ubuntu-latest-large
+      group: databricks-deco-testing-runner-group
+      labels: ubuntu-latest-deco
 
     permissions:
       checks: write
+      contents: read
+
+    environment: "test-trigger-is"
 
     steps:
+      - name: Generate GitHub App Token (check runs)
+        if: >-
+          (github.event_name == 'merge_group') ||
+          (github.event_name == 'pull_request' && !contains(fromJSON(needs.testmask.outputs.targets), 'test') && !contains(fromJSON(needs.testmask.outputs.targets), 'test-exp-ssh'))
+        id: generate-check-token
+        uses: actions/create-github-app-token@f8d387b68d61c58ab83c6c016672934102569859 # v3.0.0
+        with:
+          app-id: ${{ secrets.DECO_TEST_APPROVAL_APP_ID }}
+          private-key: ${{ secrets.DECO_TEST_APPROVAL_PRIVATE_KEY }}
+          # DECO_TEST_APPROVAL is installed on the databricks org (not databricks-eng).
+          owner: databricks
+          repositories: cli
+
+      # Skip integration tests if the primary "test" target is not triggered by this change.
+      # Use Checks API (not Statuses API) to match the required "Integration Tests" check.
       - name: Skip integration tests (pull request)
-        if: ${{ github.event_name == 'pull_request' }}
+        if: ${{ github.event_name == 'pull_request' && !contains(fromJSON(needs.testmask.outputs.targets), 'test') && !contains(fromJSON(needs.testmask.outputs.targets), 'test-exp-ssh') }}
         uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
         with:
+          github-token: ${{ steps.generate-check-token.outputs.token }}
           script: |
             await github.rest.checks.create({
               owner: context.repo.owner,
@@ -405,14 +430,16 @@ jobs:
               conclusion: 'success',
               output: {
                 title: 'Integration Tests',
-                summary: '⏭️ Skipped (integration test trigger is temporarily disabled)'
+                summary: '⏭️ Skipped (changes do not require integration tests)'
               }
             });
 
+      # Auto-approve for merge group since tests already passed on the PR.
       - name: Auto-approve for merge group
         if: ${{ github.event_name == 'merge_group' }}
         uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
         with:
+          github-token: ${{ steps.generate-check-token.outputs.token }}
           script: |
             await github.rest.checks.create({
               owner: context.repo.owner,
@@ -423,10 +450,59 @@ jobs:
               conclusion: 'success',
               output: {
                 title: 'Integration Tests',
-                summary: '⏭️ Skipped (integration test trigger is temporarily disabled)'
+                summary: '⏭️ Auto-approved for merge queue (tests already passed on PR)'
               }
             });
 
+  # Cross-org dispatch to databricks-eng/eng-dev-ecosystem. Must run on an
+  # emu-access runner because the databricks-eng org IP-allowlists only the
+  # release runner group, not deco. See databricks/databricks-sdk-go#1638.
+  trigger-tests:
+    needs:
+      - testmask
+
+    if: >-
+      (github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork && github.actor != 'dependabot[bot]' && (contains(fromJSON(needs.testmask.outputs.targets), 'test') || contains(fromJSON(needs.testmask.outputs.targets), 'test-exp-ssh'))) ||
+      (github.event_name == 'push')
+
+    runs-on:
+      group: databricks-release-runner-group-emu-access
+      labels: linux-ubuntu-latest-emu-access
+
+    permissions:
+      contents: read
+
+    environment: "test-trigger-is"
+
+    steps:
+      - name: Generate GitHub App Token
+        id: generate-token
+        uses: actions/create-github-app-token@f8d387b68d61c58ab83c6c016672934102569859 # v3.0.0
+        with:
+          app-id: ${{ secrets.DECO_WORKFLOW_TRIGGER_APP_ID }}
+          private-key: ${{ secrets.DECO_WORKFLOW_TRIGGER_PRIVATE_KEY }}
+          owner: ${{ secrets.ORG_NAME }}
+          repositories: ${{ secrets.REPO_NAME }}
+
+      - name: Trigger integration tests (pull request)
+        if: ${{ github.event_name == 'pull_request' }}
+        env:
+          GH_TOKEN: ${{ steps.generate-token.outputs.token }}
+        run: |-
+          gh workflow run cli-isolated-pr.yml -R ${{ secrets.ORG_NAME }}/${{ secrets.REPO_NAME }} \
+            --ref main \
+            -f pull_request_number=${{ github.event.pull_request.number }} \
+            -f commit_sha=${{ github.event.pull_request.head.sha }}
+
+      - name: Trigger integration tests (push to main)
+        if: ${{ github.event_name == 'push' }}
+        env:
+          GH_TOKEN: ${{ steps.generate-token.outputs.token }}
+        run: |-
+          gh workflow run cli-isolated-nightly.yml -R ${{ secrets.ORG_NAME }}/${{ secrets.REPO_NAME }} \
+            --ref main \
+            -f commit_sha=${{ github.event.after }}
+
   # Skip integration tests for dependabot PRs.
   # Dependabot has no access to the "test-trigger-is" environment secrets,
   # so we use the built-in GITHUB_TOKEN to mark the required "Integration