Speed up CI from ~11m to under 3m (#96)

## 🥞 Stacked PR
Use this [link](https://github.com/databricks/sdk-js/pull/96/files) to
review incremental changes.
- [**stack/speed-up-ci**](#96)
[[Files changed](https://github.com/databricks/sdk-js/pull/96/files)]

## Summary

Cuts the CI gate wall time from ~10m 49s to ~2m 27s (−77%) by removing
redundant build work between jobs and sharding the lint, typecheck, and
build jobs across four package-name ranges.

## Why

CI on this repo has 60+ packages and was taking over 10 minutes
end-to-end, which is painful for iteration. Two compounding problems:

1. **Every non-build job reran the full build.** Each of `lint`,
`typecheck`, `test`, and `test-browser` ran `npm run build` as its first
step, which builds all 63 packages (~3m 51s on the CI runners). Turbo's
task graph already declares `lint/typecheck/test.dependsOn: ["^build"]`,
which means Turbo will build only the upstream packages each task
actually needs (usually `core`, `databricks`, `auth` — 3 packages,
~30s). The explicit build step was pure duplicate work.

2. **Lint, typecheck, and build were single jobs processing 62 packages
on one runner.** Turbo parallelizes tasks within a job, but a CI runner
has limited cores, so ESLint in particular saturated at ~5m 27s wall
time. Nothing was sharing the load across additional runners.

A Turbo remote cache would solve both by sharing cached task outputs
across jobs and across runs, but the infra to host one isn't set up for
this repo, so this PR takes a simpler route that uses only what we
already have.

## What changed

### Interface changes

- **`npm run checks`** — New root script that runs `turbo run lint
format:check` in a single invocation, so local dev has a one-liner for
both pre-commit checks.

### Behavioral changes

None for anything the SDK exposes. Only CI runtime and parallelism
change.

### Internal changes

- **Removed the redundant `Build packages` step** from the `lint`,
`typecheck`, `test`, and `test-browser` jobs. Each of these tasks has
`dependsOn: ["^build"]` in `turbo.json`, so Turbo now builds only the ~3
upstream packages each task needs. The dedicated `build` job still
exercises a full build of every package.
- **Sharded `lint`, `typecheck`, and `build` across four package-name
ranges** using a GitHub Actions matrix: `[a-c]*`, `[d-l]*`, `[m-r]*`,
`[s-z]*`. The four ranges cover all of a-z, so a new package added under
any letter is picked up automatically. The `build` job additionally
keeps its existing `node-version: ['22', '24']` matrix, so sharding
multiplies it into 8 build jobs.
- **Shard dispatch reuses the existing npm scripts** with the filter
appended as a CLI argument: `npm run <script> --
--filter='@databricks/sdk-${{ matrix.shard }}'`. npm appends args after
`--` verbatim to the script command, so turbo receives
`--filter=<value>` as its own flag before it parses any task-args
separator.

## How is this tested?

Ran the workflow on this branch and measured:

| Stage | Gate wall time |
| --- | --- |
| Before any changes | ~10m 49s |
| After removing redundant build steps | 6m 45s (−38%) |
| After sharding lint only | 5m 40s (−48%) |
| After sharding lint + typecheck + build | **2m 27s (−77%)** |

Latest green run:
https://github.com/databricks/sdk-js/actions/runs/24778267054. The
slowest individual shard is `Lint ([s-z]*)` at ~2m 18s; the remaining
gate time is GitHub Actions runner startup and `npm ci`. Further
reductions would need a remote Turbo cache or more shards (diminishing
returns — startup alone is ~30s).

Also verified locally that the four shards together cover exactly the 62
lintable packages (union of `turbo run lint --filter=...` across the
four ranges equals the full package list).

Author: parthban-db

.github/workflows/ci.yml

@@ -17,10 +17,16 @@ concurrency:
 
 jobs:
   lint:
-    name: Lint
+    name: Lint (${{ matrix.shard }})
     runs-on:
       group: databricks-protected-runner-group
       labels: linux-ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        # Shards partition packages by the first letter after `@databricks/sdk-`.
+        # Together the four ranges cover a-z.
+        shard: ['[a-c]*', '[d-l]*', '[m-r]*', '[s-z]*']
     steps:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
 
@@ -43,20 +49,18 @@ jobs:
       - name: Install dependencies
         run: jf npm ci
 
-      - name: Build packages
-        run: npm run build
-
-      - name: Run ESLint
-        run: npm run lint
-
-      - name: Check formatting
-        run: npm run format:check
+      - name: Run lint and format checks
+        run: npm run checks -- --filter='@databricks/sdk-${{ matrix.shard }}'
 
   typecheck:
-    name: Type Check
+    name: Type Check (${{ matrix.shard }})
     runs-on:
       group: databricks-protected-runner-group
       labels: linux-ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        shard: ['[a-c]*', '[d-l]*', '[m-r]*', '[s-z]*']
     steps:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
 
@@ -79,11 +83,8 @@ jobs:
       - name: Install dependencies
         run: jf npm ci
 
-      - name: Build packages
-        run: npm run build
-
       - name: Run type check
-        run: npm run typecheck
+        run: npm run typecheck -- --filter='@databricks/sdk-${{ matrix.shard }}'
 
   test:
     name: Test (Node.js ${{ matrix.node-version }})
@@ -116,9 +117,6 @@ jobs:
       - name: Install dependencies
         run: jf npm ci
 
-      - name: Build packages
-        run: npm run build
-
       - name: Run tests
         run: npm test
 
@@ -153,24 +151,22 @@ jobs:
       - name: Install dependencies
         run: jf npm ci
 
-      - name: Build packages
-        run: npm run build
-
       - name: Install Playwright browsers
         run: npx playwright install chromium --with-deps
 
       - name: Run browser tests
         run: npm run test:browser
 
   build:
-    name: Build (Node.js ${{ matrix.node-version }})
+    name: Build (Node.js ${{ matrix.node-version }}, ${{ matrix.shard }})
     runs-on:
       group: databricks-protected-runner-group
       labels: linux-ubuntu-latest
     strategy:
       fail-fast: false
       matrix:
         node-version: ['22', '24']
+        shard: ['[a-c]*', '[d-l]*', '[m-r]*', '[s-z]*']
     steps:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
 
@@ -194,7 +190,7 @@ jobs:
         run: jf npm ci
 
       - name: Build
-        run: npm run build
+        run: npm run build -- --filter='@databricks/sdk-${{ matrix.shard }}'
 
   # This job gates merging. It depends on all CI jobs so that branch protection
   # only needs to reference this single check. When adding new jobs, append

package.json

@@ -15,6 +15,7 @@
     "lint:fix": "turbo run lint:fix",
     "format": "turbo run format",
     "format:check": "turbo run format:check",
+    "checks": "turbo run lint format:check",
     "typecheck": "turbo run typecheck",
     "clean": "turbo run clean"
   },