Skip to content

Commit cef8612

Browse files
committed
perf: optimize CI and test performance
Improve test execution performance and CI throughput for socket-cli's test suite (~48s, 2,819 tests): CI Optimizations: - Increase max-parallel from 4 to 8 for better job distribution - Set test-timeout-minutes to 20 for subprocess-heavy tests - Add commented sharded test job template for future use Test Optimizations: - Enable fileParallelism in vitest.config.mts for better thread utilization - Format test includes for consistency Documentation: - Add Performance Optimization section with timing analysis - Add CI/CD Optimization section with GitHub Actions best practices - Update Build/Dist Structure to reference dedicated docs - Update linting description (remove oxlint reference) - Document test sharding strategy for future improvements
1 parent 7ebca1f commit cef8612

File tree

3 files changed

+80
-23
lines changed

3 files changed

+80
-23
lines changed

.github/workflows/ci.yml

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,36 @@ jobs:
4747
node-versions: ${{ inputs.node-versions || '[20, 22, 24]' }}
4848
os-versions: '["ubuntu-latest", "macos-latest", "windows-latest"]'
4949
fail-fast: false
50+
# Increase parallelization for socket-cli's longer test suite (~48s).
51+
# Default is 4, we increase to 8 for better performance.
52+
max-parallel: 8
53+
# Increase timeout for slower subprocess-heavy tests.
54+
test-timeout-minutes: 20
55+
56+
# Sharded unit tests (optional, enable for faster CI).
57+
# To enable: uncomment this job and set `run-test: false` in the ci job above.
58+
# test-sharded:
59+
# name: Unit Tests (Sharded ${{ matrix.shard }}/3)
60+
# runs-on: ${{ matrix.os }}
61+
# timeout-minutes: 15
62+
# strategy:
63+
# fail-fast: false
64+
# matrix:
65+
# node-version: ${{ fromJSON(inputs.node-versions || '[20, 22, 24]') }}
66+
# os: [ubuntu-latest, macos-latest, windows-latest]
67+
# shard: [1, 2, 3]
68+
# steps:
69+
# - uses: SocketDev/socket-registry/.github/actions/setup-and-install@51be85d39d3b4a42dd9d4712948b9d30a2e04794
70+
# with:
71+
# node-version: ${{ matrix.node-version }}
72+
#
73+
# - name: Build CLI
74+
# working-directory: packages/cli
75+
# run: pnpm run build
76+
#
77+
# - name: Run unit tests (shard ${{ matrix.shard }})
78+
# working-directory: packages/cli
79+
# run: pnpm test:unit --shard=${{ matrix.shard }}/3
5080

5181
e2e:
5282
name: E2E Tests

CLAUDE.md

Lines changed: 46 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -127,32 +127,16 @@ CLI tool for Socket.dev security analysis, built with TypeScript (.mts extension
127127
- TypeScript with tsgo or tsc
128128
- Individual file compilation
129129
- Multiple environment configs
130-
- Dual linting (oxlint + eslint), Biome formatting
130+
- ESLint linting, Biome formatting
131131

132132
### Build/Dist Structure
133133

134-
**MANDATORY**: All packages follow build/dist pattern.
134+
**MANDATORY**: All packages follow build/dist pattern. See [docs/build/build-dist-structure.md](docs/build/build-dist-structure.md) for complete details.
135135

136-
**Philosophy**:
136+
**Quick summary**:
137137
- **`build/`** (gitignored) = Workspace + historical builds archive
138138
- **`dist/`** (tracked) = Blessed canonical releases
139139

140-
**Structure**:
141-
```
142-
packages/<package>/
143-
├── build/ # Gitignored
144-
│ ├── tmp/ # Current build intermediates
145-
│ ├── cache/ # Downloads, source clones
146-
│ └── archive/ # Historical builds
147-
│ ├── YYYY-MM-DD-NNN-description/
148-
│ └── latest/ # Symlink to most recent
149-
└── dist/ # Tracked releases
150-
```
151-
152-
**@socketbin Conventions**:
153-
- **Executables**: `packages/socketbin-cli-<platform>-<arch>/bin/socket` (tracked)
154-
- **Libraries**: `packages/socketbin-cli-ai/dist/` (WASM + JS, gitignored)
155-
156140
### Documentation Hierarchy
157141

158142
**Tier 1**: `/docs/` - Monorepo-wide (architecture, build systems, guides)
@@ -165,6 +149,49 @@ packages/<package>/
165149

166150
Uses Vitest with comprehensive test helpers.
167151

152+
#### Performance Optimization
153+
154+
**Test Execution Times** (measured):
155+
- Unit tests: ~48s (2,819 tests)
156+
- Bottleneck: `cmd-scan-reach.test.mts` (47s, 51 subprocess spawns)
157+
158+
**Optimization Strategies**:
159+
1. **File-level parallelization**: Enabled in vitest.config.mts
160+
2. **CPU-based thread pool**: `maxThreads: os.cpus().length`
161+
3. **Test sharding**: Use `--shard` flag for CI distribution
162+
```bash
163+
pnpm test:unit --shard=1/3 # Run 1st third
164+
pnpm test:unit --shard=2/3 # Run 2nd third
165+
pnpm test:unit --shard=3/3 # Run 3rd third
166+
```
167+
168+
**Future Improvements**:
169+
- Split cmd-scan-reach.test.mts into logical groups (dry-run, validation, execution)
170+
- Implement process pool for CLI subprocess tests
171+
- Reduce per-test subprocess overhead
172+
173+
#### CI/CD Optimization
174+
175+
**GitHub Actions** (socket-cli-specific):
176+
- **max-parallel: 8** - Higher than default (4) for longer test suites
177+
- **test-timeout-minutes: 20** - Accommodates subprocess-heavy tests
178+
- **Matrix**: 3 Node versions × 3 OS platforms = 9 parallel jobs
179+
180+
**Caching Best Practices**:
181+
1. **pnpm store**: Cached via `pnpm/action-setup` (automatic)
182+
2. **node_modules**: Implicitly cached via pnpm store
183+
3. **Build artifacts**: Consider caching `dist/` between test runs
184+
4. **Vitest cache**: `.vitest` directory (already gitignored)
185+
186+
**Test Sharding** (future):
187+
```yaml
188+
strategy:
189+
matrix:
190+
shard: [1, 2, 3]
191+
steps:
192+
- run: pnpm test:unit --shard=${{ matrix.shard }}/3
193+
```
194+
168195
#### Test Helpers
169196
1. **CLI Execution** (`test/helpers/cli-execution.mts`)
170197
- `executeCliCommand()`, `expectCliSuccess()`, `expectCliError()`, `executeCliJson()`

packages/cli/vitest.config.mts

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,10 +22,7 @@ export default defineConfig({
2222
test: {
2323
globals: false,
2424
environment: 'node',
25-
include: [
26-
'test/**/*.test.{mts,ts}',
27-
'src/**/*.test.{mts,ts}',
28-
],
25+
include: ['test/**/*.test.{mts,ts}', 'src/**/*.test.{mts,ts}'],
2926
exclude: [
3027
'**/node_modules/**',
3128
'**/dist/**',
@@ -66,6 +63,9 @@ export default defineConfig({
6663
},
6764
testTimeout: 30_000,
6865
hookTimeout: 30_000,
66+
// Enable file-level parallelization for better performance.
67+
// Large test files (like cmd-scan-reach.test.mts) will run in separate threads.
68+
fileParallelism: true,
6969
coverage: {
7070
provider: 'v8',
7171
reporter: ['text', 'json', 'html', 'lcov', 'clover'],

0 commit comments

Comments
 (0)