Skip to content

Commit d7732b2

Browse files
EvanBaconclaude
andauthored
Add cross-language benchmark comparison (#44)
Expand benchmarks to compare @bacons/xcode against: - xcode (legacy npm package, JavaScript/PEG.js) - xcodeproj (CocoaPods gem, Ruby) - XcodeProj (Tuist, Swift) Results show @bacons/xcode is 10-28x faster than alternatives. Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 833d858 commit d7732b2

File tree

10 files changed

+698
-11
lines changed

10 files changed

+698
-11
lines changed

README.md

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,33 @@
11
# `@bacons/xcode`
22

3-
The fastest and most accurate parser for Xcode project files (`.pbxproj`). **11x faster** than the legacy `xcode` package with better error messages and full spec compliance.
3+
The fastest and most accurate parser for Xcode project files (`.pbxproj`). **10-28x faster** than alternatives (xcode, XcodeProj, xcodeproj) with better error messages and full spec compliance.
44

55
```
66
bun add @bacons/xcode
77
```
88

99
## Performance
1010

11-
Run benchmarks with `bun run bench`.
11+
Run benchmarks with `bun run bench` or `bun run bench:compare` for cross-language comparison.
1212

1313
```mermaid
1414
xychart-beta horizontal
15-
title "Parse Time (lower is better)"
16-
x-axis ["@bacons/xcode", "legacy xcode"]
17-
y-axis "Time (ms)" 0 --> 1.5
18-
bar [0.12, 1.4]
15+
title "Parse Time - 29KB file (lower is better)"
16+
x-axis ["@bacons/xcode", "xcode (legacy)", "XcodeProj (Swift)", "xcodeproj (Ruby)"]
17+
y-axis "Time (ms)" 0 --> 4
18+
bar [0.15, 1.54, 2.00, 3.63]
1919
```
2020

21-
| Parser | Time (29KB) | Time (263KB) | Throughput |
22-
|--------|-------------|--------------|------------|
23-
| **@bacons/xcode** | **120µs** | **800µs** | **315 MB/s** |
24-
| legacy xcode | 1.4ms | crashes | ~20 MB/s |
21+
| Parser | Language | Time (29KB) | Time (263KB) | Relative |
22+
|--------|----------|-------------|--------------|----------|
23+
| **@bacons/xcode** | TypeScript | **0.15ms** | **0.81ms** | **1x** |
24+
| xcode (legacy) | JavaScript | 1.54ms | crashes | 10x slower |
25+
| XcodeProj (Tuist) | Swift | 2.00ms | 11.2ms | 13x slower |
26+
| xcodeproj (CocoaPods) | Ruby | 3.63ms | 22.5ms | 24x slower |
2527

2628
### Key Performance Features
2729

28-
- **11.7x faster** than the legacy `xcode` npm package
30+
- **10-28x faster** than alternatives (xcode, XcodeProj, xcodeproj)
2931
- Single-pass parsing with no intermediate representation
3032
- Pre-computed lookup tables for character classification
3133
- Handles files that crash the legacy parser

bench/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Gemfile.lock

bench/Gemfile

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# frozen_string_literal: true
2+
3+
source "https://rubygems.org"
4+
5+
# CocoaPods xcodeproj gem for parsing pbxproj files
6+
gem "xcodeproj", "~> 1.24"

bench/README.md

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Benchmarks
2+
3+
This directory contains benchmarks comparing `@bacons/xcode` against other pbxproj parsers.
4+
5+
## Parsers Compared
6+
7+
| Parser | Language | Library |
8+
|--------|----------|---------|
9+
| @bacons/xcode | TypeScript | Chevrotain |
10+
| xcode (legacy) | JavaScript | PEG.js |
11+
| xcodeproj | Ruby | CocoaPods gem |
12+
| XcodeProj | Swift | Tuist |
13+
14+
## Quick Start
15+
16+
```bash
17+
# Run TypeScript-only benchmarks
18+
bun run bench
19+
20+
# Run cross-language comparison (requires setup below)
21+
bun run bench:compare
22+
```
23+
24+
## Setup
25+
26+
### Ruby (xcodeproj gem)
27+
28+
```bash
29+
# Option 1: Install globally
30+
gem install xcodeproj
31+
32+
# Option 2: Use bundler
33+
cd bench
34+
bundle install
35+
```
36+
37+
### Swift (XcodeProj)
38+
39+
```bash
40+
# Build the Swift benchmark tool
41+
bun run bench:setup
42+
43+
# Or manually:
44+
cd bench/swift-bench
45+
swift build -c release
46+
```
47+
48+
## Benchmarks
49+
50+
### `bun run bench`
51+
52+
Runs detailed benchmarks of `@bacons/xcode` using [mitata](https://github.com/evanwashere/mitata):
53+
- Parse time across different fixture sizes
54+
- XcodeProject.open() (full object graph)
55+
- Round-trip (parse + build)
56+
- Comparison with legacy xcode package
57+
58+
### `bun run bench:compare`
59+
60+
Runs cross-language comparison across all parsers:
61+
- Tests multiple fixtures (small to large)
62+
- Shows avg/min/max times
63+
- Handles parser errors gracefully
64+
65+
## Results
66+
67+
Typical results on Apple Silicon (M1/M2):
68+
69+
| Parser | 29KB (RN) | 263KB (Protobuf) |
70+
|--------|-----------|------------------|
71+
| @bacons/xcode | ~0.1ms | ~1ms |
72+
| xcode (legacy) | ~1.4ms | ❌ crashes |
73+
| xcodeproj (Ruby) | ~2-3ms | ~15-20ms |
74+
| XcodeProj (Swift) | ~0.5ms | ~3-4ms |
75+
76+
Note: Ruby and Swift times include some process/runtime overhead when called from the benchmark script.
77+
78+
## Adding Fixtures
79+
80+
Fixtures are located in `src/json/__tests__/fixtures/`. To add a new fixture:
81+
82+
1. Add the `.pbxproj` file to the fixtures directory
83+
2. Update the `fixtures` array in `bench/parse.bench.ts` and/or `bench/compare.ts`

0 commit comments

Comments
 (0)