Skip to content

Commit fc6f5ff

Browse files
committed
Initial commit: @perryts/mysql v0.1.0
Pure-TypeScript MySQL/MariaDB wire-protocol driver. Runs on Node.js, Bun, and AOT-compiles to a native binary via Perry. - M1: Packet framing + lenenc codecs - M2: Handshake + COM_QUERY - M3: Auth plugins (native, caching_sha2, sha256, clear, ed25519) + auth-switch - M4: Prepared protocol (PREPARE/EXECUTE/CLOSE) + LRU cache - M5: 23 type codecs + Decimal, MyDate/MyTime/MyDateTime - M6: TLS, Pool, cancel via KILL QUERY, sql`` template, URL/env config - M7: Multi-resultset, LOCAL_INFILE refusal, benchmark harness 138 passing unit + integration tests, Node TLS suite, Docker matrix (MySQL 8 + MariaDB 11), benchmarks vs mysql2 in bench/RESULTS.md.
0 parents  commit fc6f5ff

93 files changed

Lines changed: 10707 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
node_modules/
2+
bench/node_modules/
3+
dist/
4+
*.log
5+
.DS_Store
6+
.env
7+
.env.local
8+
*.tsbuildinfo

CHANGELOG.md

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# Changelog
2+
3+
## v0.1 — initial release
4+
5+
Milestones M1 → M7, all shipped:
6+
7+
- **M1 — Framing + low-level codecs.** 3-byte length + 1-byte seq id packet
8+
framing, >16 MB continuation handling, length-encoded integer + string
9+
codecs, little-endian buffer cursor, NULL-bitmap helpers (with the
10+
server→client +2 reserved-bit offset baked in).
11+
- **M2 — Handshake + text query.** `HandshakeV10` decoding,
12+
`HandshakeResponse41` building, `COM_QUERY` / `COM_QUIT` / `COM_PING`,
13+
structured `MyError` (errno + SQLSTATE + message), cross-runtime
14+
socket adapter (Node, Bun, Perry).
15+
- **M3 — Auth plugins.** Dispatcher handling `AuthSwitchRequest` and
16+
`AuthMoreData` mid-handshake. Built-in plugins:
17+
`mysql_native_password`, `caching_sha2_password` (with RSA-OAEP-SHA1
18+
public-key retrieval, gated by `allowPublicKeyRetrieval`),
19+
`sha256_password`, `mysql_clear_password` (TLS-gated), MariaDB
20+
`client_ed25519`. Custom plugins pluggable via `registerAuthPlugin`.
21+
- **M4 — Prepared protocol.** `COM_STMT_PREPARE` / `COM_STMT_EXECUTE` /
22+
`COM_STMT_CLOSE` / `COM_STMT_RESET` / `COM_STMT_SEND_LONG_DATA`.
23+
Per-connection prepared-statement cache keyed on SQL text.
24+
Binary-resultset decoder with the +2 server→client NULL-bitmap offset.
25+
- **M5 — Type codec registry + 23 codecs.** Parallel-array registry
26+
keyed on MYSQL_TYPE_* + column flags. Built-in codecs for TINY,
27+
SHORT, LONG, LONGLONG, INT24, FLOAT, DOUBLE, NEWDECIMAL (+ legacy
28+
DECIMAL), NULL, YEAR, BIT, VAR_STRING, STRING, VARCHAR, BLOB (+ TINY,
29+
MEDIUM, LONG), ENUM, SET, GEOMETRY, JSON, DATE, DATETIME, TIMESTAMP,
30+
TIME. `Decimal` wrapper (string-backed, lossless). `MyDate`,
31+
`MyDateTime`, `MyTime` wrappers preserving microsecond precision.
32+
Callers can override codecs via `registerType()`.
33+
- **M6 — TLS, pool, cancel, sql tag, URL/env resolution.**
34+
`SSLRequest` + mid-stream TLS upgrade (`sslmode`:
35+
`disable` / `require` / `verify-ca` / `verify-full`) with a single
36+
Perry-vs-Node adapter. `Pool` with idle / acquire timeouts and a
37+
`withConnection` / `transaction` surface. `Connection.cancel()` opens
38+
a second connection and runs `KILL QUERY <connection_id>`.
39+
`sql\`\`` tagged template with `?` placeholders and `raw()` for
40+
identifiers. `parseConnectionString` for `mysql://` / `mariadb://`
41+
DSNs with `ssl-mode`, `charset`, `allowPublicKeyRetrieval` query
42+
params. `resolveConnectOptions` for libmysqlclient-style env
43+
precedence (`MYSQL_HOST`, `MYSQL_TCP_PORT`, `MYSQL_USER`, `MYSQL_PWD`,
44+
`MYSQL_DATABASE`, `MYSQL_SSL_MODE`).
45+
- **M7 — Multi-resultset, LOCAL_INFILE guard, polish.**
46+
Multi-statement / multi-resultset handling gated on
47+
`multipleStatements: true`; each set surfaces on
48+
`QueryResult.resultSets`. `LOCAL INFILE` requests default to a clean
49+
`MyError` refusal with the requested filename in the message.
50+
Benchmark harness + shared workloads against `mysql2` / `mysql`.
51+
Additional smoke examples (`perry-smoke-prepared`, `perry-smoke-tls`,
52+
`select-one`).
53+
54+
### Polish after M7
55+
56+
- **Positive-path TLS integration tests.** `tests-node/tls-node-tests.ts`
57+
drives full `SSLRequest` → TLS handshake → `HandshakeResponse41`
58+
round-trips against an in-process mock server with a throwaway
59+
self-signed cert. Runs under `node --import tsx --test` (Bun 1.3's
60+
`tls.connect({socket})` stalls; negative paths still covered under
61+
`bun test`).
62+
- **Cancel integration test.** Mock server supports `simulatedSleepMs`
63+
+ `KILL QUERY <id>` relay between connections; verifies
64+
`conn.cancel()` returns errno 1317 / SQLSTATE 70100 on the target.
65+
- **Bench harness vs mysql2 and legacy mysql.** `bench/bench-mysql2.ts`
66+
and `bench/bench-mysql.ts` mirror `bench-this.ts`; `bench/package.json`
67+
keeps comparison drivers out of the main package.
68+
- **Docker real-server matrix.** `scripts/test-real.sh` brings up MySQL 8
69+
+ MariaDB 11 via `docker-compose.yml`, runs
70+
`tests/integration/real-server.test.ts` against both (skipped by
71+
default without `MYSQL_REAL=1`).
72+
- **Warning event hook.** `conn.on('warning', cb)` fires a summary
73+
`MyWarning` whenever a query returns `warningCount > 0`.
74+
- **`scripts/verify.sh`** — one-shot gate: typecheck → bun test → node
75+
TLS → build.
76+
77+
### Bug fixes found via real-server testing (MySQL 8.0.45)
78+
79+
- **HandshakeResponse41 caps mismatch over TLS.** The SSLRequest
80+
advertised `CLIENT_CONNECT_ATTRS` while `HandshakeResponse41` (built
81+
after the upgrade) stripped it when no attrs were supplied. MySQL
82+
cross-checks the two and rejected with errno 1043 "Bad handshake".
83+
Now both frames are computed from a single `initialCaps` up front,
84+
guaranteeing exact equality.
85+
- **Prepared INSERT/UPDATE/DELETE hung.** `COM_STMT_EXECUTE` responses
86+
for statements that return no resultset are a single OK packet, but
87+
the driver jumped straight to the row-collection phase and tried to
88+
decode the OK as a binary row. Fixed by always entering the exec
89+
response through the column-count phase — its first-packet branch
90+
correctly recognises the OK.
91+
- **`Buffer.isBuffer()` not lowered by Perry AOT codegen.** Replaced
92+
with a duck-type check (`typeof v.readUInt8 === 'function' &&
93+
typeof v.length === 'number'`) so the driver source compiles cleanly
94+
under `perry compile`.
95+
96+
### Perry AOT status
97+
98+
- `perry check --check-deps examples/perry-aot-smoke.ts` passes.
99+
- `perry compile examples/perry-aot-smoke.ts` produces a 3.6 MB arm64
100+
Mach-O binary.
101+
- **Full end-to-end connect + text query + prepared query + close runs
102+
natively** against a real MySQL 8.0.45 server with the driver source
103+
identical across all three runtimes (Node, Bun, Perry AOT). Confirmed
104+
on perry 0.5.99.
105+
- Bringing the AOT target up surfaced eight Perry compiler issues —
106+
#78, #79, #80, #81, #82 (fixed in 0.5.95); #85 (fixed in 0.5.97); #87,
107+
#88 (fixed in 0.5.98); #91 (fixed in 0.5.99). All driver-side
108+
workarounds were removed once the upstream fixes landed.
109+
- The one driver fix that stuck (and would have been correct against
110+
every Node/Bun version too) was switching `raw[raw.length - 1]` to
111+
`raw.readUInt8(raw.length - 1)` in `decoder.decodeHandshakeV10`
112+
Perry-stdlib's Buffer doesn't lower bracket indexing, and the
113+
spurious truthiness left a stray NUL in the auth challenge that
114+
silently corrupted the SCRAM scramble.

CLAUDE.md

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# @perryts/mysql
2+
3+
Pure-TypeScript MySQL / MariaDB wire-protocol driver. Sibling of **Tusk**
4+
(the GUI that consumes it) and of **@perryts/postgres** (the other
5+
Perry-showcase driver). Published independently as `@perryts/mysql` and
6+
usable by any Perry or Node.js program that wants to talk to MySQL 8,
7+
MariaDB 11, or compatible servers.
8+
9+
## Positioning
10+
11+
- **Showcase of Perry's systems-programming capability.** No native crate
12+
in this package; all capabilities come from perry-stdlib
13+
(`net.Socket`, `tls.connect`, `socket.upgradeToTLS`, `crypto.*`, `Buffer`).
14+
- **Runs unchanged on Node.js / Bun.** The only API surface that differs
15+
between Perry and Node is TLS upgrade; a one-function adapter handles it.
16+
Everything else (Buffer, crypto, net.Socket events, little-endian reads)
17+
is Node-compatible by construction.
18+
- **Shaped for a GUI**, not an ORM. Returns raw rows plus full column
19+
metadata (type code, flags, collation, lengths, table/org-table,
20+
name/org-name). Exposes warnings, structured ErrPacket fields, server
21+
connection id, server version.
22+
23+
## Architecture
24+
25+
```
26+
TypeScript driver
27+
28+
├── src/protocol/ 3-byte+seq framing, lenenc codecs, writer/reader/decoder (M1, M2)
29+
├── src/auth/ mysql_native_password, caching_sha2_password,
30+
│ sha256_password, mysql_clear_password, client_ed25519 (M3)
31+
├── src/transport/ TCP socket adapter + TLS upgrade (M2)
32+
├── src/types/ type-code → codec registry (M5)
33+
├── src/error.ts structured MyError (errno, sqlState, message) (M2)
34+
├── src/warnings.ts MyWarning (SHOW WARNINGS surface) (M7)
35+
├── src/cancel.ts second-connection + KILL QUERY (M6)
36+
├── src/connection.ts Connection: lifecycle, text + prepared queries (M2, M4)
37+
└── src/index.ts public barrel exports
38+
```
39+
40+
## Milestones (mirror the plan at /Users/amlug/.claude/plans/akin-to-postgres-we-immutable-knuth.md)
41+
42+
- **M1** — Packet framing, lenenc codecs, buffer-cursor, null-bitmap. Unit tests green.
43+
- **M2** — TCP + HandshakeV10 + HandshakeResponse41 + COM_QUERY (`SELECT 1`).
44+
- **M3** — Auth plugins + auth-switch dispatcher (native, caching_sha2, sha256, clear, ed25519).
45+
- **M4** — Extended (prepared) protocol: COM_STMT_PREPARE/EXECUTE/CLOSE + binary rows + LRU cache.
46+
- **M5** — 20 type codecs (binary + text) + rich wrappers (Decimal, MyDate/MyTime/MyDateTime).
47+
- **M6** — TLS via SSLRequest + sslmode + pool + cancel via KILL QUERY + sql`` template + URL/env.
48+
- **M7** — Multi-resultset, LOCAL_INFILE guard, SHOW WARNINGS auto-fetch, docker matrix green.
49+
50+
## Node.js compatibility contract
51+
52+
Code under `src/` must only use APIs available both in Perry's stdlib and
53+
Node.js core:
54+
55+
- `Buffer` (same API on both), `Buffer.concat`, little-endian read/write.
56+
- `net.createConnection(host, port)` with `.on('connect'|'data'|'error'|'close')`.
57+
- `crypto.createHash / createHmac / publicEncrypt / sign / randomBytes / ...`.
58+
59+
The one divergence is TLS upgrade:
60+
61+
- Perry: `socket.upgradeToTLS(servername, verify)` returns a Promise.
62+
- Node: `tls.connect({ socket, servername, rejectUnauthorized })` returns a new TLSSocket.
63+
64+
`src/transport/upgrade-tls.ts` is a ~15-line adapter that feature-detects
65+
and picks the right path. No other code in the driver needs to know about
66+
the difference.
67+
68+
## Perry AOT constraints (apply to all source files)
69+
70+
Per the hone CLAUDE.md conventions:
71+
72+
- No `?.` optional chaining — use explicit `if (x === undefined)` / `if (x === null)`.
73+
- No `??` nullish coalescing — use explicit branching.
74+
- No `obj[variable]` dynamic key access — use `if/else if` or switch.
75+
- No `/regex/.test()` — use `indexOf` or char-code checks.
76+
- No `{ key }` ES6 shorthand — write `{ key: key }`.
77+
- No `for...of` on arrays — use `for (let i = 0; i < arr.length; i++)`.
78+
- No `setTimeout` self-recursion — use `setInterval`.
79+
- No closures capturing instance methods as `this.method` — store state in
80+
module-level `Map<id, State>` and use named module-level handlers.
81+
- No `Buffer[i]` bracket indexing — Perry's codegen doesn't lower it, the
82+
read returns undefined and scans walk off the end. Use `buf.readUInt8(i)`.
83+
84+
## MySQL-specific gotchas
85+
86+
- **Little-endian everything.** MySQL is LE; `@perryts/postgres` is BE. Don't
87+
share `BufferCursor` between the two drivers.
88+
- **3-byte length + 1-byte seq id** per packet, max 16 MB payload. A payload
89+
of exactly 16 MB triggers a trailing zero-length continuation packet.
90+
`seq = (seq + 1) & 0xFF` must wrap every 256 packets.
91+
- **0xFE is ambiguous.** (a) EOF packet (≤9-byte payload, old-style), (b)
92+
AuthSwitchRequest (during auth state), (c) LOCAL_INFILE request (during
93+
resultset state, payload is a filename). Disambiguate by (state, payload
94+
length).
95+
- **Binary-row NULL bitmap has a +2 offset.** Only for server→client rows;
96+
param bitmaps for `COM_STMT_EXECUTE` do not. Two separate helpers.
97+
- **Server speaks first.** HandshakeV10 arrives before the client writes
98+
anything. State machine is inverted from Postgres.
99+
- **Auth plugins can switch mid-handshake.** The server may reply with an
100+
`AuthSwitchRequest (0xFE)` asking for a different plugin's response.
101+
Dispatcher lives in `auth/dispatcher.ts`.
102+
- **`caching_sha2_password` full-auth over plain TCP** needs RSA pubkey
103+
retrieval. Gate behind `allowPublicKeyRetrieval: boolean` (default false,
104+
matches JDBC).
105+
- **Empty-password `mysql_native_password`.** The auth response field is
106+
zero bytes long (NOT 20 zero bytes). Common bug.
107+
- **`?` placeholders**, not `$N`. `sql.ts` doesn't need renumbering.
108+
109+
## Testing
110+
111+
```bash
112+
bun test # all tests
113+
bun test tests/unit # pure unit tests (no DB)
114+
bun test tests/integration # mock-server + docker-compose matrix
115+
```

LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026 Skelpo
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

README.md

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
# @perryts/mysql
2+
3+
Pure-TypeScript MySQL / MariaDB wire-protocol driver. Runs on Node.js and
4+
Bun, and ahead-of-time compiles to a native binary via
5+
[Perry](https://github.com/perryts/perry) (LLVM). Zero native dependencies.
6+
7+
Sibling package of
8+
[@perryts/postgres](https://github.com/perryts/postgres).
9+
10+
## Status
11+
12+
v0.1 — **all milestones shipped**:
13+
14+
- [x] M1: Packet framing (3-byte length + seq id, >16 MB chains) + lenenc codecs
15+
- [x] M2: HandshakeV10 + HandshakeResponse41 + `COM_QUERY`
16+
- [x] M3: Auth plugins (`mysql_native_password`, `caching_sha2_password`,
17+
`sha256_password`, `mysql_clear_password`, MariaDB `client_ed25519`)
18+
+ mid-handshake auth-switch
19+
- [x] M4: Prepared protocol: `COM_STMT_PREPARE` / `EXECUTE` / `CLOSE` +
20+
per-connection prepared cache + binary-resultset decoder
21+
- [x] M5: Rich type codecs — `Decimal`, `MyDate` / `MyTime` / `MyDateTime`,
22+
23 built-ins + `registerType()` extension point
23+
- [x] M6: TLS (`SSLRequest` + `sslmode`), `Pool`, `Connection.cancel()`
24+
via `KILL QUERY`, `sql\`\`` template, `parseConnectionString`,
25+
`resolveConnectOptions`
26+
- [x] M7: Multi-resultset (`resultSets` on `QueryResult`), LOCAL_INFILE
27+
refusal by default, benchmark harness, smoke examples
28+
29+
## Quickstart
30+
31+
```ts
32+
import { connect } from '@perryts/mysql';
33+
34+
const conn = await connect({
35+
host: '127.0.0.1',
36+
port: 3306,
37+
user: 'root',
38+
password: 'secret',
39+
database: 'test',
40+
});
41+
42+
const result = await conn.query('SELECT ? + ? AS sum', [40, 2]);
43+
console.log(result.rows); // [ { sum: 42 } ]
44+
45+
await conn.close();
46+
```
47+
48+
## Testing
49+
50+
```sh
51+
bun run verify # typecheck + bun test + node TLS tests + build
52+
bun test # 135 unit + integration tests against mock server
53+
bun run test:tls:node # TLS mid-stream upgrade under Node (Bun stalls on tls.connect({socket}))
54+
bun run test:real # docker MySQL 8 + MariaDB 11 integration matrix
55+
```
56+
57+
## Benchmarks
58+
59+
```sh
60+
(cd bench && bun install) # pulls mysql2 / mysql into bench/node_modules only
61+
MYSQL_HOST=127.0.0.1 MYSQL_TCP_PORT=33306 MYSQL_USER=... \
62+
bash bench/run-all.sh # runs @perryts/mysql, mysql2, mysql legacy side by side
63+
```
64+
65+
## Design notes
66+
67+
- No `pg`-style OIDs: MySQL identifies types via a single `MYSQL_TYPE_*`
68+
byte + a 2-byte flag field. Codecs key on those.
69+
- Binary protocol for parameterised queries, text protocol for parameter-less.
70+
- Rows returned as `{ fields, rows, rowsArray, rowsRaw, command, rowCount }`
71+
(raw bytes for GUI consumers + decoded objects for ergonomics).
72+
- Zero dependencies at runtime. `mysql2` / `mysql` are dev-only (bench only).
73+
74+
## License
75+
76+
MIT.

bench/README.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Benchmarks
2+
3+
Compares `@perryts/mysql` against `mysql2` and the legacy `mysql` driver.
4+
5+
## Setup
6+
7+
```sh
8+
docker compose -f ../tests/integration/docker-compose.yml up -d
9+
# Create the benchmark tables (see workloads.ts for schema expectations):
10+
mysql -h 127.0.0.1 -P 33306 -uroot -prootpw perry_test < seed.sql
11+
```
12+
13+
## Run
14+
15+
```sh
16+
MYSQL_HOST=127.0.0.1 MYSQL_TCP_PORT=33306 \
17+
MYSQL_USER=root MYSQL_PASSWORD=rootpw MYSQL_DATABASE=perry_test \
18+
bun bench/bench-this.ts
19+
```
20+
21+
Per-workload stats are printed as `min / median / mean / p95 / max` in ms.
22+
23+
## Workloads
24+
25+
- **tiny**`SELECT 1`, text protocol. Measures fixed overhead.
26+
- **param-1row**`SELECT ?`, prepared protocol, one row. Measures
27+
Parse / Execute / Sync fixed cost.
28+
- **medium-1k-x-20** — 1 000 rows × 20 columns. Measures per-row decode.
29+
- **large-10k-x-20** — 10 000 rows × 20 columns. Measures bulk throughput.

0 commit comments

Comments
 (0)