Skip to content

Commit 89e9e35

Browse files
docs(serialization): document Protobuf in API, site, and CHANGELOG
- docs/API.md: Protobuf subsection under runtime:serialization (Schema/parse/build, decoded value shape, proto3 + edition 2023). - site: re-add the Protobuf section to the API reference page, a new Protobuf guide page (/docs/serialization/protobuf) + nav link, and site-meta entries. - CHANGELOG: [Unreleased] Added — pure-JS reflective Protobuf.
1 parent 927e472 commit 89e9e35

6 files changed

Lines changed: 117 additions & 5 deletions

File tree

CHANGELOG.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,17 @@ pre-`0.1.0` and the public API is unstable.
66

77
## [Unreleased]
88

9+
### Added
10+
11+
- **`runtime:serialization``Protobuf`.** A pure-JS, reflective Protobuf
12+
implementation (no native deps, no codegen). `new Protobuf.Schema(proto)`
13+
compiles a `.proto` source string (or a `{ filename: source }` map) at runtime
14+
— proto3 and edition 2023; proto2-only constructs are rejected — and
15+
`schema.parse(messageName, bytes)` / `schema.build(messageName, value)` decode
16+
and encode the binary wire format. Decoded objects use camelCase keys, BigInt
17+
for 64-bit ints, enum value-names, and `Uint8Array` for bytes; unknown fields
18+
are preserved across re-encode. Byte-for-byte verified against protobuf-es.
19+
920
### Changed
1021

1122
- **`runtime:parsers` renamed to `runtime:serialization`** (breaking). One module

docs/API.md

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -484,18 +484,25 @@ pub/sub topics are follow-ups (D29).
484484
485485
## `runtime:serialization`
486486
487-
A high-performance parsing and serialization module for structured data formats: XML, YAML, TOML, JSONL, and MessagePack. These parsers are backed by optimized Rust implementations and are exposed via zero-cost host boundaries.
487+
A high-performance parsing and serialization module for structured data formats: XML, YAML, TOML, JSONL, MessagePack, and Protobuf. The text/binary parsers are backed by optimized Rust implementations; Protobuf is a pure-JS reflective implementation. All are exposed via zero-cost host boundaries.
488488
489489
- **Capability:** None (pure computation)
490490
- **Status:** Available
491491
492492
```js
493-
import { XML, YAML, TOML, MessagePack } from "runtime:serialization";
493+
import { XML, YAML, TOML, MessagePack, Protobuf } from "runtime:serialization";
494494

495495
const obj = XML.parse("<root><hello>world</hello></root>");
496496
const yaml = YAML.parse("hello: world");
497497
const msgpackBytes = new Uint8Array([0x81, 0xa5, 0x68, 0x65, 0x6c, 0x6c, 0x6f, 0xa5, 0x77, 0x6f, 0x72, 0x6c, 0x64]);
498498
const obj2 = MessagePack.decode(msgpackBytes);
499+
500+
const schema = new Protobuf.Schema(`
501+
syntax = "proto3";
502+
message Hello { string name = 1; }
503+
`);
504+
const pbBytes = schema.build("Hello", { name: "world" });
505+
const pbObj = schema.parse("Hello", pbBytes); // { name: "world" }
499506
```
500507
501508
### Exports
@@ -529,6 +536,16 @@ For XML, it also provides a `DecoderStream`:
529536
| --- | --- |
530537
| `new XML.DecoderStream()` | A `TransformStream` that parses XML chunks. |
531538
539+
For Protobuf, schemas are compiled from `.proto` source at runtime (pure JS, reflective — proto3 and edition 2023; proto2-only constructs are rejected):
540+
541+
| Export | Description |
542+
| --- | --- |
543+
| `new Protobuf.Schema(proto, opts?)` | Compiles a `.proto` source string (or a `{ filename: source }` map for multi-file schemas with `import`s; the `google/protobuf/*` well-known types resolve automatically). |
544+
| `schema.parse(messageName, bytes)` | Decodes a `Uint8Array` for the fully-qualified `messageName`. |
545+
| `schema.build(messageName, value)` | Encodes a JavaScript object into a `Uint8Array`. |
546+
547+
Decoded value shape: camelCase field names; 64-bit integer fields (`int64`/`uint64`/`sint64`/`fixed64`/`sfixed64`) as **BigInt**; enums as their value-name string (unknown numbers kept as numbers); `bytes` as `Uint8Array`; maps as plain objects; nested messages as plain objects. Fields absent on the wire are omitted.
548+
532549
<!-- Reference links -->
533550
[D27]: ./DECISIONS.md
534551

site/app/api/serialization/page.jsx

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ const errors = [
99
];
1010

1111
const IMPORT = `import {
12-
JSONL, XML, YAML, TOML, MessagePack
12+
JSONL, XML, YAML, TOML, MessagePack, Protobuf
1313
} from "runtime:serialization";`;
1414

1515
const sections = [
@@ -99,9 +99,18 @@ const sections = [
9999
options: [
100100
{ name: "detailed", type: "boolean", optional: true, default: "false", desc: "If true, returns an object { valid: boolean, error?: string } instead of a boolean, providing the exact syntax error if validation fails." }
101101
],
102-
ex: `MessagePack.validate(bytes, { detailed: true });`
102+
ex: `MessagePack.validate(bytes, { detailed: true });`
103103
},
104104
]
105+
},
106+
{
107+
title: "Protobuf",
108+
desc: "Schema-aware Protobuf parsing and building. Pure-JS and reflective: the .proto is compiled at runtime (proto3 and edition 2023; proto2-only constructs are rejected). Decoded objects use camelCase keys, BigInt for 64-bit ints, enum value-names, and Uint8Array for bytes.",
109+
exports: [
110+
{ sig: "new Protobuf.Schema(proto, options?)", type: "Schema", desc: "Compiles a .proto source string (or a { filename: source } map for multi-file schemas with imports; google/protobuf well-known types resolve automatically).", ex: `const schema = new Protobuf.Schema('syntax = "proto3"; message Hello { string name = 1; }');` },
111+
{ sig: "schema.parse(messageName, bytes)", type: "(string, Uint8Array) => object", desc: "Decodes a byte array into a JavaScript object for the fully-qualified message name.", ex: `schema.parse("Hello", bytes);` },
112+
{ sig: "schema.build(messageName, value)", type: "(string, object) => Uint8Array", desc: "Encodes a JavaScript object into a Protobuf byte array.", ex: `schema.build("Hello", { name: "world" });` },
113+
]
105114
}
106115
];
107116

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
import DocsShell from "../../../../components/DocsShell.jsx";
2+
import CodeBlock from "../../../../components/CodeBlock.jsx";
3+
4+
export default function ProtobufParserDoc() {
5+
return (
6+
<DocsShell active="/docs/serialization/protobuf">
7+
<p className="text-sm font-medium text-brand-600">Guides</p>
8+
<h1 className="mt-2 text-4xl font-bold tracking-tight text-zinc-900">
9+
Protobuf Processing
10+
</h1>
11+
<p className="mt-4 text-lg leading-relaxed text-zinc-600">
12+
A pure-JavaScript, reflective Protobuf implementation in <code className="rounded bg-zinc-100 px-1.5 py-0.5 text-[13px]">runtime:serialization</code>. The <code className="rounded bg-zinc-100 px-1.5 py-0.5 text-[13px]">.proto</code> schema is compiled at runtime — no codegen, no build step. proto3 and edition 2023 are supported; proto2-only constructs are rejected.
13+
</p>
14+
15+
<h2 className="mt-12 text-2xl font-semibold text-zinc-900">
16+
Compiling a schema
17+
</h2>
18+
<p className="mt-2 text-zinc-600 leading-relaxed">
19+
Construct a <code className="rounded bg-zinc-100 px-1.5 py-0.5 text-[13px]">Protobuf.Schema</code> from <code className="rounded bg-zinc-100 px-1.5 py-0.5 text-[13px]">.proto</code> source. Pass a single string, or a <code className="rounded bg-zinc-100 px-1.5 py-0.5 text-[13px]">{`{ filename: source }`}</code> map for multi-file schemas with imports.
20+
</p>
21+
<div className="mt-6">
22+
<CodeBlock code={`import { Protobuf } from "runtime:serialization";
23+
24+
const schema = new Protobuf.Schema(\`
25+
syntax = "proto3";
26+
package shop;
27+
message Book {
28+
string id = 1;
29+
double price = 5;
30+
repeated string tags = 8;
31+
}
32+
\`);`} title="protobuf_schema.js" lang="js" />
33+
</div>
34+
35+
<h2 className="mt-12 text-2xl font-semibold text-zinc-900">
36+
Building and parsing
37+
</h2>
38+
<p className="mt-2 text-zinc-600 leading-relaxed">
39+
Use <code className="rounded bg-zinc-100 px-1.5 py-0.5 text-[13px]">schema.build</code> and <code className="rounded bg-zinc-100 px-1.5 py-0.5 text-[13px]">schema.parse</code> with a fully-qualified message name.
40+
</p>
41+
<div className="mt-6">
42+
<CodeBlock code={`const bytes = schema.build("shop.Book", {
43+
id: "bk1",
44+
price: 44.95,
45+
tags: ["computer", "xml"],
46+
});
47+
48+
const book = schema.parse("shop.Book", bytes);
49+
console.log(book.price); // 44.95
50+
console.log(book.tags); // ["computer", "xml"]`} title="protobuf_roundtrip.js" lang="js" />
51+
</div>
52+
53+
<h2 className="mt-12 text-2xl font-semibold text-zinc-900">
54+
Decoded value shape
55+
</h2>
56+
<p className="mt-2 text-zinc-600 leading-relaxed">
57+
Decoded objects use camelCase field names (or the explicit <code className="rounded bg-zinc-100 px-1.5 py-0.5 text-[13px]">json_name</code>). 64-bit integer fields surface as <code className="rounded bg-zinc-100 px-1.5 py-0.5 text-[13px]">BigInt</code>; enums as their value-name string; <code className="rounded bg-zinc-100 px-1.5 py-0.5 text-[13px]">bytes</code> as <code className="rounded bg-zinc-100 px-1.5 py-0.5 text-[13px]">Uint8Array</code>; maps as plain objects. Fields absent on the wire are omitted.
58+
</p>
59+
<div className="mt-6">
60+
<CodeBlock code={`const schema = new Protobuf.Schema(\`
61+
syntax = "proto3";
62+
enum Status { ACTIVE = 0; ARCHIVED = 1; }
63+
message Account { uint64 id = 1; Status status = 2; }
64+
\`);
65+
66+
const bytes = schema.build("Account", { id: 9007199254740993n, status: "ARCHIVED" });
67+
const acct = schema.parse("Account", bytes);
68+
console.log(typeof acct.id); // "bigint"
69+
console.log(acct.status); // "ARCHIVED"`} title="protobuf_types.js" lang="js" />
70+
</div>
71+
</DocsShell>
72+
);
73+
}

site/components/DocsShell.jsx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ const NAV = [
2929
{ href: "/docs/serialization/toml", label: "TOML Parser" },
3030
{ href: "/docs/serialization/jsonl", label: "JSONL Parser" },
3131
{ href: "/docs/serialization/msgpack", label: "MessagePack Parser" },
32+
{ href: "/docs/serialization/protobuf", label: "Protobuf Parser" },
3233
],
3334
},
3435
{

site/src/site-meta.js

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ export const PAGES = {
4242
"/api/fs": { section: "API reference", title: "runtime:fs", description: "Blob-based async file I/O — `file()` handles, `write()`, `readDir`, `stat`, `mkdir`, `remove`, `rename`, and `Glob`; confined to a root jail. Gated on FileRead/FileWrite." },
4343
"/api/net": { section: "API reference", title: "runtime:net", description: "TCP sockets following the WinterTC Sockets API — `connect()` and `listen()`. Gated on Net/NetListen." },
4444
"/api/http": { section: "API reference", title: "runtime:http", description: "an HTTP/1.1 server, `serve((request) => response)`, using the Web `Request`/`Response` objects. Gated on NetListen." },
45-
"/api/serialization": { section: "API reference", title: "runtime:serialization", description: "high-performance native parsers for JSONL, XML, YAML, and TOML via streams and synchronous functions." },
45+
"/api/serialization": { section: "API reference", title: "runtime:serialization", description: "serialization for JSONL, XML, YAML, TOML, MessagePack (native), and Protobuf (pure-JS, reflective)." },
4646
"/api/websocket": { section: "API reference", title: "runtime:websocket", description: "WebSocket client and server functionality native to the engine." },
4747

4848
// Comparisons
@@ -63,6 +63,7 @@ export const PAGES = {
6363
"/docs/serialization/xml": { section: "Guides", title: "XML Parsing", description: "synchronous and streaming fast XML processing." },
6464
"/docs/serialization/yaml": { section: "Guides", title: "YAML Parsing", description: "synchronous robust YAML processing." },
6565
"/docs/serialization/msgpack": { section: "Guides", title: "MessagePack Parsing", description: "synchronous fast MessagePack processing." },
66+
"/docs/serialization/protobuf": { section: "Guides", title: "Protobuf Processing", description: "pure-JS reflective Protobuf (proto3 + edition 2023): runtime .proto compilation, BigInt 64-bit, no codegen." },
6667
};
6768

6869
// Static trailing sections of llms.txt (not derived from routes).

0 commit comments

Comments
 (0)