Skip to content

Commit 514c2c6

Browse files
committed
feat: initial extraction from mrmeeseeks/protobuf
Pure Pony protobuf runtime — wire codec (Varint, ZigZag, Tag, WireType), framing + scalar codecs (WireReader, WireWriter, Scalar, LE), UTF-8 validator, typed WireError union. Sourced from straw-hat-team/trogonai.com/mrmeeseeks/protobuf where the runtime was incubated alongside Phase 3+4 of an in-Pony codegen exploration. The codegen path was retired in favor of a Go protoc plugin (TrogonStack/protoc-gen/cmd/protoc-gen-pony, in flight as PR #47); this repo extracts the runtime so generated code can depend on it directly. 54 tests including property-based roundtrips for every protobuf scalar type and fuzz tests confirming arbitrary bytes never panic the readers.
0 parents  commit 514c2c6

19 files changed

Lines changed: 1490 additions & 0 deletions

.github/workflows/ci.yml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: ci
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
8+
jobs:
9+
test:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- uses: actions/checkout@v4
13+
- name: Install ponyc + corral via ponyup
14+
run: |
15+
sh -c "$(curl --proto '=https' --tlsv1.2 -fsSL https://raw.githubusercontent.com/ponylang/ponyup/latest-release/ponyup-init.sh)"
16+
~/.local/share/ponyup/bin/ponyup default x86_64-unknown-linux-gnu
17+
~/.local/share/ponyup/bin/ponyup update ponyc release
18+
~/.local/share/ponyup/bin/ponyup update corral release
19+
echo "$HOME/.local/share/ponyup/bin" >> "$GITHUB_PATH"
20+
- name: Build + test
21+
run: |
22+
mkdir -p build
23+
corral run -- ponyc --path . -o build test/
24+
./build/test

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
_corral/
2+
_repos/
3+
build/
4+
lock.json
5+
*.o
6+
*.dwarf

CHANGELOG.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Changelog
2+
3+
## Unreleased
4+
5+
### Features
6+
7+
- Initial extraction from `straw-hat-team/trogonai.com/mrmeeseeks/protobuf`.
8+
Wire codec (`Varint`, `ZigZag`, `Tag`, `WireType`), framing + scalar
9+
codecs (`WireReader`, `WireWriter`, `Scalar`, `LE`), UTF-8 validator,
10+
typed `WireError` union. 54 tests including property-based roundtrips
11+
for every protobuf scalar and fuzz tests on the readers.

LICENSE

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
BSD 2-Clause License
2+
3+
Copyright (c) 2026, TrogonStack
4+
5+
Redistribution and use in source and binary forms, with or without
6+
modification, are permitted provided that the following conditions are met:
7+
8+
1. Redistributions of source code must retain the above copyright notice,
9+
this list of conditions and the following disclaimer.
10+
11+
2. Redistributions in binary form must reproduce the above copyright notice,
12+
this list of conditions and the following disclaimer in the documentation
13+
and/or other materials provided with the distribution.
14+
15+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
16+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
17+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
18+
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
19+
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
20+
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
21+
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
22+
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
23+
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
24+
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
25+
POSSIBILITY OF SUCH DAMAGE.

README.md

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# protobuf-pony
2+
3+
Pure Pony protobuf runtime — wire codec, framing, scalar codecs, UTF-8
4+
validation. The runtime that generated Pony code (emitted by
5+
[`protoc-gen-pony`](https://github.com/TrogonStack/protoc-gen)) calls into.
6+
7+
## What's here
8+
9+
- **`Varint`** — varint encode/decode + zigzag encode/decode (i32/i64).
10+
- **`Tag`** + **`TagCodec`** — protobuf field tag (field number + wire type).
11+
- **`WireType`** — typed union (`WireVarint | WireFixed64 | WireLenDelim |
12+
WireFixed32`); proto2 group wire types are deliberately omitted.
13+
- **`WireReader`** — cursor-style reader over `Array[U8] val`. Reads
14+
varints, tags, fixed-width fields, length-delimited bytes, UTF-8-validated
15+
strings; supports per-wire-type skip-unknown for forward-compat.
16+
- **`WireWriter`** — accumulating writer, returns an `Array[U8] iso^` on
17+
`done()`. Includes a `write_string_field(field_num, s)` helper that
18+
internalizes proto3's "skip empty strings" rule.
19+
- **`Scalar`** — paired encode/decode for all 12 protobuf scalar types
20+
(`bool`, `int32/64`, `uint32/64`, `sint32/64`, `fixed32/64`,
21+
`sfixed32/64`, `float`, `double`). Uses `F32/F64.from_bits/.bits` LLVM
22+
intrinsics for lossless float round-trip.
23+
- **`LE`** — little-endian byte conversion helpers, shift-based (no
24+
`Platform.bigendian()` branch needed since bit shifts produce LE bytes
25+
on any host).
26+
- **`WireError`** — typed-error union (`WireTruncated | WireOverflow |
27+
WireBadTag | WireInvalidUtf8`).
28+
29+
## Install
30+
31+
```json
32+
{
33+
"deps": [
34+
{
35+
"locator": "github.com/TrogonStack/protobuf-pony",
36+
"version": "0.1.0"
37+
}
38+
]
39+
}
40+
```
41+
42+
Then in your Pony source:
43+
44+
```pony
45+
use "protobuf"
46+
```
47+
48+
## Usage
49+
50+
The typical caller is generated code from `protoc-gen-pony`. Sketch:
51+
52+
```pony
53+
let writer = WireWriter
54+
writer.write_tag(Tag(1, WireVarint))
55+
Scalar.write_int32(writer, 42)
56+
let bytes: Array[U8] val = recover val writer.done() end
57+
58+
let reader = WireReader(bytes)
59+
match reader.read_tag()
60+
| let t: Tag => /* dispatch on t.field_number, t.wire_type */
61+
| let e: WireError => /* handle */
62+
end
63+
```
64+
65+
Hand-writing decoders is supported but the expected workflow is to define
66+
your schemas in `.proto`, run `protoc-gen-pony`, and let the generated
67+
`<Msg>Codec` primitives call into this runtime.
68+
69+
## Build + test
70+
71+
Requires [Task](https://taskfile.dev), [corral](https://github.com/ponylang/corral),
72+
and `ponyc`.
73+
74+
```bash
75+
corral fetch
76+
task test
77+
```
78+
79+
## License
80+
81+
BSD-2-Clause.

Taskfile.yml

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
version: "3"
2+
3+
tasks:
4+
default:
5+
desc: Build + test
6+
deps: [test]
7+
8+
test:
9+
desc: Build and run unit + property tests
10+
cmds:
11+
- mkdir -p build
12+
- corral run -- ponyc --path . -o build test/
13+
- ./build/test
14+
15+
clean:
16+
desc: Remove build artifacts
17+
cmds:
18+
- rm -rf build _corral

corral.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"locator": "github.com/TrogonStack/protobuf-pony",
3+
"version": "0.1.0",
4+
"deps": []
5+
}

protobuf/endian.pony

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
// Bit shifts produce little-endian bytes regardless of host memory layout,
2+
// so no `Platform.bigendian()` branch is needed.
3+
4+
primitive LE
5+
fun push_u32_le(out: Array[U8] iso, v: U32): Array[U8] iso^ =>
6+
out.push((v and 0xFF).u8())
7+
out.push(((v >> 8) and 0xFF).u8())
8+
out.push(((v >> 16) and 0xFF).u8())
9+
out.push(((v >> 24) and 0xFF).u8())
10+
consume out
11+
12+
fun push_u64_le(out: Array[U8] iso, v: U64): Array[U8] iso^ =>
13+
out.push((v and 0xFF).u8())
14+
out.push(((v >> 8) and 0xFF).u8())
15+
out.push(((v >> 16) and 0xFF).u8())
16+
out.push(((v >> 24) and 0xFF).u8())
17+
out.push(((v >> 32) and 0xFF).u8())
18+
out.push(((v >> 40) and 0xFF).u8())
19+
out.push(((v >> 48) and 0xFF).u8())
20+
out.push(((v >> 56) and 0xFF).u8())
21+
consume out
22+
23+
fun u32_to_le_bytes(v: U32): Array[U8] iso^ =>
24+
push_u32_le(recover iso Array[U8](4) end, v)
25+
26+
fun u32_from_le_bytes(buf: Array[U8] box, offset: USize):
27+
(U32 | WireError)
28+
=>
29+
if buf.size() < (offset + 4) then return WireTruncated end
30+
try
31+
let b0 = buf(offset)?.u32()
32+
let b1 = buf(offset + 1)?.u32()
33+
let b2 = buf(offset + 2)?.u32()
34+
let b3 = buf(offset + 3)?.u32()
35+
b0 or (b1 << 8) or (b2 << 16) or (b3 << 24)
36+
else
37+
WireTruncated
38+
end
39+
40+
fun u64_to_le_bytes(v: U64): Array[U8] iso^ =>
41+
push_u64_le(recover iso Array[U8](8) end, v)
42+
43+
fun u64_from_le_bytes(buf: Array[U8] box, offset: USize):
44+
(U64 | WireError)
45+
=>
46+
if buf.size() < (offset + 8) then return WireTruncated end
47+
try
48+
let b0 = buf(offset)?.u64()
49+
let b1 = buf(offset + 1)?.u64()
50+
let b2 = buf(offset + 2)?.u64()
51+
let b3 = buf(offset + 3)?.u64()
52+
let b4 = buf(offset + 4)?.u64()
53+
let b5 = buf(offset + 5)?.u64()
54+
let b6 = buf(offset + 6)?.u64()
55+
let b7 = buf(offset + 7)?.u64()
56+
b0 or (b1 << 8) or (b2 << 16) or (b3 << 24) or
57+
(b4 << 32) or (b5 << 40) or (b6 << 48) or (b7 << 56)
58+
else
59+
WireTruncated
60+
end

protobuf/errors.pony

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
primitive WireTruncated
2+
fun label(): String val => "truncated"
3+
fun message(): String val =>
4+
"buffer ended before varint was complete"
5+
6+
primitive WireOverflow
7+
fun label(): String val => "overflow"
8+
fun message(): String val =>
9+
"varint exceeds 10 bytes (max for U64)"
10+
11+
primitive WireBadTag
12+
fun label(): String val => "bad_tag"
13+
fun message(): String val =>
14+
"tag has zero field number or unknown wire type"
15+
16+
primitive WireInvalidUtf8
17+
fun label(): String val => "invalid_utf8"
18+
fun message(): String val =>
19+
"string field bytes are not valid UTF-8"
20+
21+
type WireError is
22+
(WireTruncated | WireOverflow | WireBadTag | WireInvalidUtf8)

protobuf/protobuf.pony

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
"""
2+
Protocol Buffers runtime library for Pony — wire codec, framing, scalar
3+
codecs, and a UTF-8 validator. Generated message code emitted by
4+
`protoc-gen-pony` (TrogonStack/protoc-gen) calls into the types here.
5+
6+
Currently incubating inside mrmeeseeks under the package name `protobuf`;
7+
extracts to its own repo once a downstream consumer needs it.
8+
"""

0 commit comments

Comments
 (0)