|
| 1 | +<!-- markdownlint-disable MD033 --> |
| 2 | + |
1 | 3 | # distribute |
2 | 4 |
|
3 | | -<p align="center"> |
4 | | - <img src="assets/img/distribute.png" alt="distribute logo" width="200" /> |
5 | | -</p> |
| 5 | +<img src="assets/img/logo.png" alt="logo" height="218" /> |
6 | 6 |
|
7 | 7 | Typed distributed messaging for Gleam on the BEAM. |
8 | 8 |
|
9 | 9 | [](https://hex.pm/packages/distribute) |
10 | 10 | [](https://hexdocs.pm/distribute/) |
11 | 11 |
|
12 | | -## What this is |
13 | | - |
14 | | -Erlang already gives you distribution, but from Gleam you lose type info at |
15 | | -the node boundary, everything crosses the wire as raw terms. `distribute` |
16 | | -puts binary codecs in front of `:global` and `Subject` so the compiler can |
17 | | -catch mismatches before messages leave the process. |
18 | | - |
19 | | -"Typed" here means checked at encode/decode boundaries. There is no shared |
20 | | -type system across nodes. The BEAM doesn't work that way. |
| 12 | +`distribute` is a thin typed safety layer over Erlang's distribution |
| 13 | +primitives. It puts binary codecs in front of `:global` and `Subject` so |
| 14 | +the compiler catches protocol mismatches before messages leave the |
| 15 | +process. Payload size and dead-target detection are enforced at every |
| 16 | +I/O boundary, so a misbehaving peer cannot OOM your node or hang it. |
21 | 17 |
|
22 | 18 | ## Install |
23 | 19 |
|
24 | 20 | ```sh |
25 | 21 | gleam add distribute |
26 | 22 | ``` |
27 | 23 |
|
28 | | -## Usage |
29 | | - |
30 | | -### Fire-and-forget |
| 24 | +Requires Gleam 1.16 or newer, Erlang targetÉ only. |
31 | 25 |
|
32 | | -Define a `TypedName(msg)` that pairs a name with a codec, then use it on |
33 | | -both sides. The compiler won't let you register a `String` actor and look |
34 | | -it up as `Int`. |
| 26 | +## 30-second taste |
35 | 27 |
|
36 | 28 | ```gleam |
37 | 29 | import distribute |
38 | 30 | import distribute/codec |
39 | | -import distribute/registry |
40 | | -import distribute/receiver |
41 | | -
|
42 | | -// one TypedName, shared across the codebase |
43 | | -let greeter = registry.named("greeter", codec.string()) |
44 | | -
|
45 | | -// start + register |
46 | | -let assert Ok(gs) = distribute.start_actor(greeter, Nil, fn(msg, _state) { |
47 | | - io.println("Got: " <> msg) |
48 | | - receiver.Continue(Nil) |
49 | | -}) |
50 | | -let assert Ok(Nil) = distribute.register(greeter, gs) |
51 | | -
|
52 | | -// from any node |
53 | | -let assert Ok(remote) = distribute.lookup(greeter) |
54 | | -let assert Ok(Nil) = distribute.send(remote, "hello") |
55 | | -``` |
56 | | - |
57 | | -### Request / response |
58 | | - |
59 | | -Include a reply `Subject(BitArray)` in your message type and use |
60 | | -`codec.subject()` to serialize it. The `Subject` carries node info, so |
61 | | -replies route back across nodes automatically. |
62 | | - |
63 | | -```gleam |
64 | | -import distribute/codec |
65 | | -import distribute/global |
66 | 31 | import distribute/receiver |
67 | | -import gleam/erlang/process |
68 | 32 |
|
69 | | -type CounterMsg { |
70 | | - Inc(Int) |
71 | | - Get(reply: process.Subject(BitArray)) |
72 | | -} |
| 33 | +let greeter = distribute.named("greeter", codec.string()) |
73 | 34 |
|
74 | | -// handler side |
75 | | -fn handle(msg, state) { |
76 | | - case msg { |
77 | | - Inc(n) -> receiver.Continue(state + n) |
78 | | - Get(reply) -> { |
79 | | - let _ = global.reply(reply, state, codec.int_encoder()) |
80 | | - receiver.Continue(state) |
81 | | - } |
82 | | - } |
83 | | -} |
84 | | -
|
85 | | -// caller side |
86 | | -let assert Ok(count) = global.call(counter, Get, codec.int_decoder(), 5000) |
87 | | -``` |
88 | | - |
89 | | -`global.call` creates a temporary subject, sends the request, waits for |
90 | | -the response, decodes it. Same idea as `gen_server:call`. |
91 | | - |
92 | | -### Codecs |
93 | | - |
94 | | -Primitives: `codec.int()`, `codec.string()`, `codec.float()`, `codec.bool()`, |
95 | | -`codec.bitarray()`, `codec.nil()`. |
96 | | - |
97 | | -Composites: `codec.list(c)`, `codec.subject()`, `codec.map(c, wrap, unwrap)`, |
98 | | -`composite.option(c)`, `composite.result(ok, err)`, |
99 | | -`composite.tuple2(a, b)`, `composite.tuple3(a, b, c)`. |
100 | | - |
101 | | -For your own types, use `codec.map`: |
102 | | - |
103 | | -```gleam |
104 | | -type UserId { UserId(Int) } |
| 35 | +let assert Ok(_gs) = |
| 36 | + distribute.start_registered(greeter, Nil, fn(msg, _state) { |
| 37 | + io.println("got: " <> msg) |
| 38 | + receiver.Continue(Nil) |
| 39 | + }) |
105 | 40 |
|
106 | | -let user_id_codec = codec.map(codec.int(), UserId, fn(uid) { |
107 | | - let UserId(n) = uid |
108 | | - n |
109 | | -}) |
| 41 | +let assert Ok(target) = distribute.lookup(greeter) |
| 42 | +let assert Ok(Nil) = distribute.send(target, "hello") |
110 | 43 | ``` |
111 | 44 |
|
112 | | -Gleam has no derive macros or reflection, so codecs for complex types |
113 | | -are manual. The combinators handle the serialization so you just wire |
114 | | -the fields together! |
115 | | - |
116 | | -## Modules |
117 | | - |
118 | | -| Module | Does | |
119 | | -|--------|------| |
120 | | -| `distribute` | Facade: start node, connect, send, lookup | |
121 | | -| `distribute/actor` | Named actors, supervision, pools | |
122 | | -| `distribute/cluster` | `net_kernel` start/connect/ping | |
123 | | -| `distribute/codec` | Binary codecs for primitives + `subject()` | |
124 | | -| `distribute/codec/composite` | Option, Result, Tuple codecs | |
125 | | -| `distribute/codec/variant` | Build codecs for Custom Types (ADTs) | |
126 | | -| `distribute/codec/tagged` | Tagged messages with version field | |
127 | | -| `distribute/global` | `GlobalSubject(msg)`, `call`, `reply` | |
128 | | -| `distribute/cluster/monitor` | `NodeUp`, `NodeDown` typed events | |
129 | | -| `distribute/registry` | `TypedName(msg)`, `:global` registration | |
130 | | -| `distribute/receiver` | Typed receive, OTP actor wrappers | |
| 45 | +For the full walkthrough see [docs/quickstart.md](./docs/quickstart.md). |
| 46 | + |
| 47 | +## Documentation |
| 48 | + |
| 49 | +All long-form docs live in [`docs/`](./docs/README.md): |
| 50 | + |
| 51 | +- [Quickstart](./docs/quickstart.md). Boot, configure, send. |
| 52 | +- [Recipes](./docs/recipes.md). Counter, pool, versioned protocol, |
| 53 | + cluster events, custom records. |
| 54 | +- [Actors and registry](./docs/actors_and_registry.md). |
| 55 | +- [Messaging](./docs/messaging.md). `send` / `receive` / `call`. |
| 56 | +- [Codecs and types](./docs/codecs_and_types.md). Wire format, custom |
| 57 | + records, tagged messages. |
| 58 | +- [Safety and limits](./docs/safety_and_limits.md). Payload limits, |
| 59 | + threat model, recommended sizing. |
| 60 | + |
| 61 | +## What you get |
| 62 | + |
| 63 | +- **Typed boundary.** `TypedName(msg)` binds a name to a codec. |
| 64 | + Registration and lookup share the same `msg` type, and the compiler |
| 65 | + rejects mismatches. |
| 66 | +- **Hard payload caps.** `max_payload_size_bytes` is enforced before |
| 67 | + encode and before decode on every path: `send`, `receive`, `call`, |
| 68 | + `reply`, actor handlers, selectors. |
| 69 | +- **Fast-fail calls.** `global.call` monitors the target. A dead target |
| 70 | + returns `Error(TargetDown)` immediately. Late replies and DOWN |
| 71 | + messages are drained from the caller's mailbox. |
| 72 | +- **OTP-native.** Real OTP gen_server-flavored actors via |
| 73 | + `gleam_otp/actor`, real supervisors, real child specs. `observer`, |
| 74 | + `sys:get_status`, restart strategies all work. No magic. |
| 75 | +- **Single-source codec.** Encoding and decoding happen through one |
| 76 | + `Codec(a)` value. Combinators (`map`, `list`, `option`, `tuple`, |
| 77 | + `tagged`) cover the common cases without macros. |
131 | 78 |
|
132 | 79 | ### Custom Type Codecs |
133 | 80 |
|
@@ -175,26 +122,30 @@ distribute.unsubscribe(m) |
175 | 122 |
|
176 | 123 | ## Caveats |
177 | 124 |
|
178 | | -**What the types catch** — within one codebase, `TypedName` and |
179 | | -`GlobalSubject` prevent mixing up message types at compile time. |
180 | | - |
181 | | -**What they don't** — two separate codebases using different codecs for |
182 | | -the same name. The codec will reject the binary at runtime, not at compile |
183 | | -time. Same for Erlang code sending raw terms to a `distribute` actor. |
184 | | - |
185 | | -**Subject construction** — Gleam's `Subject` is opaque. To build one from |
186 | | -a remote PID and a deterministic tag (how registry lookup works), we |
187 | | -construct the `{subject, Pid, Tag}` tuple in |
188 | | -[one Erlang function](src/distribute_ffi_utils.erl). If `gleam_erlang` |
189 | | -changes the internal representation, that single function needs updating. |
190 | | - |
191 | | -**No auto-derive** — Gleam doesn't have macros. Complex message codecs |
192 | | -are manual. The combinators (`map`, `list`, `option`, `tuple2`, etc.) |
193 | | -keep it manageable, but it's not zero-boilerplate. |
| 125 | +- **Two codebases, one wire.** `TypedName` enforces type safety inside |
| 126 | + one codebase. Mismatched codecs across separate codebases produce |
| 127 | + runtime decode errors, not compile errors. |
| 128 | +- **No auto-derive.** Gleam has no macros, so complex codecs are |
| 129 | + manual. The combinators keep them short. |
| 130 | +- **`terminate` caveat in `gleam_otp/actor` 1.x.** External shutdown |
| 131 | + paths do not currently invoke an OTP-style `terminate` callback. If |
| 132 | + an actor owns files, sockets, ETS tables, ports, or other external |
| 133 | + resources, use the linked resource-owner pattern documented in |
| 134 | + [docs/recipes.md](./docs/recipes.md) and |
| 135 | + [docs/safety_and_limits.md](./docs/safety_and_limits.md). |
| 136 | +- **One internal coupling.** We construct `Subject` from a remote PID |
| 137 | + via [one Erlang FFI function](src/distribute_ffi_utils.erl). That is |
| 138 | + the single point that depends on `gleam_erlang`'s subject layout. |
194 | 139 |
|
195 | 140 | ## Development |
196 | 141 |
|
197 | 142 | ```sh |
198 | | -gleam test |
199 | | -gleam docs build |
| 143 | +gleam test # full suite (includes mandatory real-cluster Z2/Z3) |
| 144 | +epmd -daemon # start Erlang distribution daemon if not already running |
| 145 | +gleam dev # multi-node playground |
| 146 | +gleam docs build # local API docs |
200 | 147 | ``` |
| 148 | + |
| 149 | +## License |
| 150 | + |
| 151 | +MIT. See [LICENSE](./LICENSE). |
0 commit comments