Skip to content

Commit f37cb2c

Browse files
shred-proxy: add standalone binary as a workspace member (#78)
* shred-proxy: add standalone binary as workspace member Add a lightweight standalone shred-proxy binary as a new Cargo workspace member that reuses the bridge library's shred forwarder directly (doublezero_edge_connect::shred: receiver, dedup/sigverify, parser, multicast plumbing). The only new code is active-group detection via the kernel routing table (ip route get), the reconciler, and the CLI — no doublezero CLI and no solana-* crates. - Cargo workspace: root becomes [workspace] members=[".","shred-proxy"]; the bridge Docker build is scoped to -p doublezero-edge-connect so the image is unchanged (member manifest+src copied in for workspace resolution). - One-liner install: curl -fsSL https://get.doublezero.xyz/shred-proxy | bash downloads a prebuilt static (musl) binary and installs it as a systemd service. release.shred-proxy.yml builds/publishes the binary on tag shred-proxy-v*; release.install-script.yml serves the installer. * shred-proxy: fix installer one-liner abort and review findings Address code-review findings on the new shred-proxy member crate: - install.sh: SCRIPT_DIR referenced ${BASH_SOURCE[0]} under `set -u`, which is unbound when the script runs via `curl | bash` (the primary install path), aborting the installer after download. Fall back to a safe default so the packaging files are fetched from the repo instead. - install.sh: require `ip` (iproute2) in the precondition check — without it the routing-table detection silently finds no groups and the service forwards nothing. - main.rs: redact `rpc_url` in the startup log (paid RPC endpoints embed an API key in the URL; the full args dump would leak it to the journal). - release.shred-proxy.yml: pass the untrusted workflow_dispatch tag input via env instead of interpolating it into the run script (command injection). - Dockerfile: drop the unnecessary COPY of shred-proxy/src — the scoped bridge build only needs the member manifest, and copying sources busts the image build cache on unrelated shred-proxy edits. - reconcile.rs: await and log the forwarder task's result on exit instead of discarding it, so a persistent failure surfaces its error/panic. * shred-proxy: apply PR review — CI workspace, fail-open reconciler, installer self-elevate - rust.yml: build/clippy/test the whole workspace (--workspace) so the shred-proxy member is actually compiled and tested on every PR (was root-only). - reconciler: fail-open on a transient `ip route get` failure (keep current activation instead of tearing forwarding down); active_groups now signals probe failure via io::Result. One long-lived signal listener so a SIGTERM between polls isn't dropped. - installer: self-elevate privileged steps with sudo (plain `curl | bash` works for non-root); re-run restarts the running service onto the upgraded binary; fetch packaging files from the resolved release tag, not a moving `main`; document DZ_* config after the pipe. - release workflow: run tests before publish, assert the artifact is static, validate the tag is namespaced. - env.example: RUST_LOG keeps the forwarder's doublezero_edge_connect info logs. - net.rs: share parse_dev between the probe and its tests. * shred-proxy: address review lows — checksum opt-out, restart limiter, iface guard - install.sh: a missing SHA256SUMS is now an error (was a silent skip); override with SHRED_PROXY_SKIP_CHECKSUM=1 for manual/mirror installs. - shred-proxy.service: StartLimitIntervalSec=0 + RestartSec=5 so an unattended service never latches `failed` after a short crash burst. - main.rs: bail at startup when --iface is an IP in detection mode (routing-table detection matches interface names only, so it would sit "forwarder idle"). - reconcile.rs: document the dedup-window reset (duplicate burst) on a membership-change restart; now only fires on real changes since detection is fail-open. * shred-proxy: address post-approval review follow-ups - net.rs: a clean `ip route get` non-zero exit (no route to the group) is now Ok(None) — inactive, not a probe failure — so a routeless unsubscribed candidate on a host with no default route can't poison every detection tick and stall activation. Only spawn/decode failures stay Err (fail-open). - release.shred-proxy.yml: pin a dispatch-created tag to the built commit with `gh release create --target "${github.sha}"` (was default-branch HEAD). - env.example: DZ_DEDUP_MODE comment lists sigverify; add DZ_RPC_URL entry. - README: document the SHRED_PROXY_SKIP_CHECKSUM installer opt-out.
1 parent ca691fb commit f37cb2c

18 files changed

Lines changed: 1283 additions & 5 deletions

.github/workflows/release.install-script.yml

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ on:
1616
- "scripts/connect.sh"
1717
- "scripts/connect-testnet.sh"
1818
- "scripts/connect-devnet.sh"
19+
- "shred-proxy/packaging/install.sh"
1920
- ".github/workflows/release.install-script.yml"
2021
workflow_dispatch:
2122

@@ -54,8 +55,17 @@ jobs:
5455
--cache-control "public,max-age=120"
5556
done
5657
58+
# The standalone shred-proxy installer is served under the same bucket at
59+
# https://get.doublezero.xyz/shred-proxy. It downloads the prebuilt binary published by
60+
# release.shred-proxy.yml, so it is decoupled from the Docker-image release cadence.
61+
- name: Upload shred-proxy installer
62+
run: |
63+
aws s3 cp shred-proxy/packaging/install.sh "s3://${BUCKET}/shred-proxy" \
64+
--content-type text/x-shellscript \
65+
--cache-control "public,max-age=120"
66+
5767
- name: Invalidate CloudFront cache
5868
run: |
5969
aws cloudfront create-invalidation \
6070
--distribution-id "$DISTRIBUTION_ID" \
61-
--paths '/connect' '/connect-testnet' '/connect-devnet'
71+
--paths '/connect' '/connect-testnet' '/connect-devnet' '/shred-proxy'
Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
name: release.shred-proxy
2+
3+
# Builds the standalone `shred-proxy` binary as a static linux/amd64 (musl) executable and
4+
# publishes it to GitHub Releases. The install one-liner (scripts served at
5+
# https://get.doublezero.xyz/shred-proxy) downloads this asset.
6+
#
7+
# Triggered by a namespaced tag `shred-proxy-vX.Y.Z` (so it never collides with the bridge's
8+
# Docker release tags), or manually via workflow_dispatch for a smoke build.
9+
on:
10+
push:
11+
tags:
12+
- "shred-proxy-v*"
13+
workflow_dispatch:
14+
inputs:
15+
tag:
16+
description: "Release tag to publish to (e.g. shred-proxy-v0.1.0); blank = build-only"
17+
required: false
18+
default: ""
19+
20+
permissions:
21+
contents: read
22+
23+
env:
24+
TARGET: x86_64-unknown-linux-musl
25+
ASSET: shred-proxy-x86_64-unknown-linux-musl
26+
27+
jobs:
28+
build:
29+
runs-on: ubuntu-latest
30+
# `contents: write` is needed to create/upload the GitHub Release. The publish step itself is
31+
# gated on a non-empty tag, so a plain workflow_dispatch smoke build grants the scope but never
32+
# writes anything.
33+
permissions:
34+
contents: write
35+
steps:
36+
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
37+
38+
- uses: dtolnay/rust-toolchain@fa04a1451ff1842e2626ccb99004d0195b455a88 # master
39+
with:
40+
toolchain: stable
41+
targets: x86_64-unknown-linux-musl
42+
43+
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2
44+
45+
# musl-tools provides the `musl-gcc` linker driver used for the static target.
46+
- name: Install musl tools
47+
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends musl-tools
48+
49+
# Gate the release on the test suite: never publish a binary the tests haven't passed for.
50+
# Run on the host target (not musl) so the tests execute natively.
51+
- name: Test
52+
run: cargo test --locked -p shred-proxy
53+
54+
- name: Build (static, release)
55+
run: cargo build --release --locked -p shred-proxy --target "${TARGET}"
56+
57+
- name: Stage artifact + checksum
58+
id: stage
59+
run: |
60+
set -euo pipefail
61+
mkdir -p dist
62+
cp "target/${TARGET}/release/shred-proxy" "dist/${ASSET}"
63+
# Assert (not just print) that it really is static — a dynamic build would fail on hosts
64+
# without the same libc, so a non-static artifact must fail the release, not ship silently.
65+
file "dist/${ASSET}"
66+
if ! file "dist/${ASSET}" | grep -q 'statically linked'; then
67+
echo "error: ${ASSET} is not statically linked — refusing to publish" >&2
68+
exit 1
69+
fi
70+
( cd dist && sha256sum "${ASSET}" > SHA256SUMS )
71+
cat dist/SHA256SUMS
72+
73+
- name: Upload workflow artifact
74+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
75+
with:
76+
name: shred-proxy-${{ env.TARGET }}
77+
path: dist/*
78+
79+
# Resolve the release tag: the pushed tag on a tag push, or the dispatch input otherwise.
80+
- name: Resolve tag
81+
id: tag
82+
# The dispatch input is untrusted; pass it via env rather than interpolating `${{ }}` into
83+
# the shell (avoids command injection), matching the repo's other workflows.
84+
env:
85+
EVENT_NAME: ${{ github.event_name }}
86+
INPUT_TAG: ${{ github.event.inputs.tag }}
87+
run: |
88+
set -euo pipefail
89+
if [ "${EVENT_NAME}" = "push" ]; then
90+
tag="${GITHUB_REF_NAME}"
91+
else
92+
tag="${INPUT_TAG}"
93+
fi
94+
# Validate the tag before it can drive a publish: the workflow_dispatch input is arbitrary,
95+
# and `gh release create` would otherwise mint a non-namespaced tag pointing at this build's
96+
# commit. A blank tag is allowed (build-only smoke run); a non-blank one must be namespaced.
97+
if [ -n "${tag}" ] && ! printf '%s' "${tag}" | grep -Eq '^shred-proxy-v[0-9]'; then
98+
echo "error: release tag '${tag}' must match 'shred-proxy-v<major>...'; refusing to publish" >&2
99+
exit 1
100+
fi
101+
echo "tag=${tag}" >> "$GITHUB_OUTPUT"
102+
103+
- name: Publish to GitHub Releases
104+
if: steps.tag.outputs.tag != ''
105+
env:
106+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
107+
TAG: ${{ steps.tag.outputs.tag }}
108+
BUILD_SHA: ${{ github.sha }}
109+
run: |
110+
set -euo pipefail
111+
# Create the release if it doesn't exist yet, then (re)upload the assets. `--target` pins a
112+
# newly-minted tag (workflow_dispatch, where the tag doesn't exist yet) to the exact commit
113+
# that was built, not default-branch HEAD; on a tag push the tag already resolves to its
114+
# commit so --target is a no-op.
115+
if ! gh release view "${TAG}" >/dev/null 2>&1; then
116+
gh release create "${TAG}" \
117+
--target "${BUILD_SHA}" \
118+
--title "${TAG}" \
119+
--notes "Standalone shred-proxy binary (static linux/amd64). Install: curl -fsSL https://get.doublezero.xyz/shred-proxy | bash"
120+
fi
121+
gh release upload "${TAG}" "dist/${ASSET}" "dist/SHA256SUMS" --clobber

.github/workflows/rust.yml

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,10 @@ jobs:
2424
with:
2525
toolchain: stable
2626
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2
27-
- run: cargo build --release --verbose
27+
# `--workspace` is required: this is a root-package workspace (root `Cargo.toml` has both
28+
# `[package]` and `[workspace]`), so a bare `cargo build` selects only the root crate and would
29+
# never compile the `shred-proxy` member. Same for clippy/test below.
30+
- run: cargo build --workspace --release --verbose
2831

2932
rust-lint:
3033
runs-on: ubuntu-latest
@@ -46,7 +49,7 @@ jobs:
4649
# The nightly toolchain above is now the default, but clippy is only installed on
4750
# stable — pin clippy to stable so it doesn't resolve to the (clippy-less) nightly.
4851
- name: clippy
49-
run: cargo +stable clippy --all-targets -- -Dclippy::all -Dwarnings
52+
run: cargo +stable clippy --workspace --all-targets -- -Dclippy::all -Dwarnings
5053

5154
rust-test:
5255
runs-on: ubuntu-latest
@@ -56,4 +59,4 @@ jobs:
5659
with:
5760
toolchain: stable
5861
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2
59-
- run: cargo test --all-features --verbose
62+
- run: cargo test --workspace --all-features --verbose

CHANGELOG.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
## [Unreleased]
99

1010
### Added
11+
- **Standalone `shred-proxy` binary** (new workspace member `shred-proxy/`): a lightweight service
12+
that joins the DoubleZero `edge-solana-*` shred multicast feeds, deduplicates, and forwards a
13+
single copy of each shred to a local UDP port — meant to run next to a validator without the full
14+
bridge. It reuses the bridge library's shred forwarder directly (`doublezero_edge_connect::shred`:
15+
receiver, dedup/sigverify, parser, multicast plumbing); the only new code is active-group
16+
detection via the kernel routing table (`ip route get`, instead of the `doublezero` CLI), the
17+
reconciler, and the CLI. Installs via a one-liner —
18+
`curl -fsSL https://get.doublezero.xyz/shred-proxy | bash` — which downloads a prebuilt static
19+
binary published by the new `release.shred-proxy.yml` workflow (tag `shred-proxy-v*`) and installs
20+
it as a systemd service. The repo is now a Cargo workspace; the bridge's Docker build is scoped to
21+
`-p doublezero-edge-connect` so the image is unchanged.
22+
- Review hardening: PR CI (`rust.yml`) now builds/lints/tests the whole workspace (`--workspace`),
23+
not just the root crate, so the member is actually compiled and tested on every PR. The
24+
installer self-elevates with `sudo` (so a plain `curl … | bash` works for a non-root user) and
25+
a re-run now `restart`s the running service onto the upgraded binary; `DZ_*` config is documented
26+
to be set after the pipe (on the `bash` invocation), and the unit/env/sysctl files are fetched
27+
from the resolved release tag rather than a moving `main`. The reconciler is fail-open: a
28+
transient `ip route get` failure keeps the current activation instead of tearing forwarding down
29+
(was fail-empty), and one long-lived signal listener avoids dropping a SIGTERM between polls. The
30+
release workflow now runs the tests before publishing, asserts the artifact is statically linked,
31+
and validates the (dispatch) tag is namespaced. The shipped `RUST_LOG` example no longer silences
32+
the forwarder's `doublezero_edge_connect` info logs. Additional hardening: the installer now
33+
errors (rather than silently skipping) when `SHA256SUMS` is missing unless
34+
`SHRED_PROXY_SKIP_CHECKSUM=1`; the systemd unit disables the start-rate limiter
35+
(`StartLimitIntervalSec=0`, `RestartSec=5`) so an unattended service never latches `failed`; and
36+
the binary bails at startup when `--iface` is an IP in detection mode (which would never match a
37+
routing-table interface name). Detection distinguishes a candidate the kernel has no route to (a
38+
clean `ip route get` non-zero exit → treated as inactive) from a genuine probe failure
39+
(spawn/decode error → keep current), so a routeless unsubscribed candidate on a host with no
40+
default route can't stall activation. Release publishes pin a dispatch-created tag to the built
41+
commit (`--target`), and the shipped env example documents `sigverify`/`DZ_RPC_URL`.
1142
- **Per-tick win counters** `dz_quote_ticks_won_total{venue, publisher}` /
1243
`dz_depth_ticks_won_total{venue, publisher}` — the published win-rate primitive. Every
1344
`source_ts` tick counts exactly once, for the publisher class whose copy arrived first: a

Cargo.lock

Lines changed: 12 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,10 @@
1+
# The bridge crate is the workspace root; `shred-proxy/` is a lightweight member binary that reuses
2+
# this crate's library (the shred forwarder + multicast plumbing) as a path dependency. A single
3+
# workspace-level Cargo.lock keeps the `--locked` Docker build reproducible across both members.
4+
[workspace]
5+
members = [".", "shred-proxy"]
6+
resolver = "2"
7+
18
[package]
29
name = "doublezero-edge-connect"
310
version = "0.1.0"

Dockerfile

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,12 +31,18 @@ FROM rust:${RUST_VERSION}-bookworm AS build
3131
WORKDIR /src
3232
COPY Cargo.toml Cargo.lock ./
3333
COPY src ./src
34+
# The repo is a Cargo workspace: `shred-proxy/` is a sibling member crate. Cargo needs every
35+
# member's manifest present to resolve the workspace, so we copy only the member's manifest — not
36+
# its sources, which the scoped `-p doublezero-edge-connect` build never compiles. Leaving the
37+
# sources out keeps edits to shred-proxy from busting this image's build cache; the shred-proxy
38+
# binary is released separately.
39+
COPY shred-proxy/Cargo.toml ./shred-proxy/Cargo.toml
3440
# Cache the cargo registry and the target dir across builds (BuildKit cache mounts) for fast
3541
# rebuilds. The target dir lives in the cache mount (not in the image layer), so the binary
3642
# must be copied out within this same RUN before the mount goes away.
3743
RUN --mount=type=cache,target=/usr/local/cargo/registry \
3844
--mount=type=cache,target=/src/target \
39-
cargo build --release --locked \
45+
cargo build --release --locked -p doublezero-edge-connect \
4046
&& cp /src/target/release/doublezero-edge-connect /usr/local/bin/doublezero-edge-connect
4147

4248
FROM ${DZ_BASE_IMAGE}

README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -165,6 +165,18 @@ Any engine that speaks WebSocket + JSON consumes it with a thin (~50-100 line) a
165165
wire contract is in **[PROTOCOL.md](PROTOCOL.md)** (see
166166
[Consuming the feed](PROTOCOL.md#consuming-the-feed-any-engine)).
167167

168+
## Standalone shred-proxy
169+
170+
Shreds-only host and don't need the market-data bridge? The **[`shred-proxy/`](shred-proxy/)**
171+
workspace member is a lightweight service that joins the `edge-solana-*` shred feeds, deduplicates,
172+
and forwards to a local UDP port — no Docker, no `doublezero` CLI. Install it with its own one-liner:
173+
174+
```bash
175+
curl -fsSL https://get.doublezero.xyz/shred-proxy | bash
176+
```
177+
178+
It reuses this crate's shred forwarder directly; see **[shred-proxy/README.md](shred-proxy/README.md)**.
179+
168180
## Documentation
169181

170182
- **[docs/](docs/)** — operating reference:

shred-proxy/Cargo.toml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
[package]
2+
name = "shred-proxy"
3+
version = "0.1.0"
4+
edition = "2021"
5+
description = "Receives shreds from DoubleZero's edge-solana multicast feeds (active groups detected via the routing table) and forwards them deduplicated to a local UDP port"
6+
authors = ["Malbec Labs <dev@malbeclabs.com>"]
7+
license = "Apache-2.0"
8+
9+
[[bin]]
10+
name = "shred-proxy"
11+
path = "src/main.rs"
12+
13+
[dependencies]
14+
# The whole shred forwarder — receive, dedup/sigverify, fan-out, multicast bind, interface
15+
# resolution — is reused verbatim from the bridge crate's library. This binary only adds
16+
# routing-table active-group detection, the reconciler, and the CLI.
17+
doublezero-edge-connect = { path = ".." }
18+
tokio = { version = "1", features = ["full"] }
19+
anyhow = "1"
20+
clap = { version = "4", features = ["derive", "env"] }
21+
tracing = "0.1"
22+
tracing-subscriber = { version = "0.3", features = ["env-filter"] }

0 commit comments

Comments
 (0)