Commit d27b05e
authored
fix: hermes deployment fixes (env block, oauth gzip, mcp mount, branding) (#37)
* fix: hermes deployment fixes (env block, oauth gzip, mcp mount, branding)
Six fixes turned up while migrating a production deployment from
openclaw to hermes. Each had a tractable root cause and a real symptom
that broke the deploy.
- proxy/addon: OAuth response handler now decompresses gzip/br/deflate
before parsing the token JSON. The old code parsed raw \x1f\x8b
bytes as JSON, then panicked on the parse failure and took the
flow's response down with it. Reproduced live against
auth.openai.com which returns gzip by default. Wrapped the handler
in a deferred recover so future malformed responses surface as
errors instead of panics, and stripped Content-Encoding after
rewrite so the client does not try to gunzip plaintext.
- container/types: env injection no longer truncates the agent's env
file. Sluice writes its phantom tokens into a fenced
"BEGIN/END sluice-managed" block and replaces only that block on
each call. Foreign keys (set by `hermes claw migrate`, the agent's
own auth flow, or an operator) survive every injection. The
`fullReplace` flag is retained for API compatibility but no longer
affects file behavior. Keys are sorted for diff-friendly output.
- cmd/sluice: MCP /mcp endpoint mounts unconditionally, not only when
at least one upstream is registered. With zero upstreams the
gateway exposes an empty tool list, which is a valid MCP response.
This avoids the chicken-and-egg where an agent's MCP client gives
up on a 404 before the operator can register the first upstream.
- container/agent_profile: HermesProfile.WireMCPCmd now invokes
python3 through a sh wrapper that activates /opt/hermes/.venv when
present. PyYAML lives in the venv on the official Hermes Docker
image but is not on the system Python path; the previous bare
`python3 -c` failed with ModuleNotFoundError on every wire-up.
Native installs without the venv use the system python3.
- telegram/bot: agent name in approval messages is configurable via
SetAgentDisplayName; the OpenClaw label is no longer hardcoded.
Sluice's main wires this from --agent profile name at startup, so
Hermes deployments now read "Hermes wants to connect to..." in
Telegram.
- container/types: BuildEnvInjectionScript validates EnvFileRelPath
and escapes sed regex metacharacters on the marker comment so
future markers can be edited without breaking the script.
All six are covered by new unit tests.
* chore(deploy): switch repo compose/Caddy to hermes; add bootstrap.sh
The OpenClaw production compose lived in compose.yml at the repo root
since the original setup. Now that Hermes is the supported agent, the
canonical deploy files in this repo target Hermes too.
- compose.yml: new sluice (--agent hermes) + tun2proxy + hermes gateway
+ hermes-dashboard sidecar + caddy front. Sluice image pulled from
ghcr.io/nnemirovsky/sluice:latest. Hermes image pinned to
nousresearch/hermes-agent:v2026.4.30. HERMES_HOME is overridden to
/opt/data/.hermes so paths align with sluice's HermesProfile.
- compose.dev.yml: same shape but builds sluice from local source for
the dev loop. Drops caddy + dashboard sidecar.
- Caddyfile: cert paths moved off provider-specific /etc/cloudflare to
standard FHS /etc/ssl/certs/agent.pem and /etc/ssl/private/agent.key,
and reverse_proxy now targets the hermes-tun2proxy-1 dashboard port.
- bootstrap.sh: one-shot helper that imports an existing OpenClaw home
volume into the Hermes home volume via `hermes claw migrate`, then
pre-writes mcp_servers.sluice.url into ~/.hermes/config.yaml. Skips
cleanly when no legacy volume is found, so it is safe to invoke on
fresh installs too.
The old OpenClaw compose is gone. Operators still on OpenClaw can
recover it from any v0.12.x tag if needed; the migration path forward
is bootstrap.sh + the new compose.yml.
* fix: address copilot review on PR #37
- proxy/addon: snapshot body + Content-Encoding/Length/Transfer-Encoding
before decompressing the OAuth response. Restore the snapshot on any
error or panic after the decompress so a partial mid-rewrite cannot
send the client plaintext bytes with stripped encoding headers
(which, for token responses, would mean real tokens delivered with
no phantom swap). The flow now ends in one of two states: fully
swapped, or untouched.
- telegram/bot: HTML-escape agentDisplayName at render time. SetAgent-
DisplayName already filters empty values; the escape adds a second
line of defense so a future profile name containing '<', '&', etc
cannot break the message structure or inject markup.
- container/types: replace `sed '/BEGIN/,/END/d'` with an awk-based
conditional delete that runs only when both BEGIN and END markers
exist in the file. The old sed range would have happily deleted
from BEGIN to end-of-file when END was missing (truncated write,
manual edit), nuking foreign keys the marker design promises to
preserve. The awk script writes to a sibling temp file and renames
in place so a crash mid-rewrite leaves the original env file
intact. `bsdSed` is kept in the signature for API compatibility
but is no longer consulted (awk is portable across BSD and GNU).
- container/types: refresh the ContainerManager.InjectEnvVars doc
comment to describe the marker-block reconciliation semantics
instead of the old merge-vs-truncate split.
- container: rename TestHermesProfile_WireMCPUsesPython to
TestHermesProfile_WireMCPUsesVenvWrapper now that the assertion is
about the sh wrapper that activates the bundled venv.
* fix: address copilot review round 2 on PR #37
- proxy/addon: processAddonOAuthResponse now sets `modified` based on
a byte-equality check between the pre-swap and post-swap bodies. A
token endpoint that echoes already-phantom tokens (e.g. on a retry
after the upstream rotated and we already swapped on the previous
attempt) was logging "swapped to phantoms" with metrics bumps even
though zero bytes changed. The function now rolls back its decode
in the no-op case so the encoding/length headers continue to
advertise the original wire form.
- Caddyfile: reverse_proxy now targets the docker compose service
name `tun2proxy:9119` instead of the project-prefixed container
name. The dashboard sidecar shares tun2proxy's namespace, so the
service name reaches it on any internal network. Survives renames
of the compose project and replica index changes.
- bootstrap.sh: typo fix ("read read-only" -> "mounted read-only")
and BOOTSTRAP_OVERWRITE env var so a re-run can opt out of the
--overwrite flag and let migrate refuse on conflicts. Header
comment now describes the actual overwrite semantics instead of
claiming the script is unconditionally idempotent.
* fix: address copilot review round 3 on PR #37
- container/types: BuildEnvInjectionScript docstring no longer
references bsdSed/sed semantics. The implementation switched to awk
in round 1 of this PR; the doc comment lagged. Now describes the
awk-based block reconciliation accurately.
- container/types: env var values are now validated for newlines and
NUL bytes via the new validateEnvVarValue helper. The previous doc
claimed values were "single-quoted to prevent shell injection",
which is true at the shell-command layer but did not stop a value
with an embedded newline from splitting one logical entry into two
lines of the env file (the second line would either drop or be
parsed as a separate KEY=value when sourced). The escape comment
now describes the actual quoting model.
* fix: ssh proxy exit-status race + copilot round 4
Two findings, one squashed commit because they are interleaved by the
copilot review thread on this PR.
- proxy/ssh: sshHandleChannel previously called srcChan.CloseWrite from
the upstream→agent data-copy goroutine the moment it saw EOF on the
upstream channel. That CloseWrite (SSH_MSG_CHANNEL_EOF) raced the
request-forwarder goroutine writing exit-status on the same channel.
Depending on goroutine schedule, the agent could observe EOF and
channel-close BEFORE the exit-status request reached the wire, which
surfaces as `session.Wait` returning EOF instead of a clean exit.
TestCredential_SSHInjection reproduced this 10-20% of the time
locally and one in N runs in CI.
Fix: hold the agent-side stdout EOF until all three upstream→agent
goroutines (request, data, stderr) have drained, then issue
CloseWrite followed by Close. The agent now sees the documented
order: data, exit-status, EOF, close. Inputs from the agent (stdin
→ upstream) still get EOF'd as soon as the agent half-closes, so
upstream `cat`-style commands keep terminating correctly.
- e2e/credential_test: tightened the test SSH server to send
exit-status before half-closing the data side. This matches what
well-behaved SSH servers do and removes one cause of the same
ordering race on the server side.
- container/types: BuildEnvInjectionScriptForProfile now counts
non-empty values up front and skips the marker-block emission
entirely when nothing in envMap has content. The previous code
emitted a stray `BEGIN sluice-managed ... END sluice-managed` pair
with no entries when every value was empty (the per-call delete
semantic), contradicting the "skip when empty" comment.
* fix: address copilot review round 4 on PR #37
- container/types: env file values are now written as `KEY='value'`
with single-quoted values via a quoted-tag heredoc instead of
`KEY=value` (unquoted). The previous format relied on shell echo
quoting which prevented script-level injection but did NOT survive
re-sourcing the file. Production compose loads the env file with
`set -a; . file; set +a`, so an unquoted value containing `$`,
`$()`, backticks, spaces, or globs would have been re-expanded by
the shell at source time. Single quotes around values neutralize
every shell metacharacter; embedded single quotes are escaped via
the standard `'\''` idiom (close, escaped quote, reopen). New
round-trip test runs the generated script in a real shell, sources
the file, and confirms values like `pa$$word`, `it's`, `back\`tick\``
reach the environment unchanged.
- bootstrap.sh: HERMES_UID and HERMES_GID are now honored from env
vars (with the upstream-image defaults of 10000/10000) and applied
consistently to every chown across all docker run invocations. An
operator who pinned non-default UID/GID via compose.yml previously
ended up with migrated files owned by 10000:10000 and unreadable
to the runtime user.
- bootstrap.sh: the venv activation step is now guarded by a
`[ -f /opt/hermes/.venv/bin/activate ] && . ...` check that mirrors
the runtime wrapper in HermesProfile.WireMCPCmd. A custom Hermes
image without the venv now falls through to system python3 and
emits a clear error if PyYAML is missing, instead of failing on a
non-existent activate script.
- Caddyfile: comment now explicitly says the dashboard is reached
*via* tun2proxy because the dashboard sidecar uses
network_mode: "service:tun2proxy" and has no internal-network
identity of its own. Less likely to mislead a reader trying to
swap the proxy target.
* fix: address copilot review round 5 on PR #37
- container/types: refresh validateEnvVarValue and the
BuildEnvInjectionScript docstring to describe the quoted-tag
heredoc + KEY='value' format. The previous comments referenced
the old echo-based mechanism, and the example escape idiom was
rendered with a smart quote that did not match the actual code.
- container/types: harden the awk pre-pass that removes the
managed block. The old version triggered range-delete behavior
whenever both BEGIN and END markers appeared anywhere in the
file; if an operator manually shuffled the block (END before
BEGIN, or duplicated markers) it could nuke foreign keys after
an unmatched BEGIN. The new pass walks the file once, records
every well-formed BEGIN..END pair as a contiguous range to
delete, then prints every line not in the delete set.
Orphan markers stay verbatim so a corrupted file shows what is
wrong rather than silently dropping content.
- CLAUDE.md: env-block example now uses single-quoted values
(KEY1='phantom-value-1') matching the production format, and
documents the awk well-formed-pair semantic so operators
hand-editing the file know which marker layouts are honored.
- CLAUDE.md: the Hermes profile row in the Agent Profiles table
now describes the sh wrapper that activates Hermes' bundled
venv before exec'ing python3, instead of the old bare
`python3 -c <script>` form that no longer matches the code.
* fix: address copilot review round 6 on PR #37
- container/types: awkStringEscape no longer doubles backslashes.
In shell single-quoted strings, only the apostrophe needs the
break-out-and-rejoin escape; every other byte (backslash
included) is taken literally. Doubling backslashes would have
silently changed the marker text awk sees vs the bytes in the
file if EnvBlockBegin or EnvBlockEnd were ever edited to include
a backslash, breaking the pre-pass match.
- cmd/sluice: phase 3 (MCP wire-up) comments and log messages no
longer hard-code openclaw.json / mcp.servers.sluice.url. The
comment now describes the profile-specific dispatch (openclaw
uses the gateway RPC, hermes patches mcp_servers in
~/.hermes/config.yaml, future profiles plug their own
WireMCPCmd), and the success/error logs include the active
profile name so an operator reading the tail of `docker compose
logs sluice` sees which agent the wiring targeted.1 parent 9cd1a3e commit d27b05e
16 files changed
Lines changed: 1065 additions & 160 deletions
File tree
- cmd/sluice
- e2e
- internal
- container
- proxy
- telegram
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
125 | 125 | | |
126 | 126 | | |
127 | 127 | | |
128 | | - | |
| 128 | + | |
129 | 129 | | |
130 | 130 | | |
131 | 131 | | |
132 | 132 | | |
133 | 133 | | |
134 | 134 | | |
135 | | - | |
| 135 | + | |
136 | 136 | | |
137 | 137 | | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
138 | 153 | | |
139 | 154 | | |
140 | 155 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | | - | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
9 | 17 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
114 | 114 | | |
115 | 115 | | |
116 | 116 | | |
| 117 | + | |
117 | 118 | | |
118 | 119 | | |
119 | 120 | | |
| |||
461 | 462 | | |
462 | 463 | | |
463 | 464 | | |
464 | | - | |
465 | | - | |
466 | | - | |
467 | | - | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
468 | 471 | | |
469 | 472 | | |
470 | 473 | | |
471 | 474 | | |
472 | | - | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
473 | 478 | | |
474 | 479 | | |
475 | 480 | | |
| |||
546 | 551 | | |
547 | 552 | | |
548 | 553 | | |
549 | | - | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
550 | 558 | | |
551 | 559 | | |
552 | 560 | | |
| |||
597 | 605 | | |
598 | 606 | | |
599 | 607 | | |
600 | | - | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
601 | 614 | | |
602 | 615 | | |
603 | 616 | | |
604 | 617 | | |
605 | | - | |
| 618 | + | |
606 | 619 | | |
607 | | - | |
| 620 | + | |
608 | 621 | | |
609 | 622 | | |
610 | 623 | | |
| |||
913 | 926 | | |
914 | 927 | | |
915 | 928 | | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
| 937 | + | |
| 938 | + | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
| 945 | + | |
| 946 | + | |
916 | 947 | | |
917 | 948 | | |
918 | 949 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
3 | 8 | | |
4 | 9 | | |
5 | 10 | | |
6 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
7 | 22 | | |
8 | 23 | | |
9 | 24 | | |
10 | | - | |
| 25 | + | |
| 26 | + | |
11 | 27 | | |
12 | 28 | | |
13 | 29 | | |
| |||
35 | 51 | | |
36 | 52 | | |
37 | 53 | | |
38 | | - | |
39 | 54 | | |
40 | 55 | | |
41 | 56 | | |
| |||
49 | 64 | | |
50 | 65 | | |
51 | 66 | | |
52 | | - | |
53 | | - | |
| 67 | + | |
| 68 | + | |
54 | 69 | | |
55 | | - | |
| 70 | + | |
56 | 71 | | |
57 | | - | |
| 72 | + | |
| 73 | + | |
58 | 74 | | |
59 | | - | |
60 | | - | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
61 | 79 | | |
62 | 80 | | |
63 | 81 | | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | 82 | | |
68 | | - | |
| 83 | + | |
69 | 84 | | |
70 | 85 | | |
71 | 86 | | |
| |||
84 | 99 | | |
85 | 100 | | |
86 | 101 | | |
87 | | - | |
| 102 | + | |
0 commit comments