Skip to content

fix(ci): resolve macOS/bash 3.2 vitest failures on main#6137

Closed
prekshivyas wants to merge 8 commits into
mainfrom
fix/macos-vitest-failures
Closed

fix(ci): resolve macOS/bash 3.2 vitest failures on main#6137
prekshivyas wants to merge 8 commits into
mainfrom
fix/macos-vitest-failures

Conversation

@prekshivyas

@prekshivyas prekshivyas commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Five pre-existing test failures on the macos-vitest CI workflow (confirmed present before #6060) traced to bash 3.x incompatibilities, a macOS UTF-8 locale issue, and a missing fixture sync. This PR fixes all five root causes.

Related Issue

Investigation of CI run 28539883104; failures also present in run 28534214317 (before #6060), confirming #6060 is not the cause.

Changes

  • agents/hermes/start.sh — three bash 3.2 compatibility fixes:
    • Replace bash 4.1+ named-FD exec {var}<file with a { } < file grouped redirect; variables assigned inside {} remain in function scope
    • Replace mapfile -d '' -t (bash 4.x only) with while IFS= read -r -d "" elem; do arr+=("$elem"); done
    • Guard ${_HERMES_GUARD_TIMEOUT[@]} with ${arr[@]+"${arr[@]}"} so set -u does not abort the script when the array is empty (bash 3.2 treats empty [@] as unbound)
  • scripts/gateway-control.sh — add export LC_ALL=C so [a-f] character-class ranges in case patterns are byte-exact; macOS en_US.UTF-8 makes [a-f] case-insensitive, allowing uppercase hex nonces to pass the *[!0-9a-f]* check
  • scripts/lib/gateway-supervisor.sh — same LC_ALL=C fix for the sourced library's nonce validation path
  • test/gateway-supervisor-control.test.ts — pin NEMOCLAW_TEST_GATEWAY_CONTROL_CALLER_UID=0 in the nonce-rejection test so it does not depend on the CI runner's UID; filter macOS bash 3.2 SIGTERM job-notification lines from the stderr assertion
  • test/e2e/fixtures/redaction.ts — add tvly- Tavily token pattern missing since fix(policy): restore Tavily egress for managed Python #6134, fixing the e2e-redaction-parity Array(16) vs Array(17) mismatch

Type of Change

  • Code change (feature, bug fix, or refactor)

Quality Gates

  • Tests added or updated for changed behavior
  • Docs not applicable — justification: shell compatibility and test fixes, no user-facing behavior change
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) — gateway-control.sh and gateway-supervisor.sh handle nonce validation
  • Sensitive-path review completed or maintainer-approved waiver recorded — the LC_ALL=C fix tightens nonce validation (rejects uppercase hex on macOS that was previously accepted); gateway_control_stop_tracked_pid behavior is unchanged

Verification

  • Git hooks passed during commit and push, or npx prek run --from-ref main --to-ref HEAD passes
  • Targeted tests pass for changed behavior
    • test/gateway-supervisor-control.test.ts: 22/22 ✓
    • test/hermes-managed-exit-authorization.test.ts: all ✓ (was 8 failures before)
    • test/e2e/support/e2e-redaction-parity.test.ts: 3/3 ✓
    • test/hermes-gateway-supervisor-recovery.test.ts: 41/42 (1 local flake — PID 4242 alive on dev machine; unrelated to these changes, passes on CI fresh runners)
  • No secrets, API keys, or credentials committed

Remaining failures not addressed in this PR (different root class, need separate investigation):

  • install-preflight.test.ts — environment-specific
  • deepagents-code-tui-startup-check.test.ts — needs investigation
  • platform-parity-cloud-experimental.test.ts — needs investigation
  • WSL runtime-recovery-preload.test.ts, rebuild-config-hash.test.ts — different class of failure

Signed-off-by: Prekshi Vyas prekshiv@nvidia.com

Summary by CodeRabbit

  • Bug Fixes
    • Strengthened validation for Hermes endpoint settings, rejecting unsafe or malformed URLs and blocking private/internal destinations.
    • Prevented custom and user-loaded policy presets from including allowed_ips, with clearer failures when invalid presets are used.
    • Improved gateway nonce validation consistency across environments, reducing locale-related false accepts.
    • Expanded redaction to hide additional token formats in logs and test output.
    • Refined gateway control and supervisor handling for more reliable behavior on macOS and during process tracking.

prekshivyas and others added 8 commits June 30, 2026 19:01
… user presets

Fixes two SSRF gaps identified in issues #6072 and #6073.

1. src/lib/onboard/inference-providers/hermes.ts: call isPrivateHostname()
   on the user-supplied endpointUrl before passing it downstream as
   OPENAI_BASE_URL. Previously the URL bypassed all SSRF checks until
   plugin-side validateEndpointUrl() fired at inference time — too late
   to prevent credential dispatch to an internal host.

2. src/lib/policy/index.ts: reject user-supplied preset files that
   declare allowed_ips in any network_policies endpoint. The merge was
   previously a blind structural pass-through that let callers expand
   the private-IP allowlist OpenShell enforces.

Fixes #6072
Fixes #6073

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Codebase growth guardrail bans if statements in test files.
Replace requireValue mock body with a single expression.

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Drop the vi.mock factory — it contained if statements which the
codebase-growth guardrail forbids in test files. The real implementation
reads from nemoclaw-blueprint/private-networks.yaml and works correctly
in test context.

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace 'in' operator with Object.hasOwn to avoid matching inherited
prototype properties (prototype pollution guard, per PRA-2).

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tent bypass

- hermes.ts: reject non-http/https schemes and URLs with embedded
  credentials; redact raw URL from parse-error messages
- policy/index.ts: extract networkPoliciesHasAllowedIps helper and
  enforce it in applyPresetContent() when options.custom is set, closing
  the snapshot-replay bypass path (PRA-3 partial, PRA-4)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Five pre-existing failures on macos-vitest traced to bash 3.x
incompatibilities and a missing locale guard:

- agents/hermes/start.sh: replace bash 4.1+ named-FD exec ({var}<file)
  with a { } < file grouped redirect (bash 3.2 compatible); replace
  mapfile -d '' -t with while/read -r -d "" loop; guard
  ${_HERMES_GUARD_TIMEOUT[@]} with ${arr[@]+"${arr[@]}"} so set -u
  with an empty array does not abort the script
- scripts/gateway-control.sh, scripts/lib/gateway-supervisor.sh: export
  LC_ALL=C so [a-f] character-class ranges in case patterns are
  byte-exact — macOS en_US.UTF-8 treats [a-f] as case-insensitive,
  allowing uppercase hex nonces to bypass validation
- test/gateway-supervisor-control.test.ts: pin
  NEMOCLAW_TEST_GATEWAY_CONTROL_CALLER_UID=0 so the nonce-rejection test
  does not depend on the CI runner being root; suppress macOS bash 3.2
  SIGTERM job-notification lines in the stderr assertion
- test/e2e/fixtures/redaction.ts: add tvly- Tavily pattern missing since
  #6134, fixing e2e-redaction-parity Array(16) vs Array(17) mismatch

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Changes include Hermes shell script bash robustness fixes (array expansion, cmdline parsing, marker reads), locale-pinned nonce validation in gateway control scripts, a new SSRF endpoint-validation guard for the Hermes inference provider, a new policy guard rejecting allowed_ips in presets, and related test additions.

Changes

Shell Script Reliability Fixes

Layer / File(s) Summary
Hermes restart guard timeout expansion
agents/hermes/start.sh
Seal, unseal, and mutation-recovery guard calls use conditional _HERMES_GUARD_TIMEOUT array expansion instead of direct expansion.
Managed controller liveness cmdline parsing
agents/hermes/start.sh
/proc/<pid>/cmdline argv capture switches from mapfile -d '' to a NUL-delimited while read loop.
Gateway exit marker parsing refactor
agents/hermes/start.sh
Marker file reading drops the named-FD approach in favor of a grouped redirect block.
Locale-safe nonce validation
scripts/gateway-control.sh, scripts/lib/gateway-supervisor.sh
Both scripts export LC_ALL=C to make hex nonce character-class matching byte-exact.
Gateway supervisor control test stability
test/gateway-supervisor-control.test.ts
Tests add set +m, filter macOS job-control stderr noise, and set an explicit caller UID env var.

SSRF Guard for Hermes Provider Endpoint

Layer / File(s) Summary
Endpoint URL validation implementation
src/lib/onboard/inference-providers/hermes.ts
Validates endpointUrl scheme, rejects embedded credentials, and rejects private/internal hostnames via isPrivateHostname before provider setup.
SSRF guard test suite
src/lib/onboard/inference-providers/hermes.test.ts
New tests with a makeDeps helper cover rejection/acceptance cases and verify runOpenshell invocation behavior.

Policy allowed_ips Rejection Guard

Layer / File(s) Summary
allowed_ips detection and enforcement
src/lib/policy/index.ts
Adds networkPoliciesHasAllowedIps and enforces rejection in applyPresetContent (custom presets) and loadPresetFromFile.
allowed_ips guard unit tests
src/lib/policy/preset-allowed-ips.test.ts
New tests cover rejection/acceptance scenarios using temporary YAML preset files.
Live e2e negative-path test and redaction fixture
test/e2e/live/onboard-negative-paths.test.ts, test/e2e/fixtures/redaction.ts
New live test asserts CLI rejection of a malicious preset with allowed_ips; redaction fixture adds a Tavily token pattern.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Suggested labels: platform: macos, bug-fix

Suggested reviewers: ericksoa, jyaunches, cv

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning This PR does not restore Tavily egress for /opt/venv/bin/python3* or align the Tavily preset/profile allowlists required by #6134. Add /opt/venv/bin/python3* to both the tavily preset and provider-profile allowlists, and extend live tests to verify managed Python is allowed while system/project Python stays blocked.
Out of Scope Changes check ⚠️ Warning Most changes are macOS/bash compatibility and test-fixture work unrelated to #6134's Tavily allowlist restoration. Limit this PR to the Tavily preset/profile update and its live coverage, or move the CI, SSRF, and redaction changes into separate PRs.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is specific and matches the macOS/bash CI compatibility fixes in this PR.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/macos-vitest-failures

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/inference-providers/hermes.test.ts`:
- Around line 127-141: The malformed URL test in setupHermesProviderInference
only checks for the expected “valid URL” text and can still pass if the raw
endpointUrl leaks in the error message. Update the hermes.test.ts case for
setupHermesProviderInference to capture the thrown error message and assert both
that it matches the expected validation text and that it does not contain the
input value “not-a-url”. Keep the assertion at the public boundary by inspecting
the rejected error message rather than internals.

In `@src/lib/onboard/inference-providers/hermes.ts`:
- Around line 49-52: The Hermes endpoint validation in
ensureHermesProvider*Credentials() only checks isPrivateHostname() on
parsedEndpoint.hostname, so a public hostname that later resolves to private
space can still be accepted. Add the same DNS-pinning or isPrivateIp()
resolution check used in src/lib/sandbox/config.ts before persisting endpointUrl
as baseUrl, and keep the existing private-hostname rejection in Hermes as a
first-pass guard.

In `@test/e2e/live/onboard-negative-paths.test.ts`:
- Around line 185-187: The E2E assertion in the negative-path test is too broad
because `resultText(result)` is allowed to match a generic permission failure,
which can mask an unrelated error. Tighten the check in this test so the failure
message must include the `allowed_ips` guard wording (or the exact rejection
text from the preset validation path) while still asserting a non-zero exit
code. Update the assertion near `resultText`/`expect(...)` to verify the
specific denial path this scenario is meant to exercise.

In `@test/gateway-supervisor-control.test.ts`:
- Around line 176-178: The stderr cleanup in the test is too broad and can hide
real failures; tighten the filter used in the `result.stderr.replace(...)`
assertion so it only removes the specific macOS bash 3.2 job-control notice
shape. Update the regex in `test/gateway-supervisor-control.test.ts` to match
the leading signal-name plus signal-number form emitted by that noise, and keep
the `expect(...).toBe("")` check strict so unrelated stderr like genuine
`Killed` errors still fail.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1b8c1ca6-0385-4954-ad91-160ce31c70ff

📥 Commits

Reviewing files that changed from the base of the PR and between f6fac71 and 9c0f45c.

📒 Files selected for processing (10)
  • agents/hermes/start.sh
  • scripts/gateway-control.sh
  • scripts/lib/gateway-supervisor.sh
  • src/lib/onboard/inference-providers/hermes.test.ts
  • src/lib/onboard/inference-providers/hermes.ts
  • src/lib/policy/index.ts
  • src/lib/policy/preset-allowed-ips.test.ts
  • test/e2e/fixtures/redaction.ts
  • test/e2e/live/onboard-negative-paths.test.ts
  • test/gateway-supervisor-control.test.ts

Comment on lines +127 to +141
it("throws on malformed URL without leaking the raw value", async () => {
await expect(
setupHermesProviderInference(
{
sandboxName: "alpha",
model: "m",
provider: "p",
endpointUrl: "not-a-url",
credentialEnv: null,
hermesAuthMethod: null,
hermesToolGateways: [],
},
makeDeps() as never,
),
).rejects.toThrow(/valid URL/);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Assert the “no raw value leak” claim directly.

The test name says not-a-url must not leak, but Line 141 would still pass if the thrown message included the raw endpoint. Capture the error message and assert both the expected text and absence of the input value.

Suggested test tightening
   it("throws on malformed URL without leaking the raw value", async () => {
-    await expect(
-      setupHermesProviderInference(
+    let thrown: unknown;
+    try {
+      await setupHermesProviderInference(
         {
           sandboxName: "alpha",
           model: "m",
           provider: "p",
@@
           hermesToolGateways: [],
         },
         makeDeps() as never,
-      ),
-    ).rejects.toThrow(/valid URL/);
+      );
+    } catch (err) {
+      thrown = err;
+    }
+    expect(thrown).toBeInstanceOf(Error);
+    expect((thrown as Error).message).toMatch(/valid URL/);
+    expect((thrown as Error).message).not.toContain("not-a-url");
   });

As per path instructions, “Prefer observable outcomes through the public boundary over source-text, private-shape, or mock-call assertions.”

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
it("throws on malformed URL without leaking the raw value", async () => {
await expect(
setupHermesProviderInference(
{
sandboxName: "alpha",
model: "m",
provider: "p",
endpointUrl: "not-a-url",
credentialEnv: null,
hermesAuthMethod: null,
hermesToolGateways: [],
},
makeDeps() as never,
),
).rejects.toThrow(/valid URL/);
it("throws on malformed URL without leaking the raw value", async () => {
let thrown: unknown;
try {
await setupHermesProviderInference(
{
sandboxName: "alpha",
model: "m",
provider: "p",
endpointUrl: "not-a-url",
credentialEnv: null,
hermesAuthMethod: null,
hermesToolGateways: [],
},
makeDeps() as never,
);
} catch (err) {
thrown = err;
}
expect(thrown).toBeInstanceOf(Error);
expect((thrown as Error).message).toMatch(/valid URL/);
expect((thrown as Error).message).not.toContain("not-a-url");
});
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/inference-providers/hermes.test.ts` around lines 127 - 141,
The malformed URL test in setupHermesProviderInference only checks for the
expected “valid URL” text and can still pass if the raw endpointUrl leaks in the
error message. Update the hermes.test.ts case for setupHermesProviderInference
to capture the thrown error message and assert both that it matches the expected
validation text and that it does not contain the input value “not-a-url”. Keep
the assertion at the public boundary by inspecting the rejected error message
rather than internals.

Source: Path instructions

Comment on lines +49 to +52
if (isPrivateHostname(parsedEndpoint.hostname)) {
throw new Error(
`Inference endpoint URL points to a private or internal address "${parsedEndpoint.hostname}". Use a public endpoint.`,
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Inspect the downstream baseUrl consumers and private-network helpers to place
# resolved-address validation at the actual network boundary.
rg -n -C4 '\bbaseUrl\b|ensureHermesProvider(ApiKey|OAuth)Credentials|isPrivateHostname|isPrivateIp' src agents

Repository: NVIDIA/NemoClaw

Length of output: 50372


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n## hermes.ts\n'
sed -n '1,180p' src/lib/onboard/inference-providers/hermes.ts

printf '\n## hermes-provider-auth.ts\n'
sed -n '1,260p' src/lib/hermes-provider-auth.ts

printf '\n## private-networks.ts\n'
sed -n '1,240p' src/lib/private-networks.ts

printf '\n## sandbox/config.ts relevant section\n'
sed -n '620,710p' src/lib/sandbox/config.ts

printf '\n## look for DNS resolution / endpoint validation call sites\n'
rg -n -C3 'validateEndpointUrl|lookup|resolve.*hostname|dns\.|isPrivateHostname|isPrivateIp|baseUrl: endpointUrl|endpointUrl' src/lib/onboard src/lib/sandbox src/lib/hermes-provider-auth.ts

Repository: NVIDIA/NemoClaw

Length of output: 50372


Block DNS-rebinding on Hermes endpoints. isPrivateHostname() only rejects literal private/internal names here. endpointUrl is forwarded unchanged into ensureHermesProvider*Credentials(...) and persisted as baseUrl, so a public hostname that resolves to private space can still slip through. Reuse the DNS-pinning/isPrivateIp() check from src/lib/sandbox/config.ts before storing the URL.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/inference-providers/hermes.ts` around lines 49 - 52, The
Hermes endpoint validation in ensureHermesProvider*Credentials() only checks
isPrivateHostname() on parsedEndpoint.hostname, so a public hostname that later
resolves to private space can still be accepted. Add the same DNS-pinning or
isPrivateIp() resolution check used in src/lib/sandbox/config.ts before
persisting endpointUrl as baseUrl, and keep the existing private-hostname
rejection in Hermes as a first-pass guard.

Comment on lines +185 to +187
const text = resultText(result);
expect(result.exitCode, text).not.toBe(0);
expect(text).toMatch(/allowed_ips|not permitted/i);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Assert the guard-specific rejection text.

Line 187 accepts any generic “not permitted” failure, so this can pass on an unrelated permission error before preset validation runs. Please require the allowed_ips wording (or the full rejection message) so the E2E proves this new denial path is what fired. As per path instructions, tests under **/*.test.{ts,js,mts,mjs,cts,cjs} should provide “behavioral confidence rather than implementation lock-in,” and test/e2e/** should preserve the real boundary under test.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/live/onboard-negative-paths.test.ts` around lines 185 - 187, The E2E
assertion in the negative-path test is too broad because `resultText(result)` is
allowed to match a generic permission failure, which can mask an unrelated
error. Tighten the check in this test so the failure message must include the
`allowed_ips` guard wording (or the exact rejection text from the preset
validation path) while still asserting a non-zero exit code. Update the
assertion near `resultText`/`expect(...)` to verify the specific denial path
this scenario is meant to exercise.

Source: Path instructions

Comment on lines +176 to +178
// macOS bash 3.2 reports SIGTERM job-control notifications to stderr
// (e.g. "Terminated: 15 sleep 30") despite set +m; filter them out.
expect(result.stderr.replace(/^[^\n]*(?:Terminated|Killed)[^\n]*\n?/gm, "")).toBe("");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Tighten the stderr filter so it can't mask unrelated failures.

The current pattern strips any stderr line containing Terminated or Killed, which would also swallow a genuine error unrelated to bash job-control noise (e.g., an OOM Killed from a different failure), weakening the "stderr is empty" assertion. Anchor the filter to the specific job-control notice shape (leading signal-name + signal number) so only the macOS 3.2 noise is removed.

💚 Suggested tighter filter
-    expect(result.stderr.replace(/^[^\n]*(?:Terminated|Killed)[^\n]*\n?/gm, "")).toBe("");
+    // Only strip bash job-control notices of the form "Terminated: 15  <cmd>".
+    expect(result.stderr.replace(/^(?:Terminated|Killed):\s*\d+\b[^\n]*\n?/gm, "")).toBe("");

As per path instructions: "Flag ... conditionals that make a test pass without exercising its claim."

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// macOS bash 3.2 reports SIGTERM job-control notifications to stderr
// (e.g. "Terminated: 15 sleep 30") despite set +m; filter them out.
expect(result.stderr.replace(/^[^\n]*(?:Terminated|Killed)[^\n]*\n?/gm, "")).toBe("");
// macOS bash 3.2 reports SIGTERM job-control notifications to stderr
// (e.g. "Terminated: 15 sleep 30") despite set +m; filter them out.
// Only strip bash job-control notices of the form "Terminated: 15 <cmd>".
expect(result.stderr.replace(/^(?:Terminated|Killed):\s*\d+\b[^\n]*\n?/gm, "")).toBe("");
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/gateway-supervisor-control.test.ts` around lines 176 - 178, The stderr
cleanup in the test is too broad and can hide real failures; tighten the filter
used in the `result.stderr.replace(...)` assertion so it only removes the
specific macOS bash 3.2 job-control notice shape. Update the regex in
`test/gateway-supervisor-control.test.ts` to match the leading signal-name plus
signal-number form emitted by that noise, and keep the `expect(...).toBe("")`
check strict so unrelated stderr like genuine `Killed` errors still fail.

Source: Path instructions

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: hermes-e2e, gateway-guard-recovery, network-policy, onboard-negative-paths
Optional E2E: inference-routing, macos-e2e, hermes-sandbox-secret-boundary

Dispatch hint: hermes-e2e,gateway-guard-recovery,network-policy,onboard-negative-paths

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • hermes-e2e (high): Required because Hermes start.sh lifecycle/security changes touch real gateway restart, recover, managed supervisor, config hash, forward recovery, and secret-boundary behavior. This job also proves the changed Hermes provider/onboard path still works with a valid public endpoint and live inference.
  • gateway-guard-recovery (high): Required because scripts/gateway-control.sh and scripts/lib/gateway-supervisor.sh are privileged PID 1 control primitives. This live job exercises the authenticated supervisor recovery route against a real OpenShell sandbox after process/guard disruption.
  • network-policy (high): Required because src/lib/policy/index.ts changes preset parsing/application for a network-policy security boundary. Run the live allow/deny policy job to ensure normal policy merge, policy-add, and sandbox egress behavior still work.
  • onboard-negative-paths (medium): Required because this PR adds a live negative CLI contract for policy-add --from-file rejecting user-supplied allowed_ips. It directly covers the new fail-closed preset path without needing a successful sandbox onboard.

Optional E2E

  • inference-routing (medium): Useful adjacent confidence for endpoint URL handling and inference-route failure classification, although it is not Hermes-provider-specific.
  • macos-e2e (medium): Useful because several shell changes are explicitly bash 3.2/macOS compatibility fixes. This validates the repo build and available live flow on macOS, but it is adjacent rather than the main Hermes lifecycle boundary.
  • hermes-sandbox-secret-boundary (high): Additional confidence for Hermes image/startup secret isolation after changes near Hermes restart secret-boundary validation, if CI capacity permits.

New E2E recommendations

  • hermes-provider-endpoint-ssrf-guard (high): The new Hermes provider private/internal endpoint guard is currently covered by unit tests, but there is no live CLI/onboard E2E that proves a loopback or metadata endpoint is rejected before OpenShell/provider mutation and without leaking sensitive URL material.

Dispatch hint

  • Workflow: .github/workflows/e2e.yaml
  • jobs input: hermes-e2e,gateway-guard-recovery,network-policy,onboard-negative-paths

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

E2E Target Recommendation

Required E2E targets: onboard-negative-paths, e2e-all
Optional E2E targets: None

Dispatch required E2E targets:

  • gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=onboard-negative-paths
  • gh workflow run e2e.yaml --ref <pr-head-ref>

Workflow run

Full E2E target advisor summary

E2E Target Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E targets

  • onboard-negative-paths: Focused free-standing E2E job wired for changed live test test/e2e/live/onboard-negative-paths.test.ts.
    • Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=onboard-negative-paths
  • e2e-all: Shared E2E fixture redaction changed, which can affect every live target's child-process output and evidence handling. The PR also changes a wired free-standing live test, onboard-negative-paths.test.ts, plus onboarding/policy/gateway surfaces exercised by E2E, so run the canonical full E2E fan-out.
    • Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref>

Optional E2E targets

  • None.

Relevant changed files

  • agents/hermes/start.sh
  • scripts/gateway-control.sh
  • scripts/lib/gateway-supervisor.sh
  • src/lib/onboard/inference-providers/hermes.ts
  • src/lib/policy/index.ts
  • test/e2e/fixtures/redaction.ts
  • test/e2e/live/onboard-negative-paths.test.ts

@github-code-quality

github-code-quality Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in the branch is 96%. Coverage data for the branch is not yet available.

Show a code coverage summary of the most covered files.
File 9c0f45c +/-
nemoclaw/src/se...cret-scanner.ts 100%
nemoclaw/src/commands/slash.ts 100%
nemoclaw/src/li...bprocess-env.ts 100%
nemoclaw/src/bl...eprint/state.ts 98%
nemoclaw/src/onboard/config.ts 98%
nemoclaw/src/bl...int/snapshot.ts 97%
nemoclaw/src/bl...print/runner.ts 95%
nemoclaw/src/co...ration-state.ts 94%
nemoclaw/src/bl...ate-networks.ts 94%
nemoclaw/src/index.ts 94%

TypeScript / code-coverage/cli

The overall coverage in the branch is 68%. Coverage data for the branch is not yet available.

Show a code coverage summary of the most covered files.
File 9c0f45c +/-
src/lib/actions...dbox/rebuild.ts 80%
src/lib/actions...all/run-plan.ts 80%
src/lib/state/o...oard-session.ts 79%
src/lib/state/sandbox.ts 72%
src/lib/shields/index.ts 69%
src/lib/onboard/preflight.ts 69%
src/lib/onboard...er-gpu-patch.ts 59%
src/lib/actions...licy-channel.ts 58%
src/lib/policy/index.ts 55%
src/lib/onboard.ts 20%

Updated July 01, 2026 20:24 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Blocked

Merge posture: Do not merge until addressed
Primary next action: Fix PRA-3: Hermes endpoint validation still allows DNS-resolved private/internal destinations; then add or justify PRA-T1.
Open items: 1 required · 5 warnings · 0 suggestions · 8 test follow-ups
Top item: Hermes endpoint validation must reject hostnames resolving to private/internal IPs

Action checklist

  • PRA-3 Fix: Hermes endpoint validation still allows DNS-resolved private/internal destinations in src/lib/onboard/inference-providers/hermes.ts:36
  • PRA-1 Resolve or justify: Source-of-truth review needed: Hermes inference endpoint SSRF validation
  • PRA-2 Resolve or justify: Source-of-truth review needed: Custom policy preset `allowed_ips` rejection
  • PRA-4 Resolve or justify: Custom preset replay should fail closed when `network_policies` cannot be parsed in src/lib/policy/index.ts:783
  • PRA-5 Resolve or justify: This PR overlaps active security work and its title/summary understate the security-scope changes
  • PRA-6 Resolve or justify: Policy hotspot grows instead of extracting the new custom-preset validator in src/lib/policy/index.ts:111
  • PRA-T1 Add or justify test follow-up: Runtime validation
  • PRA-T2 Add or justify test follow-up: Runtime validation
  • PRA-T3 Add or justify test follow-up: Runtime validation
  • PRA-T4 Add or justify test follow-up: Runtime validation
  • PRA-T5 Add or justify test follow-up: Runtime validation
  • PRA-T6 Add or justify test follow-up: Acceptance clause
  • PRA-T7 Add or justify test follow-up: Acceptance clause
  • PRA-T8 Add or justify test follow-up: Acceptance clause

Findings index

ID Severity Category Location Required action
PRA-1 Resolve/justify architecture Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-2 Resolve/justify architecture Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-3 Required security src/lib/onboard/inference-providers/hermes.ts:36 Reuse the existing DNS-aware URL validation/pinning source of truth, or add an equivalent dependency-injected DNS validation step for Hermes endpoint setup before passing `baseUrl` to Hermes credential/provider registration. Preserve the existing literal-IP/name checks, scheme checks, and credential rejection.
PRA-4 Resolve/justify security src/lib/policy/index.ts:783 For `options.custom`, fail closed when `parseNetworkPolicies(presetContent)` returns `null` before extracting/merging entries, or otherwise parse once and pass the validated structured policy through the merge path so the custom trust boundary has a single authoritative parser.
PRA-5 Resolve/justify scope Either narrow this PR to the macOS/bash/redaction fixes or explicitly split/rename the security changes and coordinate them with the overlapping security PR so the Hermes endpoint and custom preset fixes have one reviewed source of truth.
PRA-6 Resolve/justify architecture src/lib/policy/index.ts:111 Move the custom preset structural validator into a small policy-validation helper module or offset the growth by extracting existing preset validation helpers from `index.ts`, while keeping both `loadPresetFromFile` and `applyPresetContent` using the same validator.

🚨 Required before merge

Address these before merging unless a maintainer explicitly overrides the advisor with rationale.

PRA-3 Required — Hermes endpoint validation still allows DNS-resolved private/internal destinations

  • Location: src/lib/onboard/inference-providers/hermes.ts:36
  • Category: security
  • Problem: The new guard parses the endpoint URL, rejects non-http(s), embedded credentials, and literal private/internal hostnames via `isPrivateHostname(parsedEndpoint.hostname)`, but it never resolves the hostname or pins the result. The repository already has a stronger source-of-truth pattern in `src/lib/sandbox/config.ts` (`validateUrlValueWithDnsResult` / `rewriteConfigUrlsWithDnsPinning`) that rejects public-looking hostnames resolving to private/internal IPs. As written, a public-looking attacker-controlled hostname that resolves to `169.254.169.254`, `127.0.0.1`, or RFC1918 space passes this guard and can later be dialed as the Hermes inference base URL.
  • Impact: A malicious or compromised endpoint setting can bypass the intended SSRF fix and cause Hermes/OpenShell to connect to metadata, loopback, or internal services while carrying inference credentials or trusted sandbox network context.
  • Required action: Reuse the existing DNS-aware URL validation/pinning source of truth, or add an equivalent dependency-injected DNS validation step for Hermes endpoint setup before passing `baseUrl` to Hermes credential/provider registration. Preserve the existing literal-IP/name checks, scheme checks, and credential rejection.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Read `src/lib/onboard/inference-providers/hermes.ts` around the new `new URL(endpointUrl)` block and compare it with `src/lib/sandbox/config.ts` around `validateUrlValueWithDnsResult`; the Hermes path has no `dns.lookup`, `isPrivateIp(address)`, or `rewriteConfigUrlsWithDnsPinning` equivalent.
  • Missing regression test: Add a unit test named like `setupHermesProviderInference rejects a public-looking hostname whose DNS lookup returns 169.254.169.254 and does not call runOpenshell`, plus a public-resolution success case if HTTP pinning/HTTPS preservation is implemented.
  • Done when: The required change is committed and verification passes: Read `src/lib/onboard/inference-providers/hermes.ts` around the new `new URL(endpointUrl)` block and compare it with `src/lib/sandbox/config.ts` around `validateUrlValueWithDnsResult`; the Hermes path has no `dns.lookup`, `isPrivateIp(address)`, or `rewriteConfigUrlsWithDnsPinning` equivalent.
  • Evidence: `hermes.ts` calls only `isPrivateHostname(parsedEndpoint.hostname)`. Existing tests cover literal `127.0.0.1`, `169.254.169.254`, `10.0.0.1`, `localhost`, and `.internal`, but no hostname whose DNS result is private/internal.
Review findings by urgency: 1 required fix, 5 items to resolve/justify, 0 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-1 Resolve/justify — Source-of-truth review needed: Hermes inference endpoint SSRF validation

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as missing.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: Missing: a Hermes setup test with an injected or reused lookup where `public.example` resolves to `169.254.169.254` and setup rejects before `runOpenshell`.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `hermes.ts` calls `isPrivateHostname(parsedEndpoint.hostname)` only; `sandbox/config.ts` already validates DNS answers with `isPrivateIp(address)`.

PRA-2 Resolve/justify — Source-of-truth review needed: Custom policy preset `allowed_ips` rejection

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: Missing: `applyPresetContent custom preset fails closed when network_policies cannot be parsed instead of falling through to textBasedMerge`.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `applyPresetContent` checks `if (np && networkPoliciesHasAllowedIps(np))` and then proceeds to `extractPresetEntries`; `mergePresetIntoPolicy` retains `textBasedMerge` for unparseable preset entries.

PRA-4 Resolve/justify — Custom preset replay should fail closed when `network_policies` cannot be parsed

  • Location: src/lib/policy/index.ts:783
  • Category: security
  • Problem: `applyPresetContent` blocks custom presets with `allowed_ips` only when `parseNetworkPolicies(presetContent)` returns an object. If parsing returns `null`, the function continues to `extractPresetEntries` and `mergePresetIntoPolicy`, whose text-based fallback intentionally tolerates unparseable preset entries. Normal `--from-file` input is parsed first by `loadPresetFromFile`, but `applyPresetContent` is exported and also used for custom policy replay from snapshot/registry state, so this defensive guard should not silently skip validation when the custom content cannot be parsed structurally.
  • Impact: A tampered or legacy custom policy payload could reach the merge/apply path without the new `allowed_ips` check proving that no endpoint allowlist was present, weakening the intended policy-bypass fix at a replay/direct-call boundary.
  • Recommended action: For `options.custom`, fail closed when `parseNetworkPolicies(presetContent)` returns `null` before extracting/merging entries, or otherwise parse once and pass the validated structured policy through the merge path so the custom trust boundary has a single authoritative parser.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read `src/lib/policy/index.ts` lines around `if (options.custom) { const np = parseNetworkPolicies(presetContent); if (np && networkPoliciesHasAllowedIps(np)) ... }` and then `mergePresetIntoPolicy`, which falls back to `textBasedMerge` when wrapped preset entries are not a mergeable YAML object.
  • Missing regression test: Add a test named like `applyPresetContent custom preset fails closed when network_policies cannot be parsed instead of falling through to textBasedMerge`, asserting that no policy set command or registry update occurs.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read `src/lib/policy/index.ts` lines around `if (options.custom) { const np = parseNetworkPolicies(presetContent); if (np && networkPoliciesHasAllowedIps(np)) ... }` and then `mergePresetIntoPolicy`, which falls back to `textBasedMerge` when wrapped preset entries are not a mergeable YAML object.
  • Evidence: `loadPresetFromFile` rejects invalid YAML, but `applyPresetContent` is separately exported and called from `actions/sandbox/snapshot.ts` for custom policy reconciliation; its custom guard currently treats an unparseable `network_policies` section as if the `allowed_ips` check were not applicable.

PRA-5 Resolve/justify — This PR overlaps active security work and its title/summary understate the security-scope changes

  • Location: not file-specific
  • Category: scope
  • Problem: The deterministic drift context shows open PR fix(security): validate Hermes endpoint URL and reject allowed_ips in user presets #6087 with the same security title and overlapping core files (`src/lib/onboard/inference-providers/hermes.ts`, `src/lib/policy/index.ts`, and related tests). This PR is titled as a macOS/bash 3.2 Vitest fix, but the diff also changes Hermes SSRF validation and custom policy preset trust boundaries.
  • Impact: Maintainers may review this as a CI compatibility patch and miss that it changes active security-sensitive code paths already being handled in overlapping work, increasing the risk of contradictory or incomplete security fixes landing.
  • Recommended action: Either narrow this PR to the macOS/bash/redaction fixes or explicitly split/rename the security changes and coordinate them with the overlapping security PR so the Hermes endpoint and custom preset fixes have one reviewed source of truth.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Compare the changed-file list in this PR with the drift-context overlap for PR fix(security): validate Hermes endpoint URL and reject allowed_ips in user presets #6087; the same Hermes and policy files are modified here.
  • Missing regression test: No automated test can prove PR-scope coordination; the concrete coverage to keep if split is the Hermes endpoint DNS-private rejection test and the custom `allowed_ips` rejection tests in their security-focused PR.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Compare the changed-file list in this PR with the drift-context overlap for PR fix(security): validate Hermes endpoint URL and reject allowed_ips in user presets #6087; the same Hermes and policy files are modified here.
  • Evidence: Drift context reports PR fix(security): validate Hermes endpoint URL and reject allowed_ips in user presets #6087, `fix(security): validate Hermes endpoint URL and reject allowed_ips in user presets`, with the same Hermes/policy files; this PR title is `fix(ci): resolve macOS/bash 3.2 vitest failures on main`.

PRA-6 Resolve/justify — Policy hotspot grows instead of extracting the new custom-preset validator

  • Location: src/lib/policy/index.ts:111
  • Category: architecture
  • Problem: `src/lib/policy/index.ts` is already a large policy hotspot and grows by 29 lines for the new `networkPoliciesHasAllowedIps` custom-preset validation. The validation itself is security-critical and should remain, but adding more policy-shape logic to the monolith increases review and maintenance risk.
  • Impact: Future policy trust-boundary changes become harder to audit, and security validators can be missed among unrelated CLI, merge, registry, and prompt logic.
  • Recommended action: Move the custom preset structural validator into a small policy-validation helper module or offset the growth by extracting existing preset validation helpers from `index.ts`, while keeping both `loadPresetFromFile` and `applyPresetContent` using the same validator.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read `src/lib/policy/index.ts` around `networkPoliciesHasAllowedIps`, `loadPresetFromFile`, and `applyPresetContent`; all validation, merging, CLI helper, registry, and prompt code remains in the same 1360-line file.
  • Missing regression test: Existing tests in `src/lib/policy/preset-allowed-ips.test.ts` should continue to pass after extraction; add an import-level test only if the extracted helper becomes public.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read `src/lib/policy/index.ts` around `networkPoliciesHasAllowedIps`, `loadPresetFromFile`, and `applyPresetContent`; all validation, merging, CLI helper, registry, and prompt code remains in the same 1360-line file.
  • Evidence: Drift context reports `src/lib/policy/index.ts` base 1331 lines, head 1360 lines, delta +29, and flags it as a large-file hotspot.

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

  • None.
Simplification opportunities: 1 possible cut

These are safe simplification checks only. Do not remove validation, security controls, data-loss prevention, or required tests.

  • PRA-6 shrink (src/lib/policy/index.ts:111): Cut the new custom-preset structural validation logic out of the policy monolith.
    • Replacement: Create a cohesive helper such as `src/lib/policy/custom-preset-validation.ts` and call it from both `loadPresetFromFile` and `applyPresetContent`.
    • Safety boundary: Do not remove or weaken the `allowed_ips` rejection; the extracted helper must remain fail-closed for user-supplied/custom preset content.
Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

  • PRA-T1 Runtime validation — setupHermesProviderInference rejects a public-looking hostname whose DNS lookup returns 169.254.169.254 and does not call runOpenshell. Changed code touches runtime/sandbox/infrastructure paths (`agents/hermes/start.sh`, gateway control scripts), onboarding credential/network trust boundaries, and policy application. The added unit tests cover several negative paths, but DNS-based SSRF and fail-closed custom replay behavior are not covered.
  • PRA-T2 Runtime validation — setupHermesProviderInference accepts a public hostname only after DNS validation succeeds and preserves HTTPS hostname or pins HTTP consistently with the shared validator. Changed code touches runtime/sandbox/infrastructure paths (`agents/hermes/start.sh`, gateway control scripts), onboarding credential/network trust boundaries, and policy application. The added unit tests cover several negative paths, but DNS-based SSRF and fail-closed custom replay behavior are not covered.
  • PRA-T3 Runtime validation — applyPresetContent custom preset fails closed when network_policies cannot be parsed instead of falling through to textBasedMerge. Changed code touches runtime/sandbox/infrastructure paths (`agents/hermes/start.sh`, gateway control scripts), onboarding credential/network trust boundaries, and policy application. The added unit tests cover several negative paths, but DNS-based SSRF and fail-closed custom replay behavior are not covered.
  • PRA-T4 Runtime validation — policy-add --from-dir aborts before applying later files when one custom preset contains allowed_ips. Changed code touches runtime/sandbox/infrastructure paths (`agents/hermes/start.sh`, gateway control scripts), onboarding credential/network trust boundaries, and policy application. The added unit tests cover several negative paths, but DNS-based SSRF and fail-closed custom replay behavior are not covered.
  • PRA-T5 Runtime validation — Hermes restart seal guard calls do not abort under bash 3.2 when _HERMES_GUARD_TIMEOUT is empty or unset. Changed code touches runtime/sandbox/infrastructure paths (`agents/hermes/start.sh`, gateway control scripts), onboarding credential/network trust boundaries, and policy application. The added unit tests cover several negative paths, but DNS-based SSRF and fail-closed custom replay behavior are not covered.
  • PRA-T6 Acceptance clause — No linked issue acceptance clauses were available in the deterministic context. — add test evidence or identify existing coverage. `linkedIssues` was empty in the validation context, so there were no issue-body or issue-comment clauses to map literally to the diff.
  • PRA-T7 Acceptance clause — Auto-generated PR summary claims Hermes endpoint settings reject unsafe or malformed URLs and block private/internal destinations. — add test evidence or identify existing coverage. The diff blocks malformed URLs, non-http(s), embedded credentials, literal private IPs, and reserved names, but does not reject public-looking hostnames whose DNS resolves to private/internal addresses.
  • PRA-T8 Acceptance clause — Auto-generated PR summary claims custom and user-loaded policy presets are prevented from including `allowed_ips`. — add test evidence or identify existing coverage. `loadPresetFromFile` rejects parsed custom preset files containing endpoint `allowed_ips`, and `applyPresetContent` repeats the check for custom content; however `applyPresetContent` continues when `parseNetworkPolicies` returns null, so custom replay/direct content is not fail-closed on unparseable policy content.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor (Nemotron Ultra) — Changes requested

Merge posture: Do not merge yet
Primary next action: Fix PRA-1: Monolith growth: policy/index.ts grew 29 lines (1331→1360) — extract preset SSRF guard; then add or justify PRA-T1.
Open items: 1 required · 2 warnings · 4 suggestions · 7 test follow-ups
Since last review: 0 prior items resolved · 0 still apply · 7 new items found

Action checklist

  • PRA-1 Fix: Monolith growth: policy/index.ts grew 29 lines (1331→1360) — extract preset SSRF guard in src/lib/policy/index.ts:1
  • PRA-4 Resolve or justify: LC_ALL=C fix applied only to gateway-control scripts — audit other shell scripts for [!0-9a-f] patterns in scripts/gateway-control.sh:8
  • PRA-5 Resolve or justify: bash 3.2 compatibility fixes partial — verify no remaining bash 4+ features on macOS paths in agents/hermes/start.sh:1850
  • PRA-T1 Add or justify test follow-up: Runtime validation
  • PRA-T2 Add or justify test follow-up: Runtime validation
  • PRA-T3 Add or justify test follow-up: Runtime validation
  • PRA-T4 Add or justify test follow-up: Runtime validation
  • PRA-T5 Add or justify test follow-up: Runtime validation
  • PRA-T6 Add or justify test follow-up: Tavily token pattern added — ensure parity test runs in CI
  • PRA-T7 Add or justify test follow-up: applyPresetContent allowed_ips test lacks side-effect assertions — verify zero external calls on rejection
  • PRA-2 In-scope improvement: Hermes SSRF error message leaks validated hostname — use generic message in src/lib/onboard/inference-providers/hermes.ts:33
  • PRA-3 In-scope improvement: loadPresetFromFile error message leaks file path — use generic message in src/lib/policy/index.ts:1093
  • PRA-6 In-scope improvement: Tavily token pattern added — ensure parity test runs in CI in test/e2e/fixtures/redaction.ts:40
  • PRA-7 In-scope improvement: applyPresetContent allowed_ips test lacks side-effect assertions — verify zero external calls on rejection in src/lib/policy/preset-allowed-ips.test.ts:109

Findings index

ID Severity Category Location Required action
PRA-1 Required architecture src/lib/policy/index.ts:1 Extract networkPoliciesHasAllowedIps and its two callers into src/lib/policy/preset-ssrf-guard.ts (or similar). This isolates the SSRF guard, improves test isolation, and offsets the line growth. The extraction is low-risk since the function is pure and has no external dependencies.
PRA-2 Improvement security src/lib/onboard/inference-providers/hermes.ts:33 Change error message to generic: `Inference endpoint URL points to a private or internal address. Use a public endpoint.` (without interpolating parsedEndpoint.hostname). This is a one-line change local to this PR.
PRA-3 Improvement security src/lib/policy/index.ts:1093 Change error message to: `Preset '${presetName}' contains 'allowed_ips', which is not permitted in user-supplied presets.` (drop filePath). One-line change local to this PR.
PRA-4 Resolve/justify security scripts/gateway-control.sh:8 Search all .sh files for character class patterns [!0-9a-f] or [^0-9a-f] and add export LCALL=C at script top. This is a repository-wide hardening that should be done in this PR since the pattern is now established.
PRA-5 Resolve/justify correctness agents/hermes/start.sh:1850 Audit agents/hermes/start.sh for: associative arrays (declare -A), [[ ]] with =~ or == glob, mapfile, coproc, ${var^^}. Add CI job running shell script tests on macOS/bash 3.2. The test file test/gateway-supervisor-control.test.ts:275 already handles macOS stderr noise, confirming macOS is a supported target.
PRA-6 Improvement tests test/e2e/fixtures/redaction.ts:40 Verify e2e-redaction-parity.test.ts runs in CI pipeline. No code change needed — pattern is already synced.
PRA-7 Improvement tests src/lib/policy/preset-allowed-ips.test.ts:109 Add spies/assertions in the test: verify runCapture not called, registry.updateSandbox not called, no temp files remain in os.tmpdir(). Since applyPresetContent uses real fs/runCapture, consider injecting a spy for runCapture in test or checking temp dir after call.

🚨 Required before merge

Address these before merging unless a maintainer explicitly overrides the advisor with rationale.

PRA-1 Required — Monolith growth: policy/index.ts grew 29 lines (1331→1360) — extract preset SSRF guard

  • Location: src/lib/policy/index.ts:1
  • Category: architecture
  • Problem: The drift tool flags src/lib/policy/index.ts as a blocker hotspot. The added networkPoliciesHasAllowedIps() function (lines 112-121) and its two call sites in loadPresetFromFile() (line 1091-1094) and applyPresetContent() (line 784-788) drive the growth. This file is already 1360 lines and accumulates policy logic across multiple domains.
  • Impact: Continued growth makes policy/index.ts harder to review, test, and maintain. Security-critical SSRF guard logic is mixed with preset loading, merging, registry, and CLI concerns.
  • Required action: Extract networkPoliciesHasAllowedIps and its two callers into src/lib/policy/preset-ssrf-guard.ts (or similar). This isolates the SSRF guard, improves test isolation, and offsets the line growth. The extraction is low-risk since the function is pure and has no external dependencies.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Run `git diff --stat main...HEAD -- src/lib/policy/index.ts` — shows +29 lines. Check that the new module exports networkPoliciesHasAllowedIps and the two call sites import it.
  • Missing regression test: N/A — structural concern; existing preset-allowed-ips.test.ts covers behavior
  • Done when: The required change is committed and verification passes: Run `git diff --stat main...HEAD -- src/lib/policy/index.ts` — shows +29 lines. Check that the new module exports networkPoliciesHasAllowedIps and the two call sites import it.
  • Evidence: Drift tool monolithDeltas shows severity=blocker for src/lib/policy/index.ts with delta=29. Function networkPoliciesHasAllowedIps at line 112-121, called at lines 787 and 1093.
Review findings by urgency: 1 required fix, 2 items to resolve/justify, 4 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-4 Resolve/justify — LC_ALL=C fix applied only to gateway-control scripts — audit other shell scripts for [!0-9a-f] patterns

  • Location: scripts/gateway-control.sh:8
  • Category: security
  • Problem: LC_ALL=C export added to scripts/gateway-control.sh:8 and scripts/lib/gateway-supervisor.sh:10 to fix macOS UTF-8 locale making [a-f] case-insensitive. This critical nonce validation fix should be applied consistently across all shell scripts validating hex nonces/identifiers.
  • Impact: Medium — other scripts using [!0-9a-f] or [^0-9a-f] for hex validation on macOS could accept uppercase hex nonces, bypassing format checks.
  • Recommended action: Search all .sh files for character class patterns [!0-9a-f] or [^0-9a-f] and add export LCALL=C at script top. This is a repository-wide hardening that should be done in this PR since the pattern is now established.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Run: `grep -r '\[!0-9a-f\]' --include='*.sh' /home/runner/work/NemoClaw/NemoClaw/pr-workdir` and `grep -r '\[^0-9a-f\]' --include='*.sh' /home/runner/work/NemoClaw/NemoClaw/pr-workdir`
  • Missing regression test: Add test: 'gateway-control.sh rejects uppercase hex nonce (64 A chars) on macOS locale' — runtime verification of LC_ALL=C fix
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Run: `grep -r '\[!0-9a-f\]' --include='*.sh' /home/runner/work/NemoClaw/NemoClaw/pr-workdir` and `grep -r '\[^0-9a-f\]' --include='*.sh' /home/runner/work/NemoClaw/NemoClaw/pr-workdir`.
  • Evidence: scripts/gateway-control.sh:8 adds export LC_ALL=C with comment about macOS UTF-8 locale. scripts/lib/gateway-supervisor.sh:10 adds same. No other .sh files in diff have this fix.

PRA-5 Resolve/justify — bash 3.2 compatibility fixes partial — verify no remaining bash 4+ features on macOS paths

  • Location: agents/hermes/start.sh:1850
  • Category: correctness
  • Problem: Three array expansions changed to ${arr[@]+${arr[@]}} syntax and one mapfile replaced with while-read loop. Need to verify no other bash 4+ features (associative arrays, [[ ]] extended globs, mapfile elsewhere) remain in macOS-executed code paths.
  • Impact: Medium — script failures on macOS/bash 3.2 would break Hermes gateway restart/recovery on macOS hosts.
  • Recommended action: Audit agents/hermes/start.sh for: associative arrays (declare -A), [[ ]] with =~ or == glob, mapfile, coproc, ${var^^}. Add CI job running shell script tests on macOS/bash 3.2. The test file test/gateway-supervisor-control.test.ts:275 already handles macOS stderr noise, confirming macOS is a supported target.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Search agents/hermes/start.sh for: declare -A, [[.*=~, mapfile, coproc, \$\{.*\^\^\}
  • Missing regression test: Add CI job: macOS/bash 3.2 runner executing hermes-managed-exit-authorization.test.ts and gateway-supervisor-control.test.ts
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Search agents/hermes/start.sh for: declare -A, [[.*=~, mapfile, coproc, \$\{.*\^\^\}.
  • Evidence: Diff shows 3 array expansion fixes and 1 mapfile→while-read replacement. Test file has macOS-specific stderr filter at line 275.

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

PRA-2 Improvement — Hermes SSRF error message leaks validated hostname — use generic message

  • Location: src/lib/onboard/inference-providers/hermes.ts:33
  • Category: security
  • Problem: The SSRF rejection error includes the validated hostname: `Inference endpoint URL points to a private or internal address "${parsedEndpoint.hostname}"`. While the hostname is user-provided, defense-in-depth favors generic error messages to avoid confirming internal hostname resolution to an attacker.
  • Impact: Low — hostname is already in user input. However, consistent generic error messages reduce information leakage in sandbox escape scenarios.
  • Suggested action: Change error message to generic: `Inference endpoint URL points to a private or internal address. Use a public endpoint.` (without interpolating parsedEndpoint.hostname). This is a one-line change local to this PR.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Read src/lib/onboard/inference-providers/hermes.ts line 33-35 — error message includes parsedEndpoint.hostname in template literal
  • Missing regression test: Add test: 'rejects private hostname with generic error message (no hostname leakage)' — assert error message does not contain the input hostname
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: Line 33-35: throw new Error(`Inference endpoint URL points to a private or internal address "${parsedEndpoint.hostname}". Use a public endpoint.`);

PRA-3 Improvement — loadPresetFromFile error message leaks file path — use generic message

  • Location: src/lib/policy/index.ts:1093
  • Category: security
  • Problem: The allowed_ips rejection error includes the file path: `Preset '${presetName}' contains 'allowed_ips', which is not permitted in user-supplied presets: ${filePath}`. File path is user-controlled but generic messages are safer.
  • Impact: Low — user controls the file path. Consistency with F2 recommendation.
  • Suggested action: Change error message to: `Preset '${presetName}' contains 'allowed_ips', which is not permitted in user-supplied presets.` (drop filePath). One-line change local to this PR.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Read src/lib/policy/index.ts line 1093 — error message includes filePath variable in template literal
  • Missing regression test: N/A — low-risk hygiene; existing test verifies rejection behavior
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: Line 1093: console.error(` Preset '${presetName}' contains 'allowed_ips', which is not permitted in user-supplied presets: ${filePath}`);

PRA-6 Improvement — Tavily token pattern added — ensure parity test runs in CI

  • Location: test/e2e/fixtures/redaction.ts:40
  • Category: tests
  • Problem: Tavily pattern (tvly-[A-Za-z0-9_-]{10,}) added to fixture mirror. Canonical source src/lib/security/secret-patterns.ts:45 already has it. Parity test e2e-redaction-parity.test.ts should catch divergence.
  • Impact: Low — pattern is already in canonical source. Risk is future divergence if parity test doesn't run.
  • Suggested action: Verify e2e-redaction-parity.test.ts runs in CI pipeline. No code change needed — pattern is already synced.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Run: grep 'tvly' /home/runner/work/NemoClaw/NemoClaw/pr-workdir/src/lib/security/secret-patterns.ts — confirmed at line 45. Check CI config for e2e-redaction-parity.test.ts execution.
  • Missing regression test: N/A — parity test exists; verify it runs in CI
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: test/e2e/fixtures/redaction.ts:40 adds tvly pattern. src/lib/security/secret-patterns.ts:45 already has identical pattern.

PRA-7 Improvement — applyPresetContent allowed_ips test lacks side-effect assertions — verify zero external calls on rejection

  • Location: src/lib/policy/preset-allowed-ips.test.ts:109
  • Category: tests
  • Problem: Test 'rejects custom preset content containing allowed_ips before any side effects' only asserts return value is false. It does not verify runCapture (openshell policy set) was not called, registry.updateSandbox was not called, or temp files were not created.
  • Impact: Medium — if side effects occur on rejection, an attacker could potentially trigger policy mutations or leave filesystem artifacts despite validation failure.
  • Suggested action: Add spies/assertions in the test: verify runCapture not called, registry.updateSandbox not called, no temp files remain in os.tmpdir(). Since applyPresetContent uses real fs/runCapture, consider injecting a spy for runCapture in test or checking temp dir after call.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Read test/e2e/live/onboard-negative-paths.test.ts line 165-187 — E2E test verifies exit code and error message but not side-effect freedom. Unit test at preset-allowed-ips.test.ts:109 only checks return value.
  • Missing regression test: Add assertions to preset-allowed-ips.test.ts 'rejects custom preset content containing allowed_ips' test: expect(runCapture).not.toHaveBeenCalled(), expect(registry.updateSandbox).not.toHaveBeenCalled(), expect(fs.readdirSync(tmpDir)).toEqual([]) (or similar temp-file check)
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: preset-allowed-ips.test.ts:109-120 test only expects return false. applyPresetContent uses runCapture, registry.updateSandbox, fs.writeFileSync/rmSync in try/finally.
Simplification opportunities: 1 possible cut, net -29 lines possible

These are safe simplification checks only. Do not remove validation, security controls, data-loss prevention, or required tests.

  • PRA-1 shrink (src/lib/policy/index.ts:1): networkPoliciesHasAllowedIps function (10 lines) and its two call sites in loadPresetFromFile and applyPresetContent
    • Replacement: Import from new module src/lib/policy/preset-ssrf-guard.ts
    • Net: -29 lines
    • Safety boundary: Preset SSRF guard logic must remain at both entry points (file load and in-memory apply) with identical behavior; existing tests must pass unchanged
Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

  • PRA-T1 Runtime validation — gateway-control.sh rejects uppercase hex nonce on macOS locale. Runtime/sandbox/infrastructure paths need behavioral runtime validation: agents/hermes/start.sh (bash 3.2 fixes), scripts/gateway-control.sh (LC_ALL=C), scripts/lib/gateway-supervisor.sh (LC_ALL=C), src/lib/onboard/inference-providers/hermes.ts (SSRF guard at onboarding boundary), src/lib/policy/index.ts (preset guards at policy-apply boundary). Unit tests cover logic but macOS shell behavior and onboarding/policy integration boundaries need runtime verification.
  • PRA-T2 Runtime validation — hermes/start.sh seal-restart works with unset _HERMES_GUARD_TIMEOUT array on bash 3.2. Runtime/sandbox/infrastructure paths need behavioral runtime validation: agents/hermes/start.sh (bash 3.2 fixes), scripts/gateway-control.sh (LC_ALL=C), scripts/lib/gateway-supervisor.sh (LC_ALL=C), src/lib/onboard/inference-providers/hermes.ts (SSRF guard at onboarding boundary), src/lib/policy/index.ts (preset guards at policy-apply boundary). Unit tests cover logic but macOS shell behavior and onboarding/policy integration boundaries need runtime verification.
  • PRA-T3 Runtime validation — applyPresetContent with allowed_ips makes zero external calls (runCapture, registry, fs). Runtime/sandbox/infrastructure paths need behavioral runtime validation: agents/hermes/start.sh (bash 3.2 fixes), scripts/gateway-control.sh (LC_ALL=C), scripts/lib/gateway-supervisor.sh (LC_ALL=C), src/lib/onboard/inference-providers/hermes.ts (SSRF guard at onboarding boundary), src/lib/policy/index.ts (preset guards at policy-apply boundary). Unit tests cover logic but macOS shell behavior and onboarding/policy integration boundaries need runtime verification.
  • PRA-T4 Runtime validation — Hermes SSRF error message does not leak hostname. Runtime/sandbox/infrastructure paths need behavioral runtime validation: agents/hermes/start.sh (bash 3.2 fixes), scripts/gateway-control.sh (LC_ALL=C), scripts/lib/gateway-supervisor.sh (LC_ALL=C), src/lib/onboard/inference-providers/hermes.ts (SSRF guard at onboarding boundary), src/lib/policy/index.ts (preset guards at policy-apply boundary). Unit tests cover logic but macOS shell behavior and onboarding/policy integration boundaries need runtime verification.
  • PRA-T5 Runtime validation — loadPresetFromFile error message does not leak file path. Runtime/sandbox/infrastructure paths need behavioral runtime validation: agents/hermes/start.sh (bash 3.2 fixes), scripts/gateway-control.sh (LC_ALL=C), scripts/lib/gateway-supervisor.sh (LC_ALL=C), src/lib/onboard/inference-providers/hermes.ts (SSRF guard at onboarding boundary), src/lib/policy/index.ts (preset guards at policy-apply boundary). Unit tests cover logic but macOS shell behavior and onboarding/policy integration boundaries need runtime verification.
  • PRA-T6 Tavily token pattern added — ensure parity test runs in CI — Verify e2e-redaction-parity.test.ts runs in CI pipeline. No code change needed — pattern is already synced.
  • PRA-T7 applyPresetContent allowed_ips test lacks side-effect assertions — verify zero external calls on rejection — Add spies/assertions in the test: verify runCapture not called, registry.updateSandbox not called, no temp files remain in os.tmpdir(). Since applyPresetContent uses real fs/runCapture, consider injecting a spy for runCapture in test or checking temp dir after call.
Since last review details

Current findings, using the urgency labels above:

PRA-1 Required — Monolith growth: policy/index.ts grew 29 lines (1331→1360) — extract preset SSRF guard

  • Location: src/lib/policy/index.ts:1
  • Category: architecture
  • Problem: The drift tool flags src/lib/policy/index.ts as a blocker hotspot. The added networkPoliciesHasAllowedIps() function (lines 112-121) and its two call sites in loadPresetFromFile() (line 1091-1094) and applyPresetContent() (line 784-788) drive the growth. This file is already 1360 lines and accumulates policy logic across multiple domains.
  • Impact: Continued growth makes policy/index.ts harder to review, test, and maintain. Security-critical SSRF guard logic is mixed with preset loading, merging, registry, and CLI concerns.
  • Required action: Extract networkPoliciesHasAllowedIps and its two callers into src/lib/policy/preset-ssrf-guard.ts (or similar). This isolates the SSRF guard, improves test isolation, and offsets the line growth. The extraction is low-risk since the function is pure and has no external dependencies.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Run `git diff --stat main...HEAD -- src/lib/policy/index.ts` — shows +29 lines. Check that the new module exports networkPoliciesHasAllowedIps and the two call sites import it.
  • Missing regression test: N/A — structural concern; existing preset-allowed-ips.test.ts covers behavior
  • Done when: The required change is committed and verification passes: Run `git diff --stat main...HEAD -- src/lib/policy/index.ts` — shows +29 lines. Check that the new module exports networkPoliciesHasAllowedIps and the two call sites import it.
  • Evidence: Drift tool monolithDeltas shows severity=blocker for src/lib/policy/index.ts with delta=29. Function networkPoliciesHasAllowedIps at line 112-121, called at lines 787 and 1093.

PRA-2 Improvement — Hermes SSRF error message leaks validated hostname — use generic message

  • Location: src/lib/onboard/inference-providers/hermes.ts:33
  • Category: security
  • Problem: The SSRF rejection error includes the validated hostname: `Inference endpoint URL points to a private or internal address "${parsedEndpoint.hostname}"`. While the hostname is user-provided, defense-in-depth favors generic error messages to avoid confirming internal hostname resolution to an attacker.
  • Impact: Low — hostname is already in user input. However, consistent generic error messages reduce information leakage in sandbox escape scenarios.
  • Suggested action: Change error message to generic: `Inference endpoint URL points to a private or internal address. Use a public endpoint.` (without interpolating parsedEndpoint.hostname). This is a one-line change local to this PR.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Read src/lib/onboard/inference-providers/hermes.ts line 33-35 — error message includes parsedEndpoint.hostname in template literal
  • Missing regression test: Add test: 'rejects private hostname with generic error message (no hostname leakage)' — assert error message does not contain the input hostname
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: Line 33-35: throw new Error(`Inference endpoint URL points to a private or internal address "${parsedEndpoint.hostname}". Use a public endpoint.`);

PRA-3 Improvement — loadPresetFromFile error message leaks file path — use generic message

  • Location: src/lib/policy/index.ts:1093
  • Category: security
  • Problem: The allowed_ips rejection error includes the file path: `Preset '${presetName}' contains 'allowed_ips', which is not permitted in user-supplied presets: ${filePath}`. File path is user-controlled but generic messages are safer.
  • Impact: Low — user controls the file path. Consistency with F2 recommendation.
  • Suggested action: Change error message to: `Preset '${presetName}' contains 'allowed_ips', which is not permitted in user-supplied presets.` (drop filePath). One-line change local to this PR.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Read src/lib/policy/index.ts line 1093 — error message includes filePath variable in template literal
  • Missing regression test: N/A — low-risk hygiene; existing test verifies rejection behavior
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: Line 1093: console.error(` Preset '${presetName}' contains 'allowed_ips', which is not permitted in user-supplied presets: ${filePath}`);

PRA-4 Resolve/justify — LC_ALL=C fix applied only to gateway-control scripts — audit other shell scripts for [!0-9a-f] patterns

  • Location: scripts/gateway-control.sh:8
  • Category: security
  • Problem: LC_ALL=C export added to scripts/gateway-control.sh:8 and scripts/lib/gateway-supervisor.sh:10 to fix macOS UTF-8 locale making [a-f] case-insensitive. This critical nonce validation fix should be applied consistently across all shell scripts validating hex nonces/identifiers.
  • Impact: Medium — other scripts using [!0-9a-f] or [^0-9a-f] for hex validation on macOS could accept uppercase hex nonces, bypassing format checks.
  • Recommended action: Search all .sh files for character class patterns [!0-9a-f] or [^0-9a-f] and add export LCALL=C at script top. This is a repository-wide hardening that should be done in this PR since the pattern is now established.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Run: `grep -r '\[!0-9a-f\]' --include='*.sh' /home/runner/work/NemoClaw/NemoClaw/pr-workdir` and `grep -r '\[^0-9a-f\]' --include='*.sh' /home/runner/work/NemoClaw/NemoClaw/pr-workdir`
  • Missing regression test: Add test: 'gateway-control.sh rejects uppercase hex nonce (64 A chars) on macOS locale' — runtime verification of LC_ALL=C fix
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Run: `grep -r '\[!0-9a-f\]' --include='*.sh' /home/runner/work/NemoClaw/NemoClaw/pr-workdir` and `grep -r '\[^0-9a-f\]' --include='*.sh' /home/runner/work/NemoClaw/NemoClaw/pr-workdir`.
  • Evidence: scripts/gateway-control.sh:8 adds export LC_ALL=C with comment about macOS UTF-8 locale. scripts/lib/gateway-supervisor.sh:10 adds same. No other .sh files in diff have this fix.

PRA-5 Resolve/justify — bash 3.2 compatibility fixes partial — verify no remaining bash 4+ features on macOS paths

  • Location: agents/hermes/start.sh:1850
  • Category: correctness
  • Problem: Three array expansions changed to ${arr[@]+${arr[@]}} syntax and one mapfile replaced with while-read loop. Need to verify no other bash 4+ features (associative arrays, [[ ]] extended globs, mapfile elsewhere) remain in macOS-executed code paths.
  • Impact: Medium — script failures on macOS/bash 3.2 would break Hermes gateway restart/recovery on macOS hosts.
  • Recommended action: Audit agents/hermes/start.sh for: associative arrays (declare -A), [[ ]] with =~ or == glob, mapfile, coproc, ${var^^}. Add CI job running shell script tests on macOS/bash 3.2. The test file test/gateway-supervisor-control.test.ts:275 already handles macOS stderr noise, confirming macOS is a supported target.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Search agents/hermes/start.sh for: declare -A, [[.*=~, mapfile, coproc, \$\{.*\^\^\}
  • Missing regression test: Add CI job: macOS/bash 3.2 runner executing hermes-managed-exit-authorization.test.ts and gateway-supervisor-control.test.ts
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Search agents/hermes/start.sh for: declare -A, [[.*=~, mapfile, coproc, \$\{.*\^\^\}.
  • Evidence: Diff shows 3 array expansion fixes and 1 mapfile→while-read replacement. Test file has macOS-specific stderr filter at line 275.

PRA-6 Improvement — Tavily token pattern added — ensure parity test runs in CI

  • Location: test/e2e/fixtures/redaction.ts:40
  • Category: tests
  • Problem: Tavily pattern (tvly-[A-Za-z0-9_-]{10,}) added to fixture mirror. Canonical source src/lib/security/secret-patterns.ts:45 already has it. Parity test e2e-redaction-parity.test.ts should catch divergence.
  • Impact: Low — pattern is already in canonical source. Risk is future divergence if parity test doesn't run.
  • Suggested action: Verify e2e-redaction-parity.test.ts runs in CI pipeline. No code change needed — pattern is already synced.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Run: grep 'tvly' /home/runner/work/NemoClaw/NemoClaw/pr-workdir/src/lib/security/secret-patterns.ts — confirmed at line 45. Check CI config for e2e-redaction-parity.test.ts execution.
  • Missing regression test: N/A — parity test exists; verify it runs in CI
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: test/e2e/fixtures/redaction.ts:40 adds tvly pattern. src/lib/security/secret-patterns.ts:45 already has identical pattern.

PRA-7 Improvement — applyPresetContent allowed_ips test lacks side-effect assertions — verify zero external calls on rejection

  • Location: src/lib/policy/preset-allowed-ips.test.ts:109
  • Category: tests
  • Problem: Test 'rejects custom preset content containing allowed_ips before any side effects' only asserts return value is false. It does not verify runCapture (openshell policy set) was not called, registry.updateSandbox was not called, or temp files were not created.
  • Impact: Medium — if side effects occur on rejection, an attacker could potentially trigger policy mutations or leave filesystem artifacts despite validation failure.
  • Suggested action: Add spies/assertions in the test: verify runCapture not called, registry.updateSandbox not called, no temp files remain in os.tmpdir(). Since applyPresetContent uses real fs/runCapture, consider injecting a spy for runCapture in test or checking temp dir after call.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Read test/e2e/live/onboard-negative-paths.test.ts line 165-187 — E2E test verifies exit code and error message but not side-effect freedom. Unit test at preset-allowed-ips.test.ts:109 only checks return value.
  • Missing regression test: Add assertions to preset-allowed-ips.test.ts 'rejects custom preset content containing allowed_ips' test: expect(runCapture).not.toHaveBeenCalled(), expect(registry.updateSandbox).not.toHaveBeenCalled(), expect(fs.readdirSync(tmpDir)).toEqual([]) (or similar temp-file check)
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: preset-allowed-ips.test.ts:109-120 test only expects return false. applyPresetContent uses runCapture, registry.updateSandbox, fs.writeFileSync/rmSync in try/finally.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

@prekshivyas

Copy link
Copy Markdown
Contributor Author

Closing in favor of #6140. This branch was accidentally cut from a security feature branch that hadn't merged to main yet, pulling 7 extra commits into the diff. #6140 is a clean rebase onto main with the same fixes plus the CodeRabbit SIGTERM regex tightening.

@prekshivyas prekshivyas closed this Jul 1, 2026
apurvvkumaria pushed a commit that referenced this pull request Jul 1, 2026
## Summary

Five pre-existing test failures on the `macos-vitest` CI workflow
(confirmed present before #6060) traced to bash 3.x incompatibilities, a
macOS UTF-8 locale issue, and a missing fixture sync. This PR fixes all
five root causes.

Supersedes #6137 (that PR's branch was accidentally cut from a security
feature branch, pulling in unrelated files into the diff; this is a
clean rebase onto main with the same two commits plus a CodeRabbit fix).

## Related Issue

Investigation of CI run
[28539883104](https://github.com/NVIDIA/NemoClaw/actions/runs/28539883104);
failures also present in run
[28534214317](https://github.com/NVIDIA/NemoClaw/actions/runs/28534214317)
(before #6060), confirming #6060 is not the cause.

## Changes

- **`agents/hermes/start.sh`** — three bash 3.2 compatibility fixes:
- Replace bash 4.1+ named-FD `exec {var}<file` with a `{ } < file`
grouped redirect; variables assigned inside `{}` remain in function
scope
- Replace `mapfile -d '' -t` (bash 4.x only) with `while IFS= read -r -d
"" elem; do arr+=("$elem"); done`
- Guard `${_HERMES_GUARD_TIMEOUT[@]}` with `${arr[@]+"${arr[@]}"}` so
`set -u` does not abort the script when the array is empty (bash 3.2
treats empty `[@]` as unbound)
- **`scripts/gateway-control.sh`** — add `export LC_ALL=C` so `[a-f]`
character-class ranges in `case` patterns are byte-exact; macOS
`en_US.UTF-8` makes `[a-f]` case-insensitive, allowing uppercase hex
nonces to pass the `*[!0-9a-f]*` check
- **`scripts/lib/gateway-supervisor.sh`** — same `LC_ALL=C` fix for the
sourced library's nonce validation path
- **`test/gateway-supervisor-control.test.ts`** — pin
`NEMOCLAW_TEST_GATEWAY_CONTROL_CALLER_UID=0` in the nonce-rejection test
so it does not depend on the CI runner's UID; tighten macOS bash 3.2
SIGTERM filter from broad word-match to exact `Terminated: <digits>` /
`Killed: <digits>` format so unrelated stderr still fails the assertion
- **`test/e2e/fixtures/redaction.ts`** — add `tvly-` Tavily token
pattern missing since #6134, fixing the `e2e-redaction-parity`
`Array(16)` vs `Array(17)` mismatch

## Type of Change

- [x] Code change (feature, bug fix, or refactor)

## Quality Gates

- [x] Tests added or updated for changed behavior
- [x] Docs not applicable — justification: shell compatibility and test
fixes, no user-facing behavior change
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging) —
`gateway-control.sh` and `gateway-supervisor.sh` handle nonce validation
- [x] Sensitive-path review completed or maintainer-approved waiver
recorded — the `LC_ALL=C` fix tightens nonce validation (rejects
uppercase hex on macOS that was previously accepted);
`gateway_control_stop_tracked_pid` behavior is unchanged

## Verification

- [x] Git hooks passed during commit and push
- [x] Targeted tests pass for changed behavior
  - `test/gateway-supervisor-control.test.ts`: 22/22 ✓
- `test/hermes-managed-exit-authorization.test.ts`: all ✓ (was 8
failures before)
  - `test/e2e/support/e2e-redaction-parity.test.ts`: 3/3 ✓
- `test/hermes-gateway-supervisor-recovery.test.ts`: 41/42 (1 local
flake — PID 4242 alive on dev machine; unrelated to these changes,
passes on CI fresh runners)
- [x] No secrets, API keys, or credentials committed

**Remaining failures not addressed in this PR** (different root class,
need separate investigation):
- `install-preflight.test.ts` — environment-specific
- `deepagents-code-tui-startup-check.test.ts` — needs investigation
- `platform-parity-cloud-experimental.test.ts` — needs investigation
- WSL `runtime-recovery-preload.test.ts`, `rebuild-config-hash.test.ts`
— different class of failure

---
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved gateway nonce validation to reliably accept only lowercase
hex characters, independent of locale and macOS environments.
* Fixed managed gateway startup, recovery, and live-status monitoring to
be compatible with older Bash versions.
* Tightened controller/marker parsing to reduce incorrect authorization
or status detection.
* Expanded secret redaction to cover additional Tavily-shaped tokens,
improving protection in logs and text output.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
## Summary

Five pre-existing test failures on the `macos-vitest` CI workflow
(confirmed present before NVIDIA#6060) traced to bash 3.x incompatibilities, a
macOS UTF-8 locale issue, and a missing fixture sync. This PR fixes all
five root causes.

Supersedes NVIDIA#6137 (that PR's branch was accidentally cut from a security
feature branch, pulling in unrelated files into the diff; this is a
clean rebase onto main with the same two commits plus a CodeRabbit fix).

## Related Issue

Investigation of CI run
[28539883104](https://github.com/NVIDIA/NemoClaw/actions/runs/28539883104);
failures also present in run
[28534214317](https://github.com/NVIDIA/NemoClaw/actions/runs/28534214317)
(before NVIDIA#6060), confirming NVIDIA#6060 is not the cause.

## Changes

- **`agents/hermes/start.sh`** — three bash 3.2 compatibility fixes:
- Replace bash 4.1+ named-FD `exec {var}<file` with a `{ } < file`
grouped redirect; variables assigned inside `{}` remain in function
scope
- Replace `mapfile -d '' -t` (bash 4.x only) with `while IFS= read -r -d
"" elem; do arr+=("$elem"); done`
- Guard `${_HERMES_GUARD_TIMEOUT[@]}` with `${arr[@]+"${arr[@]}"}` so
`set -u` does not abort the script when the array is empty (bash 3.2
treats empty `[@]` as unbound)
- **`scripts/gateway-control.sh`** — add `export LC_ALL=C` so `[a-f]`
character-class ranges in `case` patterns are byte-exact; macOS
`en_US.UTF-8` makes `[a-f]` case-insensitive, allowing uppercase hex
nonces to pass the `*[!0-9a-f]*` check
- **`scripts/lib/gateway-supervisor.sh`** — same `LC_ALL=C` fix for the
sourced library's nonce validation path
- **`test/gateway-supervisor-control.test.ts`** — pin
`NEMOCLAW_TEST_GATEWAY_CONTROL_CALLER_UID=0` in the nonce-rejection test
so it does not depend on the CI runner's UID; tighten macOS bash 3.2
SIGTERM filter from broad word-match to exact `Terminated: <digits>` /
`Killed: <digits>` format so unrelated stderr still fails the assertion
- **`test/e2e/fixtures/redaction.ts`** — add `tvly-` Tavily token
pattern missing since NVIDIA#6134, fixing the `e2e-redaction-parity`
`Array(16)` vs `Array(17)` mismatch

## Type of Change

- [x] Code change (feature, bug fix, or refactor)

## Quality Gates

- [x] Tests added or updated for changed behavior
- [x] Docs not applicable — justification: shell compatibility and test
fixes, no user-facing behavior change
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging) —
`gateway-control.sh` and `gateway-supervisor.sh` handle nonce validation
- [x] Sensitive-path review completed or maintainer-approved waiver
recorded — the `LC_ALL=C` fix tightens nonce validation (rejects
uppercase hex on macOS that was previously accepted);
`gateway_control_stop_tracked_pid` behavior is unchanged

## Verification

- [x] Git hooks passed during commit and push
- [x] Targeted tests pass for changed behavior
  - `test/gateway-supervisor-control.test.ts`: 22/22 ✓
- `test/hermes-managed-exit-authorization.test.ts`: all ✓ (was 8
failures before)
  - `test/e2e/support/e2e-redaction-parity.test.ts`: 3/3 ✓
- `test/hermes-gateway-supervisor-recovery.test.ts`: 41/42 (1 local
flake — PID 4242 alive on dev machine; unrelated to these changes,
passes on CI fresh runners)
- [x] No secrets, API keys, or credentials committed

**Remaining failures not addressed in this PR** (different root class,
need separate investigation):
- `install-preflight.test.ts` — environment-specific
- `deepagents-code-tui-startup-check.test.ts` — needs investigation
- `platform-parity-cloud-experimental.test.ts` — needs investigation
- WSL `runtime-recovery-preload.test.ts`, `rebuild-config-hash.test.ts`
— different class of failure

---
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved gateway nonce validation to reliably accept only lowercase
hex characters, independent of locale and macOS environments.
* Fixed managed gateway startup, recovery, and live-status monitoring to
be compatible with older Bash versions.
* Tightened controller/marker parsing to reduce incorrect authorization
or status detection.
* Expanded secret redaction to cover additional Tavily-shaped tokens,
improving protection in logs and text output.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants