fix(inference): contain shared gateway route conflicts#6338
Conversation
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
…e-containment Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in the branch is 96%. Coverage data for the branch is not yet available. Show a code coverage summary of the most covered files.
TypeScript / code-coverage/cliThe overall coverage in the branch is 75%. Coverage data for the branch is not yet available. Show a code coverage summary of the most covered files.
Updated |
|
🌿 Preview your docs: https://nvidia-preview-pr-6338.docs.buildwithfern.com/nemoclaw |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR adds gateway-scoped inference route compatibility checks across inference setup, route switching, and sandbox connect flows. It also updates missing-flag CLI messaging and documents the shared-gateway routing rules, conflict handling, and remediation paths. ChangesGateway route compatibility feature
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant runInferenceSet
participant prepareInferenceSetRoute
participant finalizeInferenceSetRoute
participant checkGatewayRouteCompatibility
participant OpenShell
runInferenceSet->>prepareInferenceSetRoute: derive route metadata
prepareInferenceSetRoute->>checkGatewayRouteCompatibility: preliminary compatibility check
runInferenceSet->>finalizeInferenceSetRoute: materialize custom endpoint identity
finalizeInferenceSetRoute->>checkGatewayRouteCompatibility: final compatibility check
runInferenceSet->>OpenShell: run inference set with gatewayName
sequenceDiagram
participant runOnboard
participant setupInferenceWithDeps
participant handleProviderInferenceState
participant checkGatewayRouteCompatibility
runOnboard->>setupInferenceWithDeps: initialize gateway-scoped setup
setupInferenceWithDeps->>checkGatewayRouteCompatibility: gate setup route
handleProviderInferenceState->>checkGatewayRouteCompatibility: validate resume route
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
PR Review Advisor (Nemotron Ultra) — Changes requestedMerge posture: Do not merge yet Action checklist
Findings index
🚨 Required before mergeAddress these before merging unless a maintainer explicitly overrides the advisor with rationale.
|
PR Review Advisor — No blocking findingsMerge posture: No blocking advisor findings Action checklist
Test follow-ups to resolve or justifyIf these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.
This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision. |
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
E2E Advisor RecommendationRequired E2E: Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
E2E Target RecommendationRequired E2E targets: Dispatch required E2E targets:
Full E2E target advisor summaryE2E Target AdvisorBase: Required E2E targets
Optional E2E targets
Relevant changed files
|
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
There was a problem hiding this comment.
🧹 Nitpick comments (2)
src/commands/inference/set.ts (1)
74-86: 📐 Maintainability & Code Quality | 🔵 TrivialDuplicate
printRequiredFlagslogic across command classes.This private helper is nearly identical to the one added in
src/commands/sandbox/inference/set.ts(Line 78-90), differing only in the command-name prefix and usage string. Per path instructions,src/commands/**should keep command classes focused on grammar/parsing/help while behavior/orchestration lives elsewhere; a shared helper (e.g. insrc/lib/cli/**, which is meant for framework/metadata/help infrastructure) would avoid the copy-paste and keep the two messages from drifting apart as this containment feature evolves.As per path instructions,
src/{commands,lib/cli}/**: "Keepsrc/lib/cli/**limited to framework, metadata, routing, and help infrastructure rather than product behavior."♻️ Sketch of a shared helper
// src/lib/cli/required-flags.ts export function requiredFlagsFailureLines(usagePrefix: string): string[] { return [ ` ${usagePrefix} requires --provider and --model.`, "", ` Run: ${usagePrefix} --provider <provider> --model <model>`, " NemoClaw must perform this operation so it can protect every sandbox sharing the target gateway.", "", ` Run '${CLI_NAME} help' for NemoClaw commands.`, ]; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/commands/inference/set.ts` around lines 74 - 86, The `printRequiredFlags` helper in `set.ts` duplicates nearly the same required-flags message logic used by the sandbox inference command, so extract the shared message-building behavior into a common helper instead of keeping copy-pasted private methods in each command. Add the reusable helper in the shared CLI support area under `src/lib/cli/**`, and have both `printRequiredFlags` callers use it with their command-specific usage prefix so the command classes only handle wiring/help text. Keep the command-specific `set.ts` method thin and delegate the actual line generation to the shared utility to prevent the two messages from drifting.Source: Path instructions
src/lib/inference/gateway-route-compatibility.test.ts (1)
217-224: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winTest doesn't exercise the "skip incomplete row" claim.
The requested route's provider (
"anthropic-prod") never matches either fixture row's provider ("nvidia-prod"/null), so this assertion would pass even without any special handling for incomplete rows — it's indistinguishable from an ordinary provider mismatch. To actually validate the "skip" behavior, use a row that shares the requested provider but is missing the model (or vice versa), e.g.:🧪 Proposed fix to exercise the skip path
it("skips registry rows without a complete provider and model (`#6315`)", () => { expect( - check(route("anthropic-prod", "claude-new"), [ - sandbox("empty", { provider: null, model: null }), - sandbox("provider-only", { provider: "nvidia-prod", model: null }), + check(route("nvidia-prod", "nvidia/model-a"), [ + sandbox("empty", { provider: null, model: null }), + sandbox("provider-only", { model: null }), ]), ).toEqual({ ok: true }); });This way the row shares the requested provider and would conflict if not skipped, so the assertion actually proves the skip behavior. As per path instructions, "Flag ... conditionals that make a test pass without exercising its claim."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/lib/inference/gateway-route-compatibility.test.ts` around lines 217 - 224, The test in check(route(...)) is not proving the “skip incomplete row” behavior because the requested provider never matches the fixture rows, so it passes for a normal mismatch. Update the fixture data in the it("skips registry rows without a complete provider and model (`#6315`)") case so one row shares the requested provider or model but is missing the other field, and keep using sandbox(...) with check(...) to verify the incomplete row is actually ignored by the route-compatibility logic.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/commands/inference/set.ts`:
- Around line 74-86: The `printRequiredFlags` helper in `set.ts` duplicates
nearly the same required-flags message logic used by the sandbox inference
command, so extract the shared message-building behavior into a common helper
instead of keeping copy-pasted private methods in each command. Add the reusable
helper in the shared CLI support area under `src/lib/cli/**`, and have both
`printRequiredFlags` callers use it with their command-specific usage prefix so
the command classes only handle wiring/help text. Keep the command-specific
`set.ts` method thin and delegate the actual line generation to the shared
utility to prevent the two messages from drifting.
In `@src/lib/inference/gateway-route-compatibility.test.ts`:
- Around line 217-224: The test in check(route(...)) is not proving the “skip
incomplete row” behavior because the requested provider never matches the
fixture rows, so it passes for a normal mismatch. Update the fixture data in the
it("skips registry rows without a complete provider and model (`#6315`)") case so
one row shares the requested provider or model but is missing the other field,
and keep using sandbox(...) with check(...) to verify the incomplete row is
actually ignored by the route-compatibility logic.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: b3b17449-4642-4973-952c-7a30cc190098
📒 Files selected for processing (28)
docs/inference/switch-inference-providers.mdxdocs/reference/commands-nemohermes.mdxdocs/reference/commands.mdxdocs/reference/troubleshooting.mdxsrc/commands/inference/set.tssrc/commands/sandbox/inference/oclif-command-adapters.test.tssrc/commands/sandbox/inference/set.tssrc/lib/actions/inference-set-compatible-provider.test.tssrc/lib/actions/inference-set-gateway-route-containment.test.tssrc/lib/actions/inference-set-hermes-run.test.tssrc/lib/actions/inference-set-openclaw-run.test.tssrc/lib/actions/inference-set.tssrc/lib/actions/sandbox/connect-route-lifecycle.test.tssrc/lib/actions/sandbox/connect-route-repair.test.tssrc/lib/actions/sandbox/connect.tssrc/lib/inference/gateway-route-compatibility.test.tssrc/lib/inference/gateway-route-compatibility.tssrc/lib/onboard.tssrc/lib/onboard/machine/core-flow-phases.test.tssrc/lib/onboard/machine/handlers/provider-inference.test.tssrc/lib/onboard/machine/handlers/provider-inference.tssrc/lib/onboard/setup-inference-route-containment.test.tssrc/lib/onboard/setup-inference.tstest/cli/list-inference.test.tstest/onboard-inference-smoke.test.tstest/onboard-selection.test.tstest/support/connect-flow-test-harness.tstest/support/setup-inference-test-harness.ts
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
…e-containment Signed-off-by: Aaron Erickson <aerickson@nvidia.com> # Conflicts: # src/lib/actions/inference-set-hermes-run.test.ts # src/lib/actions/inference-set.ts # src/lib/onboard.ts # src/lib/onboard/machine/handlers/provider-inference.ts
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/lib/onboard/inference-route.ts (1)
5-9: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winInconsistent dependency injection for the new compatibility gate.
verifyInferenceRoute/isInferenceRouteReadyboth use the factory's injectedrunCaptureOpenshell, butcheckGatewayRouteCompatibilityreaches for a directly-importedlistSandboxes()instead of taking it as a parameter. This is a security-relevant guard (contains cross-sandbox route conflicts), so it's worth keeping it consistent and fake-injectable like the rest of the module rather than requiring module-level mocking in tests.♻️ Suggested DI-consistent refactor
-export function createInferenceRouteHelpers(runCaptureOpenshell: RunCaptureOpenshell) { +export function createInferenceRouteHelpers( + runCaptureOpenshell: RunCaptureOpenshell, + listSandboxesFn: typeof listSandboxes = listSandboxes, +) { ... const checkGatewayRouteCompatibility: CurrentGatewayRouteCompatibilityCheck = (request) => checkGatewayRouteCompatibilityForRegistry({ ...request, - sandboxes: listSandboxes().sandboxes, + sandboxes: listSandboxesFn().sandboxes, });Also applies to: 29-35
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/lib/onboard/inference-route.ts` around lines 5 - 9, The new compatibility gate is mixing injected dependencies with a direct module import, so update checkGatewayRouteCompatibility in inference-route to accept listSandboxes as an injected dependency instead of importing it directly. Thread that dependency through the existing factory/entry points used by verifyInferenceRoute and isInferenceRouteReady so the gate stays fake-injectable and consistent with runCaptureOpenshell. Keep the current compatibility logic intact, but remove the module-level reliance on listSandboxes from this flow.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/lib/actions/inference-set-route-containment.ts`:
- Around line 262-342: Refresh the sandbox snapshot before the final
compatibility check in finalizeInferenceSetRoute. The current re-check still
uses the caller-provided sandboxes array, so it can miss a conflicting route
change that happens during normalizeCustomEndpointUrl. Update
finalizeInferenceSetRoute to re-read the latest registry or otherwise obtain a
fresh sandboxes snapshot immediately before calling
assertGatewayRouteCompatibility, using the existing
prepareInferenceSetRoute/finalizeInferenceSetRoute flow and gatewayName metadata
to locate the right guard.
---
Nitpick comments:
In `@src/lib/onboard/inference-route.ts`:
- Around line 5-9: The new compatibility gate is mixing injected dependencies
with a direct module import, so update checkGatewayRouteCompatibility in
inference-route to accept listSandboxes as an injected dependency instead of
importing it directly. Thread that dependency through the existing factory/entry
points used by verifyInferenceRoute and isInferenceRouteReady so the gate stays
fake-injectable and consistent with runCaptureOpenshell. Keep the current
compatibility logic intact, but remove the module-level reliance on
listSandboxes from this flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 4a7fd800-e8e2-408f-8184-cfb6159616da
📒 Files selected for processing (30)
docs/about/release-notes.mdxdocs/inference/switch-inference-providers.mdxdocs/reference/commands-nemohermes.mdxdocs/reference/commands.mdxdocs/reference/troubleshooting.mdxsrc/lib/actions/inference-set-gateway-route-containment.test.tssrc/lib/actions/inference-set-hermes-run.test.tssrc/lib/actions/inference-set-route-containment.tssrc/lib/actions/inference-set.tssrc/lib/actions/sandbox/connect-inference-gateway.tssrc/lib/actions/sandbox/connect-route-containment.test.tssrc/lib/actions/sandbox/connect-route-lifecycle.test.tssrc/lib/actions/sandbox/connect.tssrc/lib/inference/gateway-route-compatibility.test.tssrc/lib/inference/gateway-route-compatibility.tssrc/lib/onboard.tssrc/lib/onboard/gateway-provider-metadata.test.tssrc/lib/onboard/gateway-provider-metadata.tssrc/lib/onboard/inference-route.tssrc/lib/onboard/machine/core-flow-phases.test.tssrc/lib/onboard/machine/core-flow-phases.tssrc/lib/onboard/machine/handlers/provider-inference-route-containment.test.tssrc/lib/onboard/machine/handlers/provider-inference-route-containment.tssrc/lib/onboard/machine/handlers/provider-inference.test.tssrc/lib/onboard/machine/handlers/provider-inference.tssrc/lib/onboard/resume-provider-shim.tssrc/lib/onboard/setup-inference-route-containment.test.tssrc/lib/onboard/setup-inference.tstest/onboard-remote-recreate-credential-reuse.test.tstest/onboard-selection.test.ts
✅ Files skipped from review due to trivial changes (4)
- docs/about/release-notes.mdx
- src/lib/onboard/machine/core-flow-phases.test.ts
- docs/reference/troubleshooting.mdx
- docs/inference/switch-inference-providers.mdx
🚧 Files skipped from review as they are similar to previous changes (11)
- src/lib/actions/inference-set-hermes-run.test.ts
- src/lib/onboard/setup-inference-route-containment.test.ts
- src/lib/onboard/setup-inference.ts
- test/onboard-selection.test.ts
- src/lib/inference/gateway-route-compatibility.test.ts
- src/lib/onboard.ts
- src/lib/inference/gateway-route-compatibility.ts
- docs/reference/commands.mdx
- docs/reference/commands-nemohermes.mdx
- src/lib/onboard/machine/handlers/provider-inference.ts
- src/lib/actions/inference-set.ts
…e-containment Signed-off-by: Aaron Erickson <aerickson@nvidia.com> # Conflicts: # src/lib/onboard/setup-nim-flow.ts
…e-containment Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
E2E Target Results — ✅ All selected jobs passedRun: 28890883985
|
E2E Target Results — ✅ All selected jobs passedRun: 28890884098
|
|
@cv Exact-head follow-up: The only production conflict was additive in Fresh exact-head evidence:
The branch is mergeable, and all 43 commits are verified and signed off. Please rereview |
E2E Target Results — ✅ All selected jobs passedRun: 28891651353
|
E2E Target Results — ✅ All requested jobs passedRun: 28891670558
|
E2E Target Results — ✅ All requested jobs passedRun: 28891647223
|
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
E2E Target Results — ✅ All selected jobs passedRun: 28893939992
|
E2E Target Results — ✅ All requested jobs passedRun: 28893935439
|
E2E Target Results — ✅ All requested jobs passedRun: 28894237287
|
|
Maintainer disposition for PRA-1 on exact head The absence of a dedicated checked-in same-gateway conflict live target is accepted for this PR. Adding that target here would require a broader stateful registry/gateway orchestration fixture beyond the narrow merge salvage. The exact security delta is covered by behavior-specific fail-closed tests, while real OpenShell integration is covered by the complete exact-head live matrix below. Evidence:
Manual validation procedure for the residual dedicated-target gap:
A dedicated checked-in same-gateway conflict target remains desirable future hardening, but the focused no-side-effect regressions plus the successful real OpenShell matrix make that absence non-blocking for this PR. |
cv
left a comment
There was a problem hiding this comment.
Approved exact head 7df7274. The conflict resolution preserves shared-gateway locking, fail-closed incomplete-row handling, and the post-DNS fresh-peer recheck while incorporating the landed provider-alias and no-SSRF-bypass behavior. All 45 commits are Verified with DCO present; attached CI is fully green; CodeRabbit has no unresolved findings; the independent nine-category security review is clean; every refreshed E2E Advisor requirement passed in runs 28893935439, 28893939992, and 28894237287; and primary PR Review Advisor attempt 3 is merge_as_is. The contributor/approver overlap is the merge-gate advisory only and does not change readiness.
<!-- markdownlint-disable MD041 --> ## Summary Prepares the user-facing documentation for NemoClaw v0.0.76 and closes the release-prep documentation gate. It adds the release highlights, documents the arm64 Local NIM warning and expanded image cleanup behavior, and fixes agent-specific command headings in generated guides. ## Changes - Add the v0.0.76 release-notes section and move the shared-gateway route containment entry out of the v0.0.74 history where it was incorrectly placed. - Document the advisory Linux arm64 Local NIM manifest warning in the canonical platform matrix and local-inference guidance. - Document that `gc` scans both gateway-built and locally prebuilt sandbox image repositories. - Keep OpenClaw and Hermes session headings out of the generated Deep Agents command guide. - Add a focused variant regression test for the agent-specific session headings. ### Source summary | Merged sources | Documentation coverage | | --- | --- | | [#6414](#6414), [#6418](#6418), [#6416](#6416), [#6344](#6344) | v0.0.76 release notes and the Deep Agents quickstart/inference routes | | [#6340](#6340) | v0.0.76 release notes and existing Deep Agents observability guidance | | [#6338](#6338), [#6378](#6378), [#6297](#6297) | v0.0.76 release notes and existing inference/troubleshooting guidance | | [#6362](#6362) | v0.0.76 release notes and existing lifecycle, command, and credential guidance | | [#6330](#6330), [#6307](#6307), [#6008](#6008) | v0.0.76 release notes and existing security, troubleshooting, and command guidance | | [#6382](#6382) | v0.0.76 release notes and existing MCP/command guidance | | [#6326](#6326), [#5868](#5868), [#5539](#5539) | v0.0.76 release notes, platform matrix, inference options, and local-inference guidance | | [#6396](#6396), [#6390](#6390), [#6007](#6007) | v0.0.76 release notes and existing messaging guidance | | [#5388](#5388), [#6249](#6249), [#6303](#6303), [#6306](#6306) | v0.0.76 release notes and command/lifecycle guidance | ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — `npx vitest run --project integration test/generate-platform-docs.test.ts test/agent-variant-docs.test.ts test/sync-agent-variant-docs.test.ts` (3 files, 29 tests passed) - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — completed with 0 errors and 2 pre-existing Fern warnings - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added v0.0.76 release notes content, and removed an older conflicting bullet from the surrounding release history. * Expanded Local NVIDIA NIM guidance across inference/provider docs, including an advisory for Linux arm64 DGX Spark/DGX Station hosts when a matching `linux/arm64` image manifest is unavailable. * Updated the command reference for correct session-section rendering and clarified `gc` image cleanup sources. * **Tests** * Added coverage ensuring Deep Agents omits sessions headings while Hermes includes them. * **CI** * Refreshed Local NVIDIA NIM provider notes used in the platform matrix. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
…ecorded route (#6465) <!-- markdownlint-disable MD041 --> ## Summary `nemoclaw <name> status` previously showed the live host-gateway inference route as though it were the sandbox's own route. When an out-of-band operation such as direct `openshell inference set` makes that route differ from the sandbox's recorded provider/model, this PR now emits an explicit warning and safe remediation guidance. ## Related Issue Refs #6315. This complements #6338's fail-closed containment for supported flows by surfacing residual out-of-band drift in status; it follows the connect-time divergence behavior from #3726. ## Changes - Compute `routeDrift` only when a complete, readable live route differs from a complete recorded route. - Show the live and recorded routes, explain that `connect` realigns to the recorded route, and show the supported `inference set --sandbox` path for adopting the live route. - Share one control-character-sanitized warning formatter between status and connect so the messages cannot drift. - Cover divergent, aligned, unreadable, and missing-record cases plus rendered status output. - Document the warning in the existing one-route-per-gateway guidance. ## Original Author Credit @Hokonoken authored the diagnosis, implementation, tests, live proof, and documentation. The maintainer synchronization preserves those commits, and the review-fix commit records Hokonoken as co-author. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed — display-only route comparison; no route, registry, credential, or sandbox mutation is introduced; untrusted route values use the shared sanitizer - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes DCO declarations and every commit is GitHub Verified - [x] Normal pre-commit, commit-msg, and pre-push hooks passed on the current head - [x] Focused config, connect-route, status-flow, and snapshot tests — 87/87 passed with one local worker - [x] `npm run build:cli` — passed - [x] Plugin build — passed - [x] `npm run docs` — 0 errors and 2 pre-existing Fern warnings - [x] Documentation-writer review confirmed the existing page update is accurate and sufficient - [x] Quality Gates section completed with required justifications - [x] No secrets, API keys, or credentials committed - [x] Doc pages follow the style guide - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Hokonoken <41166525+Hokonoken@users.noreply.github.com> Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Sandbox status now detects “route drift” between the live gateway inference route and the sandbox’s recorded route, showing a warning with commands to realign or adopt the live route. * **Bug Fixes** * If the live route can’t be read, route-drift warnings are suppressed to avoid misleading output. * **Tests** * Added/expanded coverage for route-drift metadata, warning/sanitization, and safe shell-escaping in emitted commands. * **Documentation** * Updated “switch inference providers” docs with the new route-drift warning and resolution steps. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Hokonoken <41166525+Hokonoken@users.noreply.github.com> Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> Signed-off-by: Charan Jagwani <cjagwani@nvidia.com> Co-authored-by: Apurv Kumaria <akumaria@nvidia.com> Co-authored-by: Charan Jagwani <cjagwani@nvidia.com>
<!-- markdownlint-disable MD041 --> ## Summary Contain the current one-route-per-OpenShell-gateway behavior so onboarding, runtime inference switches, and connect-time repair cannot silently change the route used by another registered sandbox. This is a compatibility guard for the current OpenShell contract; true simultaneous per-sandbox routing remains upstream work. ## Related Issue Addresses NVIDIA#6315 ## Changes - Compare the requested provider/model and custom endpoint/API family with every durable registry row on the target gateway, including stopped sandboxes. - Allow identical routes and different gateways; fail closed for same-gateway conflicts, missing durable provider/model identity, incomplete legacy custom-route identity, and invalid gateway bindings. - Run the guard before onboarding setup, resumed setup, runtime `inference set`, connect-time route writes, endpoint probes, or repair mutations. - Serialize route-sensitive onboarding, runtime switches, connect repair, snapshot restore, and registry recovery per gateway; different gateways remain independent. - Persist a full pending route reservation before verification/smoke checks, and revalidate the target reservation under the gateway lock before creating or reusing a sandbox. - Resolve `inference set -g` from the target sandbox's recorded gateway instead of hardcoding the default gateway. - Keep missing-flag guidance inside NemoClaw so users do not bypass the shared-route protection with raw OpenShell commands. - Document the one-route-per-gateway contract and recovery options: align routes, remove the conflicting sandbox, or use another `NEMOCLAW_GATEWAY_PORT`. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: exact head `7df7274c` passed an independent nine-category security review with no findings; the maintainer disposition, exact-head E2E evidence, and manual validation procedure are recorded [here](NVIDIA#6338 (comment)). - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub — all 45 PR commits are GitHub-verified and signed off at `7df7274c138efe4080993378863a21b4567db842`. - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable — normal pre-commit, commit-msg, and pre-push hooks passed for the merge-resolution and documentation commits at exact head `7df7274c`. - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 93 conflict-focused assertions and CLI typecheck passed locally after the merge; an independent exact-head security pass reran 108 focused assertions across route locking, DNS finalization, late peers, incomplete recovered rows, aliases, SSRF no-bypass behavior, connect/onboard containment, and local-provider probes. - [x] Applicable broad gate passed — refreshed exact-head CI [run 28893800691](https://github.com/NVIDIA/NemoClaw/actions/runs/28893800691) passed all five CLI shards, aggregate tests/checks, build/typecheck, installer, plugin, security, docs, macOS/WSL, self-hosted, and growth-guard jobs; all five required contexts pass. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — 0 errors; Fern reports two pre-existing hidden warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) Additional validation: CLI typecheck, formatting, source/test budgets, the Fern docs build, and normal git hooks passed for the final merge and docs changes at `7df7274c`. The top-level `src/lib/onboard.ts` is net-neutral against the PR base (`+139/-139`). This branch includes `main` through `376ebe2ed18f71555aca0d139337e7f6a9a76e90`. The DCode default-model conflict in `setup-nim-flow.ts` was resolved additively: DCode retains its Ultra default and logger, while same-gateway discovery constraints, pre-probe checks, exact route checks, and API coercion remain intact; the combined interaction is now regression-tested. The later rebuild loader-seam merge was conflict-free. The race coverage includes same-gateway serialization, independent gateways, DNS-normalized endpoint changes between preliminary and finalized checks, gateway-binding changes while queued, pending-reservation loss, snapshot pre-delete rejection, recovered rows with incomplete durable route identity, and zero-mutation conflict exits. The earlier isolated OpenShell 0.0.72 proof `2026-07-06T23-01-11-471Z-pid-7719` remains applicable to the unchanged live containment path: identical same-gateway onboarding succeeded; a conflicting same-gateway onboard named both peers, made zero conflicting endpoint probes, and left the first route, providers, sandboxes, forwards, configs, containers, images, volumes, registry, credentials, gateway PID, and data plane unchanged; the same conflicting route succeeded on a separate gateway without changing gateway A; all cleanup checks passed. Final exact-head E2E passed every refreshed advisor requirement: [run 28893935439](https://github.com/NVIDIA/NemoClaw/actions/runs/28893935439) covered resume/repair, cloud onboarding, inference routing, both OpenClaw variants, both Hermes variants, double-onboard, and concurrent gateway ports; [run 28893939992](https://github.com/NVIDIA/NemoClaw/actions/runs/28893939992) passed the canonical OpenClaw cloud target; and [run 28894237287](https://github.com/NVIDIA/NemoClaw/actions/runs/28894237287) passed sandbox operations and snapshot commands. Exact-head GPT-5.5 attempt 2 reports zero required findings and one runtime-test warning. The explicit maintainer acceptance of that residual checked-in-target gap, its rationale, exact-head live evidence, and reproducible manual validation procedure are recorded [here](NVIDIA#6338 (comment)). Nemotron's only required item applies an unenforced 20-line growth heuristic even though the repository's 1,500-line test budget passes. An independent exact-head audit found no remaining product or security defect. <details> <summary>Advisor scope rationale</summary> - The two-phase custom-route check is intentional: the synchronous preliminary guard rejects known registry conflicts before endpoint work, then asynchronous DNS validation feeds a fresh-registry final guard before any OpenShell, config, or registry mutation. The removal condition is concrete: collapse the phases only when preparation can consume fully DNS-validated metadata without an earlier endpoint probe or mutation. - Registry recovery never invents missing route identity, probes an endpoint, or changes the live gateway route. Complete recovered identities are compatibility-checked; a durable recovered row with missing provider/model is preserved for inventory but makes every later same-gateway route mutation fail closed until the row is removed and re-onboarded. DNS pinning remains at the later endpoint-use/mutation boundary. - Snapshot restore compares the recorded route through the shared compatibility guard under the gateway mutation lock. It does not introduce or probe a new custom endpoint, so adding a second DNS-pinning path there would duplicate the endpoint-use boundary without closing a mutation gap. - Both the ambient `OPENSHELL_GATEWAY_ENDPOINT` environment path and the explicit endpoint flag are rejected or validated at their entry boundaries; neither bypasses the compatibility guard. - Further splits of `connect.ts`, provider onboarding, and the focused containment test files would be structural refactors without changing this issue's behavior. All affected files remain within the repository's enforced source/test budgets, so those splits are deferred to avoid widening the containment PR and its regression surface. - The OpenShell 0.0.72 proof above covers the same-gateway conflict contract and separate-gateway success path; the refreshed exact-head E2E run covers onboarding, repair, switching, sandbox operations, multi-gateway behavior, and stale rebuild. - Overlapping PR merge order requires maintainer coordination: this branch contains `main` through `376ebe2e`; overlapping route-changing work should rebase after this containment change lands. No change inside this PR can safely rewrite those independent branches. - Validation failures are intentionally converted to credential-safe `InferenceSetError` messages. Preserving raw URL/DNS error causes would risk exposing untrusted endpoint detail and is not required for the fail-closed behavior. </details> --- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> --------- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> Signed-off-by: Carlos Villela <cvillela@nvidia.com> Co-authored-by: Carlos Villela <cvillela@nvidia.com>
<!-- markdownlint-disable MD041 --> ## Summary Prepares the user-facing documentation for NemoClaw v0.0.76 and closes the release-prep documentation gate. It adds the release highlights, documents the arm64 Local NIM warning and expanded image cleanup behavior, and fixes agent-specific command headings in generated guides. ## Changes - Add the v0.0.76 release-notes section and move the shared-gateway route containment entry out of the v0.0.74 history where it was incorrectly placed. - Document the advisory Linux arm64 Local NIM manifest warning in the canonical platform matrix and local-inference guidance. - Document that `gc` scans both gateway-built and locally prebuilt sandbox image repositories. - Keep OpenClaw and Hermes session headings out of the generated Deep Agents command guide. - Add a focused variant regression test for the agent-specific session headings. ### Source summary | Merged sources | Documentation coverage | | --- | --- | | [NVIDIA#6414](NVIDIA#6414), [NVIDIA#6418](NVIDIA#6418), [NVIDIA#6416](NVIDIA#6416), [NVIDIA#6344](NVIDIA#6344) | v0.0.76 release notes and the Deep Agents quickstart/inference routes | | [NVIDIA#6340](NVIDIA#6340) | v0.0.76 release notes and existing Deep Agents observability guidance | | [NVIDIA#6338](NVIDIA#6338), [NVIDIA#6378](NVIDIA#6378), [NVIDIA#6297](NVIDIA#6297) | v0.0.76 release notes and existing inference/troubleshooting guidance | | [NVIDIA#6362](NVIDIA#6362) | v0.0.76 release notes and existing lifecycle, command, and credential guidance | | [NVIDIA#6330](NVIDIA#6330), [NVIDIA#6307](NVIDIA#6307), [NVIDIA#6008](NVIDIA#6008) | v0.0.76 release notes and existing security, troubleshooting, and command guidance | | [NVIDIA#6382](NVIDIA#6382) | v0.0.76 release notes and existing MCP/command guidance | | [NVIDIA#6326](NVIDIA#6326), [NVIDIA#5868](NVIDIA#5868), [NVIDIA#5539](NVIDIA#5539) | v0.0.76 release notes, platform matrix, inference options, and local-inference guidance | | [NVIDIA#6396](NVIDIA#6396), [NVIDIA#6390](NVIDIA#6390), [NVIDIA#6007](NVIDIA#6007) | v0.0.76 release notes and existing messaging guidance | | [NVIDIA#5388](NVIDIA#5388), [NVIDIA#6249](NVIDIA#6249), [NVIDIA#6303](NVIDIA#6303), [NVIDIA#6306](NVIDIA#6306) | v0.0.76 release notes and command/lifecycle guidance | ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — `npx vitest run --project integration test/generate-platform-docs.test.ts test/agent-variant-docs.test.ts test/sync-agent-variant-docs.test.ts` (3 files, 29 tests passed) - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — completed with 0 errors and 2 pre-existing Fern warnings - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added v0.0.76 release notes content, and removed an older conflicting bullet from the surrounding release history. * Expanded Local NVIDIA NIM guidance across inference/provider docs, including an advisory for Linux arm64 DGX Spark/DGX Station hosts when a matching `linux/arm64` image manifest is unavailable. * Updated the command reference for correct session-section rendering and clarified `gc` image cleanup sources. * **Tests** * Added coverage ensuring Deep Agents omits sessions headings while Hermes includes them. * **CI** * Refreshed Local NVIDIA NIM provider notes used in the platform matrix. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
…ecorded route (NVIDIA#6465) <!-- markdownlint-disable MD041 --> ## Summary `nemoclaw <name> status` previously showed the live host-gateway inference route as though it were the sandbox's own route. When an out-of-band operation such as direct `openshell inference set` makes that route differ from the sandbox's recorded provider/model, this PR now emits an explicit warning and safe remediation guidance. ## Related Issue Refs NVIDIA#6315. This complements NVIDIA#6338's fail-closed containment for supported flows by surfacing residual out-of-band drift in status; it follows the connect-time divergence behavior from NVIDIA#3726. ## Changes - Compute `routeDrift` only when a complete, readable live route differs from a complete recorded route. - Show the live and recorded routes, explain that `connect` realigns to the recorded route, and show the supported `inference set --sandbox` path for adopting the live route. - Share one control-character-sanitized warning formatter between status and connect so the messages cannot drift. - Cover divergent, aligned, unreadable, and missing-record cases plus rendered status output. - Document the warning in the existing one-route-per-gateway guidance. ## Original Author Credit @Hokonoken authored the diagnosis, implementation, tests, live proof, and documentation. The maintainer synchronization preserves those commits, and the review-fix commit records Hokonoken as co-author. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed — display-only route comparison; no route, registry, credential, or sandbox mutation is introduced; untrusted route values use the shared sanitizer - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes DCO declarations and every commit is GitHub Verified - [x] Normal pre-commit, commit-msg, and pre-push hooks passed on the current head - [x] Focused config, connect-route, status-flow, and snapshot tests — 87/87 passed with one local worker - [x] `npm run build:cli` — passed - [x] Plugin build — passed - [x] `npm run docs` — 0 errors and 2 pre-existing Fern warnings - [x] Documentation-writer review confirmed the existing page update is accurate and sufficient - [x] Quality Gates section completed with required justifications - [x] No secrets, API keys, or credentials committed - [x] Doc pages follow the style guide - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Hokonoken <41166525+Hokonoken@users.noreply.github.com> Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Sandbox status now detects “route drift” between the live gateway inference route and the sandbox’s recorded route, showing a warning with commands to realign or adopt the live route. * **Bug Fixes** * If the live route can’t be read, route-drift warnings are suppressed to avoid misleading output. * **Tests** * Added/expanded coverage for route-drift metadata, warning/sanitization, and safe shell-escaping in emitted commands. * **Documentation** * Updated “switch inference providers” docs with the new route-drift warning and resolution steps. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Hokonoken <41166525+Hokonoken@users.noreply.github.com> Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> Signed-off-by: Charan Jagwani <cjagwani@nvidia.com> Co-authored-by: Apurv Kumaria <akumaria@nvidia.com> Co-authored-by: Charan Jagwani <cjagwani@nvidia.com>
Summary
Contain the current one-route-per-OpenShell-gateway behavior so onboarding, runtime inference switches, and connect-time repair cannot silently change the route used by another registered sandbox. This is a compatibility guard for the current OpenShell contract; true simultaneous per-sandbox routing remains upstream work.
Related Issue
Addresses #6315
Changes
inference set, connect-time route writes, endpoint probes, or repair mutations.inference set -gfrom the target sandbox's recorded gateway instead of hardcoding the default gateway.NEMOCLAW_GATEWAY_PORT.Type of Change
Quality Gates
7df7274cpassed an independent nine-category security review with no findings; the maintainer disposition, exact-head E2E evidence, and manual validation procedure are recorded here.Verification
Verifiedin GitHub — all 45 PR commits are GitHub-verified and signed off at7df7274c138efe4080993378863a21b4567db842.pre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailable — normal pre-commit, commit-msg, and pre-push hooks passed for the merge-resolution and documentation commits at exact head7df7274c.npm run docsbuilds without warnings (doc changes only) — 0 errors; Fern reports two pre-existing hidden warnings.Additional validation: CLI typecheck, formatting, source/test budgets, the Fern docs build, and normal git hooks passed for the final merge and docs changes at
7df7274c. The top-levelsrc/lib/onboard.tsis net-neutral against the PR base (+139/-139). This branch includesmainthrough376ebe2ed18f71555aca0d139337e7f6a9a76e90. The DCode default-model conflict insetup-nim-flow.tswas resolved additively: DCode retains its Ultra default and logger, while same-gateway discovery constraints, pre-probe checks, exact route checks, and API coercion remain intact; the combined interaction is now regression-tested. The later rebuild loader-seam merge was conflict-free. The race coverage includes same-gateway serialization, independent gateways, DNS-normalized endpoint changes between preliminary and finalized checks, gateway-binding changes while queued, pending-reservation loss, snapshot pre-delete rejection, recovered rows with incomplete durable route identity, and zero-mutation conflict exits. The earlier isolated OpenShell 0.0.72 proof2026-07-06T23-01-11-471Z-pid-7719remains applicable to the unchanged live containment path: identical same-gateway onboarding succeeded; a conflicting same-gateway onboard named both peers, made zero conflicting endpoint probes, and left the first route, providers, sandboxes, forwards, configs, containers, images, volumes, registry, credentials, gateway PID, and data plane unchanged; the same conflicting route succeeded on a separate gateway without changing gateway A; all cleanup checks passed. Final exact-head E2E passed every refreshed advisor requirement: run 28893935439 covered resume/repair, cloud onboarding, inference routing, both OpenClaw variants, both Hermes variants, double-onboard, and concurrent gateway ports; run 28893939992 passed the canonical OpenClaw cloud target; and run 28894237287 passed sandbox operations and snapshot commands. Exact-head GPT-5.5 attempt 2 reports zero required findings and one runtime-test warning. The explicit maintainer acceptance of that residual checked-in-target gap, its rationale, exact-head live evidence, and reproducible manual validation procedure are recorded here. Nemotron's only required item applies an unenforced 20-line growth heuristic even though the repository's 1,500-line test budget passes. An independent exact-head audit found no remaining product or security defect.Advisor scope rationale
OPENSHELL_GATEWAY_ENDPOINTenvironment path and the explicit endpoint flag are rejected or validated at their entry boundaries; neither bypasses the compatibility guard.connect.ts, provider onboarding, and the focused containment test files would be structural refactors without changing this issue's behavior. All affected files remain within the repository's enforced source/test budgets, so those splits are deferred to avoid widening the containment PR and its regression surface.mainthrough376ebe2e; overlapping route-changing work should rebase after this containment change lands. No change inside this PR can safely rewrite those independent branches.InferenceSetErrormessages. Preserving raw URL/DNS error causes would risk exposing untrusted endpoint detail and is not required for the fail-closed behavior.Signed-off-by: Aaron Erickson aerickson@nvidia.com