Fix Hermes discovery and Claude Code gateway setup

Author: ForgetMeAI

README.en.md

@@ -213,16 +213,34 @@ FreeNIMAPI answers Codex model-catalog requests with Codex-specific `ModelInfo`
 
 `ANTHROPIC_BASE_URL` points to the server root, **without** `/v1`:
 
+Do not choose an item on Claude Code's login screen: an arbitrary gateway is
+not configured through that menu. Stop that screen with `Ctrl+C`, then paste
+the entire block below into the **same terminal** that will launch `claude`.
+The block supplies the guided-mode loopback key even if you skipped the shared
+`export FREENIM_LOCAL_API_KEY` above:
+
 ```bash
+export FREENIM_LOCAL_API_KEY="${FREENIM_LOCAL_API_KEY:-freenim-local}"
+export FREENIM_MODEL="${FREENIM_MODEL:-nim-auto}"
 export ANTHROPIC_BASE_URL=http://127.0.0.1:3000
 export ANTHROPIC_AUTH_TOKEN="$FREENIM_LOCAL_API_KEY"
-export ANTHROPIC_MODEL=nim-auto
-export ANTHROPIC_DEFAULT_OPUS_MODEL=nim-auto
-export ANTHROPIC_DEFAULT_SONNET_MODEL=nim-auto
-export ANTHROPIC_DEFAULT_HAIKU_MODEL=nim-auto
-claude
+export ANTHROPIC_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_DEFAULT_OPUS_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_DEFAULT_SONNET_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_DEFAULT_HAIKU_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_CUSTOM_MODEL_OPTION="$FREENIM_MODEL"
+export ANTHROPIC_CUSTOM_MODEL_OPTION_NAME="FreeNIMAPI / $FREENIM_MODEL"
+export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1
+claude --model "$FREENIM_MODEL"
 ```
 
+On the first launch, Claude Code may still ask for a theme, show its security
+notice, and ask whether you trust the folder. Those prompts are expected; the
+login-method screen is not. `/status` should show `http://127.0.0.1:3000` and
+`ANTHROPIC_AUTH_TOKEN`, while `/model` includes `FreeNIMAPI / nim-auto`. To use
+another model, set something like `export FREENIM_MODEL=minimax-m3` before the
+block.
+
 This is technical compatibility through Claude Code's official LLM gateway mechanism. Anthropic does not guarantee the behavior of non-Claude models.
 
 </details>

README.md

@@ -213,16 +213,34 @@ FreeNIMAPI отвечает на Codex-запрос каталога специ
 
 `ANTHROPIC_BASE_URL` указывает на корень сервера, **без** `/v1`:
 
+Не выбирайте пункт на экране входа Claude Code: произвольный gateway не
+добавляется через это меню. Остановите этот экран через `Ctrl+C`, затем целиком
+вставьте блок ниже в **тот же терминал**, из которого будете запускать `claude`.
+Блок сам задаёт loopback-ключ guided-режима, даже если вы не выполнили общий
+`export FREENIM_LOCAL_API_KEY` выше:
+
 ```bash
+export FREENIM_LOCAL_API_KEY="${FREENIM_LOCAL_API_KEY:-freenim-local}"
+export FREENIM_MODEL="${FREENIM_MODEL:-nim-auto}"
 export ANTHROPIC_BASE_URL=http://127.0.0.1:3000
 export ANTHROPIC_AUTH_TOKEN="$FREENIM_LOCAL_API_KEY"
-export ANTHROPIC_MODEL=nim-auto
-export ANTHROPIC_DEFAULT_OPUS_MODEL=nim-auto
-export ANTHROPIC_DEFAULT_SONNET_MODEL=nim-auto
-export ANTHROPIC_DEFAULT_HAIKU_MODEL=nim-auto
-claude
+export ANTHROPIC_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_DEFAULT_OPUS_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_DEFAULT_SONNET_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_DEFAULT_HAIKU_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_CUSTOM_MODEL_OPTION="$FREENIM_MODEL"
+export ANTHROPIC_CUSTOM_MODEL_OPTION_NAME="FreeNIMAPI / $FREENIM_MODEL"
+export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1
+claude --model "$FREENIM_MODEL"
 ```
 
+При первом запуске могут остаться выбор темы, предупреждение безопасности и
+подтверждение доверия папке — это нормально. Экрана выбора способа входа быть
+не должно. В `/status` должны отображаться адрес `http://127.0.0.1:3000` и
+источник токена `ANTHROPIC_AUTH_TOKEN`; модель также появится в `/model` как
+`FreeNIMAPI / nim-auto`. Для другой модели перед блоком задайте, например,
+`export FREENIM_MODEL=minimax-m3`.
+
 Это техническая совместимость через официальный механизм Claude Code LLM gateway. Anthropic не гарантирует поведение не-Claude моделей.
 
 </details>

README.zh-CN.md

@@ -213,16 +213,33 @@ FreeNIMAPI 会为 Codex 的模型目录请求返回专用的 `ModelInfo` 元数
 
 `ANTHROPIC_BASE_URL` 必须指向服务器根地址，**不能**包含 `/v1`：
 
+不要在 Claude Code 的登录界面中选择任何选项：任意自定义 gateway 并不是
+通过该菜单配置的。请先按 `Ctrl+C` 退出，然后把下面的完整代码块粘贴到
+之后用于启动 `claude` 的**同一个终端**。即使你跳过了上面的通用
+`export FREENIM_LOCAL_API_KEY`，此代码块也会自动设置 guided 模式的本地
+loopback 密钥：
+
 ```bash
+export FREENIM_LOCAL_API_KEY="${FREENIM_LOCAL_API_KEY:-freenim-local}"
+export FREENIM_MODEL="${FREENIM_MODEL:-nim-auto}"
 export ANTHROPIC_BASE_URL=http://127.0.0.1:3000
 export ANTHROPIC_AUTH_TOKEN="$FREENIM_LOCAL_API_KEY"
-export ANTHROPIC_MODEL=nim-auto
-export ANTHROPIC_DEFAULT_OPUS_MODEL=nim-auto
-export ANTHROPIC_DEFAULT_SONNET_MODEL=nim-auto
-export ANTHROPIC_DEFAULT_HAIKU_MODEL=nim-auto
-claude
+export ANTHROPIC_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_DEFAULT_OPUS_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_DEFAULT_SONNET_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_DEFAULT_HAIKU_MODEL="$FREENIM_MODEL"
+export ANTHROPIC_CUSTOM_MODEL_OPTION="$FREENIM_MODEL"
+export ANTHROPIC_CUSTOM_MODEL_OPTION_NAME="FreeNIMAPI / $FREENIM_MODEL"
+export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1
+claude --model "$FREENIM_MODEL"
 ```
 
+首次启动时仍可能出现主题选择、安全提示以及工作区信任确认，这些都属于
+正常流程；不应再出现登录方式选择界面。`/status` 应显示
+`http://127.0.0.1:3000` 和凭据来源 `ANTHROPIC_AUTH_TOKEN`，`/model` 中则会
+出现 `FreeNIMAPI / nim-auto`。若要使用其他模型，可在运行代码块前设置，
+例如 `export FREENIM_MODEL=minimax-m3`。
+
 该接入基于 Claude Code 官方的 LLM gateway 机制实现。Anthropic 不保证非 Claude 模型能够表现出相同的行为。
 
 </details>

docs/AGENT_E2E.md

@@ -26,6 +26,11 @@ satisfy the pass gate.
 
 The runner then checks all of the following independently:
 
+- a cold-start compatibility preflight reproduces Hermes local-server discovery,
+  lists the model catalog, and retrieves the selected model through
+  `GET /v1/models/{model}` before the first generation;
+- a runner-owned proxy emits zero unexpected `WARN` or `ERROR` entries during
+  startup, model discovery, tool preflight, and the complete agent turn;
 - client process exit code is zero;
 - a real command/edit tool appears in the client trace;
 - production source changed;

scripts/agent_e2e.js

@@ -847,6 +847,8 @@ async function clientInvocation(
         ANTHROPIC_DEFAULT_OPUS_MODEL: model,
         ANTHROPIC_DEFAULT_SONNET_MODEL: model,
         ANTHROPIC_DEFAULT_HAIKU_MODEL: model,
+        ANTHROPIC_CUSTOM_MODEL_OPTION: model,
+        ANTHROPIC_CUSTOM_MODEL_OPTION_NAME: `FreeNIMAPI / ${model}`,
         CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: "1",
         CLAUDE_TMPDIR: path.join(runRoot, "tmp"),
         CLAUDE_CODE_TMPDIR: path.join(runRoot, "tmp"),
@@ -963,9 +965,18 @@ async function startProxy(model) {
       `live-nvidia-nim requires ${OFFICIAL_NVIDIA_UPSTREAM_ORIGIN}; got ${upstreamOrigin}`,
     );
   }
+  const logCounts = { warn: 0, error: 0 };
   const server = createFreeNimServer({
     config,
-    logger: { info() {}, warn() {}, error() {} },
+    logger: {
+      info() {},
+      warn() {
+        logCounts.warn += 1;
+      },
+      error() {
+        logCounts.error += 1;
+      },
+    },
   });
   await new Promise((resolve, reject) => {
     server.once("error", reject);
@@ -975,6 +986,65 @@ async function startProxy(model) {
     server,
     baseUrl: `http://127.0.0.1:${server.address().port}`,
     upstreamOrigin,
+    logCounts,
+  };
+}
+
+async function startupCompatibilityPreflight(baseUrl, model) {
+  const headers = { authorization: `Bearer ${LOCAL_KEY}` };
+  const optionalProbePaths = [
+    "/api/v1/models",
+    "/api/tags",
+    "/v1/props",
+    "/props",
+    "/version",
+  ];
+  const optionalProbes = {};
+
+  for (const probePath of optionalProbePaths) {
+    const response = await fetch(`${baseUrl}${probePath}`, { headers });
+    optionalProbes[probePath] = response.status;
+    await response.text();
+    if (response.status !== 404) {
+      throw new Error(
+        `Local-server discovery probe ${probePath} must return a truthful HTTP 404; got ${response.status}`,
+      );
+    }
+  }
+
+  const listResponse = await fetch(`${baseUrl}/v1/models`, { headers });
+  if (!listResponse.ok) {
+    throw new Error(`Model catalog preflight failed with HTTP ${listResponse.status}`);
+  }
+  const catalog = await listResponse.json();
+  if (catalog?.object !== "list" || !Array.isArray(catalog?.data)) {
+    throw new Error("Model catalog preflight returned an invalid OpenAI-compatible payload");
+  }
+
+  const detailResponse = await fetch(
+    `${baseUrl}/v1/models/${encodeURIComponent(model)}`,
+    { headers },
+  );
+  if (!detailResponse.ok) {
+    throw new Error(
+      `Hermes model metadata preflight for '${model}' failed with HTTP ${detailResponse.status}`,
+    );
+  }
+  const detail = await detailResponse.json();
+  if (detail?.object !== "model" || typeof detail?.id !== "string") {
+    throw new Error("Hermes model metadata preflight returned an invalid model object");
+  }
+  if (!catalog.data.some((entry) => entry?.id === detail.id)) {
+    throw new Error(`Retrieved model '${detail.id}' is missing from the model catalog`);
+  }
+
+  return {
+    passed: true,
+    model_requested: model,
+    model_resolved: detail.id,
+    list_status: listResponse.status,
+    detail_status: detailResponse.status,
+    optional_probe_statuses: optionalProbes,
   };
 }
 
@@ -1317,6 +1387,8 @@ function markdownReport(report) {
       "Run the normal Hermes E2E separately for completed-tool ordering evidence.",
       "",
       `- Hermes process: ${mark(result?.checks?.client_exit)}`,
+      `- Hermes startup compatibility: ${mark(report.startup_compatibility?.passed)}`,
+      `- Clean proxy log: ${report.proxy_logs?.observable ? mark(report.proxy_logs.clean) : "NOT OBSERVABLE"}`,
       `- Initial fixture RED: ${mark(result?.checks?.initial_fixture_red)}`,
       `- Production source changed: ${mark(result?.checks?.source_changed)}`,
       `- Contract protected: ${mark(result?.checks?.contract_unchanged)}`,
@@ -1338,6 +1410,8 @@ function markdownReport(report) {
     `- Text tool fallback: disabled`,
     `- Structural tool preflight: ${report.preflight.structural_tool_call ? "PASS" : "FAIL"}`,
     `- Tool continuation preflight: ${report.preflight.tool_continuation ? "PASS" : "FAIL"}`,
+    `- Hermes startup compatibility: ${report.startup_compatibility?.passed ? "PASS" : "FAIL"}`,
+    `- Clean proxy log: ${report.proxy_logs?.observable ? (report.proxy_logs.clean ? "PASS" : "FAIL") : "NOT OBSERVABLE"}`,
     "",
     "| Client | Result | Initial RED | Failure→edit→green | Source changed | Contract protected | Independent tests | Final turn |",
     "|---|---:|---:|---:|---:|---:|---:|---:|",
@@ -1371,6 +1445,7 @@ async function main() {
   requireLoopbackProxy(baseUrl);
 
   try {
+    const startupCompatibility = await startupCompatibilityPreflight(baseUrl, options.model);
     const preflight = await livePreflight(
       baseUrl,
       options.model,
@@ -1409,6 +1484,19 @@ async function main() {
       }
     }
 
+    const proxyLogs = ownedProxy
+      ? {
+          observable: true,
+          warn_count: ownedProxy.logCounts.warn,
+          error_count: ownedProxy.logCounts.error,
+          clean: ownedProxy.logCounts.warn === 0 && ownedProxy.logCounts.error === 0,
+        }
+      : {
+          observable: false,
+          warn_count: null,
+          error_count: null,
+          clean: null,
+        };
     const report = {
       timestamp: new Date().toISOString(),
       mode: options.hermesCliDemo ? "visual-hermes-cli-demo" : options.mode,
@@ -1417,9 +1505,15 @@ async function main() {
       proxy_ownership: ownedProxy ? "runner-owned" : "external-unverified",
       upstream_provider: ownedProxy ? "nvidia-nim" : "unverified",
       upstream_origin: ownedProxy?.upstreamOrigin ?? null,
+      startup_compatibility: startupCompatibility,
+      proxy_logs: proxyLogs,
       preflight,
       results,
-      passed: results.length > 0 && results.every((result) => result.passed),
+      passed:
+        startupCompatibility.passed &&
+        (!proxyLogs.observable || proxyLogs.clean) &&
+        results.length > 0 &&
+        results.every((result) => result.passed),
     };
     await writeFile(path.join(reportDir, "report.json"), `${JSON.stringify(report, null, 2)}\n`);
     await writeFile(path.join(reportDir, "SUMMARY.md"), markdownReport(report));

scripts/check_readme_nominal.js

@@ -36,6 +36,20 @@ const installBlock = [
   "npm ci",
   "npm run start:guided",
 ].join("\n");
+const claudeGatewayBlock = [
+  'export FREENIM_LOCAL_API_KEY="${FREENIM_LOCAL_API_KEY:-freenim-local}"',
+  'export FREENIM_MODEL="${FREENIM_MODEL:-nim-auto}"',
+  "export ANTHROPIC_BASE_URL=http://127.0.0.1:3000",
+  'export ANTHROPIC_AUTH_TOKEN="$FREENIM_LOCAL_API_KEY"',
+  'export ANTHROPIC_MODEL="$FREENIM_MODEL"',
+  'export ANTHROPIC_DEFAULT_OPUS_MODEL="$FREENIM_MODEL"',
+  'export ANTHROPIC_DEFAULT_SONNET_MODEL="$FREENIM_MODEL"',
+  'export ANTHROPIC_DEFAULT_HAIKU_MODEL="$FREENIM_MODEL"',
+  'export ANTHROPIC_CUSTOM_MODEL_OPTION="$FREENIM_MODEL"',
+  'export ANTHROPIC_CUSTOM_MODEL_OPTION_NAME="FreeNIMAPI / $FREENIM_MODEL"',
+  'export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1',
+  'claude --model "$FREENIM_MODEL"',
+].join("\n");
 
 function fail(message) {
   throw new Error(`README nominal failed: ${message}`);
@@ -97,6 +111,9 @@ for (const [name, markdown] of documents) {
   }
 
   if (!markdown.includes(installBlock)) fail(`${name} does not contain the four-command install block`);
+  if (!markdown.includes(claudeGatewayBlock)) {
+    fail(`${name} does not contain the self-contained Claude Code gateway launch block`);
+  }
   if (!markdown.includes("FREENIM_LOCAL_API_KEY=freenim-local npm run health")) {
     fail(`${name} does not contain the documented health smoke`);
   }

scripts/mock_agent_e2e.js

@@ -10,7 +10,7 @@ import { createFreeNimServer } from "../src/server.js";
 
 const ROOT = path.dirname(path.dirname(fileURLToPath(import.meta.url)));
 const LOCAL_KEY = "local-e2e";
-const MODEL = "nim-auto";
+const MODEL = process.env.FREENIM_MOCK_MODEL || "nim-auto";
 const CLIENTS = process.env.FREENIM_MOCK_CLIENTS || "hermes,codex,claude,opencode-chat,opencode-responses";
 const DEBUG = process.env.FREENIM_MOCK_DEBUG === "1";
 const HERMES_CLI_DEMO = process.env.FREENIM_MOCK_HERMES_CLI_DEMO === "1";
@@ -337,6 +337,7 @@ async function main() {
     responseCache.clear();
     const clientReportDir = path.join(reportDir, client);
     await mkdir(clientReportDir, { recursive: true });
+    const proxyLogCounts = { warn: 0, error: 0 };
     const server = createFreeNimServer({
       config: {
         ...loadConfig({}),
@@ -353,7 +354,15 @@ async function main() {
         ? ({ method, path: requestPath, query }) =>
             console.error(`[FreeNIMAPI mock debug] downstream ${method} ${requestPath}${query}`)
         : null,
-      logger: { info() {}, warn() {}, error() {} },
+      logger: {
+        info() {},
+        warn() {
+          proxyLogCounts.warn += 1;
+        },
+        error() {
+          proxyLogCounts.error += 1;
+        },
+      },
     });
     await new Promise((resolve, reject) => {
       server.once("error", reject);
@@ -392,6 +401,19 @@ async function main() {
     }
 
     const report = JSON.parse(await readFile(path.join(clientReportDir, "report.json"), "utf8"));
+    const cleanProxyLog = proxyLogCounts.warn === 0 && proxyLogCounts.error === 0;
+    report.proxy_logs = {
+      observable: true,
+      warn_count: proxyLogCounts.warn,
+      error_count: proxyLogCounts.error,
+      clean: cleanProxyLog,
+    };
+    report.results[0].checks.clean_proxy_log = cleanProxyLog;
+    if (!cleanProxyLog) {
+      report.passed = false;
+      report.results[0].passed = false;
+      report.results[0].error ||= "FreeNIMAPI emitted WARN or ERROR during the client run";
+    }
     if (client === "codex" && !sawCodexApplyPatch) {
       report.passed = false;
       report.results[0].passed = false;