feat(connectors): degrade + retry declarative instances on unreachable upstream (#3017)#3049
Merged
Merged
Conversation
…e upstream (#3017) ADR-0097 made every declarative-connector materialization failure fatal at boot — right for configuration faults, wrong for operational ones: a provider: 'mcp' instance must contact its MCP server (tools/list) to materialize, so a transient network blip aborted the whole app boot. - spec: ConnectorUpstreamUnavailableError (code CONNECTOR_UPSTREAM_UNAVAILABLE, structural guard isConnectorUpstreamUnavailable) lets a provider factory mark a failure as 'upstream temporarily unreachable — degrade and retry' instead of fatal. New integration/connector-provider-errors.ts; api-surface +3. - service-automation: the reconcile degrades such instances in BOTH modes. With no live connector under the name it registers an action-less husk — state: 'degraded' + degradedReason on the GET /connectors descriptor, status: 'error' on the def — so the instance stays visible instead of silently missing; connector_action dispatch fails with the reason and a 'retries automatically' pointer. On a changed-config re-materialization the old connector keeps serving. Degraded instances retry on an exponential backoff (5s doubling to 5min, reset by config edits) and on every metadata:reloaded reconcile; recovery swaps the husk atomically. Reconcile runs (boot / reload / retry timer) are serialized; destroy() cancels the retry loop. - connector-mcp: connect / tools/list failures are classified unavailable; transport-shape validation stays a plain (fatal) throw. Configuration faults (unknown provider, invalid providerConfig, unresolvable credentialRef, name conflicts) keep the ADR-0097 fail-loud contract, verified by the existing tests. ADR-0097 gains an 'Upstream availability' section; the deferred live-mcp showcase demo and the openapi remote-URL classification are recorded as scope boundaries. Tests: 10 new reconcile cases (degrade visibility, pointed dispatch error, fake-timer recovery, exponential backoff, reload retry, removal while degraded, changed-config keeps old serving, revert cancels retry, shutdown cancels retry, plain-throw still fatal); 3 new mcp-provider classification cases; 3 new spec error-contract cases. Refs #3017 (mechanism; showcase demo deferred — see ADR scope boundaries) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbmw3pMqNJfPbkvwh9Fkjs
github-actions
Bot
added
documentation
Contributor
📓 Docs Drift CheckThis PR changes 3 package(s): 100 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This was referenced Jul 16, 2026
os-zhuang
marked this pull request as ready for review
July 16, 2026 13:15
os-zhuang
pushed a commit
that referenced
this pull request
Jul 16, 2026
#3049's squash (4f8c2d1) is content-identical to this branch's base commit, so this merge only rebases the PR surface — the three-dot diff vs main now shows the #3055 gate alone. # Conflicts: # docs/adr/0097-declarative-connector-instances.md # packages/connectors/connector-mcp/src/mcp-provider.test.ts # packages/connectors/connector-mcp/src/mcp-provider.ts
This was referenced Jul 16, 2026
feat(create-objectstack): default scaffold ships the three generic connector executors (#3056)
#3104
Merged
os-zhuang
added a commit
that referenced
this pull request
Jul 18, 2026
…c URL (#3049 follow-up) (#3179) The openapi provider's remote spec-URL fetch now classifies faults like connector-mcp's connect path: a network error or transient HTTP status (408/429/5xx) throws ConnectorUpstreamUnavailableError so the materializer degrades + retries the instance, while a wrong URL (non-retryable 4xx) or unparseable document stays a fatal config fault. Inline/file-path specs unaffected; no service-automation change (the reconcile already routes the marker generically). +14 provider-layer tests; ADR-0097 scope-boundary list trued up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
实现 #3017(ADR-0097 后续):区分配置错误(维持 boot 致命)与运行时上游不可达(降级 + 自动重试),使一个
provider: 'mcp'声明式实例的 MCP 服务器瞬时不可达不再拖垮整个应用启动。问题
ADR-0097 的 fail-loud 契约把所有 materialization 失败都定为 boot 致命。对авторing 错误(未知 provider、非法
providerConfig、credentialRef解析失败、命名冲突)这是正确的;但mcpprovider 在 materialize 时必须连接远端 MCP 服务器(tools/list),一次网络抖动就等于全站启动失败——一个集成的降级被放大成整体不可用。这也是 showcase 演示至今只敢用provider: 'rest'的原因。方案:结构化的故障分类
spec(新文件
integration/connector-provider-errors.ts,+3 导出,api-surface 已再生):ConnectorUpstreamUnavailableError/CONNECTOR_UPSTREAM_UNAVAILABLE/isConnectorUpstreamUnavailable。守卫按code做结构判断而非instanceof,跨包重复安装也能正确分类。生产者(connector 插件)与消费者(service-automation)都只依赖 spec,保持 ADR-0097 的解耦设计。service-automation(reconcile 的降级路径):
state: 'degraded'+degradedReason,def 上status: 'error'),GET /api/v1/automation/connectors诚实展示而不是静默缺失;connector_action派发时报出指向性错误(含原因 + "平台自动重试"),不再是笼统的"插件没注册?"提示。metadata:reloadedreconcile 立即重试;恢复时经registerConnector原子替换壳;降级期间被删除的实例连壳带重试一起清理。定时器unref(),destroy()取消。RegisteredConnector/ConnectorDescriptor新增state: 'ready' | 'degraded'+degradedReason?(加法字段;GET /connectors路由原样透传,objectui 选择器容忍未知字段);新增registerDegradedConnector/getConnectorDegradedReason;跨源冲突断言抽取共用(§4 规则不变)。connector-mcp:
tools/list失败 → 分类为 upstream-unavailable(保留cause);transport 形状校验错误保持普通抛出(致命)。发现失败时连接不泄漏(沿用既有 close)。为什么不是"懒连接"
Issue 里的另一选项(首个 dispatch 时才连接)被否决:MCP 连接器的 action 列表就是服务器的
tools/list——不连接就 materialize 只能注册零 action 的 def,恰好是 ADR-0097 要消灭的 plausible-but-dead 形态。已在 ADR 中记录论证。刻意的范围边界(已写入 ADR-0097)
provider: 'mcp'演示仍延后:天然的仓内目标(@objectstack/mcp平台自身端点)在 automationstart()时尚未监听,演示会确定性地先降级、数秒后自愈,使 Dogfood CI 门时序敏感。应与仓内 MCP fixture 服务器(或自连接的启动次序方案)一起落地。测试(16 新增,均通过;四包全套 28 任务绿)
connector-materialization.test.ts+10:降级可见性(descriptor 状态/原因/空 actions)、降级实例的connector_action指向性错误、假定时器下退避重试恢复、指数退避节奏(5s→10s)、reload 立即重试并恢复、降级中被删除→壳与重试全清、变更配置上游不可达→旧连接器继续服务→恢复后原子替换(旧连接仅在新 bundle 成功后关闭)、回滚到存活配置→取消挂起重试、shutdown 取消重试、普通抛出仍 boot 致命。mcp-provider.test.ts+3:连接失败/tools-list 失败 → unavailable(cause 保留、客户端关闭);transport 配置错误 → 非 unavailable。tsc --noEmit✓、check:api-surface✓(+3 导出)、全仓 ESLint ✓、turbo test(spec / service-automation / connector-mcp / runtime,28 任务)✓。Refs #3017(机制部分;showcase 演示按上述边界延后——若维护者认可,建议合并后关闭 #3017 并为演示单开小任务)。
🤖 Generated with Claude Code
https://claude.ai/code/session_01Pbmw3pMqNJfPbkvwh9Fkjs
Generated by Claude Code