diff --git a/xpertai/.changeset/tidy-codexpert-connectors.md b/xpertai/.changeset/tidy-codexpert-connectors.md new file mode 100644 index 00000000..cb2485a1 --- /dev/null +++ b/xpertai/.changeset/tidy-codexpert-connectors.md @@ -0,0 +1,5 @@ +--- +'@xpert-ai/plugin-codexpert-connector': patch +--- + +Add the Codexpert connector middleware for exposing Codexpert context tools, running Codexpert tasks, and projecting visible Codexpert output to users. diff --git a/xpertai/middlewares/codexpert-connector/.spec.swcrc b/xpertai/middlewares/codexpert-connector/.spec.swcrc new file mode 100644 index 00000000..8b445c4a --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/.spec.swcrc @@ -0,0 +1,22 @@ +{ + "jsc": { + "target": "es2017", + "parser": { + "syntax": "typescript", + "decorators": true, + "dynamicImport": true + }, + "transform": { + "decoratorMetadata": true, + "legacyDecorator": true + }, + "keepClassNames": true, + "externalHelpers": true, + "loose": true + }, + "module": { + "type": "commonjs" + }, + "sourceMaps": true, + "exclude": [] +} diff --git a/xpertai/middlewares/codexpert-connector/README.md b/xpertai/middlewares/codexpert-connector/README.md new file mode 100644 index 00000000..3f18c896 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/README.md @@ -0,0 +1,383 @@ +# XpertAI Plugin: Codexpert Connector Middleware + +`@xpert-ai/plugin-codexpert-connector` connects an Xpert agent to Codexpert coding sessions. + +It exposes Codexpert context tools to the agent, runs Codexpert tasks synchronously, streams Codexpert-visible output to the user, and returns compact task metadata back to the agent. + +For the Chinese documentation, see [README_zh.md](./README_zh.md). + +## Current Runtime Model + +- The plugin is a connector middleware, not a Codexpert runtime. +- Codexpert remains the source of truth for coding sessions, tasks, threads, executions, and environments. +- Xpert only owns agent orchestration and user-visible projection. +- The agent receives context tools plus final task metadata. +- The user sees Codexpert-visible text directly, after light cleanup. +- The plugin stores only a lightweight run mapping table for recovery and diagnostics. + +## Design Principles + +### Codexpert Owns Coding State + +Codexpert owns MCP tools, coding session anchors, task execution, environment setup, and environment reuse. + +The plugin does not clone repositories, create coding environments, or copy Codexpert task details into Xpert. + +### The Plugin Connects And Projects + +The plugin reads the current business principal from the Xpert runtime context, injects identity headers into Codexpert requests, and converts Codexpert stream events into Xpert-compatible visible text chunks. + +### The Agent Receives Metadata + +Codexpert text is projected to the user. The agent receives only compact terminal metadata, such as session id, task id, thread id, execution id, repository, branch, environment id, summary, and error. + +The agent should not consume raw Codexpert stream events or summarize Codexpert output again. + +## Code Entry Points + +| Module | File | +| --- | --- | +| Plugin entry | `src/index.ts` | +| Middleware strategy | `src/lib/codexpert-connector.middleware.ts` | +| Codexpert HTTP / MCP client | `src/lib/codexpert-connector.client.ts` | +| Visible projection | `src/lib/codexpert-visible-projector.ts` | +| Types and config schema | `src/lib/types.ts` | +| Run mapping service | `src/lib/codexpert-connector-run.service.ts` | +| Run mapping entity | `src/lib/entities/codexpert-connector-run.entity.ts` | + +The middleware strategy name is: + +```text +CodexpertConnector +``` + +## Configuration + +| Field | Type | Default | Description | +| --- | --- | --- | --- | +| `codexpertMcpUrl` | `string` | none | Codexpert MCP URL, for example `http://localhost:3001/v1/mcp`. | +| `codexpertConnectorBaseUrl` | `string` | none | Codexpert connector API base URL, for example `http://localhost:3001/api`. Do not include `/api/codexpert-connector`; the client appends the connector path. | +| `serviceToken` | `string` | none | Service token used for Codexpert MCP and connector stream requests. | +| `timeoutMs` | `number` | `600000` | Timeout for MCP calls and synchronous stream execution, in milliseconds. | +| `enableVisibleProjection` | `boolean` | `true` | Whether to project Codexpert-visible output to the user. | +| `enableStatusEvents` | `boolean` | `true` | Whether to project compact status milestones. | +| `defaultXpertId` | `string` | none | Default Codexpert coding assistant id. | +| `defaultRepoId` | `string` | none | Default repository id. | +| `defaultConnectionId` | `string` | none | Default Git connection id. | +| `defaultBranchName` | `string` | none | Default branch name. | + +Example: + +```json +{ + "codexpertMcpUrl": "http://localhost:3001/v1/mcp", + "codexpertConnectorBaseUrl": "http://localhost:3001/api", + "serviceToken": "", + "timeoutMs": 600000, + "enableVisibleProjection": true, + "enableStatusEvents": true, + "defaultXpertId": "", + "defaultRepoId": "", + "defaultConnectionId": "", + "defaultBranchName": "main" +} +``` + +If `codexpertMcpUrl`, `codexpertConnectorBaseUrl`, or `serviceToken` is missing, the middleware name is still registered, but no Codexpert tools are exposed. + +## Identity Propagation + +The plugin resolves the business principal from the current Xpert runtime context: + +- `tenantId` +- `organizationId` +- `userId` + +Every request to Codexpert includes: + +```text +Authorization: Bearer +tenant-id: +organization-id: +x-principal-user-id: +``` + +Rules: + +- Missing `tenantId`, `organizationId`, or `userId` fails immediately. +- The plugin does not infer Codexpert users from external-platform open ids. +- Lark/Feishu users must already be bound to real business users on the Xpert side. + +## Agent Tools + +The plugin passes through these Codexpert MCP tools: + +- `listCodingAssistants` +- `listCodexpertConversations` +- `listGitConnections` +- `listGitRepositories` +- `listGitBranches` +- `selectCodingContext` +- `resolveCodexpertConversationContext` +- `resumeCodexpertSession` + +Pull request and issue-comment publishing tools are intentionally not exposed through this external connector. Codexpert/code agent owns repository writeback and PR delivery inside the coding task. If Codexpert creates a PR, the connector returns its URL as result metadata. + +External Xpert agents should use this connector middleware instead of attaching the Codexpert MCP server as a standalone MCP toolset. A direct MCP toolset bypasses the connector filter and is reserved for Codexpert-side coding agents that own delivery actions. + +The plugin also provides: + +```text +runCodexpertTask +``` + +`runCodexpertTask` input: + +| Field | Required | Description | +| --- | --- | --- | +| `prompt` | yes | Task prompt to send to Codexpert. | +| `taskTitle` | no | Optional task title. | +| `codingSessionId` | no | Existing Codexpert coding session. | +| `conversationId` | no | Context recovery key. | +| `threadId` | no | Context recovery key. | +| `taskId` | no | Context recovery key. | +| `xpertId` | no | Codexpert coding assistant id; falls back to `defaultXpertId`. | +| `repoId` | no | Codexpert repository id; falls back to `defaultRepoId`. | +| `connectionId` | no | Git connection id; falls back to `defaultConnectionId`. | +| `branchName` | no | Branch name; falls back to `defaultBranchName`. | +| `timeoutMs` | no | Per-call timeout override. | + +Result returned to the agent: + +```ts +type CodexpertTaskResult = { + status: 'success' | 'failed' | 'timeout' | 'canceled' + codingSessionId: string | null + taskId: string | null + threadId: string | null + executionId: string | null + repo: { + id?: string | null + name?: string | null + owner?: string | null + slug?: string | null + } | null + branch: string | null + environmentId: string | null + environmentReused: boolean | null + summary: string | null + error: string | null + prUrl?: string | null +} +``` + +## Recommended Agent Flow + +Without default context: + +1. `listCodingAssistants` +2. `listGitConnections` +3. `listGitRepositories` +4. `listGitBranches` +5. `selectCodingContext` +6. `runCodexpertTask` + +For an existing task or session: + +1. `listCodexpertConversations` or `resolveCodexpertConversationContext` +2. `resumeCodexpertSession` +3. `runCodexpertTask` + +If `defaultXpertId`, `defaultConnectionId`, `defaultRepoId`, and `defaultBranchName` are configured, `runCodexpertTask` can create a session without an explicit `codingSessionId`. + +## Visible Projection + +Codexpert stream events are converted into Xpert-compatible text chunks: + +```ts +{ + type: 'text', + text, + xpertName: 'Codexpert', + agentKey: 'codexpert' +} +``` + +Projection behavior: + +- `text_delta` preserves spaces and Markdown as much as possible. +- The `thought` stream is filtered. +- `assistant_snapshot` is filtered once normal output text has already appeared. +- Raw metadata, debug messages, and setup logs are filtered. +- Setup noise such as dependency installation and clone progress is filtered. +- `done.summary` is only shown when no output text was projected. + +Status milestones are enabled by default and deduplicated: + +- `正在准备编码环境。` +- `编码环境已就绪。` +- `已开始处理。` +- `需要补充信息。` +- `已完成。` +- `Codexpert failed: ` + +## Run Mapping Table + +The plugin maintains a lightweight table: + +```text +plugin_codexpert_connector_run +``` + +It records: + +- `tenant_id` +- `organization_id` +- `user_id` +- `xpert_id` +- `conversation_id` +- `execution_id` +- `coding_session_id` +- `task_id` +- `thread_id` +- `codexpert_execution_id` +- `status` +- `last_error` +- `metadata` +- `created_at` +- `updated_at` + +The table maps Xpert executions to Codexpert sessions, tasks, threads, and executions. It does not copy Codexpert task details or raw stream events. + +## Local Development + +Local testing needs three running pieces: + +- Codexpert API, which owns coding sessions and task execution. +- Xpert API, which loads this middleware and provides the current business principal. +- The local `@xpert-ai/plugin-codexpert-connector` workspace. + +### Codexpert Environment + +Configure the Codexpert API environment first. The exact ports are up to your local setup, but the plugin URLs must point to the same Codexpert API instance. + +For example, if Codexpert runs on `3001` and Xpert API runs on `3000`, then `codexpertMcpUrl` is `http://localhost:3001/v1/mcp`, `codexpertConnectorBaseUrl` is `http://localhost:3001/api`, and `XPERTAI_API_URL` is `http://localhost:3000/api`. + +```env +# Codexpert API listen port. +PORT= + +# Xpert API used by Codexpert when it creates tasks, environments, and related Xpert-side resources. +XPERTAI_API_URL=http://localhost:/api + +# Optional. Set this too when running the Codexpert web UI locally. +VITE_XPERTAI_API_URL=http://localhost:/api + +# Workspace API key used by Codexpert when calling Xpert APIs. +XPERTAI_WORKSPACE_API_KEY= + +# Token accepted by the Codexpert MCP endpoint. +MCP_SERVER_TOKEN= + +# Optional additional MCP tokens. Use this when the connector should use a token +# different from MCP_SERVER_TOKEN. +MCP_SERVER_TOKENS= + +# Token accepted by /api/codexpert-connector/*. +# If this is omitted, the connector endpoint also accepts MCP_SERVER_TOKEN, +# CODEXPERT_ACP_SERVICE_TOKEN, ACP_SERVICE_TOKEN, and MCP_SERVER_TOKENS. +CODEXPERT_CONNECTOR_SERVICE_TOKEN= + +# Optional local user-id mapping when Xpert and Codexpert development data use +# different user ids for the same tester. +CODEXPERT_DEV_PRINCIPAL_USER_MAP={"":""} +``` + +The middleware `serviceToken` must be accepted by both Codexpert request paths it calls: + +- `POST /v1/mcp` +- `POST /api/codexpert-connector/sessions` +- `POST /api/codexpert-connector/sessions/:sessionId/prompts/stream` + +For the simplest local setup, use the same value for `MCP_SERVER_TOKEN`, `CODEXPERT_CONNECTOR_SERVICE_TOKEN`, and the middleware `serviceToken`. + +### Xpert Host Environment + +The Xpert API must be able to install and load the local plugin workspace. Add the plugin workspace to `PLUGIN_WORKSPACE_ROOTS` in the Xpert API environment: + +```env +PLUGIN_WORKSPACE_ROOTS=/xpertai/middlewares/codexpert-connector +``` + +`PLUGIN_WORKSPACE_ROOTS` may contain multiple roots separated by `;` or `,`. The local install `workspacePath` must be an absolute path on the Xpert API host and must be inside one of these allowed roots. + +### Build The Plugin + +From the `xpertai` workspace in the `xpert-plugins` repository: + +```bash +cd +pnpm -C xpertai install +pnpm -C xpertai exec tsc -p middlewares/codexpert-connector/tsconfig.lib.json +``` + +### Install The Local Plugin Into Xpert + +Install or refresh the local plugin in the Xpert API host: + +```bash +cd + +pnpm plugin:reinstall:local \ + @xpert-ai/plugin-codexpert-connector \ + /xpertai/middlewares/codexpert-connector \ + --api-url http://localhost: \ + --org-id \ + --token \ + --build-cwd /xpertai \ + --build-command "./node_modules/.bin/tsc -p middlewares/codexpert-connector/tsconfig.lib.json" +``` + +You can also install it from the Xpert plugin settings UI by using: + +```text +Plugin package name: @xpert-ai/plugin-codexpert-connector +Workspace path: /xpertai/middlewares/codexpert-connector +``` + +### Configure The Agent Middleware + +Add the `CodexpertConnector` middleware to the target Xpert agent and configure it with the Codexpert URLs and service token: + +```json +{ + "codexpertMcpUrl": "http://localhost:3001/v1/mcp", + "codexpertConnectorBaseUrl": "http://localhost:3001/api", + "serviceToken": "", + "timeoutMs": 600000, + "enableVisibleProjection": true, + "enableStatusEvents": true, + "defaultXpertId": "", + "defaultRepoId": "", + "defaultConnectionId": "", + "defaultBranchName": "main" +} +``` + +If the default Codexpert context fields are omitted, the agent must first use the context tools to select or resume a coding context before calling `runCodexpertTask`. + +### Smoke Test Checklist + +- Xpert API is running and has loaded `@xpert-ai/plugin-codexpert-connector`. +- Codexpert API is running and accepts `Authorization: Bearer `. +- The middleware config points to the Codexpert API port, not the Xpert API port. +- The current Xpert request has `tenantId`, `organizationId`, and `userId`. +- `XPERTAI_WORKSPACE_API_KEY` lets Codexpert call the configured Xpert API. + +## Boundaries + +- The plugin is not a replacement for the Codexpert runtime. +- It does not implement coding environment creation, repository cloning, or task execution inside Xpert. +- It does not store Codexpert raw events. +- It requires a real business principal in the current Xpert context. +- `serviceToken` is a service-to-service credential and should not be provided by the agent or end user. diff --git a/xpertai/middlewares/codexpert-connector/README_zh.md b/xpertai/middlewares/codexpert-connector/README_zh.md new file mode 100644 index 00000000..d19dcc08 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/README_zh.md @@ -0,0 +1,383 @@ +# XpertAI 插件:Codexpert 连接器中间件 + +`@xpert-ai/plugin-codexpert-connector` 用来把 Xpert Agent 连接到 Codexpert 编码会话。 + +它向 Agent 暴露 Codexpert 上下文工具,同步执行 Codexpert 编码任务,把 Codexpert 的用户可见输出实时投影给用户,并把精简后的任务元信息返回给 Agent。 + +英文文档见 [README.md](./README.md)。 + +## 当前运行模型 + +- 插件是连接型 middleware,不是 Codexpert 运行时。 +- Codexpert 仍然是 coding session、task、thread、execution、environment 的事实源。 +- Xpert 只负责 Agent 编排和用户可见投影。 +- Agent 拿到上下文工具和最终任务元信息。 +- 用户看到 Codexpert 可见正文,插件只做轻量清洗。 +- 插件只保存一张轻量运行映射表,用于恢复和排查。 + +## 设计思路 + +### Codexpert 负责编码状态 + +Codexpert 负责 MCP 工具、编码会话锚点、任务执行、环境创建和环境复用。 + +插件不 clone 仓库,不创建编码环境,也不把 Codexpert task 详情复制到 Xpert。 + +### 插件负责连接和投影 + +插件从 Xpert 当前运行上下文读取真实业务用户身份,把身份头注入到 Codexpert MCP 和 connector stream 请求里,并把 Codexpert stream event 转成 Xpert 用户消息流支持的文本 chunk。 + +### Agent 只拿元信息 + +Codexpert 正文直接投影给用户。Agent 最终只拿精简元信息,例如 session id、task id、thread id、execution id、仓库、分支、环境 id、summary 和 error。 + +Agent 不需要消费 Codexpert raw stream event,也不负责重新总结 Codexpert 正文。 + +## 代码入口 + +| 模块 | 文件 | +| --- | --- | +| 插件入口 | `src/index.ts` | +| Middleware strategy | `src/lib/codexpert-connector.middleware.ts` | +| Codexpert HTTP / MCP client | `src/lib/codexpert-connector.client.ts` | +| 用户可见投影 | `src/lib/codexpert-visible-projector.ts` | +| 类型与配置 schema | `src/lib/types.ts` | +| 运行映射服务 | `src/lib/codexpert-connector-run.service.ts` | +| 运行映射实体 | `src/lib/entities/codexpert-connector-run.entity.ts` | + +middleware strategy 名称是: + +```text +CodexpertConnector +``` + +## 配置项 + +| 字段 | 类型 | 默认值 | 说明 | +| --- | --- | --- | --- | +| `codexpertMcpUrl` | `string` | 无 | Codexpert MCP 地址,例如 `http://localhost:3001/v1/mcp`。 | +| `codexpertConnectorBaseUrl` | `string` | 无 | Codexpert connector API 基础地址,例如 `http://localhost:3001/api`。不要配到 `/api/codexpert-connector`,客户端会自动拼接后续路径。 | +| `serviceToken` | `string` | 无 | 插件调用 Codexpert MCP 和 connector stream 接口使用的服务令牌。 | +| `timeoutMs` | `number` | `600000` | MCP 调用和同步任务 stream 的超时时间,单位毫秒。 | +| `enableVisibleProjection` | `boolean` | `true` | 是否把 Codexpert 可见输出投影给用户。 | +| `enableStatusEvents` | `boolean` | `true` | 是否投影精简状态里程碑。 | +| `defaultXpertId` | `string` | 无 | 默认 Codexpert 编码助手 id。 | +| `defaultRepoId` | `string` | 无 | 默认仓库 id。 | +| `defaultConnectionId` | `string` | 无 | 默认 Git connection id。 | +| `defaultBranchName` | `string` | 无 | 默认分支名。 | + +示例: + +```json +{ + "codexpertMcpUrl": "http://localhost:3001/v1/mcp", + "codexpertConnectorBaseUrl": "http://localhost:3001/api", + "serviceToken": "", + "timeoutMs": 600000, + "enableVisibleProjection": true, + "enableStatusEvents": true, + "defaultXpertId": "", + "defaultRepoId": "", + "defaultConnectionId": "", + "defaultBranchName": "main" +} +``` + +如果缺少 `codexpertMcpUrl`、`codexpertConnectorBaseUrl` 或 `serviceToken`,插件只注册 middleware 名称,不会暴露 Codexpert 工具。 + +## 身份透传 + +插件会从 Xpert 当前运行上下文解析业务用户身份: + +- `tenantId` +- `organizationId` +- `userId` + +每次请求 Codexpert 都会注入: + +```text +Authorization: Bearer +tenant-id: +organization-id: +x-principal-user-id: +``` + +规则: + +- 缺 `tenantId`、`organizationId` 或 `userId` 时,插件直接失败。 +- 插件不根据外部平台 open id 推断 Codexpert 用户。 +- 飞书用户必须先在 Xpert 侧绑定并解析成真实业务用户。 + +## 暴露给 Agent 的工具 + +插件透传以下 Codexpert MCP 工具: + +- `listCodingAssistants` +- `listCodexpertConversations` +- `listGitConnections` +- `listGitRepositories` +- `listGitBranches` +- `selectCodingContext` +- `resolveCodexpertConversationContext` +- `resumeCodexpertSession` + +PR 发布和 issue 评论工具不会通过这个外部连接器暴露。仓库写回和 PR 交付由 Codexpert/code agent 在编码任务内部负责。如果 Codexpert 创建了 PR,连接器只会把 PR URL 作为结果元信息返回。 + +外部 Xpert 应使用这个 connector middleware,不要再把 Codexpert MCP server 作为独立 MCP toolset 直接挂到智能体上。直连 MCP 会绕过 connector 的工具过滤,只适合 Codexpert 侧负责交付动作的编码智能体使用。 + +插件额外提供: + +```text +runCodexpertTask +``` + +`runCodexpertTask` 输入: + +| 字段 | 必填 | 说明 | +| --- | --- | --- | +| `prompt` | 是 | 要交给 Codexpert 执行的任务内容。 | +| `taskTitle` | 否 | 任务标题。 | +| `codingSessionId` | 否 | 已有 Codexpert coding session。 | +| `conversationId` | 否 | 用于恢复上下文。 | +| `threadId` | 否 | 用于恢复上下文。 | +| `taskId` | 否 | 用于恢复上下文。 | +| `xpertId` | 否 | Codexpert 编码助手 id;缺省时使用 `defaultXpertId`。 | +| `repoId` | 否 | Codexpert 仓库 id;缺省时使用 `defaultRepoId`。 | +| `connectionId` | 否 | Git connection id;缺省时使用 `defaultConnectionId`。 | +| `branchName` | 否 | 分支名;缺省时使用 `defaultBranchName`。 | +| `timeoutMs` | 否 | 覆盖本次任务的超时时间。 | + +返回给 Agent 的结果: + +```ts +type CodexpertTaskResult = { + status: 'success' | 'failed' | 'timeout' | 'canceled' + codingSessionId: string | null + taskId: string | null + threadId: string | null + executionId: string | null + repo: { + id?: string | null + name?: string | null + owner?: string | null + slug?: string | null + } | null + branch: string | null + environmentId: string | null + environmentReused: boolean | null + summary: string | null + error: string | null + prUrl?: string | null +} +``` + +## 推荐 Agent 调用流程 + +没有默认上下文时: + +1. `listCodingAssistants` +2. `listGitConnections` +3. `listGitRepositories` +4. `listGitBranches` +5. `selectCodingContext` +6. `runCodexpertTask` + +已有任务或会话时: + +1. `listCodexpertConversations` 或 `resolveCodexpertConversationContext` +2. `resumeCodexpertSession` +3. `runCodexpertTask` + +如果配置了 `defaultXpertId`、`defaultConnectionId`、`defaultRepoId`、`defaultBranchName`,`runCodexpertTask` 可以在没有 `codingSessionId` 的情况下创建 session。 + +## 用户可见投影 + +Codexpert stream event 会被转换成 Xpert 支持的文本 chunk: + +```ts +{ + type: 'text', + text, + xpertName: 'Codexpert', + agentKey: 'codexpert' +} +``` + +投影规则: + +- `text_delta` 尽量保留原始空格和 Markdown。 +- 过滤 `thought` stream。 +- 已经收到正文后,过滤重复的 `assistant_snapshot`。 +- 过滤 raw metadata、debug、setup log 等内部文本。 +- 过滤安装依赖、clone progress 等 setup 噪音。 +- `done.summary` 只在没有正文输出时显示,避免重复刷屏。 + +状态里程碑默认开启,并会去重: + +- `正在准备编码环境。` +- `编码环境已就绪。` +- `已开始处理。` +- `需要补充信息。` +- `已完成。` +- `Codexpert failed: ` + +## 运行映射表 + +插件维护一张轻量表: + +```text +plugin_codexpert_connector_run +``` + +记录字段包括: + +- `tenant_id` +- `organization_id` +- `user_id` +- `xpert_id` +- `conversation_id` +- `execution_id` +- `coding_session_id` +- `task_id` +- `thread_id` +- `codexpert_execution_id` +- `status` +- `last_error` +- `metadata` +- `created_at` +- `updated_at` + +这张表用于记录 Xpert execution 到 Codexpert session、task、thread、execution 的映射。它不复制 Codexpert task 详情,也不保存 raw stream event。 + +## 本地开发 + +本地测试需要三部分同时可用: + +- Codexpert API:负责 coding session 和任务执行。 +- Xpert API:负责加载这个 middleware,并提供当前业务用户身份。 +- 本地 `@xpert-ai/plugin-codexpert-connector` 插件工作区。 + +### Codexpert 环境变量 + +先配置 Codexpert API 环境。端口可以按你的本地环境调整,但插件里的 URL 必须指向同一个 Codexpert API 实例。 + +例如 Codexpert 运行在 `3001`、Xpert API 运行在 `3000` 时,`codexpertMcpUrl` 是 `http://localhost:3001/v1/mcp`,`codexpertConnectorBaseUrl` 是 `http://localhost:3001/api`,`XPERTAI_API_URL` 是 `http://localhost:3000/api`。 + +```env +# Codexpert API 监听端口。 +PORT= + +# Codexpert 创建任务、环境以及相关 Xpert 侧资源时访问的 Xpert API。 +XPERTAI_API_URL=http://localhost:/api + +# 可选。如果本地也运行 Codexpert Web UI,也把这个字段配成同一个 Xpert API。 +VITE_XPERTAI_API_URL=http://localhost:/api + +# Codexpert 调用 Xpert API 时使用的 workspace API key。 +XPERTAI_WORKSPACE_API_KEY= + +# Codexpert MCP endpoint 接受的 token。 +MCP_SERVER_TOKEN= + +# 可选的额外 MCP tokens。如果 connector 使用的 token 和 MCP_SERVER_TOKEN 不同, +# 可以把 connector token 放到这里。 +MCP_SERVER_TOKENS= + +# /api/codexpert-connector/* 接受的 token。 +# 如果不配置这个字段,connector endpoint 也会接受 MCP_SERVER_TOKEN、 +# CODEXPERT_ACP_SERVICE_TOKEN、ACP_SERVICE_TOKEN 和 MCP_SERVER_TOKENS。 +CODEXPERT_CONNECTOR_SERVICE_TOKEN= + +# 可选。本地 Xpert 和 Codexpert 开发数据里的同一个测试用户 id 不一致时, +# 可以用这个映射到 Codexpert 侧真正有效的用户 id。 +CODEXPERT_DEV_PRINCIPAL_USER_MAP={"":""} +``` + +middleware 的 `serviceToken` 必须能同时通过 Codexpert 的两类请求路径: + +- `POST /v1/mcp` +- `POST /api/codexpert-connector/sessions` +- `POST /api/codexpert-connector/sessions/:sessionId/prompts/stream` + +最简单的本地配置是让 `MCP_SERVER_TOKEN`、`CODEXPERT_CONNECTOR_SERVICE_TOKEN` 和 middleware 里的 `serviceToken` 使用同一个值。 + +### Xpert 宿主环境变量 + +Xpert API 必须能安装并加载本地插件工作区。在 Xpert API 环境里加入: + +```env +PLUGIN_WORKSPACE_ROOTS=/xpertai/middlewares/codexpert-connector +``` + +`PLUGIN_WORKSPACE_ROOTS` 可以用 `;` 或 `,` 分隔多个 root。本地安装时传入的 `workspacePath` 必须是 Xpert API 宿主机可见的绝对路径,并且必须位于这些允许的 root 之内。 + +### 编译插件 + +在 `xpert-plugins` 仓库的 `xpertai` 工作区里执行: + +```bash +cd +pnpm -C xpertai install +pnpm -C xpertai exec tsc -p middlewares/codexpert-connector/tsconfig.lib.json +``` + +### 安装本地插件到 Xpert + +在 Xpert API 宿主里安装或刷新本地插件: + +```bash +cd + +pnpm plugin:reinstall:local \ + @xpert-ai/plugin-codexpert-connector \ + /xpertai/middlewares/codexpert-connector \ + --api-url http://localhost: \ + --org-id \ + --token \ + --build-cwd /xpertai \ + --build-command "./node_modules/.bin/tsc -p middlewares/codexpert-connector/tsconfig.lib.json" +``` + +也可以在 Xpert 的插件设置页面安装本地插件,填写: + +```text +Plugin package name: @xpert-ai/plugin-codexpert-connector +Workspace path: /xpertai/middlewares/codexpert-connector +``` + +### 配置 Agent Middleware + +给目标 Xpert Agent 添加 `CodexpertConnector` middleware,并填写 Codexpert URL 和 service token: + +```json +{ + "codexpertMcpUrl": "http://localhost:3001/v1/mcp", + "codexpertConnectorBaseUrl": "http://localhost:3001/api", + "serviceToken": "", + "timeoutMs": 600000, + "enableVisibleProjection": true, + "enableStatusEvents": true, + "defaultXpertId": "", + "defaultRepoId": "", + "defaultConnectionId": "", + "defaultBranchName": "main" +} +``` + +如果不配置默认 Codexpert 上下文字段,Agent 必须先使用上下文工具选择或恢复 coding context,然后才能调用 `runCodexpertTask`。 + +### 冒烟检查 + +- Xpert API 正在运行,并且已经加载 `@xpert-ai/plugin-codexpert-connector`。 +- Codexpert API 正在运行,并接受 `Authorization: Bearer `。 +- middleware 配置指向 Codexpert API 端口,不是 Xpert API 端口。 +- 当前 Xpert 请求能解析出 `tenantId`、`organizationId` 和 `userId`。 +- `XPERTAI_WORKSPACE_API_KEY` 能让 Codexpert 调用配置的 Xpert API。 + +## 边界 + +- 插件不是 Codexpert 运行时的替代品。 +- 插件不在 Xpert 侧实现编码环境创建、仓库 clone 或任务执行。 +- 插件不保存 Codexpert raw event。 +- 插件依赖 Xpert 当前上下文里已经有真实业务用户身份。 +- `serviceToken` 是服务间认证令牌,不应由 Agent 或终端用户直接输入。 diff --git a/xpertai/middlewares/codexpert-connector/eslint.config.mjs b/xpertai/middlewares/codexpert-connector/eslint.config.mjs new file mode 100644 index 00000000..9859f079 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/eslint.config.mjs @@ -0,0 +1 @@ +export default [] diff --git a/xpertai/middlewares/codexpert-connector/jest.config.ts b/xpertai/middlewares/codexpert-connector/jest.config.ts new file mode 100644 index 00000000..c8ce75f4 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/jest.config.ts @@ -0,0 +1,18 @@ +/* eslint-disable */ +import { readFileSync } from 'node:fs' +import { dirname, join } from 'node:path' +import { fileURLToPath } from 'node:url' + +const moduleDir = dirname(fileURLToPath(import.meta.url)) +const swcJestConfig = JSON.parse(readFileSync(join(moduleDir, '.spec.swcrc'), 'utf-8')) +swcJestConfig.swcrc = false + +export default { + displayName: 'codexpert-connector', + preset: '../../jest.preset.js', + testEnvironment: 'node', + transform: { + '^.+\\.[tj]s$': ['@swc/jest', swcJestConfig] + }, + moduleFileExtensions: ['ts', 'js', 'html'] +} diff --git a/xpertai/middlewares/codexpert-connector/package.json b/xpertai/middlewares/codexpert-connector/package.json new file mode 100644 index 00000000..37d57a90 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/package.json @@ -0,0 +1,48 @@ +{ + "name": "@xpert-ai/plugin-codexpert-connector", + "version": "0.0.1", + "author": { + "name": "XpertAI", + "url": "https://xpertai.cn" + }, + "license": "AGPL-3.0", + "repository": { + "type": "git", + "url": "https://github.com/xpert-ai/xpert-plugins.git" + }, + "bugs": { + "url": "https://github.com/xpert-ai/xpert-plugins/issues" + }, + "type": "module", + "main": "./dist/index.js", + "module": "./dist/index.js", + "types": "./dist/index.d.ts", + "exports": { + "./package.json": "./package.json", + ".": { + "@xpert-plugins-starter/source": "./src/index.ts", + "types": "./dist/index.d.ts", + "import": "./dist/index.js", + "default": "./dist/index.js" + } + }, + "files": [ + "dist", + "!**/*.tsbuildinfo" + ], + "dependencies": { + "tslib": "^2.3.0" + }, + "peerDependencies": { + "@langchain/core": "0.3.72", + "@metad/contracts": "^3.8.0", + "@modelcontextprotocol/sdk": "^1.17.5", + "@nestjs/common": "^11.1.6", + "@nestjs/typeorm": "^11.0.0", + "@xpert-ai/chatkit-types": "^0.0.12", + "@xpert-ai/plugin-sdk": "^3.8.0", + "chalk": "4.1.2", + "typeorm": "^0.3.20", + "zod": "3.25.67" + } +} diff --git a/xpertai/middlewares/codexpert-connector/src/index.ts b/xpertai/middlewares/codexpert-connector/src/index.ts new file mode 100644 index 00000000..efccf6d3 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/src/index.ts @@ -0,0 +1,44 @@ +import { readFileSync } from 'fs' +import { dirname, join } from 'path' +import { fileURLToPath } from 'url' +import { z } from 'zod' +import type { XpertPlugin } from '@xpert-ai/plugin-sdk' +import { CodexpertConnectorPlugin } from './lib/codexpert-connector.module.js' +import { CodexpertConnectorConfigSchema } from './lib/types.js' + +const __filename = fileURLToPath(import.meta.url) +const __dirname = dirname(__filename) + +const packageJson = JSON.parse(readFileSync(join(__dirname, '../package.json'), 'utf8')) as { + name: string + version: string +} + +const plugin: XpertPlugin> = { + meta: { + name: packageJson.name, + version: packageJson.version, + category: 'middleware', + displayName: 'Codexpert Connector', + description: 'A middleware connector that exposes Codexpert context tools and runs Codexpert coding tasks with user-visible output projection.', + keywords: ['codexpert', 'coding', 'connector', 'middleware'], + author: 'XpertAI Team', + }, + config: { + schema: CodexpertConnectorConfigSchema, + }, + register(ctx) { + ctx.logger.log('register codexpert connector middleware plugin') + return { module: CodexpertConnectorPlugin, global: true } + }, + async onStart(ctx) { + ctx.logger.log('codexpert connector middleware plugin started') + }, + async onStop(ctx) { + ctx.logger.log('codexpert connector middleware plugin stopped') + }, +} + +export default plugin +export * from './lib/types.js' +export * from './lib/codexpert-connector.middleware.js' diff --git a/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector-run.service.ts b/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector-run.service.ts new file mode 100644 index 00000000..6f2de43f --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector-run.service.ts @@ -0,0 +1,146 @@ +import { Injectable, Logger, Optional } from '@nestjs/common' +import { InjectDataSource, InjectRepository } from '@nestjs/typeorm' +import { DataSource, Repository, Table, TableColumn, TableIndex, type TableColumnOptions } from 'typeorm' +import { PluginCodexpertConnectorRunEntity } from './entities/codexpert-connector-run.entity.js' + +type UpsertRunInput = { + tenantId: string + organizationId: string + userId: string + xpertId?: string | null + conversationId?: string | null + executionId?: string | null + codingSessionId?: string | null + taskId?: string | null + threadId?: string | null + codexpertExecutionId?: string | null + status: string + lastError?: string | null + metadata?: Record | null +} + +@Injectable() +export class CodexpertConnectorRunService { + private readonly logger = new Logger(CodexpertConnectorRunService.name) + private schemaEnsured = false + + constructor( + @Optional() + @InjectDataSource() + private readonly dataSource?: DataSource, + @Optional() + @InjectRepository(PluginCodexpertConnectorRunEntity) + private readonly repository?: Repository, + ) {} + + async ensureSchema(): Promise { + if (this.schemaEnsured || !this.dataSource?.isInitialized) { + return + } + + const queryRunner = this.dataSource.createQueryRunner() + try { + await queryRunner.connect() + const table = await queryRunner.getTable(PluginCodexpertConnectorRunEntity.tableName) + if (!table) { + await queryRunner.createTable( + new Table({ + name: PluginCodexpertConnectorRunEntity.tableName, + columns: [ + { name: 'id', type: 'uuid', isPrimary: true, generationStrategy: 'uuid', default: 'gen_random_uuid()' }, + { name: 'tenant_id', type: 'varchar', length: '64' }, + { name: 'organization_id', type: 'varchar', length: '64' }, + { name: 'user_id', type: 'varchar', length: '64' }, + { name: 'xpert_id', type: 'varchar', length: '64', isNullable: true }, + { name: 'conversation_id', type: 'varchar', length: '128', isNullable: true }, + { name: 'execution_id', type: 'varchar', length: '128', isNullable: true }, + { name: 'coding_session_id', type: 'varchar', length: '128', isNullable: true }, + { name: 'task_id', type: 'varchar', length: '128', isNullable: true }, + { name: 'thread_id', type: 'varchar', length: '128', isNullable: true }, + { name: 'codexpert_execution_id', type: 'varchar', length: '128', isNullable: true }, + { name: 'status', type: 'varchar', length: '32' }, + { name: 'last_error', type: 'text', isNullable: true }, + { name: 'metadata', type: 'jsonb', isNullable: true }, + { name: 'created_at', type: 'timestamp with time zone', default: 'now()' }, + { name: 'updated_at', type: 'timestamp with time zone', default: 'now()' }, + ], + }), + true, + ) + } else { + await this.ensureColumn(table, 'codexpert_execution_id', { type: 'varchar', length: '128', isNullable: true }) + await this.ensureColumn(table, 'last_error', { type: 'text', isNullable: true }) + await this.ensureColumn(table, 'metadata', { type: 'jsonb', isNullable: true }) + } + + await this.ensureIndex('IDX_plugin_codexpert_run_scope', ['tenant_id', 'organization_id', 'user_id']) + await this.ensureIndex('IDX_plugin_codexpert_run_execution', ['execution_id']) + await this.ensureIndex('IDX_plugin_codexpert_run_session', ['coding_session_id']) + this.schemaEnsured = true + } catch (error) { + this.logger.warn(`Failed to ensure Codexpert connector run schema: ${describeError(error)}`) + } finally { + await queryRunner.release() + } + } + + async record(input: UpsertRunInput): Promise { + if (!this.repository) { + return + } + try { + await this.ensureSchema() + const existing = input.executionId + ? await this.repository.findOne({ where: { executionId: input.executionId } }) + : null + const entity = this.repository.create({ + ...(existing ?? {}), + ...input, + updatedAt: new Date(), + }) + await this.repository.save(entity) + } catch (error) { + this.logger.warn(`Failed to record Codexpert connector run: ${describeError(error)}`) + } + } + + private async ensureColumn(table: Table, name: string, definition: Omit) { + if (table.findColumnByName(name) || !this.dataSource?.isInitialized) { + return + } + const queryRunner = this.dataSource.createQueryRunner() + try { + await queryRunner.connect() + await queryRunner.addColumn( + PluginCodexpertConnectorRunEntity.tableName, + new TableColumn({ name, ...definition }), + ) + } finally { + await queryRunner.release() + } + } + + private async ensureIndex(name: string, columnNames: string[]) { + if (!this.dataSource?.isInitialized) { + return + } + const queryRunner = this.dataSource.createQueryRunner() + try { + await queryRunner.connect() + const table = await queryRunner.getTable(PluginCodexpertConnectorRunEntity.tableName) + if (!table || table.indices.some((index) => index.name === name)) { + return + } + await queryRunner.createIndex( + PluginCodexpertConnectorRunEntity.tableName, + new TableIndex({ name, columnNames }), + ) + } finally { + await queryRunner.release() + } + } +} + +function describeError(error: unknown): string { + return error instanceof Error ? error.message : String(error) +} diff --git a/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector.client.ts b/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector.client.ts new file mode 100644 index 00000000..9e929313 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector.client.ts @@ -0,0 +1,191 @@ +import { Client } from '@modelcontextprotocol/sdk/client/index.js' +import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js' +import { + type CodexpertConnectorEvent, + type CodexpertConnectorConfig, + type PrincipalContext, +} from './types.js' + +type ConnectorHeadersInput = { + serviceToken: string + principal: PrincipalContext +} + +export async function callCodexpertMcpTool( + config: Required>, + principal: PrincipalContext, + name: string, + args: Record, + timeoutMs: number, +): Promise { + const client = new Client({ name: 'xpert-codexpert-connector', version: '0.1.0' }) + const headers = buildConnectorHeaders({ serviceToken: config.serviceToken, principal }) + const transport = new StreamableHTTPClientTransport(new URL(config.codexpertMcpUrl), { + requestInit: { + headers, + }, + fetch: createConnectorFetch(headers), + }) + try { + await client.connect(transport) + const result = await client.callTool( + { + name, + arguments: args, + }, + undefined, + { timeout: timeoutMs }, + ) + if (result.isError) { + throw new Error(readMcpText(result.content) || `Codexpert MCP tool ${name} failed`) + } + return (result.structuredContent ?? readMcpPayload(result.content)) as T + } finally { + await client.close().catch(() => undefined) + } +} + +export async function createCodexpertSession( + config: Required>, + principal: PrincipalContext, + body: Record, + timeoutMs: number, +): Promise> { + const response = await fetchWithTimeout( + `${trimRight(config.codexpertConnectorBaseUrl, '/')}/codexpert-connector/sessions`, + { + method: 'POST', + headers: { + ...buildConnectorHeaders({ serviceToken: config.serviceToken, principal }), + 'content-type': 'application/json', + }, + body: JSON.stringify(body), + }, + timeoutMs, + ) + if (!response.ok) { + throw new Error(`Codexpert connector session failed (${response.status}): ${await response.text()}`) + } + return response.json() as Promise> +} + +export async function* streamCodexpertPrompt( + config: Required>, + principal: PrincipalContext, + sessionId: string, + body: Record, + timeoutMs: number, +): AsyncIterable { + const response = await fetchWithTimeout( + `${trimRight(config.codexpertConnectorBaseUrl, '/')}/codexpert-connector/sessions/${encodeURIComponent(sessionId)}/prompts/stream`, + { + method: 'POST', + headers: { + ...buildConnectorHeaders({ serviceToken: config.serviceToken, principal }), + 'content-type': 'application/json', + }, + body: JSON.stringify(body), + }, + timeoutMs, + ) + if (!response.ok) { + throw new Error(`Codexpert connector stream failed (${response.status}): ${await response.text()}`) + } + if (!response.body) { + throw new Error('Codexpert connector stream returned an empty body') + } + + const reader = response.body.getReader() + const decoder = new TextDecoder() + let buffer = '' + try { + while (true) { + const { done, value } = await reader.read() + if (done) { + break + } + buffer += decoder.decode(value, { stream: true }) + let newlineIndex = buffer.indexOf('\n') + while (newlineIndex >= 0) { + const line = buffer.slice(0, newlineIndex).trim() + buffer = buffer.slice(newlineIndex + 1) + if (line) { + yield JSON.parse(line) as CodexpertConnectorEvent + } + newlineIndex = buffer.indexOf('\n') + } + } + const tail = buffer.trim() + if (tail) { + yield JSON.parse(tail) as CodexpertConnectorEvent + } + } finally { + reader.releaseLock() + } +} + +function createConnectorFetch(connectorHeaders: Record): typeof fetch { + return (input, init) => { + const headers = new Headers(init?.headers) + for (const [key, value] of Object.entries(connectorHeaders)) { + headers.set(key, value) + } + return fetch(input, { + ...init, + headers, + }) + } +} + +function buildConnectorHeaders(input: ConnectorHeadersInput): Record { + return { + authorization: `Bearer ${input.serviceToken}`, + 'tenant-id': input.principal.tenantId, + 'organization-id': input.principal.organizationId, + 'x-principal-user-id': input.principal.userId, + } +} + +async function fetchWithTimeout(url: string, init: RequestInit, timeoutMs: number): Promise { + const controller = new AbortController() + const timer = setTimeout(() => controller.abort(), timeoutMs) + try { + return await fetch(url, { + ...init, + signal: controller.signal, + }) + } finally { + clearTimeout(timer) + } +} + +function readMcpPayload(content: unknown): T { + const text = readMcpText(content) + if (!text) { + return content as T + } + try { + return JSON.parse(text) as T + } catch { + return text as T + } +} + +function readMcpText(content: unknown): string | null { + if (!Array.isArray(content)) { + return null + } + return content + .map((item) => (item && typeof item === 'object' && 'text' in item ? String((item as { text?: unknown }).text ?? '') : '')) + .filter(Boolean) + .join('\n') + .trim() || null +} + +function trimRight(value: string, char: string): string { + let current = value + while (current.endsWith(char)) { + current = current.slice(0, -char.length) + } + return current +} diff --git a/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector.middleware.ts b/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector.middleware.ts new file mode 100644 index 00000000..e25c8f69 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector.middleware.ts @@ -0,0 +1,506 @@ +import { tool } from '@langchain/core/tools' +import type { RunnableConfig } from '@langchain/core/runnables' +import { getToolCallFromConfig, type TAgentMiddlewareMeta } from '@metad/contracts' +import { Injectable, Logger } from '@nestjs/common' +import { + AgentMiddleware, + AgentMiddlewareStrategy, + IAgentMiddlewareContext, + IAgentMiddlewareStrategy, + PromiseOrValue, + RequestContext, +} from '@xpert-ai/plugin-sdk' +import { ChatMessageTypeEnum, type TMessageContentText } from '@xpert-ai/chatkit-types' +import { z } from 'zod/v3' +import { + callCodexpertMcpTool, + createCodexpertSession, + streamCodexpertPrompt, +} from './codexpert-connector.client.js' +import { CodexpertConnectorRunService } from './codexpert-connector-run.service.js' +import { + createProjectionState, + flushVisibleCodexpertProjection, + projectVisibleCodexpertEvent, +} from './codexpert-visible-projector.js' +import { + CODEXPERT_AGENT_KEY, + CODEXPERT_CONNECTOR_MIDDLEWARE_NAME, + CODEXPERT_XPERT_NAME, + CodexpertConnectorConfigSchema, + RunCodexpertTaskInputSchema, + type CodexpertConnectorConfig, + type CodexpertConnectorEvent, + type CodexpertTaskResult, + type PrincipalContext, + type RunCodexpertTaskInput, +} from './types.js' + +const MCP_TOOL_NAMES = [ + 'listCodingAssistants', + 'listCodexpertConversations', + 'listGitConnections', + 'listGitRepositories', + 'listGitBranches', + 'selectCodingContext', + 'resolveCodexpertConversationContext', + 'resumeCodexpertSession', +] as const + +const passthroughToolSchema = z.record(z.string(), z.unknown()).optional().default({}) + +const CONNECTOR_DESCRIPTION = + 'Codexpert Connector lets the agent delegate real coding work to Codexpert while keeping the current user informed in chat. Use it when the user clearly asks to inspect, modify, implement, debug, or continue work in a code repository. First use the context tools to select or resume the coding assistant, Git connection, repository, branch, and coding session. Then call runCodexpertTask for actual coding execution. Codexpert owns repository preparation, coding environment reuse, task execution, and final coding output. Do not use this connector for general discussion, planning, configuration explanation, or status-only questions.' + +const RUN_CODEXPERT_TASK_DESCRIPTION = [ + 'Run a Codexpert coding task in the selected or resumed coding session.', + 'Use this only when the user wants Codexpert to actually perform or continue coding work, such as inspecting, modifying, implementing, debugging, or continuing a repository task.', + 'Before calling this tool, make sure the coding context is known. If codingSessionId or resumed task context is missing, first use listCodingAssistants, listGitConnections, listGitRepositories, listGitBranches, selectCodingContext, listCodexpertConversations, or resumeCodexpertSession as needed.', + 'During execution, Codexpert visible progress and output are streamed directly to the user. The agent should not restate or rewrite that live output unless the user asks.', + 'After the tool returns, use the metadata it returns, including status, codingSessionId, taskId, threadId, executionId, environmentId, summary, error, and prUrl, to decide whether to continue, recover, report a Codexpert-produced PR, or report failure.', + 'Do not call this tool for general discussion, planning, explanation, configuration questions, or status-only lookup.', +].join('\n') + +function buildMcpToolDescription(toolName: string): string { + return [ + `Call Codexpert MCP context tool ${toolName}.`, + 'Use Codexpert context tools before runCodexpertTask to identify or resume the coding assistant, Git connection, repository, branch, conversation, and coding session.', + 'The connector injects the service token and current business-user identity headers automatically.', + 'Do not use context tools as a substitute for runCodexpertTask when the user has already asked Codexpert to execute coding work.', + ].join('\n') +} + +@Injectable() +@AgentMiddlewareStrategy(CODEXPERT_CONNECTOR_MIDDLEWARE_NAME) +export class CodexpertConnectorMiddleware implements IAgentMiddlewareStrategy> { + private readonly logger = new Logger(CodexpertConnectorMiddleware.name) + + readonly meta: TAgentMiddlewareMeta = { + name: CODEXPERT_CONNECTOR_MIDDLEWARE_NAME, + label: { + en_US: 'Codexpert Connector', + zh_Hans: 'Codexpert 连接器', + }, + description: { + en_US: CONNECTOR_DESCRIPTION, + zh_Hans: + 'Codexpert 连接器让 Agent 把真实编码工作委托给 Codexpert,同时把 Codexpert 的可见进度和结果直接投影给当前用户。用户明确要求检查、修改、实现、调试或继续仓库代码任务时使用。先用上下文工具选择或恢复编码助手、Git 连接、仓库、分支和编码会话,再用 runCodexpertTask 执行真实编码任务。Codexpert 负责仓库准备、编码环境复用、任务执行和最终编码输出。不要把它用于普通讨论、规划、配置解释或只查询状态的问题。', + }, + configSchema: { + type: 'object', + properties: { + codexpertMcpUrl: { type: 'string', title: { en_US: 'Codexpert MCP URL', zh_Hans: 'Codexpert MCP 地址' } }, + codexpertConnectorBaseUrl: { type: 'string', title: { en_US: 'Connector Base URL', zh_Hans: 'Connector 基础地址' } }, + serviceToken: { type: 'string', title: { en_US: 'Service Token', zh_Hans: '服务令牌' } }, + timeoutMs: { type: 'number', default: 600000, title: { en_US: 'Timeout (ms)', zh_Hans: '超时时间 (ms)' } }, + enableVisibleProjection: { type: 'boolean', default: true, title: { en_US: 'Visible Projection', zh_Hans: '用户可见投影' } }, + enableStatusEvents: { type: 'boolean', default: true, title: { en_US: 'Status Events', zh_Hans: '状态事件' } }, + defaultXpertId: { type: 'string', title: { en_US: 'Default Coding Assistant', zh_Hans: '默认编码助手' } }, + defaultRepoId: { type: 'string', title: { en_US: 'Default Repository', zh_Hans: '默认仓库' } }, + defaultConnectionId: { type: 'string', title: { en_US: 'Default Git Connection', zh_Hans: '默认 Git 连接' } }, + defaultBranchName: { type: 'string', title: { en_US: 'Default Branch', zh_Hans: '默认分支' } }, + }, + }, + } + + constructor(private readonly runService: CodexpertConnectorRunService) {} + + createMiddleware( + options: Partial, + context: IAgentMiddlewareContext, + ): PromiseOrValue { + const parsed = CodexpertConnectorConfigSchema.safeParse(options ?? {}) + if (!parsed.success) { + throw new Error(`Invalid Codexpert connector config: ${parsed.error.message}`) + } + const config = parsed.data + + if (!config.codexpertMcpUrl || !config.codexpertConnectorBaseUrl || !config.serviceToken) { + return { + name: CODEXPERT_CONNECTOR_MIDDLEWARE_NAME, + } + } + + const mcpTools = MCP_TOOL_NAMES.map((toolName) => + tool( + async (args, runnableConfig) => { + const principal = resolvePrincipal(context, runnableConfig) + return callCodexpertMcpTool( + { + codexpertMcpUrl: config.codexpertMcpUrl!, + serviceToken: config.serviceToken!, + }, + principal, + toolName, + normalizeRecord(args), + config.timeoutMs ?? 600_000, + ) + }, + { + name: toolName, + description: buildMcpToolDescription(toolName), + schema: passthroughToolSchema, + }, + ), + ) + + const runTool = tool( + async (input, runnableConfig) => { + return this.runCodexpertTask(config, context, input, runnableConfig) + }, + { + name: 'runCodexpertTask', + description: RUN_CODEXPERT_TASK_DESCRIPTION, + schema: RunCodexpertTaskInputSchema, + }, + ) + + return { + name: CODEXPERT_CONNECTOR_MIDDLEWARE_NAME, + tools: [...mcpTools, runTool], + } + } + + private async runCodexpertTask( + config: CodexpertConnectorConfig, + context: IAgentMiddlewareContext, + input: RunCodexpertTaskInput, + runnableConfig?: RunnableConfig, + ): Promise { + const timeoutMs = input.timeoutMs ?? config.timeoutMs ?? 600_000 + const executionId = pickString((runnableConfig?.configurable as Record | undefined)?.executionId) + const toolCall = getToolCallFromConfig(runnableConfig) + let sessionId = pickString(input.codingSessionId) + let sessionSnapshot: Record | null = null + let finalEvent: CodexpertConnectorEvent | null = null + let result: CodexpertTaskResult = buildInitialResult(input) + let principal: PrincipalContext | null = null + const projectionState = createProjectionState() + + try { + principal = resolvePrincipal(context, runnableConfig) + if (!sessionId) { + sessionSnapshot = await this.resolveOrCreateSession(config, principal, input, timeoutMs) + sessionId = pickString(sessionSnapshot.codingSessionId, sessionSnapshot.id) + } + if (!sessionId) { + throw new Error('Missing Codexpert codingSessionId. Use selectCodingContext or resumeCodexpertSession before runCodexpertTask.') + } + + await this.runService.record({ + tenantId: principal.tenantId, + organizationId: principal.organizationId, + userId: principal.userId, + xpertId: pickString(input.xpertId, sessionSnapshot?.xpertId), + conversationId: pickString(input.conversationId, sessionSnapshot?.sourceConversationId), + executionId, + codingSessionId: sessionId, + status: 'running', + metadata: { toolCallId: toolCall?.id ?? null }, + }) + + for await (const event of streamCodexpertPrompt( + { + codexpertConnectorBaseUrl: config.codexpertConnectorBaseUrl!, + serviceToken: config.serviceToken!, + }, + principal, + sessionId, + { + prompt: input.prompt, + title: input.taskTitle, + requestId: toolCall?.id ?? executionId ?? undefined, + }, + timeoutMs, + )) { + finalEvent = event.type === 'done' || event.type === 'error' ? event : finalEvent + if (config.enableVisibleProjection ?? true) { + await projectVisibleCodexpertEvent( + event, + projectionState, + runnableConfig, + config.enableStatusEvents ?? true, + ) + } + } + flushVisibleCodexpertProjection(projectionState, runnableConfig) + + result = buildResult(input, sessionId, finalEvent, null, sessionSnapshot) + await this.runService.record({ + tenantId: principal.tenantId, + organizationId: principal.organizationId, + userId: principal.userId, + xpertId: pickString(input.xpertId, sessionSnapshot?.xpertId), + conversationId: pickString(input.conversationId, sessionSnapshot?.sourceConversationId), + executionId, + codingSessionId: result.codingSessionId, + taskId: result.taskId, + threadId: result.threadId, + codexpertExecutionId: result.executionId, + status: result.status, + lastError: result.error, + metadata: { toolCallId: toolCall?.id ?? null }, + }) + return result + } catch (error) { + const message = error instanceof Error ? error.message : String(error) + flushVisibleCodexpertProjection(projectionState, runnableConfig) + projectImmediateError(message, runnableConfig) + result = buildResult(input, sessionId ?? null, finalEvent, message, sessionSnapshot) + if (!principal) { + return result + } + await this.runService.record({ + tenantId: principal.tenantId, + organizationId: principal.organizationId, + userId: principal.userId, + xpertId: pickString(input.xpertId, sessionSnapshot?.xpertId), + conversationId: pickString(input.conversationId, sessionSnapshot?.sourceConversationId), + executionId, + codingSessionId: result.codingSessionId, + taskId: result.taskId, + threadId: result.threadId, + codexpertExecutionId: result.executionId, + status: result.status, + lastError: message, + metadata: { toolCallId: toolCall?.id ?? null }, + }) + return result + } + } + + private async resolveOrCreateSession( + config: CodexpertConnectorConfig, + principal: PrincipalContext, + input: RunCodexpertTaskInput, + timeoutMs: number, + ): Promise> { + if (input.taskId || input.conversationId || input.threadId) { + const resolved = await callCodexpertMcpTool>( + { + codexpertMcpUrl: config.codexpertMcpUrl!, + serviceToken: config.serviceToken!, + }, + principal, + 'resumeCodexpertSession', + { + taskId: input.taskId, + conversationId: input.conversationId, + threadId: input.threadId, + }, + timeoutMs, + ) + return resolved + } + + const xpertId = pickString(input.xpertId, config.defaultXpertId) + const connectionId = pickString(input.connectionId, config.defaultConnectionId) + const repoId = pickString(input.repoId, config.defaultRepoId) + const branchName = pickString(input.branchName, config.defaultBranchName) + if (!xpertId || !connectionId || !repoId || !branchName) { + throw new Error('Missing Codexpert context. Select assistant, git connection, repository, and branch before runCodexpertTask.') + } + + const selected = await callCodexpertMcpTool>( + { + codexpertMcpUrl: config.codexpertMcpUrl!, + serviceToken: config.serviceToken!, + }, + principal, + 'selectCodingContext', + { + xpertId, + connectionId, + repoId, + branchName, + }, + timeoutMs, + ) + + return createCodexpertSession( + { + codexpertConnectorBaseUrl: config.codexpertConnectorBaseUrl!, + serviceToken: config.serviceToken!, + }, + principal, + { + ...selected, + xpertId, + repoId, + branchName, + metadata: { + repoConnectionId: connectionId, + }, + }, + timeoutMs, + ) + } +} + +function resolvePrincipal(context: IAgentMiddlewareContext, config?: RunnableConfig): PrincipalContext { + const configurable = config?.configurable as Record | undefined + const currentUser = RequestContext.currentUser?.() as Record | null | undefined + const contextRecord = context as unknown as Record + const tenantId = pickString( + RequestContext.currentTenantId?.(), + currentUser?.tenantId, + contextRecord.tenantId, + configurable?.tenantId, + ) + const organizationId = pickString( + RequestContext.getOrganizationId?.(), + currentUser?.organizationId, + contextRecord.organizationId, + configurable?.organizationId, + ) + const userId = pickString( + RequestContext.currentUserId?.(), + currentUser?.id, + contextRecord.userId, + configurable?.userId, + ) + if (!tenantId || !organizationId || !userId) { + throw new Error('Codexpert connector requires tenantId, organizationId, and userId. Bind the external user to a real business user before running Codexpert.') + } + return { tenantId, organizationId, userId } +} + +function buildInitialResult(input: RunCodexpertTaskInput): CodexpertTaskResult { + return { + status: 'failed', + codingSessionId: pickString(input.codingSessionId), + taskId: pickString(input.taskId), + threadId: pickString(input.threadId), + executionId: null, + repo: input.repoId ? { id: input.repoId } : null, + branch: pickString(input.branchName), + environmentId: null, + environmentReused: null, + summary: null, + error: null, + } +} + +function buildResult( + input: RunCodexpertTaskInput, + sessionId: string | null, + event: CodexpertConnectorEvent | null, + error: string | null, + sessionSnapshot: Record | null, +): CodexpertTaskResult { + const terminal = resolveTerminalResult(event, error) + const repo = buildRepoResult(input, sessionSnapshot) + return { + status: terminal.status, + codingSessionId: pickString( + event && 'codingSessionId' in event ? event.codingSessionId : null, + sessionId, + sessionSnapshot?.codingSessionId, + sessionSnapshot?.id, + ), + taskId: pickString(event && 'taskId' in event ? event.taskId : null, input.taskId), + threadId: pickString(event && 'threadId' in event ? event.threadId : null, input.threadId), + executionId: pickString(event && 'executionId' in event ? event.executionId : null), + repo, + branch: pickString(input.branchName, sessionSnapshot?.branchName), + environmentId: pickString(event && 'environmentId' in event ? event.environmentId : null, sessionSnapshot?.environmentId), + environmentReused: null, + summary: event?.type === 'done' ? pickString(event.summary, event.output) : null, + error: terminal.error, + ...(event?.type === 'done' && event.prUrl ? { prUrl: event.prUrl } : {}), + } +} + +function resolveTerminalResult( + event: CodexpertConnectorEvent | null, + error: string | null, +): Pick { + if (error) { + return { status: isTimeoutError(error) ? 'timeout' : 'failed', error } + } + if (!event) { + return { status: 'failed', error: 'Codexpert stream ended without a terminal event.' } + } + if (event.type === 'error') { + return { status: 'failed', error: event.message } + } + if (event.type !== 'done') { + return { status: 'failed', error: 'Codexpert stream ended without a terminal done event.' } + } + + const status = normalizeTerminalStatus(event.status) + if (status === 'success') { + return { status, error: null } + } + return { + status, + error: event.status ? `Codexpert finished with status: ${event.status}` : 'Codexpert finished unsuccessfully.', + } +} + +function normalizeTerminalStatus(status: unknown): CodexpertTaskResult['status'] { + const value = typeof status === 'string' ? status.trim().toLowerCase() : '' + if (!value || ['success', 'succeeded', 'completed', 'complete', 'done', 'ok'].includes(value)) { + return 'success' + } + if (['timeout', 'timed_out', 'timed-out'].includes(value)) { + return 'timeout' + } + if (['canceled', 'cancelled', 'interrupted', 'aborted'].includes(value)) { + return 'canceled' + } + return 'failed' +} + +function isTimeoutError(message: string): boolean { + const lower = message.toLowerCase() + return lower.includes('timeout') || lower.includes('timed out') || lower.includes('abort') +} + +function buildRepoResult( + input: RunCodexpertTaskInput, + sessionSnapshot: Record | null, +): CodexpertTaskResult['repo'] { + const repo = { + id: pickString(input.repoId, sessionSnapshot?.repoId), + name: pickString(sessionSnapshot?.repoName), + owner: pickString(sessionSnapshot?.repoOwner), + slug: pickString(sessionSnapshot?.repoSlug), + } + return repo.id || repo.name || repo.owner || repo.slug ? repo : null +} + +function normalizeRecord(value: unknown): Record { + return value && typeof value === 'object' && !Array.isArray(value) ? value as Record : {} +} + +function pickString(...values: unknown[]): string | null { + for (const value of values) { + if (typeof value === 'string' && value.trim()) { + return value.trim() + } + } + return null +} + +function projectImmediateError(message: string, config?: RunnableConfig) { + const subscriber = (config?.configurable as Record | undefined)?.subscriber + if (!subscriber || typeof subscriber.next !== 'function') { + return + } + try { + subscriber.next({ + data: { + type: ChatMessageTypeEnum.MESSAGE, + data: { + type: 'text', + text: `Codexpert failed: ${message}`, + xpertName: CODEXPERT_XPERT_NAME, + agentKey: CODEXPERT_AGENT_KEY, + } satisfies TMessageContentText, + }, + }) + } catch { + // Ignore callback pipeline errors so the tool can still return the failure metadata. + } +} diff --git a/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector.module.ts b/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector.module.ts new file mode 100644 index 00000000..4a72ba3e --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/src/lib/codexpert-connector.module.ts @@ -0,0 +1,24 @@ +import { IOnPluginBootstrap, IOnPluginDestroy, XpertServerPlugin } from '@xpert-ai/plugin-sdk' +import { TypeOrmModule } from '@nestjs/typeorm' +import chalk from 'chalk' +import { CodexpertConnectorMiddleware } from './codexpert-connector.middleware.js' +import { CodexpertConnectorRunService } from './codexpert-connector-run.service.js' +import { PluginCodexpertConnectorRunEntity } from './entities/codexpert-connector-run.entity.js' + +@XpertServerPlugin({ + imports: [TypeOrmModule.forFeature([PluginCodexpertConnectorRunEntity])], + providers: [CodexpertConnectorMiddleware, CodexpertConnectorRunService], + entities: [PluginCodexpertConnectorRunEntity], +}) +export class CodexpertConnectorPlugin implements IOnPluginBootstrap, IOnPluginDestroy { + constructor(private readonly runService: CodexpertConnectorRunService) {} + + async onPluginBootstrap(): Promise { + console.log(chalk.green(`${CodexpertConnectorPlugin.name} is being bootstrapped...`)) + await this.runService.ensureSchema() + } + + onPluginDestroy(): void { + console.log(chalk.green(`${CodexpertConnectorPlugin.name} is being destroyed...`)) + } +} diff --git a/xpertai/middlewares/codexpert-connector/src/lib/codexpert-visible-projector.ts b/xpertai/middlewares/codexpert-connector/src/lib/codexpert-visible-projector.ts new file mode 100644 index 00000000..dd2e14b5 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/src/lib/codexpert-visible-projector.ts @@ -0,0 +1,299 @@ +import { ChatMessageTypeEnum, type TMessageContentText } from '@xpert-ai/chatkit-types' +import type { RunnableConfig } from '@langchain/core/runnables' +import { + CODEXPERT_AGENT_KEY, + CODEXPERT_XPERT_NAME, + type CodexpertConnectorEvent, +} from './types.js' +import { formatVisibleMarkdown } from './markdown-format.js' + +type ProjectionState = { + lastTextHash?: string + milestoneKeys: Set + textStreamId: string + hasOutputText: boolean + pendingText: string + pendingTextTimer?: ReturnType +} + +let projectionSequence = 0 +const TEXT_FLUSH_INTERVAL_MS = 600 +const TEXT_FLUSH_MAX_CHARS = 320 +const TEXT_FLUSH_NATURAL_MIN_CHARS = 80 + +export function createProjectionState(): ProjectionState { + return { + milestoneKeys: new Set(), + textStreamId: `codexpert-visible-text-${++projectionSequence}`, + hasOutputText: false, + pendingText: '', + } +} + +export async function projectVisibleCodexpertEvent( + event: CodexpertConnectorEvent, + state: ProjectionState, + config?: RunnableConfig, + enableStatusEvents = true, +) { + if (event.type !== 'text_delta') { + flushVisibleCodexpertProjection(state, config) + } + + const text = resolveVisibleText(event, state, enableStatusEvents) + if (!text) { + return + } + + if (event.type === 'text_delta') { + appendBufferedText(state, text) + if (shouldFlushBufferedText(state.pendingText)) { + flushVisibleCodexpertProjection(state, config) + } else { + scheduleBufferedTextFlush(state, config) + } + return + } + + emitSubscriberMessage(config, text) +} + +export function flushVisibleCodexpertProjection( + state: ProjectionState, + config?: RunnableConfig, +) { + if (state.pendingTextTimer) { + clearTimeout(state.pendingTextTimer) + state.pendingTextTimer = undefined + } + + const text = dedupeText(state.pendingText, state) + state.pendingText = '' + if (!text) { + return + } + + emitSubscriberMessage(config, text, state.textStreamId) +} + +function resolveVisibleText( + event: CodexpertConnectorEvent, + state: ProjectionState, + enableStatusEvents: boolean, +): string | null { + if (event.type === 'text_delta') { + if (event.stream === 'thought') { + return null + } + if (event.tag === 'assistant_snapshot' && state.hasOutputText) { + return null + } + const text = cleanDeltaText(event.text) + if (text) { + state.hasOutputText = true + } + return text + } + + if (event.type === 'error') { + return dedupeMilestone(`failed:${event.message}`, `Codexpert failed: ${cleanText(event.message)}`, state) + } + + if (event.type === 'done') { + if (state.hasOutputText) { + return null + } + return dedupeMilestone( + `done:${event.taskId ?? event.executionId ?? event.codingSessionId ?? 'unknown'}`, + cleanText(event.summary ?? 'Codexpert task completed.'), + state, + ) + } + + if (!enableStatusEvents || event.type !== 'status' || !event.isMilestone) { + return null + } + + const phase = cleanText(event.phase ?? '') + const headline = cleanText(event.headline ?? event.text ?? '') + if (!headline || isSetupNoise(headline)) { + return null + } + const milestoneText = formatStatusMilestone(phase, headline, state) + if (milestoneText) { + return milestoneText + } + + if (phase === 'setup') { + return null + } + if (phase === 'running') { + return dedupeMilestone(`running:${headline}`, headline, state) + } + if (phase === 'waiting_input') { + return dedupeMilestone(`waiting:${headline}`, headline, state) + } + if (phase === 'completed') { + return dedupeMilestone(`completed:${headline}`, headline, state) + } + return null +} + +function formatStatusMilestone(phase: string, headline: string, state: ProjectionState): string | null { + const lower = headline.toLowerCase() + if (phase === 'setup') { + if (lower.includes('ready')) { + return dedupeMilestone('setup:ready', '编码环境已就绪。', state) + } + if (lower.includes('queued') || lower.includes('setup')) { + return dedupeMilestone('setup:queued', '正在准备编码环境。', state) + } + } + if (phase === 'running') { + return dedupeMilestone('running:start', '已开始处理。', state) + } + if (phase === 'waiting_input') { + return dedupeMilestone('waiting:input', '需要补充信息。', state) + } + if (phase === 'completed' && !state.hasOutputText) { + return dedupeMilestone('completed:done', '已完成。', state) + } + return null +} + +function cleanText(value: unknown): string { + const text = typeof value === 'string' ? value : value == null ? '' : JSON.stringify(value) + return text + .replace(/\r\n/g, '\n') + .split('\n') + .map((line) => line.trimEnd()) + .join('\n') + .trim() +} + +function cleanDeltaText(value: unknown): string | null { + const text = typeof value === 'string' ? value : value == null ? '' : JSON.stringify(value) + const normalized = text.replace(/\r\n/g, '\n') + if (!normalized || isInternalText(normalized.trim())) { + return null + } + return normalized +} + +function dedupeText(text: string, state: ProjectionState): string | null { + if (!text || isInternalText(text)) { + return null + } + const hash = stableHash(text) + if (hash === state.lastTextHash) { + return null + } + state.lastTextHash = hash + return text +} + +function dedupeMilestone(key: string, text: string, state: ProjectionState): string | null { + if (!text || state.milestoneKeys.has(key) || isInternalText(text)) { + return null + } + state.milestoneKeys.add(key) + return text +} + +function isInternalText(text: string): boolean { + const lower = text.toLowerCase() + return ( + lower.startsWith('thought:') || + lower.includes('"metadata"') || + lower.includes('raw metadata') || + lower.includes('setup log') || + lower.includes('sandbox setup log') || + lower.includes('[debug]') + ) +} + +function isSetupNoise(text: string): boolean { + const lower = text.toLowerCase() + return ( + lower.startsWith('setup:') || + lower.includes('installing dependencies') || + lower.includes('clone progress') || + lower.includes('synchronizing repository') || + lower.includes('repository synchronized') || + lower.includes('acquiring runtime') || + lower.includes('runtime acquired') || + lower.includes('preparing workspace') || + lower.includes('workspace prepared') || + lower.includes('running setup lifecycle') + ) +} + +function appendBufferedText(state: ProjectionState, text: string) { + state.pendingText = `${state.pendingText}${text}` +} + +function shouldFlushBufferedText(text: string): boolean { + if (text.length >= TEXT_FLUSH_MAX_CHARS) { + return true + } + if (text.length < TEXT_FLUSH_NATURAL_MIN_CHARS) { + return false + } + return /[\n。!?!?;;::]$/.test(text.trimEnd()) +} + +function scheduleBufferedTextFlush(state: ProjectionState, config?: RunnableConfig) { + if (state.pendingTextTimer) { + return + } + + state.pendingTextTimer = setTimeout(() => { + state.pendingTextTimer = undefined + flushVisibleCodexpertProjection(state, config) + }, TEXT_FLUSH_INTERVAL_MS) +} + +function emitSubscriberMessage(config: RunnableConfig | undefined, text: string, streamId?: string) { + const subscriber = (config?.configurable as Record | undefined)?.subscriber + if (!subscriber || typeof subscriber.next !== 'function') { + return + } + + try { + subscriber.next({ + data: { + type: ChatMessageTypeEnum.MESSAGE, + data: buildVisibleTextChunk(text, streamId), + }, + }) + } catch { + try { + subscriber.next({ + data: { + type: ChatMessageTypeEnum.MESSAGE, + data: text, + }, + }) + } catch { + // Projection delivery is best-effort; keep the Codexpert task flow alive. + } + } +} + +function buildVisibleTextChunk(text: string, streamId?: string): TMessageContentText { + return { + ...(streamId ? { id: streamId } : {}), + type: 'text', + text: formatVisibleMarkdown(text, { standalone: !streamId }), + xpertName: CODEXPERT_XPERT_NAME, + agentKey: CODEXPERT_AGENT_KEY, + } +} + +function stableHash(text: string): string { + let hash = 0 + for (let index = 0; index < text.length; index += 1) { + hash = (hash * 31 + text.charCodeAt(index)) >>> 0 + } + return hash.toString(16) +} diff --git a/xpertai/middlewares/codexpert-connector/src/lib/entities/codexpert-connector-run.entity.ts b/xpertai/middlewares/codexpert-connector/src/lib/entities/codexpert-connector-run.entity.ts new file mode 100644 index 00000000..309266e7 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/src/lib/entities/codexpert-connector-run.entity.ts @@ -0,0 +1,64 @@ +import { + Column, + CreateDateColumn, + Entity, + Index, + PrimaryGeneratedColumn, + UpdateDateColumn +} from 'typeorm' + +@Entity(PluginCodexpertConnectorRunEntity.tableName) +@Index('IDX_plugin_codexpert_run_scope', ['tenantId', 'organizationId', 'userId']) +@Index('IDX_plugin_codexpert_run_execution', ['executionId']) +@Index('IDX_plugin_codexpert_run_session', ['codingSessionId']) +export class PluginCodexpertConnectorRunEntity { + static readonly tableName = 'plugin_codexpert_connector_run' + + @PrimaryGeneratedColumn('uuid') + id!: string + + @Column({ name: 'tenant_id', type: 'varchar', length: 64 }) + tenantId!: string + + @Column({ name: 'organization_id', type: 'varchar', length: 64 }) + organizationId!: string + + @Column({ name: 'user_id', type: 'varchar', length: 64 }) + userId!: string + + @Column({ name: 'xpert_id', type: 'varchar', length: 64, nullable: true }) + xpertId?: string | null + + @Column({ name: 'conversation_id', type: 'varchar', length: 128, nullable: true }) + conversationId?: string | null + + @Column({ name: 'execution_id', type: 'varchar', length: 128, nullable: true }) + executionId?: string | null + + @Column({ name: 'coding_session_id', type: 'varchar', length: 128, nullable: true }) + codingSessionId?: string | null + + @Column({ name: 'task_id', type: 'varchar', length: 128, nullable: true }) + taskId?: string | null + + @Column({ name: 'thread_id', type: 'varchar', length: 128, nullable: true }) + threadId?: string | null + + @Column({ name: 'codexpert_execution_id', type: 'varchar', length: 128, nullable: true }) + codexpertExecutionId?: string | null + + @Column({ name: 'status', type: 'varchar', length: 32 }) + status!: string + + @Column({ name: 'last_error', type: 'text', nullable: true }) + lastError?: string | null + + @Column({ name: 'metadata', type: 'jsonb', nullable: true }) + metadata?: Record | null + + @CreateDateColumn({ name: 'created_at' }) + createdAt!: Date + + @UpdateDateColumn({ name: 'updated_at' }) + updatedAt!: Date +} diff --git a/xpertai/middlewares/codexpert-connector/src/lib/markdown-format.spec.ts b/xpertai/middlewares/codexpert-connector/src/lib/markdown-format.spec.ts new file mode 100644 index 00000000..e49d7188 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/src/lib/markdown-format.spec.ts @@ -0,0 +1,29 @@ +import { formatVisibleMarkdown, preserveMarkdownLineBreaks } from './markdown-format.js' + +describe('preserveMarkdownLineBreaks', () => { + it('keeps visible single line breaks in markdown text', () => { + expect(preserveMarkdownLineBreaks('第一行\n第二行\n第三行')).toBe('第一行 \n第二行 \n第三行') + }) + + it('keeps a trailing line break when a flush boundary ends after newline', () => { + expect(preserveMarkdownLineBreaks('第一行\n')).toBe('第一行 \n') + }) + + it('keeps paragraph breaks and fenced code blocks unchanged', () => { + expect(preserveMarkdownLineBreaks('说明\n\n```ts\nconst a = 1\nconst b = 2\n```\n结束')).toBe( + '说明\n\n```ts\nconst a = 1\nconst b = 2\n```\n结束' + ) + }) +}) + +describe('formatVisibleMarkdown', () => { + it('separates standalone milestone text from following streamed text', () => { + expect(formatVisibleMarkdown('已开始处理。', { standalone: true })).toBe('已开始处理。\n\n') + }) + + it('normalizes common compact markdown blocks', () => { + expect(formatVisibleMarkdown('已完成。##文档内容概述###1.改动概述\n-重构目的')).toBe( + '已完成。\n\n## 文档内容概述\n\n### 1.改动概述\n- 重构目的' + ) + }) +}) diff --git a/xpertai/middlewares/codexpert-connector/src/lib/markdown-format.ts b/xpertai/middlewares/codexpert-connector/src/lib/markdown-format.ts new file mode 100644 index 00000000..650780b9 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/src/lib/markdown-format.ts @@ -0,0 +1,71 @@ +type VisibleMarkdownOptions = { + standalone?: boolean +} + +export function formatVisibleMarkdown(text: string, options: VisibleMarkdownOptions = {}): string { + const formatted = preserveMarkdownLineBreaks(normalizeMarkdownBlockSyntax(text)) + return options.standalone ? ensureTrailingParagraphBreak(formatted) : formatted +} + +export function preserveMarkdownLineBreaks(text: string): string { + const normalized = text.replace(/\r\n/g, '\n') + const lines = normalized.split('\n') + if (lines.length <= 1) { + return normalized + } + + let inFence = false + let result = '' + for (let index = 0; index < lines.length; index += 1) { + const line = lines[index] + result += line + if (index === lines.length - 1) { + break + } + + const isFenceLine = /^(```|~~~)/.test(line.trimStart()) + const nextLine = lines[index + 1] ?? '' + const needsHardBreak = + !inFence && + !isFenceLine && + !isMarkdownBlockLine(line) && + !isMarkdownBlockLine(nextLine) && + line.trim().length > 0 && + (nextLine.trim().length > 0 || index + 1 === lines.length - 1) && + !line.endsWith(' ') + result += needsHardBreak ? ' \n' : '\n' + + if (isFenceLine) { + inFence = !inFence + } + } + + return result +} + +function normalizeMarkdownBlockSyntax(text: string): string { + return text + .replace(/\r\n/g, '\n') + .replace(/([^#\n])(?=#{2,6}\S)/g, '$1\n\n') + .replace(/(^|\n)(#{1,6})(?=\S)/g, '$1$2 ') + .replace(/(^|\n)(\d+\.)(?=\S)/g, '$1$2 ') + .replace(/(^|\n)([-*+])(?=\S)/g, '$1$2 ') +} + +function ensureTrailingParagraphBreak(text: string): string { + if (!text || /\n\n$/.test(text)) { + return text + } + return text.endsWith('\n') ? `${text}\n` : `${text}\n\n` +} + +function isMarkdownBlockLine(line: string): boolean { + const trimmed = line.trimStart() + return ( + /^#{1,6}\s+/.test(trimmed) || + /^[-*+]\s+/.test(trimmed) || + /^\d+\.\s+/.test(trimmed) || + /^>/.test(trimmed) || + /^\|/.test(trimmed) + ) +} diff --git a/xpertai/middlewares/codexpert-connector/src/lib/types.ts b/xpertai/middlewares/codexpert-connector/src/lib/types.ts new file mode 100644 index 00000000..fb37e72a --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/src/lib/types.ts @@ -0,0 +1,109 @@ +import { z } from 'zod/v3' + +export const CODEXPERT_CONNECTOR_MIDDLEWARE_NAME = 'CodexpertConnector' +export const CODEXPERT_AGENT_KEY = 'codexpert' +export const CODEXPERT_XPERT_NAME = 'Codexpert' + +export const CodexpertConnectorConfigSchema = z.object({ + codexpertMcpUrl: z.string().url().optional(), + codexpertConnectorBaseUrl: z.string().url().optional(), + serviceToken: z.string().trim().min(1).optional(), + timeoutMs: z.number().int().positive().optional().default(600_000), + enableVisibleProjection: z.boolean().optional().default(true), + enableStatusEvents: z.boolean().optional().default(true), + defaultXpertId: z.string().trim().min(1).optional(), + defaultRepoId: z.string().trim().min(1).optional(), + defaultConnectionId: z.string().trim().min(1).optional(), + defaultBranchName: z.string().trim().min(1).optional() +}) + +export type CodexpertConnectorConfig = z.infer + +export const RunCodexpertTaskInputSchema = z.object({ + prompt: z.string().trim().min(1), + taskTitle: z.string().trim().min(1).optional(), + codingSessionId: z.string().trim().min(1).optional(), + conversationId: z.string().trim().min(1).optional(), + threadId: z.string().trim().min(1).optional(), + taskId: z.string().trim().min(1).optional(), + xpertId: z.string().trim().min(1).optional(), + repoId: z.string().trim().min(1).optional(), + connectionId: z.string().trim().min(1).optional(), + branchName: z.string().trim().min(1).optional(), + timeoutMs: z.number().int().positive().optional() +}) + +export type RunCodexpertTaskInput = z.infer + +export type PrincipalContext = { + tenantId: string + organizationId: string + userId: string +} + +export type CodexpertConnectorEvent = + | { + type: 'status' + text?: string + headline?: string + phase?: string + isMilestone?: boolean + details?: Record + } + | { + type: 'text_delta' + text: string + stream?: string + tag?: string + } + | { + type: 'tool_call_update' + toolName?: string + name?: string + status?: string + message?: string + error?: string + } + | { + type: 'done' + status?: string + summary?: string | null + output?: string | null + taskId?: string | null + codingSessionId?: string | null + threadId?: string | null + executionId?: string | null + environmentId?: string | null + prUrl?: string | null + } + | { + type: 'error' + message: string + phase?: string + headline?: string + taskId?: string | null + codingSessionId?: string | null + threadId?: string | null + executionId?: string | null + environmentId?: string | null + } + +export type CodexpertTaskResult = { + status: 'success' | 'failed' | 'timeout' | 'canceled' + codingSessionId: string | null + taskId: string | null + threadId: string | null + executionId: string | null + repo: { + id?: string | null + name?: string | null + owner?: string | null + slug?: string | null + } | null + branch: string | null + environmentId: string | null + environmentReused: boolean | null + summary: string | null + error: string | null + prUrl?: string | null +} diff --git a/xpertai/middlewares/codexpert-connector/tsconfig.json b/xpertai/middlewares/codexpert-connector/tsconfig.json new file mode 100644 index 00000000..62ebbd94 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/tsconfig.json @@ -0,0 +1,13 @@ +{ + "extends": "../../tsconfig.base.json", + "files": [], + "include": [], + "references": [ + { + "path": "./tsconfig.lib.json" + }, + { + "path": "./tsconfig.spec.json" + } + ] +} diff --git a/xpertai/middlewares/codexpert-connector/tsconfig.lib.json b/xpertai/middlewares/codexpert-connector/tsconfig.lib.json new file mode 100644 index 00000000..aee870b5 --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/tsconfig.lib.json @@ -0,0 +1,16 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "baseUrl": ".", + "rootDir": "src", + "outDir": "dist", + "tsBuildInfoFile": "dist/tsconfig.lib.tsbuildinfo", + "emitDeclarationOnly": false, + "importHelpers": false, + "forceConsistentCasingInFileNames": true, + "types": ["node"] + }, + "include": ["src/**/*.ts"], + "references": [], + "exclude": ["jest.config.ts", "src/**/*.spec.ts", "src/**/*.test.ts"] +} diff --git a/xpertai/middlewares/codexpert-connector/tsconfig.spec.json b/xpertai/middlewares/codexpert-connector/tsconfig.spec.json new file mode 100644 index 00000000..73c3e7fc --- /dev/null +++ b/xpertai/middlewares/codexpert-connector/tsconfig.spec.json @@ -0,0 +1,7 @@ +{ + "extends": "./tsconfig.lib.json", + "compilerOptions": { + "outDir": "dist-spec" + }, + "include": ["src/**/*.spec.ts", "src/**/*.test.ts"] +} diff --git a/xpertai/pnpm-lock.yaml b/xpertai/pnpm-lock.yaml index 7acadc05..67afa09f 100644 --- a/xpertai/pnpm-lock.yaml +++ b/xpertai/pnpm-lock.yaml @@ -793,6 +793,42 @@ importers: specifier: ^2.3.0 version: 2.8.1 + middlewares/codexpert-connector: + dependencies: + '@langchain/core': + specifier: 0.3.72 + version: 0.3.72(@opentelemetry/api@1.9.0)(openai@5.12.2(ws@8.19.0)(zod@3.25.67)) + '@metad/contracts': + specifier: ^3.8.0 + version: 3.8.3(@langchain/core@0.3.72(@opentelemetry/api@1.9.0)(openai@5.12.2(ws@8.19.0)(zod@3.25.67)))(signal-polyfill@0.2.2) + '@modelcontextprotocol/sdk': + specifier: ^1.17.5 + version: 1.27.1(@cfworker/json-schema@4.1.1)(zod@3.25.67) + '@nestjs/common': + specifier: ^11.1.6 + version: 11.1.14(class-validator@0.14.3)(reflect-metadata@0.2.2)(rxjs@7.8.2) + '@nestjs/typeorm': + specifier: ^11.0.0 + version: 11.0.0(@nestjs/common@11.1.14(class-validator@0.14.3)(reflect-metadata@0.2.2)(rxjs@7.8.2))(@nestjs/core@11.1.14(@nestjs/common@11.1.14(class-validator@0.14.3)(reflect-metadata@0.2.2)(rxjs@7.8.2))(reflect-metadata@0.2.2)(rxjs@7.8.2))(reflect-metadata@0.2.2)(rxjs@7.8.2)(typeorm@0.3.28(@sap/hana-client@2.27.23)(babel-plugin-macros@3.1.0)(ioredis@5.9.3)(mysql2@3.17.5(@types/node@20.19.9))(ts-node@10.9.1(@swc/core@1.5.29(@swc/helpers@0.5.19))(@types/node@20.19.9)(typescript@5.9.3))) + '@xpert-ai/chatkit-types': + specifier: ^0.0.12 + version: 0.0.12(@a2ui/lit@0.8.1(signal-polyfill@0.2.2))(@langchain/core@0.3.72(@opentelemetry/api@1.9.0)(openai@5.12.2(ws@8.19.0)(zod@3.25.67))) + '@xpert-ai/plugin-sdk': + specifier: ^3.8.0 + version: 3.9.1(899dc4fc6495635b951ed001752cf0ac) + chalk: + specifier: 4.1.2 + version: 4.1.2 + tslib: + specifier: ^2.3.0 + version: 2.8.1 + typeorm: + specifier: ^0.3.20 + version: 0.3.28(@sap/hana-client@2.27.23)(babel-plugin-macros@3.1.0)(ioredis@5.9.3)(mysql2@3.17.5(@types/node@20.19.9))(ts-node@10.9.1(@swc/core@1.5.29(@swc/helpers@0.5.19))(@types/node@20.19.9)(typescript@5.9.3)) + zod: + specifier: 3.25.67 + version: 3.25.67 + middlewares/context-editing: dependencies: '@langchain/core':