|
| 1 | +--- |
| 2 | +title: AI Service |
| 3 | +description: LLMs, embedders, RAG, and MCP — pluggable across providers, swappable at runtime. |
| 4 | +--- |
| 5 | + |
| 6 | +# AI Service |
| 7 | + |
| 8 | +ObjectOS treats AI as a first-class capability with **three pluggable |
| 9 | +layers**: |
| 10 | + |
| 11 | +| Layer | Package | What it does | |
| 12 | +|---|---|---| |
| 13 | +| **Chat / generation** | `@objectstack/service-ai` | Conversations, tool calls, streaming | |
| 14 | +| **Embeddings** | `@objectstack/service-embedder` + adapter | Text → vectors for semantic search and RAG | |
| 15 | +| **Knowledge / RAG** | `@objectstack/service-knowledge` + adapter | Documents → indexed knowledge bases | |
| 16 | + |
| 17 | +All three are optional, all three are provider-agnostic, and all three |
| 18 | +can be reconfigured at runtime from **Console → Configuration** |
| 19 | +without a restart. |
| 20 | + |
| 21 | +## Chat / generation |
| 22 | + |
| 23 | +Powered by the [Vercel AI SDK](https://ai-sdk.dev). Install the |
| 24 | +provider(s) you want as peer deps: |
| 25 | + |
| 26 | +```bash |
| 27 | +pnpm add @ai-sdk/openai # OpenAI |
| 28 | +pnpm add @ai-sdk/anthropic # Claude |
| 29 | +pnpm add @ai-sdk/google # Gemini |
| 30 | +pnpm add @ai-sdk/gateway # AI gateway / OpenRouter / proxies |
| 31 | +``` |
| 32 | + |
| 33 | +Then register the service with one or more models: |
| 34 | + |
| 35 | +```ts |
| 36 | +import { ServiceAI } from '@objectstack/service-ai'; |
| 37 | +import { openai } from '@ai-sdk/openai'; |
| 38 | +import { anthropic } from '@ai-sdk/anthropic'; |
| 39 | + |
| 40 | +ServiceAI.configure({ |
| 41 | + defaultModel: 'fast', |
| 42 | + models: { |
| 43 | + fast: openai('gpt-4o-mini'), |
| 44 | + smart: openai('gpt-4o'), |
| 45 | + claude: anthropic('claude-3-5-sonnet-latest'), |
| 46 | + }, |
| 47 | + enableStreaming: true, |
| 48 | + maxHistoryLength: 50, |
| 49 | +}); |
| 50 | +``` |
| 51 | + |
| 52 | +Provider API keys come from each SDK's normal env vars: |
| 53 | + |
| 54 | +| Provider | Env var | |
| 55 | +|---|---| |
| 56 | +| OpenAI | `OPENAI_API_KEY` | |
| 57 | +| Anthropic | `ANTHROPIC_API_KEY` | |
| 58 | +| Google | `GOOGLE_GENERATIVE_AI_API_KEY` | |
| 59 | +| AI Gateway | `AI_GATEWAY_API_KEY` | |
| 60 | + |
| 61 | +In Console you can paste these as runtime settings instead — they go |
| 62 | +through the same precedence (env > settings) but live editing means |
| 63 | +no restart. |
| 64 | + |
| 65 | +### Using the AI service from code |
| 66 | + |
| 67 | +```ts |
| 68 | +const ai = kernel.getService<IAIService>('ai'); |
| 69 | + |
| 70 | +const convo = await ai.createConversation({ |
| 71 | + model: 'smart', |
| 72 | + systemPrompt: 'You are a helpful assistant.', |
| 73 | +}); |
| 74 | + |
| 75 | +const reply = await ai.sendMessage({ |
| 76 | + conversationId: convo.id, |
| 77 | + message: 'Summarize ObjectStack in two sentences.', |
| 78 | +}); |
| 79 | +``` |
| 80 | + |
| 81 | +### Using it from a flow |
| 82 | + |
| 83 | +The `automation` capability exposes an `ai_call` step type: |
| 84 | + |
| 85 | +```ts |
| 86 | +{ |
| 87 | + type: 'action', |
| 88 | + action: 'ai_call', |
| 89 | + inputs: { |
| 90 | + model: 'fast', |
| 91 | + prompt: 'Categorize this ticket: {!trigger.record.subject}', |
| 92 | + schema: { category: 'string', priority: 'string' }, |
| 93 | + }, |
| 94 | + output: 'classified', |
| 95 | +} |
| 96 | +``` |
| 97 | + |
| 98 | +See [Flows & Automation](/docs/build/flows) for the surrounding |
| 99 | +context. |
| 100 | + |
| 101 | +## Embedders |
| 102 | + |
| 103 | +The embedder service converts text to dense vectors. ObjectStack |
| 104 | +ships adapters for many providers — chosen because the embedding |
| 105 | +landscape moves fast and your "best" choice depends on cost, latency, |
| 106 | +and language. |
| 107 | + |
| 108 | +| Provider | Adapter | Notes | |
| 109 | +|---|---|---| |
| 110 | +| OpenAI | `@objectstack/embedder-openai` | `text-embedding-3-small`/`-large` | |
| 111 | +| Azure OpenAI | `@objectstack/embedder-openai` (Azure config) | Enterprise, regional | |
| 112 | +| 阿里通义 DashScope | `@objectstack/embedder-dashscope` | `text-embedding-v3` | |
| 113 | +| 智谱 GLM | `@objectstack/embedder-zhipu` | `embedding-2` | |
| 114 | +| 硅基流动 SiliconFlow | `@objectstack/embedder-siliconflow` | Aggregator of OSS models | |
| 115 | +| 火山 Doubao | `@objectstack/embedder-doubao` | ByteDance | |
| 116 | +| MiniMax | `@objectstack/embedder-minimax` | — | |
| 117 | +| Ollama (self-hosted) | `@objectstack/embedder-ollama` | Air-gapped friendly | |
| 118 | +| Custom | `@objectstack/embedder-custom` | Bring your own HTTP endpoint | |
| 119 | +| None | _builtin_ | Disable embeddings entirely | |
| 120 | + |
| 121 | +Configure in code: |
| 122 | + |
| 123 | +```ts |
| 124 | +import { ServiceEmbedder } from '@objectstack/service-embedder'; |
| 125 | +import { OpenAIEmbedderPlugin } from '@objectstack/embedder-openai'; |
| 126 | + |
| 127 | +ServiceEmbedder.configure({ defaultModel: 'small' }); |
| 128 | +// then register the adapter plugin: |
| 129 | +new OpenAIEmbedderPlugin({ |
| 130 | + model: 'text-embedding-3-small', |
| 131 | + // OPENAI_API_KEY from env |
| 132 | +}); |
| 133 | +``` |
| 134 | + |
| 135 | +Or pick at runtime from **Console → Configuration → AI → Embedder**. |
| 136 | +Switch providers without restart; existing vectors stay searchable |
| 137 | +(you can re-index in the background). |
| 138 | + |
| 139 | +## Knowledge / RAG |
| 140 | + |
| 141 | +The knowledge service orchestrates document ingestion, chunking, |
| 142 | +embedding (via the embedder service), and retrieval. The actual |
| 143 | +storage and search backend is pluggable: |
| 144 | + |
| 145 | +| Adapter | Backend | Good for | |
| 146 | +|---|---|---| |
| 147 | +| `@objectstack/knowledge-memory` | In-process | Dev, demos, small KBs | |
| 148 | +| `@objectstack/knowledge-turso` | Turso/libSQL + sqlite-vss | Single-region production, embedded vectors | |
| 149 | +| `@objectstack/knowledge-ragflow` | [RAGFlow](https://github.com/infiniflow/ragflow) | High-quality OSS RAG with chunking + reranking | |
| 150 | + |
| 151 | +```ts |
| 152 | +import { ServiceKnowledge } from '@objectstack/service-knowledge'; |
| 153 | +import { TursoKnowledgePlugin } from '@objectstack/knowledge-turso'; |
| 154 | + |
| 155 | +ServiceKnowledge.configure(); |
| 156 | +new TursoKnowledgePlugin({ |
| 157 | + databaseUrl: process.env.TURSO_DATABASE_URL, |
| 158 | + authToken: process.env.TURSO_AUTH_TOKEN, |
| 159 | +}); |
| 160 | +``` |
| 161 | + |
| 162 | +Indexed knowledge bases become first-class objects — query them from |
| 163 | +flows, surface them in Studio, attach them to AI assistants as |
| 164 | +retrieval context. |
| 165 | + |
| 166 | +## MCP — Model Context Protocol |
| 167 | + |
| 168 | +ObjectOS can expose itself as a tool server to AI agents (Claude |
| 169 | +Desktop, IDEs, custom agents) via the open [Model Context |
| 170 | +Protocol](https://modelcontextprotocol.io). |
| 171 | + |
| 172 | +```ts |
| 173 | +import { McpServerPlugin } from '@objectstack/plugin-mcp-server'; |
| 174 | + |
| 175 | +new McpServerPlugin({ |
| 176 | + // expose specific objects + actions as MCP tools |
| 177 | + expose: ['todo_task', 'support_ticket'], |
| 178 | +}); |
| 179 | +``` |
| 180 | + |
| 181 | +Agents discover and invoke ObjectOS actions through MCP — subject to |
| 182 | +the calling user's permission set. The MCP server also exposes a |
| 183 | +small set of universal tools: `search_records`, `get_record`, |
| 184 | +`create_record`, `invoke_action`. |
| 185 | + |
| 186 | +## Operational guarantees |
| 187 | + |
| 188 | +- **No mandatory cloud dependency.** Use Ollama for chat + Ollama |
| 189 | + embedder + memory knowledge — entirely air-gapped. |
| 190 | +- **Live swappable.** Change provider in Console; new requests use the |
| 191 | + new provider on next call. No restart. |
| 192 | +- **Per-tenant config.** Each Environment has its own AI settings. |
| 193 | + Tenant A on OpenAI, tenant B on Anthropic — same runtime. |
| 194 | +- **Audit log entries.** Every conversation, tool call, and embedder |
| 195 | + request can be audited (`@objectstack/plugin-audit`). |
| 196 | +- **Cost-aware.** Token counts and provider IDs flow through to the |
| 197 | + audit log for chargeback / cost analysis. |
| 198 | + |
| 199 | +## Where to go next |
| 200 | + |
| 201 | +- [Flows & Automation](/docs/build/flows) — call AI from declarative business logic |
| 202 | +- [Marketplace](/docs/build/marketplace) — AI-powered apps in the default catalog |
| 203 | +- [Security & Compliance](/docs/reference/security) — how AI data flows are isolated |
| 204 | +- [`@objectstack/service-ai` source](https://github.com/objectstack-ai/framework/tree/main/packages/services/service-ai) |
| 205 | +- [`@objectstack/service-embedder` source](https://github.com/objectstack-ai/framework/tree/main/packages/services/service-embedder) |
| 206 | +- [`@objectstack/service-knowledge` source](https://github.com/objectstack-ai/framework/tree/main/packages/services/service-knowledge) |
0 commit comments