Skip to content

Commit a5c2f60

Browse files
grypezgrypez
authored
feat: Add kernel language model service (#586)
Lays out a strategy for language model integration into the ocap-kernel. ## Changes - **New package**: `@ocap/kernel-language-model-service`. Implementation-specific dependencies are isolated into distinct exports. ### `@ocap/kernel-language-model-service` - Provides a streaming language model interface. There is a `makeInstance` method, which returns an object with a `sample` method. The name sample is chosen to align with the semantics of `sample` in the model context protocol. - Provides a nodejs ollama client implementation, `OllamaNodejsService`. - Adds an e2e test which can be used to connect to and test integration with an ollama server running locally on the development workstation.
1 parent cc8006a commit a5c2f60

25 files changed

Lines changed: 1642 additions & 127 deletions

‎packages/kernel-language-model-service/CHANGELOG.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Changelog
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7+
8+
## [Unreleased]
9+
10+
[Unreleased]: https://github.com/MetaMask/ocap-kernel/

‎packages/kernel-language-model-service/README.md‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# `@ocap/kernel-language-model-service`
2+
3+
A package providing language model service implementations for the ocap kernel. This package defines interfaces and implementations for integrating various language model providers (like Ollama) into the kernel's object capability system.
4+
5+
## Overview
6+
7+
This package provides:
8+
9+
- **Generic interfaces** for language model services that can be implemented by different providers
10+
- **Ollama integration** for local language model inference
11+
- **Object capability security** through hardened instances and endowment patterns
12+
- **Type-safe configuration** using Superstruct validation
13+
14+
## Architecture
15+
16+
The package follows the object capability pattern with clear separation of concerns:
17+
18+
- `LanguageModelService` - Factory interface for creating model instances
19+
- `LanguageModel` - Interface for individual model instances
20+
- Provider-specific implementations (e.g., `OllamaNodejsService`)
21+
22+
All model instances are hardened using `harden()` from `@endo/ses` for security.
23+
24+
## Installation
25+
26+
`yarn add @ocap/kernel-language-model-service`
27+
28+
or
29+
30+
`npm install @ocap/kernel-language-model-service`
31+
32+
## Usage
33+
34+
### Basic Ollama Integration
35+
36+
```typescript
37+
import { OllamaNodejsService } from '@ocap/kernel-language-model-service/ollama/nodejs';
38+
39+
// Create a service instance with required endowments
40+
const service = new OllamaNodejsService({
41+
endowments: { fetch: global.fetch },
42+
});
43+
44+
// Create a model instance
45+
const model = await service.makeInstance({
46+
model: 'llama2',
47+
options: { temperature: 0.7 },
48+
});
49+
50+
// (Optional) Load the model into memory
51+
await model.load();
52+
53+
// Generate a response
54+
const response = await model.sample('Hello, world!');
55+
for await (const chunk of response) {
56+
console.log(chunk.response);
57+
}
58+
59+
// (Optional) Unload the model when done
60+
await model.unload();
61+
```
62+
63+
### Using Host-Restricted Fetch
64+
65+
For enhanced security, you can use the host-restricted fetch utility:
66+
67+
```typescript
68+
import { makeHostRestrictedFetch } from '@ocap/kernel-language-model-service/ollama/fetch';
69+
70+
const restrictedFetch = makeHostRestrictedFetch(
71+
['localhost:11434'],
72+
global.fetch,
73+
);
74+
75+
const service = new OllamaNodejsService({
76+
endowments: { fetch: restrictedFetch },
77+
});
78+
```
79+
80+
### Listing Available Models
81+
82+
```typescript
83+
const models = await service.getModels();
84+
console.log(
85+
'Available models:',
86+
models.models.map((m) => m.name),
87+
);
88+
```
89+
90+
## Security Considerations
91+
92+
- **Object Capabilities**: All model instances are hardened and can be safely passed between vats
93+
- **Endowment Pattern**: External dependencies (like `fetch`) must be explicitly provided
94+
- **Host Restrictions**: Use `makeHostRestrictedFetch` to limit network access
95+
- **Validation**: All configurations are validated using Superstruct schemas
96+
97+
## API Reference
98+
99+
### Core Types
100+
101+
- `LanguageModelService<Config, Options, Response>` - Factory for creating model instances
102+
- `LanguageModel<Options, Response>` - Interface for model instances
103+
- `ModelInfo<Options>` - Configuration information for a model
104+
- `InstanceConfig<Options>` - Configuration for creating model instances
105+
106+
### Ollama Types
107+
108+
- `OllamaNodejsService` - Node.js implementation of Ollama service
109+
- `OllamaModelOptions` - Valid options for Ollama model generation
110+
- `OllamaInstanceConfig` - Configuration for Ollama model instances
111+
112+
## Contributing
113+
114+
This package is part of a monorepo. Instructions for contributing can be found in the [monorepo README](https://github.com/MetaMask/ocap-kernel#readme).

‎packages/kernel-language-model-service/package.json‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
{
2+
"name": "@ocap/kernel-language-model-service",
3+
"version": "0.0.0",
4+
"private": true,
5+
"description": "A place for implementations providing language model services to the ocap kernel",
6+
"homepage": "https://github.com/MetaMask/ocap-kernel/tree/main/packages/kernel-language-model-service#readme",
7+
"bugs": {
8+
"url": "https://github.com/MetaMask/ocap-kernel/issues"
9+
},
10+
"repository": {
11+
"type": "git",
12+
"url": "https://github.com/MetaMask/ocap-kernel.git"
13+
},
14+
"type": "module",
15+
"exports": {
16+
"./ollama/nodejs": {
17+
"import": {
18+
"types": "./dist/ollama/nodejs.d.mts",
19+
"default": "./dist/ollama/nodejs.mjs"
20+
},
21+
"require": {
22+
"types": "./dist/ollama/nodejs.d.cts",
23+
"default": "./dist/ollama/nodejs.cjs"
24+
}
25+
},
26+
"./package.json": "./package.json"
27+
},
28+
"files": [
29+
"dist/"
30+
],
31+
"scripts": {
32+
"build": "ts-bridge --project tsconfig.build.json --clean",
33+
"build:docs": "typedoc",
34+
"changelog:validate": "../../scripts/validate-changelog.sh @ocap/kernel-language-model-service",
35+
"clean": "rimraf --glob './*.tsbuildinfo' ./.eslintcache ./coverage ./dist",
36+
"lint": "yarn lint:eslint && yarn lint:misc --check && yarn constraints && yarn lint:dependencies",
37+
"lint:dependencies": "depcheck",
38+
"lint:eslint": "eslint . --cache",
39+
"lint:fix": "yarn lint:eslint --fix && yarn lint:misc --write && yarn constraints --fix && yarn lint:dependencies",
40+
"lint:misc": "prettier --no-error-on-unmatched-pattern '**/*.json' '**/*.md' '**/*.html' '!**/CHANGELOG.old.md' '**/*.yml' '!.yarnrc.yml' '!merged-packages/**' --ignore-path ../../.gitignore",
41+
"publish:preview": "yarn npm publish --tag preview",
42+
"test": "vitest run --config vitest.config.ts",
43+
"test:e2e": "vitest run --config vitest.config.e2e.ts",
44+
"test:clean": "yarn test --no-cache --coverage.clean",
45+
"test:dev": "yarn test --mode development",
46+
"test:verbose": "yarn test --reporter verbose",
47+
"test:watch": "vitest --config vitest.config.ts"
48+
},
49+
"devDependencies": {
50+
"@arethetypeswrong/cli": "^0.17.4",
51+
"@metamask/auto-changelog": "^5.0.1",
52+
"@metamask/eslint-config": "^14.0.0",
53+
"@metamask/eslint-config-nodejs": "^14.0.0",
54+
"@metamask/eslint-config-typescript": "^14.0.0",
55+
"@metamask/streams": "workspace:^",
56+
"@ocap/test-utils": "workspace:^",
57+
"@ts-bridge/cli": "^0.6.3",
58+
"@ts-bridge/shims": "^0.1.1",
59+
"@types/chrome": "^0.0.313",
60+
"@typescript-eslint/eslint-plugin": "^8.29.0",
61+
"@typescript-eslint/parser": "^8.29.0",
62+
"@typescript-eslint/utils": "^8.29.0",
63+
"@vitest/eslint-plugin": "^1.1.44",
64+
"depcheck": "^1.4.7",
65+
"eslint": "^9.23.0",
66+
"eslint-config-prettier": "^10.1.1",
67+
"eslint-import-resolver-typescript": "^4.3.1",
68+
"eslint-plugin-import-x": "^4.10.0",
69+
"eslint-plugin-jsdoc": "^50.6.9",
70+
"eslint-plugin-n": "^17.17.0",
71+
"eslint-plugin-prettier": "^5.2.6",
72+
"eslint-plugin-promise": "^7.2.1",
73+
"prettier": "^3.5.3",
74+
"rimraf": "^6.0.1",
75+
"typedoc": "^0.28.1",
76+
"typescript": "~5.8.2",
77+
"typescript-eslint": "^8.29.0",
78+
"vite": "^6.3.5",
79+
"vitest": "^3.2.4"
80+
},
81+
"engines": {
82+
"node": "^20 || >=22"
83+
},
84+
"dependencies": {
85+
"@metamask/superstruct": "^3.2.1",
86+
"ollama": "^0.5.16",
87+
"ses": "^1.13.0"
88+
}
89+
}

0 commit comments

Comments
 (0)