Skip to content

Commit d1b69f5

Browse files
committed
feat(api-server): implement generation service and standalone server
- Added GenerationService class for handling response generation with support for tool calls and conversation history. - Introduced standalone server entry point for starting the API server with configurable options. - Created types for API server options, request/response structures, and transcript items. - Implemented utility functions for handling HTTP requests and responses, including CORS support. - Added support for processing '@<path>' commands to read file contents dynamically. - Updated CLI package to include the new API server as a dependency and adjusted import paths.
1 parent 445be72 commit d1b69f5

23 files changed

Lines changed: 2593 additions & 1138 deletions

package-lock.json

Lines changed: 1310 additions & 1134 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/api-server/README.md

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# @kolosal-ai/api-server
2+
3+
A standalone HTTP API server for Kolosal AI generation services.
4+
5+
## Installation
6+
7+
```bash
8+
npm install @kolosal-ai/api-server
9+
```
10+
11+
## Usage
12+
13+
### Basic Usage
14+
15+
```typescript
16+
import { startApiServer } from '@kolosal-ai/api-server';
17+
import { Config } from '@kolosal-ai/kolosal-ai-core';
18+
19+
const config = new Config(/* your config options */);
20+
21+
const server = await startApiServer(config, {
22+
port: 8080,
23+
host: '127.0.0.1',
24+
enableCors: true
25+
});
26+
27+
console.log(`Server running on http://127.0.0.1:${server.port}`);
28+
29+
// Later, to close the server
30+
await server.close();
31+
```
32+
33+
### API Endpoints
34+
35+
#### Health Check
36+
- **GET** `/healthz` - Returns server health status
37+
38+
#### Status
39+
- **GET** `/status` - Returns server status and configuration info
40+
41+
#### Generate
42+
- **POST** `/v1/generate` - Generate AI responses
43+
44+
**Request Body:**
45+
```json
46+
{
47+
"input": "Your prompt here",
48+
"stream": false,
49+
"prompt_id": "optional-prompt-id",
50+
"history": [],
51+
"model": "optional-custom-model",
52+
"api_key": "optional-custom-api-key",
53+
"base_url": "optional-custom-base-url",
54+
"working_directory": "/optional/working/directory"
55+
}
56+
```
57+
58+
**Response:**
59+
```json
60+
{
61+
"output": "AI generated response",
62+
"prompt_id": "prompt-id",
63+
"messages": [],
64+
"history": []
65+
}
66+
```
67+
68+
### Advanced Usage
69+
70+
You can also use the lower-level components for custom setups:
71+
72+
```typescript
73+
import { ApiServerFactory, Router } from '@kolosal-ai/api-server';
74+
75+
// Create a custom server with additional routes
76+
const config = /* your config */;
77+
const router = new Router();
78+
79+
// Add custom routes
80+
router.addRoute('GET', '/custom', customHandler);
81+
82+
// Create server with custom router
83+
const server = await ApiServerFactory.create(config, options);
84+
```
85+
86+
## Features
87+
88+
- **File Reference Support**: Use `@filename` syntax in prompts to include file contents
89+
- **Streaming Support**: Real-time response streaming
90+
- **CORS Support**: Configurable cross-origin resource sharing
91+
- **Custom Models**: Support for custom model configurations
92+
- **Working Directory**: Set custom working directories for file operations (automatically created if it doesn't exist)
93+
- **History Management**: Conversation history support
94+
95+
## Dependencies
96+
97+
This package requires:
98+
- `@kolosal-ai/kolosal-ai-core` - Core Kolosal AI functionality
99+
- `@google/genai` - Google Generative AI client
100+
101+
## License
102+
103+
Apache-2.0

packages/api-server/package.json

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
{
2+
"name": "@kolosal-ai/api-server",
3+
"version": "0.1.3",
4+
"description": "Kolosal AI API Server - HTTP server for AI generation services",
5+
"type": "module",
6+
"main": "dist/index.js",
7+
"types": "dist/index.d.ts",
8+
"exports": {
9+
".": {
10+
"import": "./dist/index.js",
11+
"types": "./dist/index.d.ts"
12+
}
13+
},
14+
"scripts": {
15+
"build": "node ../../scripts/build_package.js",
16+
"start": "node dist/standalone.js",
17+
"start:server": "node dist/standalone.js",
18+
"start:server:debug": "DEBUG=1 node dist/standalone.js",
19+
"start:server:port": "node dist/standalone.js",
20+
"test": "vitest",
21+
"test:watch": "vitest --watch",
22+
"lint": "eslint src",
23+
"clean": "rm -rf dist"
24+
},
25+
"dependencies": {
26+
"@kolosal-ai/kolosal-ai-core": "file:../core",
27+
"@google/genai": "1.9.0"
28+
},
29+
"devDependencies": {
30+
"@types/node": "^22.9.0",
31+
"typescript": "^5.6.3",
32+
"vitest": "^2.1.4"
33+
},
34+
"license": "Apache-2.0",
35+
"keywords": [
36+
"ai",
37+
"api",
38+
"server",
39+
"http",
40+
"kolosal"
41+
],
42+
"repository": {
43+
"type": "git",
44+
"url": "https://github.com/kolosalAI/kolosal-cli.git",
45+
"directory": "packages/api-server"
46+
},
47+
"bugs": {
48+
"url": "https://github.com/kolosalAI/kolosal-cli/issues"
49+
},
50+
"homepage": "https://github.com/kolosalAI/kolosal-cli/tree/main/packages/api-server"
51+
}
File renamed without changes.
File renamed without changes.
File renamed without changes.

packages/api-server/src/index.ts

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
/**
2+
* @license
3+
* Copyright 2025 Kolosal Inc.
4+
* SPDX-License-Identifier: Apache-2.0
5+
*/
6+
7+
// Main API server exports
8+
export { startApiServer } from './server.js';
9+
export type { ApiServerOptions, ApiServer } from './types/index.js';
10+
11+
// Additional exports for advanced usage
12+
export { ApiServerFactory } from './server.factory.js';
13+
export { Router } from './router.js';
14+
export type {
15+
HttpContext,
16+
RouteHandler,
17+
Middleware,
18+
GenerateRequest,
19+
GenerateResponse,
20+
GenerationResult,
21+
TranscriptItem,
22+
StreamEventCallback,
23+
ContentStreamCallback
24+
} from './types/index.js';
File renamed without changes.

0 commit comments

Comments
 (0)