Skip to content

Commit bf88610

Browse files
committed
feat: add Gemini API support with complete implementation
- Implement Gemini API endpoints (generateContent, streamGenerateContent, countTokens) - Add request/response translation between Gemini and OpenAI formats - Support function calling with proper tool_call_id mapping - Handle streaming responses with chunk-by-chunk translation - Add debug logging system for troubleshooting - Implement tool call utilities for parameter accumulation - Add comprehensive test coverage for all Gemini features This implementation provides full compatibility with Google's Gemini API specification while using GitHub Copilot as the underlying LLM provider.
1 parent 9c3cbf8 commit bf88610

23 files changed

Lines changed: 4102 additions & 1343 deletions

CLAUDE.md

Lines changed: 0 additions & 135 deletions
Original file line numberDiff line numberDiff line change
@@ -1,135 +0,0 @@
1-
# CLAUDE.md
2-
3-
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4-
5-
## Development Commands
6-
7-
- **Build**: `bun run build` (uses tsdown)
8-
- **Development**: `bun run dev` (with file watching)
9-
- **Production**: `bun run start` (sets NODE_ENV=production)
10-
- **Lint**: `bun run lint` (uses @echristian/eslint-config with cache)
11-
- **Lint fix**: `bunx lint-staged` (fixes staged files)
12-
- **Typecheck**: `bun run typecheck` (runs TypeScript compiler)
13-
- **Test all**: `bun test`
14-
- **Test single file**: `bun test tests/[filename].test.ts`
15-
- **Package**: `bun run prepack` (builds before packaging)
16-
17-
## Project Architecture
18-
19-
### High-Level Structure
20-
This is a GitHub Copilot API proxy server that exposes Copilot as OpenAI-compatible, Anthropic-compatible, and Gemini-compatible APIs. The server is built with Hono framework and uses Bun as the runtime.
21-
22-
### Core Architecture Components
23-
24-
**API Translation Layer** (`src/routes/messages/`):
25-
- Translates between Anthropic Messages API format and OpenAI Chat Completions format
26-
- Translates between Gemini API format and OpenAI Chat Completions format
27-
- Handles both streaming and non-streaming responses
28-
- Key files: `handler.ts`, `anthropic-types.ts`, `stream-translation.ts`, `non-stream-translation.ts`
29-
- Gemini files: `gemini-handler.ts`, `gemini-translation.ts`, `gemini-types.ts`, `gemini-route.ts`
30-
31-
**Token Counting for Anthropic Models** (`src/lib/tokenizer.ts`):
32-
- Uses `gpt-tokenizer/model/gpt-4o` for token counting
33-
- Separates input tokens (all messages except last assistant message) from output tokens (last assistant message)
34-
- Filters out tool messages and extracts text content from multipart messages
35-
- Used by `/v1/messages/count_tokens` endpoint for Anthropic compatibility
36-
37-
**GitHub Copilot Integration** (`src/services/`):
38-
- Authentication flow using device code OAuth
39-
- Token management and refresh
40-
- API requests to GitHub Copilot endpoints
41-
- Usage monitoring and quota tracking
42-
43-
**Rate Limiting & Controls** (`src/lib/`):
44-
- Rate limiting between requests (`rate-limit.ts`)
45-
- Manual approval system for requests (`approval.ts`)
46-
- State management for server configuration (`state.ts`)
47-
48-
### API Endpoints Structure
49-
50-
**OpenAI Compatible**:
51-
- `/v1/chat/completions` - Chat completions
52-
- `/v1/models` - Available models
53-
- `/v1/embeddings` - Text embeddings
54-
55-
**Anthropic Compatible**:
56-
- `/v1/messages` - Message completions (translates to/from OpenAI format)
57-
- `/v1/messages/count_tokens` - Token counting for Anthropic format
58-
59-
**Gemini Compatible**:
60-
- `/v1beta/models/{model}:generateContent` - Standard generation
61-
- `/v1beta/models/{model}:streamGenerateContent` - Streaming generation
62-
- `/v1beta/models/{model}:countTokens` - Token counting
63-
64-
**Monitoring**:
65-
- `/usage` - GitHub Copilot usage dashboard
66-
- `/token` - Current Copilot token info
67-
68-
### Key Implementation Details
69-
70-
**Anthropic Token Counting**:
71-
The `getTokenCount()` function in `src/lib/tokenizer.ts` implements token counting specifically for Anthropic compatibility:
72-
- Converts multipart content to text-only for counting
73-
- Splits messages into input (all except last assistant) and output (last assistant message only)
74-
- Uses GPT-4o tokenizer as the underlying counting mechanism
75-
- Returns `{input: number, output: number}` format
76-
77-
**Message Translation**:
78-
- OpenAI → Anthropic: Converts chat completion responses to Anthropic message format
79-
- Anthropic → OpenAI: Converts Anthropic message requests to OpenAI chat completion format
80-
- OpenAI → Gemini: Converts chat completion responses to Gemini response format
81-
- Gemini → OpenAI: Converts Gemini requests to OpenAI chat completion format
82-
- Handles tool calls, system messages, and content blocks appropriately for all formats
83-
84-
**Streaming Translation**:
85-
Real-time conversion of OpenAI SSE chunks to both Anthropic streaming events and Gemini streaming responses, maintaining state for proper message reconstruction.
86-
87-
**Gemini API Implementation**:
88-
The Gemini integration (`src/routes/messages/gemini-*`) provides:
89-
- Full compatibility with Google's Gemini API specification
90-
- Comprehensive request/response translation between Gemini and OpenAI formats
91-
- Support for function calling, multimodal content (text + images), and streaming
92-
- Extensive debug logging with file-based logs in `logs/` directory
93-
- Error handling with appropriate HTTP status codes and Gemini-formatted error responses
94-
- Support for generation configuration (temperature, max tokens, top-p, stop sequences)
95-
96-
**Critical Gemini Translation Details**:
97-
- Gemini CLI sends function responses as **nested arrays** in contents, requiring special handling
98-
- `parametersJsonSchema` field takes precedence over `parameters` in function declarations
99-
- Tool call ID mapping must be maintained between assistant tool calls and user tool responses
100-
- Function response arrays need extraction with `processFunctionResponseArray()` helper
101-
- Debug logs in `logs/gemini-*.log` files are essential for troubleshooting translation issues
102-
103-
## Code Style & Conventions
104-
105-
- **TypeScript**: Strict mode enabled, avoid `any` types
106-
- **Imports**: Use `~/*` path aliases for `src/*` imports
107-
- **Error Handling**: Use explicit error classes from `src/lib/error.ts`
108-
- **Testing**: Place tests in `tests/` directory with `*.test.ts` naming
109-
- **Formatting**: Prettier with package.json plugin
110-
- **Linting**: @echristian/eslint-config with strict rules
111-
112-
## Important Notes
113-
114-
- Server uses GitHub Copilot as the underlying LLM provider
115-
- Rate limiting and manual approval features help avoid GitHub abuse detection
116-
- Token counting uses GPT-4o tokenizer regardless of the actual model being proxied
117-
- All API translations maintain compatibility with OpenAI, Anthropic, and Gemini client libraries
118-
- Gemini API debugging logs are written to `logs/` directory for troubleshooting translation issues
119-
120-
## Debugging & Troubleshooting
121-
122-
**Common Gemini API Issues**:
123-
- **Function calls fail while text prompts work**: Check `logs/gemini-translation.log` for missing `parameters` in translated tools
124-
- **Tool response mapping errors**: Verify tool_call_id consistency between assistant tool calls and user tool responses
125-
- **Nested array handling**: Gemini CLI sends function responses as nested arrays requiring `processFunctionResponseArray()` extraction
126-
- **HTTPError from create-chat-completions**: Usually indicates parameter validation failure in OpenAI translation layer
127-
128-
**Key Log Files**:
129-
- `logs/gemini-errors.log`: HTTP errors and stack traces
130-
- `logs/gemini-debug.log`: Request/response flow with full JSON payloads
131-
- `logs/gemini-translation.log`: Translation pipeline details showing input/output transformations
132-
133-
**Debugging Commands**:
134-
- `bun run lint && bun run typecheck && bun run build`: Full validation pipeline
135-
- Check error reports in `C:\Users\39764\AppData\Local\Temp\gemini-client-error-*.json` for client-side failures

src/lib/debug-logger.ts

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
import { existsSync, mkdirSync } from "node:fs"
2+
import { writeFile } from "node:fs/promises"
3+
import { join } from "node:path"
4+
5+
import type { GeminiRequest } from "~/routes/generate-content/types"
6+
import type {
7+
ChatCompletionsPayload,
8+
ChatCompletionResponse,
9+
} from "~/services/copilot/create-chat-completions"
10+
11+
interface DebugLogData {
12+
timestamp: string
13+
requestId: string
14+
originalGeminiPayload: GeminiRequest
15+
translatedOpenAIPayload: ChatCompletionsPayload | null
16+
error?: string
17+
processingTime?: number
18+
}
19+
20+
export class DebugLogger {
21+
private static instance: DebugLogger | undefined
22+
private logDir: string
23+
24+
private constructor() {
25+
this.logDir = process.env.DEBUG_LOG_DIR || join(process.cwd(), "debug-logs")
26+
this.ensureLogDir()
27+
}
28+
29+
static getInstance(): DebugLogger {
30+
if (!DebugLogger.instance) {
31+
DebugLogger.instance = new DebugLogger()
32+
}
33+
return DebugLogger.instance
34+
}
35+
36+
private ensureLogDir(): void {
37+
if (!existsSync(this.logDir)) {
38+
mkdirSync(this.logDir, { recursive: true })
39+
}
40+
}
41+
42+
private generateLogFileName(requestId: string): string {
43+
const timestamp = new Date().toISOString().replaceAll(/[:.]/g, "-")
44+
return join(this.logDir, `debug-gemini-${timestamp}-${requestId}.log`)
45+
}
46+
47+
async logRequest(data: {
48+
requestId: string
49+
geminiPayload: GeminiRequest
50+
openAIPayload?: ChatCompletionsPayload | null
51+
error?: string
52+
processingTime?: number
53+
}): Promise<void> {
54+
const logData: DebugLogData = {
55+
timestamp: new Date().toISOString(),
56+
requestId: data.requestId,
57+
originalGeminiPayload: data.geminiPayload,
58+
translatedOpenAIPayload: data.openAIPayload ?? null,
59+
error: data.error,
60+
processingTime: data.processingTime,
61+
}
62+
63+
const logPath = this.generateLogFileName(data.requestId)
64+
65+
try {
66+
await writeFile(logPath, JSON.stringify(logData, null, 2), "utf8")
67+
console.log(`[DEBUG] Logged request data to: ${logPath}`)
68+
} catch (writeError) {
69+
console.error(`[DEBUG] Failed to write log file ${logPath}:`, writeError)
70+
}
71+
}
72+
73+
// For backward compatibility during development
74+
static async logGeminiRequest(
75+
geminiPayload: GeminiRequest,
76+
openAIPayload?: ChatCompletionsPayload,
77+
error?: string,
78+
): Promise<void> {
79+
const logger = DebugLogger.getInstance()
80+
const requestId = Math.random().toString(36).slice(2, 8)
81+
await logger.logRequest({ requestId, geminiPayload, openAIPayload, error })
82+
}
83+
84+
// Log GitHub Copilot API Response
85+
static async logCopilotResponse(
86+
response: ChatCompletionResponse,
87+
context?: string,
88+
): Promise<void> {
89+
const logger = DebugLogger.getInstance()
90+
const requestId = Math.random().toString(36).slice(2, 8)
91+
const timestamp = new Date().toISOString().replaceAll(/[:.]/g, "-")
92+
const logPath = join(
93+
logger.logDir,
94+
`debug-copilot-response-${timestamp}-${requestId}.log`,
95+
)
96+
97+
const logData = {
98+
timestamp: new Date().toISOString(),
99+
context: context || "GitHub Copilot API Response",
100+
response,
101+
}
102+
103+
try {
104+
await writeFile(logPath, JSON.stringify(logData, null, 2), "utf8")
105+
console.log(`[DEBUG] Logged Copilot response to: ${logPath}`)
106+
} catch (writeError) {
107+
console.error(
108+
`[DEBUG] Failed to write Copilot response log file ${logPath}:`,
109+
writeError,
110+
)
111+
}
112+
}
113+
114+
// Log any object for debugging purposes
115+
static async logDebugData(
116+
data: unknown,
117+
context: string,
118+
filePrefix = "debug-data",
119+
): Promise<void> {
120+
const logger = DebugLogger.getInstance()
121+
const requestId = Math.random().toString(36).slice(2, 8)
122+
const timestamp = new Date().toISOString().replaceAll(/[:.]/g, "-")
123+
const logPath = join(
124+
logger.logDir,
125+
`${filePrefix}-${timestamp}-${requestId}.log`,
126+
)
127+
128+
const logData = {
129+
timestamp: new Date().toISOString(),
130+
context,
131+
data,
132+
}
133+
134+
try {
135+
await writeFile(logPath, JSON.stringify(logData, null, 2), "utf8")
136+
console.log(`[DEBUG] Logged ${context} to: ${logPath}`)
137+
} catch (writeError) {
138+
console.error(
139+
`[DEBUG] Failed to write debug log file ${logPath}:`,
140+
writeError,
141+
)
142+
}
143+
}
144+
145+
// Log original and translated response comparison
146+
static async logResponseComparison(
147+
originalResponse: unknown,
148+
translatedResponse: unknown,
149+
options: { context: string; filePrefix?: string } = {
150+
context: "Response Comparison",
151+
},
152+
): Promise<void> {
153+
const { context, filePrefix = "debug-comparison" } = options
154+
const logger = DebugLogger.getInstance()
155+
const requestId = Math.random().toString(36).slice(2, 8)
156+
const timestamp = new Date().toISOString().replaceAll(/[:.]/g, "-")
157+
const logPath = join(
158+
logger.logDir,
159+
`${filePrefix}-${timestamp}-${requestId}.log`,
160+
)
161+
162+
const logData = {
163+
timestamp: new Date().toISOString(),
164+
context,
165+
originalResponse,
166+
translatedResponse,
167+
}
168+
169+
try {
170+
await writeFile(logPath, JSON.stringify(logData, null, 2), "utf8")
171+
console.log(`[DEBUG] Logged ${context} comparison to: ${logPath}`)
172+
} catch (writeError) {
173+
console.error(
174+
`[DEBUG] Failed to write comparison log file ${logPath}:`,
175+
writeError,
176+
)
177+
}
178+
}
179+
}

0 commit comments

Comments
 (0)