Skip to content

Commit 081e375

Browse files
asimclaude
andauthored
Add AI provider integration guide and new providers support (#2900)
* docs: add AI provider integration guide and Supported AI Providers section Add a step-by-step guide for AI infrastructure companies to implement ai.Model and contribute a provider to go-micro. Covers the full lifecycle: skeleton, tool call handling, tests, registration, and PR checklist. Add a "Supported AI Providers" section to the project README that lists current providers (Anthropic, OpenAI) in a table and links to the integration guide with a call-to-action for new providers and sponsors. Streamline the "Adding a New Provider" section in ai/README.md to point to the new guide instead of duplicating a full code listing. * fix: remove nonexistent Discord link from README * fix(website): set content container width to 800px on desktop Move the 800px max-width from .markdown-body up to .content so the entire content pane (not just the inner body) is sized correctly. The container now fills up to 800px beside the sidebar. * feat(ai): wire Atlas Cloud into server and auto-detection Import atlascloud provider in the micro server so it is available when running micro run / micro server. Add atlascloud to AutoDetectProvider so --ai_base_url with an atlascloud domain selects the right provider automatically. * feat(ai): add Google Gemini provider Add ai/gemini implementing ai.Model for Google's Gemini API. Uses the native generateContent endpoint with system_instruction, contents/parts, and functionDeclarations — not an OpenAI shim. Default model gemini-2.5-flash, auth via x-goog-api-key header. Wire into micro server imports and AutoDetectProvider (matches googleapis.com and google in base URL). Update README.md and ai/README.md with provider listing. * feat(ai): add Groq, Mistral, and Together AI providers Add three new OpenAI-compatible providers: - ai/groq: ultra-fast inference, default model llama-3.3-70b-versatile - ai/mistral: Mistral AI, default model mistral-large-latest - ai/together: Together AI, default model Llama-3.3-70B-Instruct-Turbo All three are wired into the micro server imports and AutoDetectProvider. README and ai/README updated with the full provider table. * feat(ai): add ai/tools helper and 'micro chat' interactive agent Extract the registry-discovery + RPC-execution loop from the web agent playground into a reusable ai/tools package: - tools.New(reg) creates a Set bound to a registry - Set.Discover() walks the registry and returns []ai.Tool with LLM-safe (underscored) names, remembering the mapping back to the original dotted form - Set.Handler(client) returns an ai.ToolHandler that resolves the safe name and issues the RPC Add cmd/micro/chat — an interactive 'micro chat' REPL that uses ai/tools to let users talk to their services through any registered AI provider. Supports --prompt for single-shot use, auto-detects the provider from --base_url, and falls back to the provider's conventional env var (ANTHROPIC_API_KEY, etc). Update README with the new command and the programmatic example. * feat(examples): add gRPC interop example Add examples/grpc-interop showing that any standard gRPC client can call a go-micro service — no go-micro SDK required on the client side. Includes: - proto/greeter.proto with generated Go, gRPC, and micro stubs - server/ using go-micro gRPC transport - client/ using stock google.golang.org/grpc (no go-micro imports) - README with Python example and explanation of how routing works Addresses the confusion from issue #2818 where users didn't know that go-micro gRPC services are callable by any gRPC client. --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 2f78103 commit 081e375

26 files changed

Lines changed: 2728 additions & 5 deletions

File tree

README.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,34 @@ micro run
149149

150150
Use `micro mcp serve` for local AI tools like Claude Code, or connect any MCP-compatible agent to the HTTP endpoint.
151151

152+
### micro chat
153+
154+
For an interactive terminal session that lets you talk to your services through an LLM:
155+
156+
```bash
157+
ANTHROPIC_API_KEY=sk-ant-... micro chat --provider anthropic
158+
> list all users
159+
> create an order for product 42
160+
```
161+
162+
`micro chat` discovers every service in the registry, exposes each endpoint as a tool, and lets the model orchestrate calls. The same building blocks (`ai/tools`) work from your own services:
163+
164+
```go
165+
import "go-micro.dev/v5/ai/tools"
166+
167+
set := tools.New(service.Registry())
168+
discovered, _ := set.Discover()
169+
170+
m := ai.New("anthropic",
171+
ai.WithAPIKey(key),
172+
ai.WithToolHandler(set.Handler(service.Client())),
173+
)
174+
resp, _ := m.Generate(ctx, &ai.Request{
175+
Prompt: userInput,
176+
Tools: discovered,
177+
})
178+
```
179+
152180
See the [MCP guide](https://go-micro.dev/docs/mcp.html) for authentication, scopes, and advanced usage.
153181

154182
## Multi-Service Binaries
@@ -247,6 +275,7 @@ Every service gets `Client()`, `Server()`, and `Model()` — call services, hand
247275

248276
Check out [/examples](examples/) for runnable code:
249277
- [hello-world](examples/hello-world/) - Basic RPC service
278+
- [grpc-interop](examples/grpc-interop/) - Call go-micro from any gRPC client
250279
- [web-service](examples/web-service/) - HTTP REST API
251280
- [multi-service](examples/multi-service/) - Multiple services in one binary
252281
- [mcp](examples/mcp/) - MCP integration with AI agents
@@ -396,7 +425,11 @@ Go Micro’s `ai` package gives every provider the same interface: `Init`, `Gene
396425
| Provider | Import | Default Model |
397426
|----------|--------|---------------|
398427
| **Anthropic** | `go-micro.dev/v5/ai/anthropic` | `claude-sonnet-4-20250514` |
428+
| **Google Gemini** | `go-micro.dev/v5/ai/gemini` | `gemini-2.5-flash` |
429+
| **Groq** | `go-micro.dev/v5/ai/groq` | `llama-3.3-70b-versatile` |
430+
| **Mistral** | `go-micro.dev/v5/ai/mistral` | `mistral-large-latest` |
399431
| **OpenAI** | `go-micro.dev/v5/ai/openai` | `gpt-4o` |
432+
| **Together AI** | `go-micro.dev/v5/ai/together` | `Llama-3.3-70B-Instruct-Turbo` |
400433
| **Atlas Cloud** | `go-micro.dev/v5/ai/atlascloud` | `llama-3.3-70b` |
401434

402435
Any provider that exposes an OpenAI-compatible API can also be used directly:

ai/README.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,62 @@ m := ai.New("openai",
152152
Default model: `gpt-4o`
153153
Default base URL: `https://api.openai.com`
154154

155+
### Google Gemini
156+
157+
```go
158+
m := ai.New("gemini",
159+
ai.WithAPIKey("your-key"),
160+
ai.WithModel("gemini-2.5-flash"), // default
161+
)
162+
```
163+
164+
Default model: `gemini-2.5-flash`
165+
Default base URL: `https://generativelanguage.googleapis.com`
166+
167+
Google Gemini uses its own API format with `system_instruction`, `contents` (not `messages`), and `functionDeclarations` for tool calling. The provider handles the translation automatically.
168+
169+
### Groq
170+
171+
```go
172+
m := ai.New("groq",
173+
ai.WithAPIKey("your-key"),
174+
ai.WithModel("llama-3.3-70b-versatile"), // default
175+
)
176+
```
177+
178+
Default model: `llama-3.3-70b-versatile`
179+
Default base URL: `https://api.groq.com/openai`
180+
181+
Groq provides ultra-fast inference for open-weight models via an OpenAI-compatible endpoint.
182+
183+
### Mistral
184+
185+
```go
186+
m := ai.New("mistral",
187+
ai.WithAPIKey("your-key"),
188+
ai.WithModel("mistral-large-latest"), // default
189+
)
190+
```
191+
192+
Default model: `mistral-large-latest`
193+
Default base URL: `https://api.mistral.ai`
194+
195+
Mistral AI is a European AI company offering high-performance models via an OpenAI-compatible endpoint.
196+
197+
### Together AI
198+
199+
```go
200+
m := ai.New("together",
201+
ai.WithAPIKey("your-key"),
202+
ai.WithModel("meta-llama/Llama-3.3-70B-Instruct-Turbo"), // default
203+
)
204+
```
205+
206+
Default model: `meta-llama/Llama-3.3-70B-Instruct-Turbo`
207+
Default base URL: `https://api.together.xyz`
208+
209+
Together AI provides fast inference for open-weight models via an OpenAI-compatible endpoint.
210+
155211
### Atlas Cloud
156212

157213
```go

ai/gemini/gemini.go

Lines changed: 226 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,226 @@
1+
// Package gemini implements the Google Gemini model provider.
2+
//
3+
// Usage:
4+
//
5+
// import _ "go-micro.dev/v5/ai/gemini"
6+
//
7+
// m := ai.New("gemini",
8+
// ai.WithAPIKey("your-api-key"),
9+
// )
10+
package gemini
11+
12+
import (
13+
"bytes"
14+
"context"
15+
"encoding/json"
16+
"fmt"
17+
"io"
18+
"net/http"
19+
"strings"
20+
21+
"go-micro.dev/v5/ai"
22+
)
23+
24+
func init() {
25+
ai.Register("gemini", func(opts ...ai.Option) ai.Model {
26+
return NewProvider(opts...)
27+
})
28+
}
29+
30+
// Provider implements the ai.Model interface for Google Gemini.
31+
type Provider struct {
32+
opts ai.Options
33+
}
34+
35+
// NewProvider creates a new Gemini provider.
36+
func NewProvider(opts ...ai.Option) *Provider {
37+
options := ai.NewOptions(opts...)
38+
39+
if options.Model == "" {
40+
options.Model = "gemini-2.5-flash"
41+
}
42+
if options.BaseURL == "" {
43+
options.BaseURL = "https://generativelanguage.googleapis.com"
44+
}
45+
46+
return &Provider{opts: options}
47+
}
48+
49+
func (p *Provider) Init(opts ...ai.Option) error {
50+
for _, o := range opts {
51+
o(&p.opts)
52+
}
53+
return nil
54+
}
55+
56+
func (p *Provider) Options() ai.Options { return p.opts }
57+
func (p *Provider) String() string { return "gemini" }
58+
59+
func (p *Provider) Generate(ctx context.Context, req *ai.Request, opts ...ai.GenerateOption) (*ai.Response, error) {
60+
var tools []map[string]any
61+
for _, t := range req.Tools {
62+
tools = append(tools, map[string]any{
63+
"name": t.Name,
64+
"description": t.Description,
65+
"parameters": map[string]any{
66+
"type": "object",
67+
"properties": t.Properties,
68+
},
69+
})
70+
}
71+
72+
contents := []map[string]any{
73+
{"role": "user", "parts": []map[string]any{{"text": req.Prompt}}},
74+
}
75+
76+
apiReq := map[string]any{
77+
"contents": contents,
78+
}
79+
80+
if req.SystemPrompt != "" {
81+
apiReq["system_instruction"] = map[string]any{
82+
"parts": []map[string]any{{"text": req.SystemPrompt}},
83+
}
84+
}
85+
86+
if len(tools) > 0 {
87+
apiReq["tools"] = []map[string]any{
88+
{"functionDeclarations": tools},
89+
}
90+
}
91+
92+
resp, rawParts, err := p.callAPI(ctx, apiReq)
93+
if err != nil {
94+
return nil, err
95+
}
96+
97+
if len(resp.ToolCalls) == 0 {
98+
return resp, nil
99+
}
100+
101+
if p.opts.ToolHandler != nil {
102+
var resultParts []map[string]any
103+
for _, tc := range resp.ToolCalls {
104+
result, _ := p.opts.ToolHandler(tc.Name, tc.Input)
105+
resultParts = append(resultParts, map[string]any{
106+
"functionResponse": map[string]any{
107+
"name": tc.Name,
108+
"id": tc.ID,
109+
"response": result,
110+
},
111+
})
112+
}
113+
114+
followUpContents := append(contents,
115+
map[string]any{"role": "model", "parts": rawParts},
116+
map[string]any{"role": "user", "parts": resultParts},
117+
)
118+
119+
followUpReq := map[string]any{
120+
"contents": followUpContents,
121+
}
122+
if req.SystemPrompt != "" {
123+
followUpReq["system_instruction"] = map[string]any{
124+
"parts": []map[string]any{{"text": req.SystemPrompt}},
125+
}
126+
}
127+
128+
followUpResp, _, err := p.callAPI(ctx, followUpReq)
129+
if err == nil && followUpResp.Reply != "" {
130+
resp.Answer = followUpResp.Reply
131+
}
132+
}
133+
134+
return resp, nil
135+
}
136+
137+
func (p *Provider) Stream(ctx context.Context, req *ai.Request, opts ...ai.GenerateOption) (ai.Stream, error) {
138+
return nil, fmt.Errorf("streaming not yet implemented for gemini provider")
139+
}
140+
141+
func (p *Provider) callAPI(ctx context.Context, req map[string]any) (*ai.Response, []map[string]any, error) {
142+
reqBody, err := json.Marshal(req)
143+
if err != nil {
144+
return nil, nil, fmt.Errorf("failed to marshal request: %w", err)
145+
}
146+
147+
apiURL := strings.TrimRight(p.opts.BaseURL, "/") +
148+
"/v1beta/models/" + p.opts.Model + ":generateContent"
149+
150+
httpReq, err := http.NewRequestWithContext(ctx, "POST", apiURL, bytes.NewReader(reqBody))
151+
if err != nil {
152+
return nil, nil, fmt.Errorf("failed to create request: %w", err)
153+
}
154+
155+
httpReq.Header.Set("Content-Type", "application/json")
156+
httpReq.Header.Set("x-goog-api-key", p.opts.APIKey)
157+
158+
httpResp, err := http.DefaultClient.Do(httpReq)
159+
if err != nil {
160+
return nil, nil, fmt.Errorf("API request failed: %w", err)
161+
}
162+
defer httpResp.Body.Close()
163+
164+
respBody, _ := io.ReadAll(httpResp.Body)
165+
if httpResp.StatusCode != 200 {
166+
return nil, nil, fmt.Errorf("API error (%s): %s", httpResp.Status, string(respBody))
167+
}
168+
169+
var geminiResp struct {
170+
Candidates []struct {
171+
Content struct {
172+
Parts []struct {
173+
Text string `json:"text"`
174+
FunctionCall *functionCallPB `json:"functionCall"`
175+
} `json:"parts"`
176+
} `json:"content"`
177+
} `json:"candidates"`
178+
}
179+
180+
if err := json.Unmarshal(respBody, &geminiResp); err != nil {
181+
return nil, nil, fmt.Errorf("failed to parse response: %w", err)
182+
}
183+
184+
if len(geminiResp.Candidates) == 0 {
185+
return nil, nil, fmt.Errorf("no response from API")
186+
}
187+
188+
parts := geminiResp.Candidates[0].Content.Parts
189+
response := &ai.Response{}
190+
191+
var replyParts []string
192+
var rawParts []map[string]any
193+
194+
for _, part := range parts {
195+
if part.Text != "" {
196+
replyParts = append(replyParts, part.Text)
197+
rawParts = append(rawParts, map[string]any{"text": part.Text})
198+
}
199+
if part.FunctionCall != nil {
200+
response.ToolCalls = append(response.ToolCalls, ai.ToolCall{
201+
ID: part.FunctionCall.ID,
202+
Name: part.FunctionCall.Name,
203+
Input: part.FunctionCall.Args,
204+
})
205+
rawParts = append(rawParts, map[string]any{
206+
"functionCall": map[string]any{
207+
"id": part.FunctionCall.ID,
208+
"name": part.FunctionCall.Name,
209+
"args": part.FunctionCall.Args,
210+
},
211+
})
212+
}
213+
}
214+
215+
if len(replyParts) > 0 {
216+
response.Reply = strings.Join(replyParts, "\n")
217+
}
218+
219+
return response, rawParts, nil
220+
}
221+
222+
type functionCallPB struct {
223+
ID string `json:"id"`
224+
Name string `json:"name"`
225+
Args map[string]any `json:"args"`
226+
}

0 commit comments

Comments
 (0)