Skip to content

Commit 4918653

Browse files
SniderSnider
committed
docs(go): flesh out ai + ml + mlx, promote to content sidebar
ai - Facade purpose + multi-package repo layout (ai/ai, ai/mcp, ai/lab) - ProviderRouter with declarative routes + tag-based selection - QueryRAGForTask helper for retrieval-augmented prompts - MCP transport stack: stdio / TCP / Unix domain socket - JSONL metrics — append-only, no external dep - RegisterMetricsCommand example ml - Three pluggable backends: InferenceAdapter (Metal via go-mlx), LlamaBackend (managed llama-server subprocess), HTTPBackend (OpenAI-compatible) - 23 capability probes with RunCapabilityProbes + Full variant - Content probes for prose/summary/translation quality - Judge + InfluxClient pipeline — ScoreCapabilityAndPush / ScoreContentAndPush write to InfluxDB + DuckDB - Agent orchestrator running eval over SSH fleet - InferenceAdapter bridge to go/inference TextModel mlx - Apple Silicon Metal backend via mlx-c CGO - Two API shapes — inference contract (blank import) vs direct mlx.LoadModel for advanced features - LoadModelFromMedium for remote-storage-backed models - Sub-package map: adapter / compute / lora / agent / gguf / safetensors / pack / merge / kv / memory / eval / bundle / probe / chat / mlxlm / openai — table of what each covers - Three training surfaces: SFT, LoRA, GRPO, Distillation - Checkpoint metadata round-trip pattern - Frame compute API beyond LLMs — Session, kernels, CRT filter - mlxlm Python subprocess CGO-free fallback Sidebar - Promoted ai + ml + mlx into "Packages — content" — total now 17 - Removed all three from "Packages — index" Co-Authored-By: Virgil <virgil@lethean.io>
1 parent ff8bebd commit 4918653

4 files changed

Lines changed: 389 additions & 19 deletions

File tree

‎astro.config.mjs‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,9 @@ export default defineConfig({
6363
{ label: 'forge', slug: 'go/forge' },
6464
{ label: 'orm', slug: 'go/orm' },
6565
{ label: 'inference', slug: 'go/inference' },
66+
{ label: 'ai', slug: 'go/ai' },
67+
{ label: 'ml', slug: 'go/ml' },
68+
{ label: 'mlx', slug: 'go/mlx' },
6669
{ label: 'agent', slug: 'go/agent' },
6770
{ label: 'mcp', slug: 'go/mcp' },
6871
{ label: 'miner', slug: 'go/miner' },
@@ -79,7 +82,6 @@ export default defineConfig({
7982
label: 'Packages — index',
8083
collapsed: true,
8184
items: [
82-
{ label: 'ai', slug: 'go/ai' },
8385
{ label: 'ansible', slug: 'go/ansible' },
8486
{ label: 'api', slug: 'go/api' },
8587
{ label: 'build', slug: 'go/build' },
@@ -96,8 +98,6 @@ export default defineConfig({
9698
{ label: 'i18n', slug: 'go/i18n' },
9799
{ label: 'ide', slug: 'go/ide' },
98100
{ label: 'log', slug: 'go/log' },
99-
{ label: 'ml', slug: 'go/ml' },
100-
{ label: 'mlx', slug: 'go/mlx' },
101101
{ label: 'netops', slug: 'go/netops' },
102102
{ label: 'p2p', slug: 'go/p2p' },
103103
{ label: 'rag', slug: 'go/rag' },

‎src/content/docs/go/ai.mdx‎

Lines changed: 105 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: ai
3-
description: AI runtime primitives — model adapters, lab, mcp transport.
3+
description: Unified AI surface — facade, provider router, JSONL metrics, MCP transports.
44
head:
55
- tag: meta
66
attrs:
@@ -12,7 +12,13 @@ head:
1212
content: 'dappco.re/go/ai https://github.com/dappcore/go-ai https://github.com/dappcore/go-ai/tree/main{/dir} https://github.com/dappcore/go-ai/blob/main{/dir}/{file}#L{line}'
1313
---
1414

15-
AI runtime primitives — model adapters, lab, mcp transport. Built on the [`core/go`](/go/) primitives.
15+
The thin AI facade for everything in the Core ecosystem that needs a
16+
language model. Sits on top of [`go/inference`](/go/inference/) (local
17+
backends) and remote providers (OpenAI-compatible HTTP, Anthropic,
18+
Ollama), then layers in JSONL metrics, RAG query helpers, and an MCP
19+
transport stack so the same call site talks to whichever shape is closest.
20+
Built on [`core/go`](/go/) — every constructor returns a
21+
[`Result`](/go/result/).
1622

1723
## Install
1824

@@ -23,10 +29,103 @@ go get dappco.re/go/ai@latest
2329
## Import
2430

2531
```go
26-
import "dappco.re/go/ai"
32+
import "dappco.re/go/ai/ai" // facade package — common case
2733
```
2834

29-
## Status
35+
The repo is multi-package: `ai/ai/` is the facade, `ai/mcp/` is the MCP
36+
transport stack, `ai/lab/` is the local lab dashboard, and so on.
3037

31-
Page in flight — content fills in as the package converges. Source of truth
32-
today is the README and inline docstrings in the repo.
38+
## Provider router — one call, many backends
39+
40+
`ProviderRouter` is the load-balanced front for a heterogeneous fleet —
41+
local Metal model alongside a managed llama-server subprocess alongside
42+
an OpenAI-compatible HTTP endpoint. Routes are declarative; the router
43+
picks based on declared capability + health:
44+
45+
```go
46+
r := ai.NewProviderRouter(
47+
ai.ProviderRoute{
48+
Name: "local-metal",
49+
Provider: ai.ProviderInference, // go/inference Backend
50+
Model: "gemma-4-e2b",
51+
Tags: []string{"fast", "private"},
52+
},
53+
ai.ProviderRoute{
54+
Name: "remote-claude",
55+
Provider: ai.ProviderAnthropic,
56+
Model: "claude-opus-4-7",
57+
Tags: []string{"frontier"},
58+
},
59+
)
60+
if !r.OK { return r }
61+
router := r.Value.(*ai.ProviderRouter)
62+
63+
resp, _ := router.Chat(ctx, ai.ChatRequest{
64+
Tags: []string{"frontier"}, // pick remote-claude
65+
Messages: []ai.Message{{Role: "user", Content: "hello"}},
66+
})
67+
```
68+
69+
`NewProviderRouterWithOptions` exposes timeout, retry, fallback chain,
70+
and metrics-sink knobs for production deployments where one route
71+
flapping shouldn't take the whole router down.
72+
73+
## RAG query helper
74+
75+
`QueryRAGForTask` is the canonical entry for retrieval-augmented prompts
76+
— give it a `TaskInfo` (title, description, optional code context) and
77+
get back the assembled context text + provenance to splice into the
78+
prompt:
79+
80+
```go
81+
r := ai.QueryRAGForTask(ai.TaskInfo{
82+
Title: "Investigate build failure",
83+
Description: "CI compile step fails on go-store after duckdb bump",
84+
Sources: []string{"recent-commits", "rfc-windows-compile"},
85+
})
86+
if !r.OK { return r }
87+
ctx := r.Value.(ai.RAGContext)
88+
89+
prompt := ctx.Render() + "\n\n## Question\n" + question
90+
```
91+
92+
## MCP transport stack
93+
94+
The `ai/mcp/` sub-package implements the Model Context Protocol with three
95+
transport flavours so a Core can host MCP tools over whatever shape the
96+
client supports:
97+
98+
| Transport | Use case |
99+
|---|---|
100+
| `mcp/transport_stdio.go` | Sub-process MCP servers (Claude Desktop pattern) |
101+
| `mcp/transport_tcp.go` | Remote MCP servers over plain or TLS sockets |
102+
| `mcp/transport_unix.go` | Same-host MCP via Unix domain socket |
103+
104+
Each transport exposes the same `Server` / `Client` shapes so consumer
105+
code is transport-agnostic. The `tools_core.go` + `tools_external.go`
106+
split lets a Core register both its own tool surface and any external
107+
tool sets in one MCP server.
108+
109+
## JSONL metrics
110+
111+
Token counts, latency, cost, and routing decisions stream to a JSONL
112+
file at a configurable path — one line per inference call, append-only,
113+
no external dependency. The CLI command `ai metrics` consumes the same
114+
file to render daily roll-ups:
115+
116+
```go
117+
ai.RegisterMetricsCommand(c, "/var/log/ai/metrics.jsonl")
118+
```
119+
120+
## Sibling packages
121+
122+
- [`go/inference`](/go/inference/) — the local-backend contract `ai`
123+
routes through for on-host models
124+
- [`go/mlx`](/go/mlx/) — Metal backend that `inference` exposes to `ai`
125+
- [`go/rag`](/go/rag/) — the RAG corpus + retrieval `QueryRAGForTask` consumes
126+
- [`go/agent`](/go/agent/) — the agentic runtime that drives `ai` for
127+
dispatched coding tasks
128+
129+
## Source
130+
131+
[`github.com/dappcore/go-ai`](https://github.com/dappcore/go-ai) — full source, the facade + MCP + lab + CLI commands.

‎src/content/docs/go/ml.mdx‎

Lines changed: 115 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: ml
3-
description: Machine-learning runtime — workers, types, training adapters.
3+
description: Multi-suite scoring engine + agent orchestrator for the Lethean AI stack.
44
head:
55
- tag: meta
66
attrs:
@@ -12,7 +12,13 @@ head:
1212
content: 'dappco.re/go/ml https://github.com/dappcore/go-ml https://github.com/dappcore/go-ml/tree/main{/dir} https://github.com/dappcore/go-ml/blob/main{/dir}/{file}#L{line}'
1313
---
1414

15-
Machine-learning runtime — workers, types, training adapters. Built on the [`core/go`](/go/) primitives.
15+
The orchestration + evaluation layer above [`go/inference`](/go/inference/).
16+
Pluggable backends (Apple Metal via [`go/mlx`](/go/mlx/), managed
17+
llama-server subprocesses, OpenAI-compatible HTTP), a concurrent scoring
18+
engine that grades model outputs across heuristic / semantic / content /
19+
standard-benchmark suites, 23 capability probes, GGUF model management,
20+
and an SSH-based agent orchestrator that streams checkpoint evaluations
21+
to InfluxDB + DuckDB.
1622

1723
## Install
1824

@@ -26,7 +32,111 @@ go get dappco.re/go/ml@latest
2632
import "dappco.re/go/ml"
2733
```
2834

29-
## Status
35+
## Backends
3036

31-
Page in flight — content fills in as the package converges. Source of truth
32-
today is the README and inline docstrings in the repo.
37+
Three pluggable backend shapes implement the shared `Backend` contract —
38+
pick the one closest to where your model lives:
39+
40+
```go
41+
// Apple Silicon, Metal
42+
adapter := ml.NewInferenceAdapter(metalModel, "gemma-4-e2b")
43+
44+
// Managed llama-server subprocess
45+
llama := ml.NewLlamaBackend("./model.gguf", ml.WithContextLen(8192))
46+
47+
// Any OpenAI-compatible HTTP endpoint (Ollama, vLLM, hosted APIs)
48+
http := ml.NewHTTPBackend("http://localhost:11434", "qwen3-8b")
49+
```
50+
51+
Each backend exposes the same `TextModel` shape — `NewLlamaTextModel`
52+
and `NewHTTPTextModel` wrap them in the `go/inference` `TextModel`
53+
interface so they slot directly into the rest of the stack.
54+
55+
## Capability probes
56+
57+
23 standardised probes that measure what a model can actually do — tool
58+
use, structured output, multi-turn coherence, refusal calibration, code
59+
synthesis, etc.:
60+
61+
```go
62+
result := ml.RunCapabilityProbes(ctx, backend)
63+
fmt.Printf("Score: %.2f (passed %d of %d probes)\n",
64+
result.Score, result.Passed, result.Total)
65+
66+
// Full variant emits per-probe response + lets a callback observe each
67+
// step (useful for live UIs during long runs):
68+
result, responses := ml.RunCapabilityProbesFull(ctx, backend, func(p ml.Probe, r ml.CapResponseEntry) {
69+
log.Printf("[%s] %s — %s", p.ID, p.Title, r.Outcome)
70+
})
71+
```
72+
73+
The companion `RunContentProbes` covers content-quality dimensions
74+
(prose, summary, translation, structured rewrite) on the same shape.
75+
76+
## Scoring + persistence
77+
78+
Probe responses become checkpoint scores via the Judge — a separate
79+
model graded against rubrics. Results stream to InfluxDB for time-series
80+
analysis and DuckDB for cross-checkpoint joins:
81+
82+
```go
83+
judge := ml.NewJudge(ml.JudgeConfig{
84+
Backend: ml.NewHTTPBackend("...", "claude-opus-4-7"),
85+
Rubric: "rubric/v1.yaml",
86+
})
87+
influx := ml.NewInfluxClient(...)
88+
89+
ml.ScoreCapabilityAndPush(ctx, judge, influx, checkpoint, responses)
90+
ml.ScoreContentAndPush(ctx, judge, influx, checkpoint, runID, contentResponses)
91+
```
92+
93+
The DuckDB tables `checkpoint_scores` and `probe_results` come from
94+
[`go/store`](/go/store/) so any consumer with the Core can join scoring
95+
data against arbitrary local data.
96+
97+
## Agent orchestrator
98+
99+
`Agent` runs the eval loop end-to-end across a fleet of remote workers
100+
over SSH — fetch a checkpoint, run probes locally on the worker, ship
101+
responses back, score, persist, repeat:
102+
103+
```go
104+
agent := ml.NewAgent(&ml.AgentConfig{
105+
Fleet: []ml.WorkerSpec{ /* SSH targets + GPU specs */ },
106+
Backends: []ml.BackendSpec{ /* per-worker backend assignments */ },
107+
Cadence: 10 * time.Minute,
108+
OnReport: func(report ml.Report) { /* update dashboard */ },
109+
})
110+
111+
ml.RunAgentLoop(agent.Config())
112+
```
113+
114+
The orchestrator multiplexes the SSH transport so one local process can
115+
drive dozens of workers without per-host shell juggling.
116+
117+
## Adapters
118+
119+
`InferenceAdapter` is the bridge that turns a `go/inference` TextModel
120+
into an `ml.Backend` — useful when you want to point the ml scoring
121+
engine at any model registered through inference:
122+
123+
```go
124+
ir := inference.LoadModel(path, inference.WithBackend("metal"))
125+
model := ir.Value.(inference.TextModel)
126+
backend := ml.NewInferenceAdapter(model, "gemma-4-e2b")
127+
128+
result := ml.RunCapabilityProbes(ctx, backend)
129+
```
130+
131+
## Sibling packages
132+
133+
- [`go/inference`](/go/inference/) — the local-backend contract `ml`
134+
adapts via `NewInferenceAdapter`
135+
- [`go/mlx`](/go/mlx/) — Apple Silicon backend, the Metal default
136+
- [`go/ai`](/go/ai/) — facade above ml when consumers want chat
137+
ergonomics rather than scoring infrastructure
138+
- [`go/store`](/go/store/) — DuckDB scoring tables `ml.Score*AndPush` writes to
139+
140+
## Source
141+
142+
[`github.com/dappcore/go-ml`](https://github.com/dappcore/go-ml) — full source, the 23 probes, the scoring engine, the SSH agent orchestrator.

0 commit comments

Comments
 (0)