Skip to content

Commit cb55ce2

Browse files
JAORMXJAORMX
committed
RFC: ToolHive Core Shared Library with Package Graduation Criteria
Proposes creating toolhive-core, a shared Go library with explicit API stability guarantees for the ToolHive ecosystem. Establishes formal graduation criteria for promoting packages from internal to shared status. Key points: - Graduation criteria: stability, quality, dependencies, governance - Tier 1 packages: logger, errors, validation, healthcheck, versions, env - Based on actual exploration of toolhive, toolhive-registry-server, dockyard
1 parent 895f8eb commit cb55ce2

1 file changed

Lines changed: 353 additions & 0 deletions

File tree

‎rfcs/THV-0032-toolhive-core-shared-library.md‎

Lines changed: 353 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,353 @@
1+
# THV-0032: ToolHive Core Shared Library with Package Graduation Criteria
2+
3+
- **Status**: Draft
4+
- **Author(s)**: Juan Antonio Osorio (@JAORMX)
5+
- **Created**: 2026-01-29
6+
- **Last Updated**: 2026-01-29
7+
- **Target Repository**: multiple (new repository: `toolhive-core`)
8+
- **Related Issues**: N/A
9+
10+
## Summary
11+
12+
This RFC proposes creating a shared Go library (`toolhive-core`) to provide stable, well-tested utilities with explicit API guarantees for the ToolHive Go ecosystem. The proposal establishes formal graduation criteria for promoting packages from internal/unstable status to the shared library, enabling projects like `dockyard` (which already imports from toolhive) to depend on stable APIs without risk of unexpected breakage.
13+
14+
## Problem Statement
15+
16+
The ToolHive ecosystem spans multiple Go repositories:
17+
- **toolhive** - Core runtime, CLI, operator, proxy-runner, virtual MCP
18+
- **toolhive-registry** - Registry data and CLI tools
19+
- **toolhive-registry-server** - Registry API server
20+
- **dockyard** - Container packaging for MCP servers
21+
22+
Additionally, TypeScript projects exist (toolhive-studio, toolhive-cloud-ui) but are out of scope for a Go shared library.
23+
24+
**Current state of code sharing:**
25+
26+
| Project | Imports from toolhive | Pattern Divergence |
27+
|---------|----------------------|-------------------|
28+
| **dockyard** | `pkg/logger`, `pkg/container/images`, `pkg/runner` | Aligned - uses toolhive utilities |
29+
| **toolhive-registry** | `pkg/logger`, `pkg/permissions`, `pkg/registry/*`, `pkg/container/verifier` | Aligned - heavily uses toolhive utilities |
30+
| **toolhive-registry-server** | Only `pkg/versions` + `pkg/registry` types | Uses stdlib `log/slog` instead of toolhive's zap logger; custom error handling |
31+
32+
**Current limitations:**
33+
34+
1. **No stability guarantees**: Projects like dockyard import toolhive's `pkg/` packages directly, but these are internal packages with no API stability commitment. A breaking change in toolhive could break dockyard unexpectedly.
35+
36+
2. **Pattern divergence without clear rationale**: toolhive-registry-server uses Go stdlib `log/slog` while toolhive uses zap-based logging. Both are valid choices, but there's no documented decision about when to use which approach.
37+
38+
3. **Unclear package maturity**: The `toolhive` repository contains 93+ packages in `pkg/`, with `logger` imported 185 times internally. Contributors and downstream consumers have no way to know which packages are stable vs. experimental.
39+
40+
4. **Friction for new Go projects**: When creating new ToolHive ecosystem projects, developers must decide whether to import from toolhive (risking breakage) or reimplement utilities (causing divergence).
41+
42+
5. **No graduation path**: There's no process for promoting battle-tested internal packages to "blessed" shared status with stability guarantees.
43+
44+
## Goals
45+
46+
- **Establish `toolhive-core`**: Create a new repository for shared, stable Go packages with explicit API stability guarantees
47+
- **Define graduation criteria**: Formalize how packages move from internal/unstable to shared/stable
48+
- **Provide stability guarantees**: Implement semantic versioning so downstream projects (dockyard, future services) can depend on toolhive-core without fear of breakage
49+
- **Document architectural decisions**: Clarify when to use toolhive-core utilities vs. stdlib alternatives (e.g., zap vs. slog)
50+
- **Enable independent versioning**: Allow the shared library to evolve on its own release cadence, decoupled from toolhive releases
51+
- **Create a graduation path**: Provide a clear process for promoting mature internal packages to shared status
52+
53+
## Non-Goals
54+
55+
- **Migrate all packages immediately**: Only graduated packages move to `toolhive-core`
56+
- **Break existing APIs**: The initial release must maintain compatibility with current usage
57+
- **Force adoption**: Projects may continue using internal implementations (e.g., toolhive-registry-server can keep using slog)
58+
- **Include domain-specific code**: Container runtime, MCP protocol, VMCP logic stay in their respective repos
59+
- **Create a monorepo**: `toolhive-core` is a standalone library, not a consolidation of all projects
60+
- **TypeScript/frontend utilities**: toolhive-studio and toolhive-cloud-ui are TypeScript projects and out of scope
61+
62+
## Proposed Solution
63+
64+
### High-Level Design
65+
66+
```mermaid
67+
flowchart TB
68+
subgraph "Package Lifecycle"
69+
Internal[Internal Package<br/>pkg/foo in toolhive]
70+
Candidate[Graduation Candidate<br/>Meets criteria]
71+
Core[toolhive-core/foo<br/>Stable, versioned]
72+
73+
Internal -->|"Meets graduation criteria"| Candidate
74+
Candidate -->|"RFC + Review"| Core
75+
end
76+
77+
subgraph "Go Consumers"
78+
TH[toolhive]
79+
RS[toolhive-registry-server]
80+
DY[dockyard]
81+
Future[future Go services]
82+
end
83+
84+
Core --> TH
85+
Core --> RS
86+
Core --> DY
87+
Core --> Future
88+
89+
style Core fill:#81c784
90+
style Candidate fill:#ffb74d
91+
style Internal fill:#90caf9
92+
```
93+
94+
### Package Graduation Criteria
95+
96+
Packages can graduate via two tracks depending on complexity:
97+
98+
#### Fast Track (Simple Packages)
99+
100+
For small, focused packages with minimal dependencies (e.g., `env`, `errors`, `validation`):
101+
102+
| Criterion | Requirement |
103+
|-----------|-------------|
104+
| **Production usage** | Deployed in production for ≥1 month |
105+
| **No internal dependencies** | Cannot depend on non-graduated internal packages |
106+
| **No global state** | No singletons, global variables for state, or `init()` side effects |
107+
| **Test coverage** | ≥70% line coverage |
108+
| **Documentation** | Package-level godoc |
109+
| **Approval** | GitHub issue approved by one maintainer |
110+
111+
#### Standard Track (Complex Packages)
112+
113+
For packages with external dependencies, multiple types, or broader API surface:
114+
115+
| Criterion | Requirement |
116+
|-----------|-------------|
117+
| **Production usage** | Deployed in production for ≥2 months without breaking changes |
118+
| **API stability** | No breaking changes in the last 2 minor releases |
119+
| **Interface design** | Uses Go interfaces for dependency injection and testability |
120+
| **Error handling** | Returns typed errors; no panics except for programming bugs |
121+
| **No global state** | No singletons, global variables for state, or `init()` side effects |
122+
| **Test coverage** | ≥70% line coverage with meaningful assertions |
123+
| **Documentation** | Package-level godoc with usage examples |
124+
| **Linting** | Passes `golangci-lint` with project configuration |
125+
| **Minimal dependencies** | Only essential external dependencies |
126+
| **No circular imports** | Must not create import cycles when extracted |
127+
| **No internal dependencies** | Cannot depend on non-graduated internal packages |
128+
| **Stable external deps** | External dependencies must be v1.0+ or widely adopted |
129+
| **Sponsorship** | At least one maintainer sponsors the graduation |
130+
| **Approval** | RFC or detailed GitHub issue reviewed and approved |
131+
132+
### Graduation Process
133+
134+
1. **Proposal**: Open a GitHub issue identifying the graduation candidate and proposed track (fast/standard)
135+
2. **Track determination**: Maintainers confirm which track applies based on package complexity
136+
3. **Evaluation**: Assess against the relevant track's criteria
137+
4. **Approval**: Fast track requires one maintainer approval; standard track requires RFC or detailed issue review
138+
5. **Extraction**: Move package to `toolhive-core` with necessary adaptations
139+
6. **Release**: Tag a new semver release of `toolhive-core`
140+
7. **Migration**: Update consuming projects to import from `toolhive-core`
141+
142+
### Stability Levels
143+
144+
Each package in `toolhive-core` is marked with a stability level in its godoc:
145+
146+
| Level | Meaning | API Guarantees |
147+
|-------|---------|----------------|
148+
| **Stable** | Production-ready, fully supported | No breaking changes without major version bump |
149+
| **Beta** | Feature-complete, may have minor changes | Breaking changes possible with deprecation notice |
150+
| **Alpha** | Experimental, subject to significant changes | No stability guarantees |
151+
152+
### Versioning Strategy
153+
154+
`toolhive-core` follows [Semantic Versioning 2.0.0](https://semver.org/):
155+
156+
- **Major (vX.0.0)**: Breaking API changes
157+
- **Minor (v0.X.0)**: New features, backward-compatible
158+
- **Patch (v0.0.X)**: Bug fixes, backward-compatible
159+
160+
**Release cadence**:
161+
- Patch releases: As needed for critical fixes
162+
- Minor releases: Monthly or when significant features graduate
163+
- Major releases: Rare, only when breaking changes are necessary
164+
165+
### Initial Package Tiers
166+
167+
Based on analysis of the `toolhive` codebase:
168+
169+
#### Tier 1: Immediate Graduation Candidates
170+
171+
These packages meet all graduation criteria (zero ToolHive-specific coupling, well-tested, minimal dependencies) and should be included in the initial `toolhive-core` release:
172+
173+
| Package | Current Location | Stability | Rationale |
174+
|---------|------------------|-----------|-----------|
175+
| **errors** | `pkg/errors/` | Beta | HTTP-aware error handling, zero deps, 10mo stable |
176+
| **oauth** | `pkg/oauth/` | Beta | RFC-compliant OAuth/OIDC types, fosite dep only |
177+
| **env** | `pkg/env/` | Beta | Testable environment access, zero deps |
178+
| **permissions** | `pkg/permissions/` | Beta | Container permission profiles, stdlib-only, security validations |
179+
| **validation** | `pkg/validation/` | Beta | RFC 7230 HTTP header validation, security-focused |
180+
| **versions** | `pkg/versions/` | Beta | Build metadata, User-Agent generation |
181+
182+
> **Note**: All packages start as Beta in v0.x releases. Once the library reaches v1.0.0, packages meeting all graduation criteria will be promoted to Stable.
183+
184+
#### Tier 2: Minor Refactoring Required
185+
186+
These packages have solid implementations but need minor changes to remove ToolHive-specific coupling:
187+
188+
| Package | Current Location | Status | Required Changes |
189+
|---------|------------------|--------|------------------|
190+
| **healthcheck** | `pkg/healthcheck/` | Beta | Remove unused logger import |
191+
| **logger** | `pkg/logger/` | Beta | Remove `viper.GetBool("debug")` coupling; accept config via parameters |
192+
| **auth/tokenexchange** | `pkg/auth/tokenexchange/` | Beta | Split core RFC 8693 logic from ToolHive middleware |
193+
| **networking** | `pkg/networking/` | Beta | Replace ToolHive logger with slog or interface |
194+
| **lockfile** | `pkg/lockfile/` | Beta | Accept logger interface instead of importing pkg/logger |
195+
196+
#### Tier 3: Partial Extraction or Future Work
197+
198+
These packages have useful components but require significant refactoring or are too coupled:
199+
200+
| Package | Current Location | Status | Notes |
201+
|---------|------------------|--------|-------|
202+
| **labels** | `pkg/labels/` | Beta | Extract K8s validation funcs only; keep ToolHive constants |
203+
| **secrets** | `pkg/secrets/` | Beta | Multiple providers, needs interface review |
204+
| **transport/types** | `pkg/transport/types/` | Beta | Middleware abstraction, widely used but coupled |
205+
| **mcp** | `pkg/mcp/` | Beta | MCP primitives, protocol-specific |
206+
207+
#### Not Recommended for Extraction
208+
209+
These packages are too coupled to ToolHive internals:
210+
211+
| Package | Reason |
212+
|---------|--------|
213+
| **auth** (heavy pieces) | Product-level wiring: token.go, oauth/*, remote/*, secrets/* |
214+
| **runner**, **workloads** | Deep orchestration coupling, RunConfig migrations |
215+
| **config** | Viper global state, ToolHive-specific env vars |
216+
| **ignore** | Hardcoded `.thvignore`, ToolHive paths |
217+
| **process** | Hardcoded paths, imports pkg/container/runtime |
218+
219+
## Security Considerations
220+
221+
### Threat Model
222+
223+
- **Supply chain attacks**: Malicious code injected into shared library affects all consumers
224+
- **Dependency confusion**: Typosquatting or namespace confusion attacks
225+
- **Credential exposure**: Logging or error handling inadvertently exposing secrets
226+
227+
### Authentication and Authorization
228+
229+
- No direct auth changes; auth packages remain in Tier 2/3 until fully reviewed
230+
- Logger and error packages must not log sensitive data by default
231+
232+
### Data Security
233+
234+
- Validation package must reject inputs that could cause injection attacks
235+
- Error wrapping must not expose internal implementation details in error messages
236+
237+
### Input Validation
238+
239+
- The `validation` package provides security-focused validation for:
240+
- HTTP headers (preventing header injection)
241+
- Resource URIs (preventing path traversal)
242+
- Group names (preventing null byte injection)
243+
244+
### Secrets Management
245+
246+
- Logger package must provide redaction capabilities for sensitive fields
247+
- Error messages must not include credential values
248+
249+
### Audit and Logging
250+
251+
- All package releases are tagged and signed
252+
- CHANGELOG documents all changes
253+
- Security advisories published via GitHub Security Advisories
254+
255+
### Mitigations
256+
257+
| Threat | Mitigation |
258+
|--------|------------|
259+
| Supply chain attacks | Signed releases, dependency review, SLSA provenance |
260+
| Dependency confusion | Use `github.com/stacklok/` namespace, Go module proxy |
261+
| Credential exposure | Redaction functions, sensitive field markers |
262+
263+
## Compatibility
264+
265+
### Backward Compatibility
266+
267+
- **Initial release**: API-compatible with existing `toolhive/pkg/*` packages
268+
- **Migration path**:
269+
1. Add `toolhive-core` dependency
270+
2. Update imports (can be automated with `gofmt -r`)
271+
3. Remove now-unused internal packages
272+
- **Deprecation**: Internal packages deprecated but kept for 2 minor versions (serves as rollback window if issues arise)
273+
274+
### Forward Compatibility
275+
276+
- **Extensibility**: Interfaces allow adding implementations without breaking changes
277+
- **Optional features**: New features use functional options pattern
278+
- **Version constraints**: Consumers can pin to major versions for stability
279+
280+
## Testing Strategy
281+
282+
### Unit Tests
283+
284+
- All packages must have ≥70% coverage
285+
- Table-driven tests for validation functions
286+
- Mock-based tests for interface implementations
287+
288+
### Integration Tests
289+
290+
- Cross-package integration tests (e.g., validation + errors)
291+
- Import cycle detection in CI
292+
293+
### Compatibility Tests
294+
295+
- Build `toolhive` and `dockyard` against new versions
296+
- Automated compatibility matrix testing
297+
298+
### Performance Tests
299+
300+
- Benchmark validation functions
301+
- Memory allocation tests for hot paths
302+
303+
### Security Tests
304+
305+
- Static analysis with `gosec`
306+
- Dependency vulnerability scanning with `govulncheck`
307+
308+
## Documentation
309+
310+
- **README.md**: Quick start, installation, package overview
311+
- **STABILITY.md**: Graduation criteria, stability levels explained
312+
- **CHANGELOG.md**: All changes in keep-a-changelog format
313+
- **Package godoc**: Usage examples, API documentation
314+
- **Migration guide**: How to migrate from internal packages
315+
- **Architecture docs**: Updated to reference `toolhive-core`
316+
317+
## Open Questions
318+
319+
1. **Release cadence**: Monthly minor releases, or release when ready?
320+
2. **Tier 2 timeline**: When should we start refactoring healthcheck/tokenexchange for graduation?
321+
3. **Logger approach**: When decoupling logger from Viper, should we use zap with config injection, switch to slog (stdlib), or provide an interface that supports both?
322+
4. **permissions package scope**: Is `pkg/permissions` (container security profiles) generic enough for a shared library, or is it too domain-specific to ToolHive?
323+
324+
## References
325+
326+
- [Go Modules Reference](https://go.dev/ref/mod)
327+
- [Semantic Versioning 2.0.0](https://semver.org/)
328+
- [Keep a Changelog](https://keepachangelog.com/)
329+
- [Go API Stability](https://go.dev/doc/modules/version-numbers)
330+
- [SLSA Supply Chain Security](https://slsa.dev/)
331+
- [ToolHive Architecture Documentation](https://github.com/stacklok/toolhive/tree/main/docs/arch)
332+
333+
---
334+
335+
## RFC Lifecycle
336+
337+
<!-- This section is maintained by RFC reviewers -->
338+
339+
### Review History
340+
341+
| Date | Reviewer | Decision | Notes |
342+
|------|----------|----------|-------|
343+
| 2026-01-29 | @JAORMX | Draft | Initial submission |
344+
345+
### Implementation Tracking
346+
347+
| Repository | PR | Status |
348+
|------------|-----|--------|
349+
| toolhive-core | N/A | Not started |
350+
| toolhive | N/A | Not started |
351+
| dockyard | N/A | Not started |
352+
| toolhive-registry | N/A | Not started |
353+
| toolhive-registry-server | N/A | Optional - may continue using slog |

0 commit comments

Comments
 (0)