Skip to content

Commit dc8d0bd

Browse files
coreydaleyclaude
andcommitted
refactor(claude): convert provider-compat from command to project-local skill
Replaces .claude/commands/provider-compat.md with a proper skill under .claude/skills/provider-compat/. Adds provider_compat.py which infers claimed scope by parsing Go source + server/server.go route registrations, scaffolds the compat_test.go header, and lists the provider registry. The skill delegates large spec payloads to the spec-analyst subagent. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 060121f commit dc8d0bd

4 files changed

Lines changed: 360 additions & 106 deletions

File tree

.claude/commands/provider-compat.md

Lines changed: 0 additions & 105 deletions
This file was deleted.

.claude/settings.local.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,9 @@
22
"permissions": {
33
"allow": [
44
"Bash(rm -f /Users/corey/Code/github.com/coreydaley/messagepit/main/.git/index.lock)",
5-
"Bash(git stash *)"
5+
"Bash(git stash *)",
6+
"Bash(rm /Users/corey/Code/github.com/coreydaley/messagepit/main/.claude/commands/provider-compat.md)",
7+
"Bash(mkdir -p /Users/corey/Code/github.com/coreydaley/messagepit/main/.claude/skills/provider-compat/scripts)"
68
]
79
}
810
}
Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
---
2+
name: provider-compat
3+
description: Audit MessagePit provider stub fidelity against the real provider API spec. Infers claimed scope from Go source, fetches the real spec, generates compat_test.go, runs tests, and reports a fidelity score.
4+
triggers:
5+
- /provider-compat
6+
allowed-tools: Bash, Read, Write, Edit, Agent, WebFetch, WebSearch
7+
---
8+
9+
# /provider-compat
10+
11+
Audit how closely the MessagePit stub for `$ARGUMENTS` matches the real provider API.
12+
13+
## Script
14+
15+
```bash
16+
SKILL_DIR=$(find . -path "*/.claude/skills/provider-compat/scripts/provider_compat.py" | head -1)
17+
```
18+
19+
All heavy lifting goes through the script. Run from the project root.
20+
21+
## Workflow
22+
23+
### 1. Resolve Provider
24+
25+
```bash
26+
python3 "$SKILL_DIR" --providers
27+
```
28+
29+
Match `$ARGUMENTS` (lowercase, trimmed) against the registry. If not found, print the registry and exit.
30+
31+
### 2. Infer Claimed Scope
32+
33+
```bash
34+
python3 "$SKILL_DIR" --scope <name>
35+
```
36+
37+
This reads all non-test `.go` files in the provider package and returns JSON:
38+
39+
```json
40+
{
41+
"provider": "sendgrid",
42+
"package": "internal/sendgrid",
43+
"port": 8101,
44+
"spec_url": "https://...",
45+
"scope": {
46+
"routes": [{"path": "/v3/mail/send", "methods": ["POST"]}],
47+
"auth": "Bearer token",
48+
"request_fields": ["content", "from", "personalizations", "subject", ...],
49+
"status_codes": [202, 400, 401, 413, 500]
50+
}
51+
}
52+
```
53+
54+
Read this output — it is the **Claimed Scope** for the rest of the workflow.
55+
56+
### 3. Fetch the Real Spec
57+
58+
Delegate to the `spec-analyst` subagent (`.claude/agents/spec-analyst.md`), passing:
59+
- The `spec_url` from step 2
60+
- The full claimed scope JSON
61+
62+
The subagent returns a **Spec Snapshot** — the real API's contract for the claimed endpoints, distilled to the fields and status codes that matter.
63+
64+
If the `spec-analyst` agent is unavailable, use `WebFetch` inline and extract the relevant paths yourself.
65+
66+
### 4. Generate `compat_test.go`
67+
68+
Get the file header:
69+
70+
```bash
71+
python3 "$SKILL_DIR" --write-header <name>
72+
```
73+
74+
Write `internal/<name>/compat_test.go`**fully regenerated on every run**.
75+
76+
Structure:
77+
78+
```go
79+
// Code generated by /provider-compat. DO NOT EDIT.
80+
// Re-run: /provider-compat <name>
81+
82+
package <name>_test
83+
84+
import (
85+
"net/http"
86+
"net/http/httptest"
87+
"strings"
88+
"testing"
89+
90+
"github.com/coreydaley/messagepit/internal/<name>"
91+
"github.com/coreydaley/messagepit/internal/storage"
92+
"github.com/coreydaley/messagepit/logger"
93+
)
94+
95+
func init() {
96+
logger.NoLogging = true
97+
}
98+
99+
func setupCompat(t *testing.T) {
100+
t.Helper()
101+
storage.InitDB()
102+
t.Cleanup(storage.Close)
103+
}
104+
```
105+
106+
Then one `TestCompat_*` function per claimed endpoint. Each test:
107+
108+
1. Calls `setupCompat(t)`
109+
2. Builds the exact request the real spec requires (correct method, path, auth header, content-type, body with required fields)
110+
3. Posts it to `httptest.NewRecorder()` + the handler directly
111+
4. Asserts the success status code matches the spec
112+
5. Has a sub-test (`t.Run`) for each expected error case (missing auth → 401, malformed body → 400, etc.)
113+
114+
Name tests `TestCompat_<METHOD>_<PathSuffix>` (e.g., `TestCompat_POST_MailSend`).
115+
116+
Do **not** duplicate assertions already in `<name>_test.go`. Compat tests cover spec conformance; behavioral edge cases stay human-owned.
117+
118+
### 5. Run the Tests
119+
120+
```bash
121+
go test ./internal/<name>/... -run TestCompat_ -v 2>&1
122+
```
123+
124+
Capture output. Parse pass/fail per test.
125+
126+
### 6. Report
127+
128+
Print a fidelity report:
129+
130+
```
131+
=== Provider Compat Report: <name> ===
132+
133+
Claimed scope
134+
Endpoints : GET /path, POST /path ...
135+
Auth : Bearer token
136+
Fields : N request fields
137+
Statuses : 200, 400, 401 ...
138+
139+
Spec coverage
140+
Matched : N/M claimed endpoints found in spec
141+
Gaps : <claimed items not in spec — possible drift or extension>
142+
143+
Test results
144+
Passed : N
145+
Failed : N
146+
<for each failure: test name, assertion that failed, spec citation>
147+
148+
Not yet claimed (in spec, not in twin)
149+
<paths in scope domain not yet implemented>
150+
151+
Fidelity score: <passed>/<total> (<percent>%)
152+
```
153+
154+
End with `Twin is in sync.` if all pass, or `Action required: see failures above.` if any fail.

0 commit comments

Comments
 (0)