Skip to content

Commit b194db4

Browse files
Shahinyanmclaude
andcommitted
feat(classifier): subscription-native agent-sdk backend via Haiku
Add a CLI-backed classifier that reaches an LLM through the local, already-authenticated `claude` binary — no ANTHROPIC_API_KEY needed. Resurrects the v0.7.x cli path that was dropped in v0.8.0, this time documented honestly: since 2026-06-15 a headless `claude -p` draws from the separate Agent SDK credit pool, not the interactive Pro/Max pool. - ClaudeCliClassifier (classifier/agent_sdk.rs) runs `claude -p <prompt> --model claude-haiku-4-5 --output-format json --strict-mcp-config`, parses the envelope's `result`, reuses the shared parse_verdict. Command execution injected via a CommandRunner trait so the parse path is unit-testable without shelling out. from_env() returns None unless `claude` is on PATH; model overridable via TJ_AGENT_SDK_MODEL. - Factor http.rs's fence-strip+serde parse into classifier::parse_verdict, reused by both http and agent_sdk so they never diverge. - Hybrid is now an ordered LLM chain: heuristic >= 0.7 -> agent-sdk (if claude on PATH) -> api (if key) -> pending/. Reorder via TJ_HYBRID_LLM_ORDER (default agent-sdk,api). - Wire `agent-sdk` into --backend for ingest-hook and classify-worker, and add --backend to install-hooks (baked into the generated hook command). - README Configuration + doctor note + --backend help document the backend and the Agent SDK credit caveat. Tests: agent_sdk parse/threshold/fence/error (fake runner, no live claude); hybrid proves uncertain -> agent-sdk wins and http is never touched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent d4f9470 commit b194db4

7 files changed

Lines changed: 518 additions & 59 deletions

File tree

CHANGELOG.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Added
11+
- **`agent-sdk` classifier backend** — subscription-native LLM classification
12+
via the local, already-authenticated `claude` binary, no `ANTHROPIC_API_KEY`
13+
required. `tj_core::classifier::agent_sdk::ClaudeCliClassifier` invokes
14+
`claude -p <prompt> --model claude-haiku-4-5 --output-format json
15+
--strict-mcp-config`, parses the JSON envelope's `result`, and reuses the
16+
shared verdict parser. Command execution is injected via a `CommandRunner`
17+
trait so the path is unit-testable without shelling out. `from_env()` returns
18+
`None` unless `claude` is on PATH; model overridable with `TJ_AGENT_SDK_MODEL`.
19+
- Wired into `--backend` selection (`ingest-hook`, `classify-worker`) and
20+
added to `install-hooks --backend`, alongside `hybrid` | `api` | `heuristic`.
21+
- **Hybrid fallback is now an ordered chain**: heuristic (≥ 0.7) → `agent-sdk`
22+
(if `claude` on PATH) → `api` (if key) → `pending/`. Reorder via
23+
`TJ_HYBRID_LLM_ORDER` (default `agent-sdk,api`) to prefer the API key.
24+
- **Honest cost note**: since **2026-06-15** a headless `claude -p` run draws
25+
from the separate Agent SDK monthly credit pool (~$20 Pro / $100 Max 5x /
26+
$200 Max 20x at API rates), not the interactive pool. Documented in the
27+
README, `--backend` help, and `doctor`.
28+
1029
## [0.12.0]
1130

1231
### Added

README.md

Lines changed: 25 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,7 @@ task-journal pack tj-x9rz1f --mode full
165165
| `doctor` | Self-check the install |
166166
| `rebuild-state` | Rebuild SQLite from JSONL |
167167
| `migrate-project` | Re-key data when a project moves on disk |
168-
| `install-hooks [--scope user\|project]` | Wire Claude Code auto-capture hooks |
168+
| `install-hooks [--scope user\|project] [--backend hybrid\|agent-sdk\|api\|heuristic]` | Wire Claude Code auto-capture hooks |
169169

170170
## MCP tools
171171

@@ -181,10 +181,32 @@ The MCP server exposes five tools to Claude Code (and any MCP client):
181181

182182
## Configuration
183183

184+
### Classifier backends
185+
186+
The auto-capture classifier (a best-effort backstop — explicit self-tagging via the
187+
MCP tools is the primary path) has a heuristic stage plus an optional LLM stage. The
188+
LLM stage has **two** ways to reach a model, pick via `--backend` on `install-hooks`
189+
or `ingest-hook`:
190+
191+
- **`agent-sdk`** — classify via the local, already-logged-in `claude` binary. **No
192+
`ANTHROPIC_API_KEY` needed** — it rides your Claude subscription. Pinned to Haiku.
193+
⚠️ Since **2026-06-15** a headless `claude -p` run draws from the separate **Agent
194+
SDK** monthly credit pool (~$20 Pro / $100 Max 5x / $200 Max 20x, at API rates),
195+
not the interactive pool. Classification is Haiku-class and tiny (a few hundred
196+
tokens per chunk), so the credit lasts a long time — but it is not strictly free.
197+
- **`api`** — call the Anthropic API directly. Requires `ANTHROPIC_API_KEY`.
198+
199+
`--backend=hybrid` (the default) runs the heuristic first, then falls through the LLM
200+
chain `agent-sdk → api`, using whichever backends are available. Reorder the chain
201+
with `TJ_HYBRID_LLM_ORDER` (e.g. `api,agent-sdk` to prefer the API key). With no LLM
202+
backend available, the heuristic still runs and ambiguous chunks queue in `pending/`.
203+
184204
| Env var | Effect | Default |
185205
|---------|--------|---------|
186-
| `ANTHROPIC_API_KEY` | Powers the API stage of `--backend=hybrid` (default) and is required for `--backend=api`. Without it, only the offline heuristic runs and ambiguous chunks land in the local pending queue. | _unset_ |
187-
| `TJ_CLASSIFIER_MODEL` | Override the Anthropic model used by the API stage. | `claude-haiku-4-5-20251001` |
206+
| `ANTHROPIC_API_KEY` | Enables the `api` LLM backend (and the `api` link of the hybrid chain). Optional — the `agent-sdk` backend needs no key. | _unset_ |
207+
| `TJ_AGENT_SDK_MODEL` | Override the model the `agent-sdk` backend passes to `claude --model`. | `claude-haiku-4-5` |
208+
| `TJ_HYBRID_LLM_ORDER` | Comma-separated fallback order for `--backend=hybrid`. | `agent-sdk,api` |
209+
| `TJ_CLASSIFIER_MODEL` | Override the Anthropic model used by the `api` backend. | `claude-haiku-4-5-20251001` |
188210
| `TJ_AUTO_OPEN_TASKS` | Set to `0` / `false` to disable auto-opening a task from `UserPromptSubmit` when no open task exists. | `1` |
189211

190212
## Event types

crates/tj-cli/src/main.rs

Lines changed: 64 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -506,9 +506,10 @@ fn run_doctor() -> Result<DoctorReport> {
506506
}
507507
Ok(_) | Err(_) => {
508508
notes.push(
509-
"claude CLI not on PATH — that's fine if you use the API backend \
510-
(set ANTHROPIC_API_KEY). For the CLI backend (free with Pro/Max), \
511-
install Claude Code from https://claude.com/claude-code"
509+
"claude CLI not on PATH — that's fine if you use the `api` backend \
510+
(set ANTHROPIC_API_KEY). For the `agent-sdk` backend (no API key; \
511+
uses your Claude login, drawing the Agent SDK credit pool since \
512+
2026-06-15), install Claude Code from https://claude.com/claude-code"
512513
.into(),
513514
);
514515
(false, None)
@@ -736,6 +737,12 @@ enum Commands {
736737
/// `task-journal backfill` afterwards. Onboarding shortcut.
737738
#[arg(long)]
738739
backfill: bool,
740+
/// Classifier backend baked into the installed hook command:
741+
/// "hybrid" (default), "agent-sdk", "api", or "heuristic". Use
742+
/// "agent-sdk" to classify via the local `claude` login without an
743+
/// ANTHROPIC_API_KEY (see `ingest-hook --help` for the credit note).
744+
#[arg(long, default_value = "hybrid")]
745+
backend: String,
739746
},
740747
/// Show local classifier and journal statistics.
741748
Stats,
@@ -838,13 +845,18 @@ enum Commands {
838845
#[arg(long)]
839846
text: Option<String>,
840847
/// Classifier backend:
841-
/// - "hybrid" (default) — keyword heuristic first (free, offline);
842-
/// Anthropic API fallback when uncertain (needs ANTHROPIC_API_KEY).
843-
/// - "api" — always call the Anthropic API. Best quality, paid.
848+
/// - "hybrid" (default) — keyword heuristic first (free, offline),
849+
/// then the configured LLM fallback chain (agent-sdk, then api;
850+
/// reorder with TJ_HYBRID_LLM_ORDER). Only available backends run.
851+
/// - "agent-sdk" — classify via the local, already-logged-in `claude`
852+
/// binary; no ANTHROPIC_API_KEY needed. Pinned to Haiku (override
853+
/// with TJ_AGENT_SDK_MODEL). NOTE: since 2026-06-15 a headless
854+
/// `claude -p` draws from the separate Agent SDK monthly credit
855+
/// pool (~$20 Pro / $100 Max 5x / $200 Max 20x at API rates), not
856+
/// the interactive pool. Classification is tiny, so it lasts.
857+
/// - "api" — always call the Anthropic API. Needs ANTHROPIC_API_KEY.
844858
/// - "heuristic" — heuristic only, no LLM. Fastest, lowest coverage.
845-
/// - "cli" — deprecated alias for hybrid. The old `claude -p` path
846-
/// was removed in v0.8.0 because Anthropic now bills it
847-
/// separately from Pro/Max.
859+
/// - "cli" — removed in v0.8.0; use "agent-sdk" (its resurrection).
848860
#[arg(long, default_value = "hybrid")]
849861
backend: String,
850862
/// Test/dev override: bypass classifier and force this event type. Hidden from --help.
@@ -864,7 +876,8 @@ enum Commands {
864876
/// at a time. Hidden from --help; not a public API.
865877
#[command(hide = true)]
866878
ClassifyWorker {
867-
/// Classifier backend: "hybrid", "api", or "heuristic". Defaults to hybrid.
879+
/// Classifier backend: "hybrid", "agent-sdk", "api", or "heuristic".
880+
/// Defaults to hybrid.
868881
#[arg(long, default_value = "hybrid")]
869882
backend: String,
870883
},
@@ -1378,6 +1391,7 @@ fn main() -> Result<()> {
13781391
scope,
13791392
uninstall,
13801393
backfill,
1394+
backend,
13811395
} => {
13821396
let settings_path = match scope.as_str() {
13831397
"user" => {
@@ -1473,11 +1487,25 @@ fn main() -> Result<()> {
14731487
// at env vars Claude Code never sets and therefore always
14741488
// fed the classifier empty text. Stdin-only is the correct
14751489
// wiring (see claude-memory-rsw).
1476-
// No --backend flag: the binary's default (hybrid) wins.
1477-
// Hybrid = free heuristic first, Anthropic API fallback when
1478-
// uncertain. Users wanting always-api can edit settings.json
1479-
// and add `--backend=api`.
1480-
let cmd = "task-journal ingest-hook || true";
1490+
// Bake the selected backend into the hook command. Default
1491+
// "hybrid" stays flag-free (heuristic first, then the agent-sdk
1492+
// → api fallback chain). A non-default backend — e.g.
1493+
// `--backend=agent-sdk` for subscription users with no API key
1494+
// — is passed through so the spawned classify-worker honors it.
1495+
if !matches!(
1496+
backend.as_str(),
1497+
"hybrid" | "agent-sdk" | "api" | "heuristic"
1498+
) {
1499+
anyhow::bail!(
1500+
"unknown --backend: {backend} (expected `hybrid`, `agent-sdk`, `api`, or `heuristic`)"
1501+
);
1502+
}
1503+
let cmd_string = if backend == "hybrid" {
1504+
"task-journal ingest-hook || true".to_string()
1505+
} else {
1506+
format!("task-journal ingest-hook --backend={backend} || true")
1507+
};
1508+
let cmd = cmd_string.as_str();
14811509
let entries = serde_json::json!({
14821510
"UserPromptSubmit": [{ "matcher": "", "hooks": [{ "type": "command", "command": cmd }] }],
14831511
"PostToolUse": [{ "matcher": "", "hooks": [{ "type": "command", "command": cmd }] }],
@@ -2241,6 +2269,15 @@ fn main() -> Result<()> {
22412269
Box::new(tj_core::classifier::hybrid::HybridClassifier::from_env())
22422270
}
22432271
"api" => Box::new(tj_core::classifier::http::AnthropicClassifier::from_env()?),
2272+
"agent-sdk" => Box::new(
2273+
tj_core::classifier::agent_sdk::ClaudeCliClassifier::from_env()
2274+
.ok_or_else(|| {
2275+
anyhow::anyhow!(
2276+
"agent-sdk backend selected but no `claude` binary on PATH — \
2277+
install Claude Code (https://claude.com/claude-code) or pick another --backend"
2278+
)
2279+
})?,
2280+
),
22442281
"heuristic" => {
22452282
// Heuristic-only: no LLM at all. Trades coverage
22462283
// for absolute zero-cost / offline operation.
@@ -2262,7 +2299,7 @@ fn main() -> Result<()> {
22622299
Box::new(HeuristicOnly)
22632300
}
22642301
other => anyhow::bail!(
2265-
"unknown backend: {other} (expected `hybrid`, `api`, or `heuristic`)"
2302+
"unknown backend: {other} (expected `hybrid`, `agent-sdk`, `api`, or `heuristic`)"
22662303
),
22672304
};
22682305
let input = tj_core::classifier::ClassifyInput {
@@ -3859,6 +3896,14 @@ fn process_pending_entry(
38593896
let classifier: Box<dyn Classifier> = match backend {
38603897
"hybrid" | "" => Box::new(tj_core::classifier::hybrid::HybridClassifier::from_env()),
38613898
"api" => Box::new(tj_core::classifier::http::AnthropicClassifier::from_env()?),
3899+
"agent-sdk" => Box::new(
3900+
tj_core::classifier::agent_sdk::ClaudeCliClassifier::from_env().ok_or_else(|| {
3901+
anyhow::anyhow!(
3902+
"agent-sdk backend selected but no `claude` binary on PATH — \
3903+
install Claude Code (https://claude.com/claude-code) or pick another --backend"
3904+
)
3905+
})?,
3906+
),
38623907
"heuristic" => {
38633908
use tj_core::classifier::heuristic::try_heuristic;
38643909
use tj_core::classifier::{ClassifyInput, ClassifyOutput};
@@ -3874,9 +3919,9 @@ fn process_pending_entry(
38743919
}
38753920
Box::new(HeuristicOnly)
38763921
}
3877-
other => {
3878-
anyhow::bail!("unknown backend: {other} (expected `hybrid`, `api`, or `heuristic`)")
3879-
}
3922+
other => anyhow::bail!(
3923+
"unknown backend: {other} (expected `hybrid`, `agent-sdk`, `api`, or `heuristic`)"
3924+
),
38803925
};
38813926
let input = tj_core::classifier::ClassifyInput {
38823927
text: text.clone(),

0 commit comments

Comments
 (0)