Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ Use `logan_report_finding` for each distinct finding, then send a summary via `l
| Tool | Purpose |
|------|---------|
| `logan_status` | Check if file is open, get line count and state |
| `logan_evidence_pack` | **Fetch FIRST**: one compact briefing (severity, levels, grouped crashes, top components/gaps, discovered field vocabulary, filter hints, optional baseline delta) as `viewerLine` refs + counts — not raw text. Drill down from it instead of many exploratory calls |
| `logan_report_finding` | **Pin a finding**: annotate + navigate + chat message in one call |
| `logan_send_message` | Send chat message to user (for summaries, questions, greetings) |
| `logan_wait_for_message` | Block until user replies (SSE-backed) |
Expand Down
19 changes: 12 additions & 7 deletions docs/GRANULARIZATION_DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,10 +128,10 @@ Conventions layered onto the **existing** MCP tools so the agent stays cheap:
| Trends / transitions / correlate / time-gaps / crashes | ✅ built |
| Baseline compare (level shift / new crash / error-rate spike) | ✅ built |
| Click-to-line finding system (`logan_report_finding`) | ✅ built |
| **Evidence-pack assembler + single MCP fetch call** (`logan_evidence_pack`) | ✅ built — composes analyze + time-gaps + trend-fields + baseline, returns refs not raw text |
| **Record stitching → stable `unit_id`s** | 🔨 new |
| **Persisted per-file structured index** | 🔨 new (persist what `trend_fields` computes) |
| **Layered map-reduce summaries** | 🔨 new |
| **Evidence-pack assembler + single MCP fetch call** | 🔨 new |
| **Payload caps + by-reference conventions across tools** | 🔨 wiring |

The new pieces are mostly *composition and persistence* of primitives LOGAN already has
Expand All @@ -142,13 +142,18 @@ The new pieces are mostly *composition and persistence* of primitives LOGAN alre
## Build order (recommended)

1. **This spec** (done).
2. **Structured index + record stitching** — extend the existing timestamp/level pass to
2. **Evidence-pack assembler** ✅ **done** — `logan_evidence_pack` MCP tool →
`POST /api/evidence-pack` composes analyze + time-gaps + trend-fields + optional
baseline delta into one compact briefing (severity, level counts, grouped crashes,
top components, top gaps, field vocabulary, filter hints), returning `viewerLine`
references + counts rather than raw log text. Journaled as one replayable step.
Built first because it delivers the headline value with zero changes to the hot
parsing path.
3. **Structured index + record stitching** — extend the existing timestamp/level pass to
emit stitched records with stable `unit_id`s; persist the `trend_fields` output as a
per-file index sidecar.
3. **Layered summaries** — whole-file / per-component / per-chunk rollups over the units.
4. **Evidence-pack assembler** — one function that gathers fields + rollups + crashes +
gaps + baseline deltas + anomaly flags, exposed as a single MCP tool the agent calls
first (e.g. `logan_evidence_pack`).
per-file index sidecar. This lets the evidence pack reference stable `unit_id`s.
4. **Layered summaries** — whole-file / per-component / per-chunk rollups over the units,
folded into the evidence pack as the "zoom" layers.
5. **Payload caps + by-reference drill-down** conventions across the existing MCP tools.

Net: same analytical power, a fraction of the tokens — LOGAN hands the LLM a **briefing,
Expand Down
107 changes: 106 additions & 1 deletion src/main/api-server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ const INVESTIGATIVE_PATHS = new Set<string>([
'/api/search', '/api/filter', '/api/clear-filter', '/api/analyze', '/api/time-gaps',
'/api/trend-fields', '/api/trend-series', '/api/trend-transitions', '/api/trend-correlate',
'/api/trend-show', '/api/investigate-crashes', '/api/investigate-component',
'/api/investigate-timerange', '/api/triage', '/api/navigate',
'/api/investigate-timerange', '/api/triage', '/api/navigate', '/api/evidence-pack',
]);
const JOURNAL_CAP = 200;
let agentJournal: JournalEntry[] = [];
Expand All @@ -34,6 +34,7 @@ function journalLabel(p: string, body: Record<string, any>): string {
if (p.startsWith('/api/trend-')) return `${name} ${body.field ?? body.pattern ?? ''}`.trim();
if (p === '/api/investigate-component') return `investigate component ${body.component ?? ''}`;
if (p === '/api/triage') return `triage ${body.symptom ?? ''}`.trim();
if (p === '/api/evidence-pack') return `evidence-pack${body.baselineId ? ' (vs baseline)' : ''}`;
return name;
}

Expand Down Expand Up @@ -939,6 +940,110 @@ export function startApiServer(ctx: ApiContext): void {
return;
}

// Compose a compact "evidence pack" — one briefing the agent fetches
// FIRST, instead of dozens of exploratory round-trips. Reuses existing
// primitives (analyze / time-gaps / trend-fields / baseline) in-process;
// returns counts + references (viewerLine), not raw log text.
if (url === '/api/evidence-pack') {
const filePath = ctx.getCurrentFilePath();
const handler = ctx.getFileHandler();
if (!filePath || !handler) return sendError(res, 'No file open');
const totalLines = handler.getTotalLines();

const thresholdSeconds = body.thresholdSeconds ?? 60;
const topFieldsN = body.topFields ?? 25;
const topGapsN = body.topGaps ?? 8;
const topComponentsN = body.topComponents ?? 10;

// 1. Analysis (also caches getAnalysisResult() for the baseline step)
const analysisResp = await ctx.analyze(body.analyzerName);
const aresult = analysisResp?.success ? analysisResp.result : (analysisResp?.result ?? null);
const levelCounts = aresult?.levelCounts || {};
const totalAnalyzed = aresult?.stats?.analyzedLines || totalLines;
const errorCount = levelCounts['error'] || 0;
const warningCount = levelCounts['warning'] || 0;
const errorPercent = totalAnalyzed > 0 ? (errorCount / totalAnalyzed) * 100 : 0;
const warningPercent = totalAnalyzed > 0 ? (warningCount / totalAnalyzed) * 100 : 0;
const crashes = aresult?.insights?.crashes || [];
const topFailingComponents = aresult?.insights?.topFailingComponents || [];
const filterSuggestions = aresult?.insights?.filterSuggestions || [];

// 2. Time gaps (top N, with 1-based viewerLine)
const gapsResp = await ctx.detectTimeGaps({ thresholdSeconds });
const allGaps = gapsResp?.success ? (gapsResp.gaps || []) : [];
const timeGaps = allGaps.slice(0, topGapsN).map((g: any) => ({
viewerLine: g.lineNumber + 1,
gapSeconds: Math.round(g.gapSeconds),
from: g.prevTimestamp,
to: g.currTimestamp,
preview: g.linePreview,
}));

// 3. Discovered fields (the agent's vocabulary) — top N by frequency
const fieldsResp = await ctx.trendDiscoverFields({ sampleSize: body.fieldSampleSize });
const allFields = fieldsResp?.success ? (fieldsResp.fields || []) : [];
const fields = allFields.slice(0, topFieldsN).map((f: any) => ({
name: f.name, type: f.type, occurrences: f.occurrences,
distinct: f.distinct, examples: f.examples,
}));

// Group crashes by keyword, keep first-occurrence viewerLine
const crashGroups: Record<string, { keyword: string; count: number; viewerLine: number; sample: string }> = {};
for (const c of crashes) {
if (!crashGroups[c.keyword]) {
crashGroups[c.keyword] = { keyword: c.keyword, count: 0, viewerLine: c.lineNumber + 1, sample: c.text };
}
crashGroups[c.keyword].count++;
}

// Severity + one-line summary (same rubric as logan_triage)
let severity: 'healthy' | 'warning' | 'critical' = 'healthy';
if (crashes.length > 0 || errorPercent > 20) {
severity = 'critical';
} else if (errorPercent > 5 || timeGaps.some((g: any) => g.gapSeconds > 300) || topFailingComponents.length > 3) {
severity = 'warning';
}
const parts = [`${totalLines.toLocaleString()} lines`];
if (errorCount > 0) parts.push(`${errorCount.toLocaleString()} errors (${errorPercent.toFixed(1)}%)`);
if (crashes.length > 0) parts.push(`${crashes.length} crashes`);
if (allGaps.length > 0) parts.push(`${allGaps.length} time gaps`);
if (allFields.length > 0) parts.push(`${allFields.length} fields`);

// 4. Optional baseline delta (analysis just ran, so getAnalysisResult() is set)
let baselineDelta: any = null;
if (body.baselineId) {
const ar = ctx.getAnalysisResult();
if (ar) {
const fp = buildFingerprint(filePath, ar, handler);
baselineDelta = ctx.getBaselineStore().compare(fp, body.baselineId) || null;
}
}

const pack = {
file: { path: filePath, totalLines, timeRange: aresult?.timeRange || null },
severity,
summary: parts.join(', '),
levels: {
...levelCounts,
errorPercent: Math.round(errorPercent * 100) / 100,
warningPercent: Math.round(warningPercent * 100) / 100,
},
crashes: Object.values(crashGroups),
topComponents: topFailingComponents.slice(0, topComponentsN),
timeGaps,
fields,
filterSuggestions: filterSuggestions.slice(0, 5),
baselineDelta,
caps: {
fields: { shown: fields.length, total: allFields.length, truncated: allFields.length > fields.length },
timeGaps: { shown: timeGaps.length, total: allGaps.length, truncated: allGaps.length > timeGaps.length },
note: 'Compact briefing. Drill into any viewerLine with logan_get_lines; chart any field with logan_trend_show; pin issues with logan_report_finding.',
},
};
sendJson(res, { success: true, pack });
return;
}

sendError(res, 'Not found', 404);
return;
}
Expand Down
28 changes: 28 additions & 0 deletions src/mcp-server/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -866,6 +866,34 @@ server.tool(
}
);

// === Tool: logan_evidence_pack ===
server.tool(
'logan_evidence_pack',
'Fetch this FIRST. Assembles one compact "evidence pack" for the open log — severity, level counts, grouped crashes, top failing components, top time gaps, the discovered field vocabulary, filter hints, and (optionally) a baseline delta — in a single call. Everything is references (viewerLine) + counts, NOT raw log text, so you spend tokens on judgement, not on ingesting the file. Then drill down with logan_get_lines / logan_search / logan_trend_show and pin issues with logan_report_finding.',
{
thresholdSeconds: z.number().min(1).default(60).describe('Minimum time gap (seconds) to report'),
fieldSampleSize: z.number().min(100).optional().describe('Lines to sample for field discovery (default ~3000)'),
topFields: z.number().min(1).default(25).describe('Max discovered fields to include (most frequent first)'),
topGaps: z.number().min(1).default(8).describe('Max time gaps to include (largest first)'),
baselineId: z.string().optional().describe('If set, include a delta vs. this saved baseline'),
redact: z.boolean().default(true).describe('Whether to redact sensitive data'),
},
async ({ thresholdSeconds, fieldSampleSize, topFields, topGaps, baselineId, redact }) => {
try {
const result = await apiCall('POST', '/api/evidence-pack', {
thresholdSeconds, fieldSampleSize, topFields, topGaps, baselineId,
});
if (!result.success) {
return { content: [{ type: 'text', text: `Error: ${result.error || 'evidence pack failed'}` }], isError: true };
}
const output = redact ? maybeRedact(result.pack, true) : result.pack;
return { content: [{ type: 'text', text: JSON.stringify(output, null, 2) }] };
} catch (err: any) {
return { content: [{ type: 'text', text: `Error: ${err.message}` }], isError: true };
}
}
);

// === Tool: logan_investigate_crashes ===
server.tool(
'logan_investigate_crashes',
Expand Down
Loading