Skip to content

feat(agent/analyze): add pattern_search tool for catalog browsing#242

Open
nghiadaulau wants to merge 1 commit into
VersusControl:mainfrom
nghiadaulau:feat/analyze-tool-pattern-search
Open

feat(agent/analyze): add pattern_search tool for catalog browsing#242
nghiadaulau wants to merge 1 commit into
VersusControl:mainfrom
nghiadaulau:feat/analyze-tool-pattern-search

Conversation

@nghiadaulau

Copy link
Copy Markdown
Member

Summary

Adds a read-only tool to the analyze agent: pattern_search. It
complements pattern_history (which requires a known pattern id) by
letting the model browse the learned-pattern catalog with a small
filter / sort surface.

Why this is useful

The analyze agent's catalog-side tools today:

Existing tool What it does Gap
pattern_history One pattern by id Needs the id up front
describe_service One service summary Only one service at a time

That leaves no way for the model to answer questions like:

  • "Are there other patterns on this service that fired recently?"
  • "Show me every spike-verdict pattern in the last day."
  • "Has anything matching this template ever been classified?"

The model can't list and filter the catalog without one — today it
either asks the operator or guesses ids. pattern_search closes that
gap.

Filter / sort surface

{
  "query":      "connection refused",  // case-insensitive substring on template
  "service":    "orders",              // exact, case-insensitive
  "verdict":    "spike",               // known | unknown | spike
  "rule_name":  "oom",                 // exact, case-sensitive (operator-authored ids)
  "order_by":   "count_desc",          // count_desc (default) | last_seen_desc | first_seen_desc
  "limit":      20                     // default 20, capped at 100
}

Output uses the standard ToolResult envelope. Found stays true
on zero matches (matching the convention used by recent_incidents
and recent_changes — the empty list is itself an answer). The
response carries count, total_matched, and truncated so the
model knows when to narrow.

Constraints honoured

  • Strictly read-only. Only reads from PatternCatalog via the
    existing local interface in tools.go — no new imports, no storage
    mutations, no provider calls. The import-graph guard
    (TestTools_ImportGraphStaysReadOnly) passes.
  • No tools.yaml changes — purely catalog-backed, no external
    config required. Registered in tools.Default next to
    pattern_history, so it's available whenever a catalog is wired —
    same zero-config story as the other catalog tools.
  • No detect-mode change — analyze-only addition.
  • Rebased on the find_runbook-era Default signature (embedder
    • runbooks params); both tools coexist, registration capacity 7→8.

Tests

TestDefault_PatternSearchRegistration (registration convention from
tools_default_test.go) + TestPatternSearch_* (11 cases): metadata
shape, nil catalog, bad args, invalid enums, case-insensitive query,
combined service + verdict filter, case-sensitive rule_name, both
orderings, limit truncation with total_matched/truncated,
limit clamp to 100, empty result still Found=true.

A hand-rolled in-memory fakePatternCatalog keeps the test file free
of any pkg/agent import — the analyze/tools package stays
import-cycle-free.

Docs

  • src/agent/ai-analyze-mode.md — added pattern_search to the
    inline tool list.
  • src/agent/analyze-tools/tools.md — per-arg reference, use-case
    prompts, and the relationship to pattern_history.

Test plan

  • gofmt -l clean
  • go vet ./... clean
  • go build ./... clean
  • go test -race -count=1 ./... green on every package
    (TestGitChangeFeed_RealRepo_Eino needs live network to clone
    cloudwego/eino and fails identically on a pristine
    upstream/main checkout in a sandboxed env — unrelated)

Adds a read-only tool to the analyze agent: `pattern_search`. It
complements `pattern_history` (lookup-by-id only) by letting the
model browse the learned-pattern catalog when it doesn't already
have an id.

Why it matters
--------------
The analyze agent's catalog-side tools are:

  - `pattern_history` — needs a specific pattern id.
  - `describe_service` — limited to one service at a time.

That leaves no way to answer questions like:

  - "Are there other patterns on this service that fired recently?"
  - "Show me every spike-verdict pattern in the last day."
  - "Has this template ever been classified before?"

The model can't list and filter the catalog without one. Today it
ends up asking the operator or making up ids. `pattern_search`
closes that gap.

Filter / sort surface
---------------------
- `query` — case-insensitive substring on the template.
- `service` — exact, case-insensitive.
- `verdict` — enum: `known | unknown | spike`.
- `rule_name` — exact (case-sensitive; rule names are operator
  authored identifiers).
- `order_by` — enum: `count_desc` (default), `last_seen_desc`,
  `first_seen_desc`. Ties broken by id ascending so output is
  deterministic.
- `limit` — default 20, clamped at 100. The result carries
  `total_matched` + `truncated` so the model knows when to narrow.

Output uses the standard ToolResult envelope with `Found: true` even
on zero matches, matching the convention used by `recent_incidents`
and `recent_changes` (the empty list is itself a meaningful answer).

Constraints honored
-------------------
- Strictly read-only: only reads from `PatternCatalog` via the
  existing local interface in tools.go — no new imports, no
  storage mutations, no provider calls. The import-graph guard
  (TestTools_ImportGraphStaysReadOnly) passes.
- No changes to `tools.yaml` — purely catalog-backed, no external
  config needed. Registered in `tools.Default` next to
  `pattern_history` so it's available whenever a catalog is wired.
- No worker / detect-mode behaviour change — analyze-only.
- Rebased on the find_runbook-era `Default` signature (embedder +
  runbooks params); registration slot capacity bumped 7 -> 8.

Tests
-----
- TestDefault_PatternSearchRegistration — registered when a catalog
  is wired, omitted when nil (same convention as the find_runbook
  registration test).
- TestPatternSearch_Metadata — Name/Description/ArgsSchema shape.
- TestPatternSearch_NilCatalog / BadArgs / InvalidEnums — input
  validation surfaces clear errors.
- TestPatternSearch_QuerySubstringCaseInsensitive — `query=PANIC`
  matches `panic: runtime error <*>`.
- TestPatternSearch_ServiceAndVerdictFilters — combined AND
  filtering.
- TestPatternSearch_RuleNameExactMatch — case-sensitive rule_name
  (proves we do NOT case-fold operator-authored rule names).
- TestPatternSearch_OrderByCountDescDefault / LastSeenDesc — both
  orderings produce the expected sequence.
- TestPatternSearch_LimitTruncation — count/total_matched/truncated
  triple is filled correctly.
- TestPatternSearch_LimitClamp — 9999 clamps to 100.
- TestPatternSearch_EmptyResultStillFound — list-style "no match"
  returns Found=true with count=0 + truncated=false.

The fake catalog used by the tests is hand-rolled in the test file
(no dependency on pkg/agent) so the analyze/tools package stays
import-cycle-free.

Docs
----
- src/agent/ai-analyze-mode.md — added pattern_search to the inline
  tool list.
- src/agent/analyze-tools/tools.md — full per-arg reference,
  use-case prompts, and the relationship to pattern_history.

Pre-checks: gofmt clean, go vet clean, go build clean,
go test -race green on every package (the one exception is
TestGitChangeFeed_RealRepo_Eino, which requires live network access
to clone github.com/cloudwego/eino and fails identically on a
pristine upstream/main checkout in a sandboxed environment).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant