Add natural-language query interface (/ask)

[skearnes]

Summary

Adds a natural-language search page (/ask) that translates a free-text question — e.g. "reactions using benzene as an input with yield over 70%" — into the existing QueryParams and runs it through the current index-accelerated search path. No ORM or schema changes; a thin translation layer over the existing engine. Full-stack (backend + frontend).

The model emits a structured query via a single forced tool call — never SQL, never invented SMILES. Specific compound names resolve to SMILES via ord_schema.resolvers (PubChem → NCI/CADD CIR → OPSIN); compound classes and functional groups (e.g. "an aryl boronic acid", "brominated products") become model-authored SMARTS/SUBSTRUCTURE patterns and are never sent to the resolver.

Backend (ord_interface/api/nl_query.py)

• Forced build_query tool call → NLQuery; model via ORD_NL_QUERY_MODEL (default claude-haiku-4-5); prompt in nl_query_prompt.md.
• Match modes: EXACT (named compound, resolved), SUBSTRUCTURE/SMARTS (model-authored fragment), SIMILAR. Model-authored SMARTS and reaction_smarts are RDKit-validated up front, so a bad pattern is a 422 (with the pattern echoed) in both normal and dry-run mode.
• Error mapping: rate limit → 429, other Anthropic API errors → 503, no/malformed tool call → 502, unresolvable compound / invalid SMARTS / empty interpretation → 422, missing key → 503. No uncaught 500s.
• Redis caching (best-effort, fast-fail): translation only (1h), name→SMILES separately (30d). Search results are never cached. Concurrent name resolution.
• GET /api/nl_query?q=... (length-bounded, dry_run supported) → {query, interpretation, resolved_components, query_components, results, dry_run}.

Frontend (/ask)

• Text box, example chips, and an interpretation panel that separates provenance: a "From the model" section (the tool call, raw JSON available) and a "Resolved to structures" section listing only identifiers a resolver actually handled.
• Dry-run toggle (?dry_run=1); shareable ?q= URLs synced to the input; in-development banner; route unlisted in the nav.

Eval & tests

• nl_query_eval.py + 15 YAML cases. Identifiers compared by structure (RDKit-canonicalized SMARTS/SMILES; string fallback for names). Translation accuracy 15/15 on Haiku 4.5; --search also runs each query with a per-phase timing breakdown.
• 25 unit tests, fully stubbed (no network/DB/key). ruff/ty/tsc/eslint clean.

Deploy note

Requires ANTHROPIC_API_KEY in the API environment (provisioned in ord-infrastructure#33). Follow-up: offline name→SMILES table and per-IP rate limiting.

🤖 Generated with Claude Code

Greptile Summary

This PR adds a natural-language search page (/ask) that translates a chemist's free-text question into the existing QueryParams via a forced Anthropic tool call, then runs it through the current search backend. The implementation is full-stack (FastAPI endpoint + React UI) with Redis caching for translations, concurrent name resolution via asyncio.gather, and thorough error mapping across all Anthropic/resolution/query failure modes.

• Backend (nl_query.py): Translation, SMARTS/reaction-SMARTS validation, name→SMILES resolution via ord_schema.resolvers, and a Redis cache for translations (1 h TTL) and resolutions (30 d TTL) are all cleanly separated; previously-flagged issues (empty-interpretation 500, missing query-length guard, ValidationError 500 on bad tool payload, sequential resolution latency) have been fixed.
• Frontend (MainNLSearch.tsx, useNLQuery.ts): URL-keyed query, dry-run toggle, and a two-layer interpretation panel (model output vs. resolver output) are implemented correctly; the route is reachable but intentionally unlisted while the feature is in development.
• Eval harness (nl_query_eval.py): Reports 15/15 accuracy with Haiku 4.5, but CaseExpectation and check_interpretation do not score the reaction_smarts or use_stereochemistry fields, so accuracy for those two NLQuery fields is unverified by the harness.

Confidence Score: 5/5

Safe to merge; the new endpoint is well-guarded and the UI is unlisted from the nav while in development.

The core backend is solid: every realistic failure mode (Anthropic errors, bad SMARTS, unresolvable names, empty interpretation) is mapped to an appropriate HTTP status code, and 25 unit tests cover all those paths without needing a live API key or database. The only gap is in the offline eval harness, which does not score reaction_smarts or use_stereochemistry — a test-coverage shortcoming that does not affect production correctness.

ord_interface/api/nl_query_eval.py — CaseExpectation and check_interpretation are missing coverage for reaction_smarts and use_stereochemistry.

Important Files Changed

Filename	Overview
ord_interface/api/nl_query.py	Core NL→query translation layer. Error handling is thorough (429/502/503 from Anthropic, 422 for bad SMARTS/unresolvable names, 422 for empty constraints), caching is correctly scoped to translations only, and concurrent resolution via asyncio.gather is in place. No issues found.
ord_interface/api/nl_query_eval.py	Eval harness for translation accuracy. CaseExpectation and check_interpretation cover components and numeric filters but omit reaction_smarts and use_stereochemistry, so accuracy for those two NLQuery fields is unverified.
app/src/views/nl-search/MainNLSearch.tsx	React UI for /ask page. Text input synced to URL params, dry-run toggle, example chips, two-layer interpretation panel (model output + resolver output). Logic and state management look correct.
app/src/hooks/useNLQuery.ts	TanStack Query hook for /api/nl_query. Query key includes both query string and dryRun flag, retry disabled, staleTime Infinity for URL-keyed results. Proto deserialization matches existing search hook pattern.
ord_interface/api/nl_query_test.py	25 unit tests covering translation, error mapping (429/502/503), cache hit/miss, invalid SMARTS 422, dry-run, and empty-interpretation 422. All paths from the PR description are covered; tests are fully stubbed.
ord_interface/api/main.py	Adds nl_query router to FastAPI app; one-line change, no issues.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant UI as React /ask
    participant Hook as useNLQuery
    participant API as FastAPI /api/nl_query
    participant Redis as Redis Cache
    participant LLM as Anthropic (Haiku)
    participant Resolver as ord_schema.resolvers
    participant DB as PostgreSQL

    UI->>Hook: "submit query (q=...)"
    Hook->>API: "GET /api/nl_query?q=..."
    API->>Redis: get translation cache
    alt cache hit
        Redis-->>API: NLQuery (cached)
    else cache miss
        API->>LLM: messages.create (forced build_query tool call)
        LLM-->>API: "ToolUseBlock { NLQuery }"
        API->>Redis: set translation cache (1h TTL)
    end
    API->>Resolver: asyncio.gather(_resolve_component x N)
    Note over Resolver: SMARTS validate with RDKit, SMILES canonicalize, name PubChem/CIR/OPSIN (cached 30d)
    Resolver-->>API: ResolvedComponent[]
    alt not dry_run
        API->>DB: run_query(QueryParams)
        DB-->>API: QueryResult[]
    end
    API-->>Hook: NLQueryResponse
    Hook-->>UI: NLQueryData (protos deserialized)
    UI->>UI: render Interpretation panel + SearchResults

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant UI as React /ask
    participant Hook as useNLQuery
    participant API as FastAPI /api/nl_query
    participant Redis as Redis Cache
    participant LLM as Anthropic (Haiku)
    participant Resolver as ord_schema.resolvers
    participant DB as PostgreSQL

    UI->>Hook: "submit query (q=...)"
    Hook->>API: "GET /api/nl_query?q=..."
    API->>Redis: get translation cache
    alt cache hit
        Redis-->>API: NLQuery (cached)
    else cache miss
        API->>LLM: messages.create (forced build_query tool call)
        LLM-->>API: "ToolUseBlock { NLQuery }"
        API->>Redis: set translation cache (1h TTL)
    end
    API->>Resolver: asyncio.gather(_resolve_component x N)
    Note over Resolver: SMARTS validate with RDKit, SMILES canonicalize, name PubChem/CIR/OPSIN (cached 30d)
    Resolver-->>API: ResolvedComponent[]
    alt not dry_run
        API->>DB: run_query(QueryParams)
        DB-->>API: QueryResult[]
    end
    API-->>Hook: NLQueryResponse
    Hook-->>UI: NLQueryData (protos deserialized)
    UI->>UI: render Interpretation panel + SearchResults

Loading

_{Reviews (9): Last reviewed commit: "Tighten verbose comments in nl_query" | Re-trigger Greptile}

[skearnes]

Good catch on the SMARTS ; collision — fixed in 3dff42f. One correction: the suggested split(";", 2) would mis-parse a SMARTS whose ; is near the start (e.g. [#6;R] → ["[#6", "R]", "input;smarts"]), since it splits left-to-right. Used rsplit(";", 2) instead — target and mode are the trailing fields and never contain ;. Added a regression test (test_query_smarts_with_semicolon).

Note: we're also planning to replace the pattern;target;mode string encoding with JSON entirely (separate PR), which removes this class of delimiter bug.

🤖 Generated with Claude Code