Skip to content

IA-2: tacit_knowledge_curator returns empty results because ChromaDB corpus is not indexed #13

Description

@adris-misra

Description

The tacit_knowledge_curator agent, when invoked from IABENCH-v1 task
IA-2, returns zero retrieved documents for every query. nDCG@5 is 0.0 not
because retrieval is wrong but because the ChromaDB instance has no
documents indexed at runtime — the agent has nothing to retrieve from.

Surfaced by the PR 2 health-check run.

Steps to reproduce

  1. Check out main with PR 2 merged
  2. $env:PYTHONPATH = "."
  3. industrial-agents bench --suite all --provider ollama --model llama3.1:8b
  4. Inspect IA-2's per-query details — retrieved_docs: [] for all queries

Expected behaviour

On framework startup or first agent invocation, the tacit_knowledge_curator
should index the synthetic SOPs (data/synthetic/sops/) and expert notes
(data/synthetic/expert_notes/) into its configured vector store, so that
queries return ranked results.

Actual behaviour

ChromaDB is empty. The agent runs without errors but every retrieval
returns []. IA-2 mean nDCG@5 = 0.0 across all queries.

Framework version

v0.1.0-pre (bench/iabench-retrieval-hallucination @ 2d2834f)

LLM provider

ollama

Environment

Windows 11, Python 3.12

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions