fix(config): read the KB metadata database path from config instead of hardcoding it

[lfnothias]

Summary

The vector store honours database.chroma_path, but the KB-metadata SQLite was hardcoded to a working-directory-relative ./data/perspicacite.db. Isolation between instances therefore depended entirely on the process working directory, which is undocumented and easy to get wrong.

A second instance configured with its own database.path and chroma_path listed the main hub's ~795 knowledge bases (metadata read from the repo-root SQLite) while its Chroma directory was empty. Any KB it created registered metadata in the production database while its vectors landed in the isolated store — producing knowledge bases that exist in the listing and return nothing.

The bug was in four places, not one

web/state.py was the reported site. The same hardcoding exists in:

• mcp/server.py, which also hardcoded its provenance sidecar to ./data/provenance while reading config.database.chroma_path two lines earlier.
• cli.py, twice (create-kb --from-bibtex and add-to-kb), whose --session-db help text advertised data/perspicacite.db as the default while --chroma-dir already defaulted to the configured value.

Fixing state.py alone would have left the MCP server and CLI mis-isolated.

A trap worth naming

Config().database.path evaluates to the literal PosixPath('~/.local/share/perspicacite/memory.db') — pydantic v2 does not run field validators on defaults unless validate_default=True, so the expand_home validator never fires for the default. SessionStore does not call expanduser either. Reading config.database.path naively would have created a directory literally named ~ under the working directory.

Changes

• config/paths.py (new): resolve_session_db_path(config) returns database.path expanded, but only when the value was actually supplied — by config file or PERSPICACITE_DB_PATH, detected via pydantic's model_fields_set. Otherwise it returns the legacy ./data/perspicacite.db.

  This matters: applying the schema default unconditionally would silently relocate the metadata of every existing deployment, whose knowledge bases live in the legacy path. Existing installs are unaffected unless they opt in.

• web/state.py, mcp/server.py, cli.py: all four sites use the shared resolver. The MCP provenance sidecar now derives from the resolved database's parent, matching web/state.py.

• Both resolved paths are logged at startup, so a mis-isolated instance is obvious immediately:

  Session store initialized at /path/to/memory.db (chroma: /path/to/chroma)
  

Behaviour change for operators

If your config.yml sets database.path (as config.example.yml documents), this instance will now read that database instead of ./data/perspicacite.db. That is the point of the fix, but it means an existing hub whose KBs live in ./data/perspicacite.db must either drop the key or point it at that file. The new startup log makes the resolved location explicit.

Verification

• New tests/unit/test_config_database_path.py (5 tests): an unset path keeps the legacy location; setting only chroma_path does not move the metadata store; an explicit ~ path is expanded; PERSPICACITE_DB_PATH reaches the resolver through the real loader; and AppState.initialize() hands the configured path to SessionStore.
• The AppState test fails on main and passes with this change, so it is a real regression test rather than a tautology.
• Full unit suite with CI's exclusions: 2407 passed, 0 failed.
• ruff: no new findings against the main baseline for the touched files.
• Confirmed no other module derives the metadata SQLite path independently: LEGACY_SESSION_DB_PATH in config/paths.py is now the single definition.

Context

Found while standing up an isolated, locally-embedded instance (port 8010, all-MiniLM-L6-v2) for the Consilium rediscovery benchmark. The previous workaround was to launch with the working directory set to the isolated data directory.