Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .claude/skills/perspicacite-mcp/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,9 @@ Known databases: `semantic_scholar`, `openalex`, `pubmed`, `europepmc`,
| Build then query a reasoning graph of claims across a KB | `build_claim_graph` → `query_claim_graph` (`claim_graph_export` → JSON-LD; `claim_graph_status` to poll) |

The claim tools (`extract_claims_from_passages`, the `*_claim_graph` family, `export_astra`)
require the server's optional `indicia` extra (`uv sync --extra indicia`); they return a clear
error when it isn't installed.
require the private `indicium` package, which is unpublished and therefore maintainer-only;
they return a clear error when it isn't installed. The claim graph's RDF backend is public
(`uv sync --extra graph`).

Use `adaptive=True` on `get_relevant_passages` for terse or jargon-heavy
queries: the server runs the optimizer once and retries if the first pass
Expand Down
28 changes: 23 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,23 @@ jobs:
- name: Ruff (report-only, --exit-zero)
run: ruff check src/ tests/ --exit-zero --statistics

uv-resolve:
# Issue #29: uv resolves every extra and every [tool.uv.sources] entry when
# it locks, so one local-path source breaks `uv sync` on every fresh clone.
# The `test` job below installs with pip, which ignores [tool.uv.sources]
# outright and so cannot catch this. A GitHub runner never has the private
# sibling checkouts, which is exactly an external user's environment.
name: uv resolves on a clean checkout
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- uses: astral-sh/setup-uv@v5
- name: uv lock
run: uv lock

test:
runs-on: ubuntu-latest
steps:
Expand Down Expand Up @@ -57,11 +74,12 @@ jobs:
-v --timeout=30 --timeout-method=signal --no-header

- name: Unit tests (safe subset)
# The indicium knowledge-graph layer depends on the local sibling
# packages indicium / indicium-adapters (see [tool.uv.sources]) plus
# rdflib; indicium-adapters is not published on PyPI, so these modules
# cannot import under CI's plain `pip install -e ".[dev]"`. Excluded
# here until CI provisions the KG stack.
# The indicium knowledge-graph layer depends on the private sibling
# packages indicium / indicium-adapters, which are unpublished and so
# cannot be declared in pyproject.toml at all (see the comment there,
# and issue #29). These modules cannot import under CI's plain
# `pip install -e ".[dev]"`. Excluded here until CI provisions the
# KG stack.
#
# The trailing --deselect block additionally drops individual tests
# that cannot pass in CI without the KG stack or network:
Expand Down
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]

### Changed
- **BREAKING:** the `indicia` and `adapters` optional extras are removed. They required the
private, unpublished `indicium` stack, and `uv lock` resolves every extra whether or not it
is requested, so their mere presence broke `uv sync` on every fresh clone. Maintainers now
install the sibling checkouts into the synced environment with
`uv pip install -e ../indicium ...`, and pass `--inexact` on subsequent syncs. See `CLAUDE.md`.
- **BREAKING:** `extract_claims_from_passages` MCP tool: `domain: str | None` renamed to
`domains: list[str] | None`. Pass a single domain as `["metabolomics"]`. Multiple
domain IDs are resolved and composed into a `CompositeAdapter` so all adapters'
context, qualifier, enrichment, and SHACL shapes are applied together.
- **BREAKING:** `generate_report` MCP tool: same `domain` → `domains` rename and composition logic.

### Added
- `graph` optional extra (`rdflib`, `pyoxigraph`) — the RDF/SPARQL backend behind the claim
graph. Both packages are on PyPI, so `uv sync --extra graph` works for everyone. `rdflib` was
previously an undeclared dependency, reaching the environment only through `indicium`.
- `uv-resolve` CI job and `tests/unit/test_pyproject_packaging.py`, which fail if a local-path
`[tool.uv.sources]` entry or a private distribution is reintroduced.
- `domains` multi-adapter support for both claim-extraction MCP tools (see Changed above).
- `claims_to_graph()` now serializes the `ontology_terms` dict from enriched claims as
`asb:{slot}_ontology_term` RDF literals, enabling SHACL property-shape validation on
ontology identifiers.

### Fixed
- `uv sync` failed on every fresh clone with `Distribution not found at: .../indicium`, because
`[tool.uv.sources]` pointed at private sibling checkouts that only exist on maintainer machines
(#29, reintroducing #5). The sources are removed and CI now exercises `uv lock` on a clean
checkout; the previous CI installed with `pip`, which ignores `[tool.uv.sources]` entirely.
- `domain_adapter` is now correctly passed into `extract_claims()` (not applied as a manual
post-processing loop), enabling LLM context enrichment and domain qualifier acceptance during
extraction rather than only after.
Expand Down
12 changes: 11 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,17 @@ FastAPI app is defined in [src/perspicacite/web/app.py](src/perspicacite/web/app

Defined in [src/perspicacite/mcp/server.py](src/perspicacite/mcp/server.py) using `fastmcp`. It has its own `MCPState` singleton (separate from `AppState`) and is mounted at `/mcp` by the CLI. Tool usage patterns are documented in [docs/perspicacite_skills.md](docs/perspicacite_skills.md) (and the live `get_usage_guide` tool is the source of truth).

The claim/standardization tools — `extract_claims_from_passages` and `export_astra` — require the optional **`indicia`** extra (`uv sync --extra indicia`), which pulls in the local [`indicium`](../indicium) standard package (typed claims: Bucur 5-slot SuperPattern + ECO/CiTO/SEPIO; SHACL-validated). They degrade with a clear error when the extra isn't installed.
The claim/standardization tools — `extract_claims_from_passages` and `export_astra` — need the [`indicium`](../indicium) standard package (typed claims: Bucur 5-slot SuperPattern + ECO/CiTO/SEPIO; SHACL-validated). They degrade with a clear error when it is absent, which is the normal state for anyone outside the lab.

`indicium` and its two adapter siblings are private and unpublished, so they are **not** declared in `pyproject.toml` — `uv lock` resolves every extra, so one undeclarable requirement breaks `uv sync` on every fresh clone (issue #29). Maintainers who hold the sibling checkouts install them into the synced venv instead:

```bash
uv sync --extra dev --extra graph
uv pip install -e ../indicium -e ../indicium-adapters -e ../indicium-adapters-metabolomics
uv sync --inexact --extra dev --extra graph # --inexact, or uv prunes the editable installs
```

The RDF backend behind the claim graph (`rdflib` + `pyoxigraph`) *is* public: `uv sync --extra graph`.

For using Perspicacité over MCP intelligently (query shaping, tool/mode choice), follow [.claude/skills/perspicacite-mcp/SKILL.md](.claude/skills/perspicacite-mcp/SKILL.md) or call the `get_usage_guide` tool for the live source of truth.

Expand Down
66 changes: 33 additions & 33 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -123,33 +123,41 @@ browser = [
"playwright>=1.40",
]

# indicium — Knowledge Graph / SHACL validation layer for structured claim
# representation. Local sibling package (not yet on PyPI). Pulls in linkml,
# pyshacl, and rdflib. Install only when KG-backed workflows are needed.
#
# Local packages are available at ../indicium, ../indicium-adapters, and
# ../indicium-adapters-metabolomics. The `extract_claims_from_passages`
# and `export_astra` MCP tools degrade gracefully without these extras.
# graph — RDF/SPARQL backend for the claim-graph layer. indicium_layer/store.py
# imports rdflib at module level and pyoxigraph for the production store. Both
# are on PyPI, so this extra resolves for every user.
#
# Install with:
# uv sync --extra indicia --extra adapters
indicia = [
# Floor: 1.8.0 is the first release that ships the verify_quote kernel that
# the R3 anchor path (anchor_claims / annotate_anchor_status) imports. An
# older indicium would make anchoring fail-soft to "unchecked" silently.
"indicium>=1.8.0",
"pyoxigraph>=0.4",
]

# indicium-adapters — optional domain adapter protocol. Core package (>=0.4.0)
# no longer ships the metabolomics adapters; those live in the sibling package
# indicium-adapters-metabolomics (MetabolomicsAdapter, MetaboKGAdapter,
# EMIAdapter). Both are needed so discover_adapters() returns all three.
# Enables domain_adapter= kwarg in extract_claims().
adapters = [
"indicium-adapters>=0.4.0",
"indicium-adapters-metabolomics>=0.1.0",
]
# uv sync --extra graph
graph = [
"rdflib>=6.0",
"pyoxigraph>=0.4",
]

# The indicium stack — indicium, indicium-adapters,
# indicium-adapters-metabolomics — backs validate_claims(), build_claim_graph()
# and export_astra(). Those packages are private and absent from PyPI, so they
# are deliberately NOT declared here, in any extra, and NOT wired up through
# [tool.uv.sources]. `uv lock` resolves every extra regardless of which one is
# requested, so a single undeclarable requirement breaks `uv sync` on every
# fresh clone — issue #29, and issue #5 before it.
#
# Never re-add them. tests/unit/test_pyproject_packaging.py and the
# `uv-resolve` CI job fail if they come back.
#
# The PyPI name `indicium` belongs to an unrelated project (a key-value store,
# latest 0.1.0a3), so a bare `indicium` requirement installs the wrong package.
#
# Maintainers holding the private sibling checkouts install them into the
# synced venv, without editing this file:
#
# uv sync --extra dev --extra graph
# uv pip install -e ../indicium -e ../indicium-adapters \
# -e ../indicium-adapters-metabolomics
#
# Every later sync must pass --inexact, or uv prunes those editable installs:
#
# uv sync --inexact --extra dev --extra graph

# docling — high-fidelity PDF -> structured document conversion (layout,
# tables, sections) for the content pipeline. Heavier than the other
Expand Down Expand Up @@ -191,14 +199,6 @@ packages = ["src/perspicacite"]
# Safe while the [scilex] extra is unused by the core / KB / MCP paths.
override-dependencies = ["httpx>=0.28.1"]

[tool.uv.sources]
# Local-path sources for the indicium stack. All three sibling packages are
# available at the paths below (editable installs). Pair with the `indicia`
# and `adapters` extras above: uv sync --extra indicia --extra adapters
indicium = { path = "../indicium", editable = true }
indicium-adapters = { path = "../indicium-adapters", editable = true }
indicium-adapters-metabolomics = { path = "../indicium-adapters-metabolomics", editable = true }

[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
Expand Down
16 changes: 8 additions & 8 deletions src/perspicacite/mcp/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -6286,7 +6286,7 @@ async def extract_claims_from_passages(
try:
from perspicacite.pipeline.claims import extract_claims, validate_claims
except ImportError:
return _json_error("indicium not installed; reinstall with the 'indicia' extra")
return _json_error("indicium is not installed; it is a private, maintainer-only package")

# Resolve domain adapter(s) — compose if multiple (best-effort; skip if not installed)
adapter = None
Expand Down Expand Up @@ -6347,7 +6347,7 @@ async def export_astra(claims: list[dict]) -> str:
try:
from indicium import claim_to_insight
except ImportError:
return _json_error("indicium not installed; reinstall with the 'indicia' extra")
return _json_error("indicium is not installed; it is a private, maintainer-only package")
insights = []
for i, c in enumerate(claims):
c = {**c, "id": c.get("id", f"c{i}")}
Expand Down Expand Up @@ -6385,7 +6385,7 @@ async def build_claim_graph(
try:
import indicium # noqa: F401
except ImportError:
return _json_error("indicia extra not installed; uv sync --extra indicia")
return _json_error("indicium is not installed; it is a private, maintainer-only package")

import pathlib

Expand Down Expand Up @@ -6466,7 +6466,7 @@ async def claim_graph_status(kb_name: str) -> str:
try:
import indicium # noqa: F401
except ImportError:
return _json_error("indicia extra not installed; uv sync --extra indicia")
return _json_error("indicium is not installed; it is a private, maintainer-only package")
from perspicacite.indicium_layer.invalidation import schema_version_changed
from perspicacite.indicium_layer.manifest import read_manifest

Expand Down Expand Up @@ -6519,7 +6519,7 @@ async def query_claim_graph(
try:
import indicium # noqa: F401
except ImportError:
return _json_error("indicia extra not installed; uv sync --extra indicia")
return _json_error("indicium is not installed; it is a private, maintainer-only package")

from perspicacite.indicium_layer import queries as _q

Expand Down Expand Up @@ -6570,7 +6570,7 @@ async def get_claim_figures(
try:
import indicium # noqa: F401
except ImportError:
return _json_error("indicia extra not installed; uv sync --extra indicia")
return _json_error("indicium is not installed; it is a private, maintainer-only package")

from perspicacite.indicium_layer.queries import figures_for_claim

Expand Down Expand Up @@ -6616,7 +6616,7 @@ async def get_claim_links(
try:
import indicium # noqa: F401
except ImportError:
return _json_error("indicia extra not installed; uv sync --extra indicia")
return _json_error("indicium is not installed; it is a private, maintainer-only package")

from perspicacite.indicium_layer.queries import claim_links_for_claim

Expand Down Expand Up @@ -6657,7 +6657,7 @@ async def claim_graph_export(
try:
import indicium # noqa: F401
except ImportError:
return _json_error("indicia extra not installed; uv sync --extra indicia")
return _json_error("indicium is not installed; it is a private, maintainer-only package")

_valid_formats = ("nquads", "turtle", "jsonld", "rocrate")
if format not in _valid_formats:
Expand Down
9 changes: 5 additions & 4 deletions src/perspicacite/rag/modes/reasoning/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
Subplan A ships ``provenance`` and ``contradiction``; unshipped strategies
return a single error StreamEvent pointing to the planned sprint.

The indicia extra (``indicium`` + ``pyoxigraph``) is required. When missing,
we yield a friendly error event explaining how to enable.
The private ``indicium`` package is required, alongside ``pyoxigraph`` from the
``graph`` extra. When either is missing we yield a friendly error event.
"""

from __future__ import annotations
Expand Down Expand Up @@ -70,8 +70,9 @@ async def execute_stream( # type: ignore[override]
data=json.dumps(
{
"message": (
"Reasoning mode requires the 'indicia' extra. "
"Install with: uv sync --extra indicia"
"Reasoning mode requires indicium, a private "
"maintainer-only package, plus pyoxigraph "
"(uv sync --extra graph)."
)
}
),
Expand Down
2 changes: 1 addition & 1 deletion src/perspicacite/web/routers/kb.py
Original file line number Diff line number Diff line change
Expand Up @@ -1710,7 +1710,7 @@ async def export_claim_graph(
except ImportError:
raise HTTPException(
status_code=501,
detail="indicia extra not installed; run: uv sync --extra indicia",
detail="indicium is not installed; it is a private, maintainer-only package",
) from None

_valid = ("nquads", "turtle", "jsonld", "rocrate")
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/test_claims_validation.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ def test_claims_to_graph_routes_domain_qualifier_to_domain_slot():
def test_domain_qualifier_claim_conforms_end_to_end():
"""A domain-qualifier claim round-trips through indicium SHACL (Reading 1).

Requires the `indicia` extra; skipped otherwise. Proves the routed
Requires the private indicium package; skipped otherwise. Proves the routed
asb:domainQualifier value is accepted by indicium's open snake_case slot."""
pytest.importorskip("indicium")
claims = [{"context": "c", "subject": "s", "qualifier": "aligned_with",
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/test_mcp_claim_graph_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import json
import pytest

pyoxigraph = pytest.importorskip("pyoxigraph", reason="pyoxigraph (indicia extra) not installed")
pyoxigraph = pytest.importorskip("pyoxigraph", reason="pyoxigraph (graph extra) not installed")


async def test_build_claim_graph_returns_summary(monkeypatch, tmp_path):
Expand Down
75 changes: 75 additions & 0 deletions tests/unit/test_pyproject_packaging.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
"""Guards the packaging metadata that keeps `uv sync` working on fresh clones.

Regression test for issue #29 (and #5 before it). uv resolves *every* extra and
*every* [tool.uv.sources] entry when it locks, regardless of which extra the
user asked for. So a local-path source, or a requirement on a package that is
not published, makes `uv sync` fail for anyone without the private sibling
checkouts — before a single line of Python runs.
"""

import re
import tomllib
from pathlib import Path

PYPROJECT_PATH = Path(__file__).resolve().parents[2] / "pyproject.toml"

# Private, unpublished siblings of this repository. See the pyproject comment.
PRIVATE_DISTRIBUTIONS = frozenset(
{"indicium", "indicium-adapters", "indicium-adapters-metabolomics"}
)


def _load_pyproject() -> dict:
"""Parse the repository's pyproject.toml."""
return tomllib.loads(PYPROJECT_PATH.read_text(encoding="utf-8"))


def _uses_local_path(source_spec) -> bool:
"""True when a [tool.uv.sources] entry points at a filesystem path."""
entries = source_spec if isinstance(source_spec, list) else [source_spec]
return any(isinstance(entry, dict) and "path" in entry for entry in entries)


def _distribution_name(requirement: str) -> str:
"""Return the normalised distribution name of a PEP 508 requirement."""
name = re.split(r"[\s\[<>=!~;@]", requirement, maxsplit=1)[0]
return name.strip().lower().replace("_", "-")


def _declared_requirements(pyproject: dict) -> list[str]:
"""Every requirement uv resolves: base deps, extras, and dependency-groups."""
project = pyproject["project"]
requirements = list(project.get("dependencies", []))
for extra_requirements in project.get("optional-dependencies", {}).values():
requirements.extend(extra_requirements)
for group_requirements in pyproject.get("dependency-groups", {}).values():
# PEP 735 groups may also hold {"include-group": ...} tables; skip those.
requirements.extend(r for r in group_requirements if isinstance(r, str))
return requirements


def test_no_local_path_uv_sources():
"""Local-path sources break `uv sync` for clones without the siblings."""
sources = _load_pyproject().get("tool", {}).get("uv", {}).get("sources", {})
offenders = sorted(name for name, spec in sources.items() if _uses_local_path(spec))
assert not offenders, (
f"[tool.uv.sources] entries {offenders} point at local paths, which makes "
"`uv sync` fail on any fresh clone lacking those sibling directories"
)


def test_no_private_distributions_declared():
"""Unpublished packages must not appear in any uv-resolved requirement."""
declared = {_distribution_name(req) for req in _declared_requirements(_load_pyproject())}
leaked = sorted(declared & PRIVATE_DISTRIBUTIONS)
assert not leaked, (
f"{leaked} are private and absent from PyPI; declaring them in any extra "
"or dependency-group makes `uv lock` unsatisfiable for every external user"
)


def test_graph_extra_declares_the_rdf_backend():
"""indicium_layer/store.py imports rdflib and pyoxigraph directly."""
extras = _load_pyproject()["project"]["optional-dependencies"]
graph_distributions = {_distribution_name(req) for req in extras["graph"]}
assert {"rdflib", "pyoxigraph"} <= graph_distributions
Loading
Loading