Harden packaging: audit out-of-package runtime data + clean-install smoke test - #82
Merged
Merged
Conversation
…l smoke test
Audit (no runtime change): traced every user-reachable CLI entry point
(permea, permea bench, permea bench diagnose, and the separate permea-drylab
script) for reads of data living outside the shipped package trees
(permea_core, permea_explain, permea_ui). Verdict: none. `permea bench
diagnose` reads only user-supplied dataset/clusters; its downstream
run()/diagnose()/warnings registry are pure Python. The many repo-root-relative
path constants (ROOT = parents[3], Path("docs/examples/generated"),
Path("benchmarks/*.yaml"), ...) all live in artifact-generation modules driven
by scripts/, not exposed through the CLI. A clean-venv sweep from outside the
repo confirmed no FileNotFoundError / ModuleNotFoundError / missing-resource on
any command. Because nothing broke, no files were moved and no package-data rule
was added -- moving unused data would ship files nothing needs.
Changes:
- pyproject.toml: add a `dev` optional-dependency group (build, twine, pytest)
so a canonical sdist+wheel can be produced and CI can run the suite. No upload.
- pyproject.toml: register `slow` and `packaging` pytest markers.
- tests/test_packaging_smoke.py: one clean-install smoke test (marked
slow+packaging) that builds a real wheel, installs it into a throwaway venv,
and drives `permea bench diagnose` from outside the repo on files produced by
permea_ui.fixtures.write_example, asserting exit 0 and a fired warning code.
The existing 562 tests run against an editable install and structurally cannot
catch a packaging regression; this closes that gap. It gates on a real
`python -m build` capability probe (not `import build`) so it skips cleanly
when the dev extra is absent and runs in CI where `.[dev]` is installed.
Hygiene:
- Remove stray src/permea_core/.DS_Store (untracked; .DS_Store already ignored).
- Delete finished untracked doc docs/pr/PR-permea-explain-and-packaging.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The prior commit's tests/test_packaging_smoke.py claimed "CI installs the dev extra", but the repo had no .github/workflows. The test skips whenever `python -m build` is absent, so on the maintainer machine it skipped and ran nowhere else -- it guarded nothing and that comment was not verifiable. This adds the CI that makes the claim true. Audit (Phase A): the suite is fully offline -- no test makes a live LLM provider call and none requires a secret. The provider tests clear/monkeypatch PERMEA_LLM_* and use fakes; test_explain_provider_configured is "strictly offline" by construction; test_explain_boundary asserts openai/anthropic/httpx are NOT imported. So CI needs no secrets. Findings while wiring CI (pip install ".[dev]" alone could not even COLLECT the suite): - The test suite imports feature-extra deps unconditionally (fastapi in test_ui_drylab, jsonschema in several test_explain_*), and PyYAML -- an OPTIONAL runtime dep of permea_core guarded behind `try: import yaml` -- is declared in no extra at all. With only `.[dev]` these produced 14 collection errors, not skips. - Fix: make `dev` a self-referential superset -- `permea-core[explain,ui]` plus pyyaml plus build/twine/pytest -- so `pip install .[dev]` yields an environment that can run the ENTIRE suite in one command. Base `dependencies` are untouched: PyYAML stays optional for CLI users; the diagnose path never needs it. Changes: - .github/workflows/ci.yml: CI on pull_request + push to main, ubuntu-latest, matrix over Python 3.11/3.12/3.13 (per requires-python >=3.11). Steps: checkout, setup-python, `pip install ".[dev]"`, then two pytest steps -- `-m "not packaging"` (fast) and `-m packaging` (the wheel build/install test, kept separate so a packaging regression is distinguishable in the log). No secrets; only actions/checkout and actions/setup-python. - pyproject.toml: expand the `dev` extra as above. - tests/test_packaging_smoke.py: the "CI installs the dev extra" statement is now true (workflow added); docstring now states plainly that the test SKIPS when `python -m build` is unavailable, so a green local run does not imply it ran, and points at the CI workflow where it actually executes. Test not weakened. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase A verdict
Does anything break in a clean install? NO. No user-reachable CLI command reads data that lives outside the shipped package trees (
permea_core,permea_explain,permea_ui).Audited every entry point —
permea,permea bench,permea bench diagnose, and the separatepermea-drylabscript:permea/permea bench/--helppermea bench diagnose DATASET CLUSTERSrun()→diagnose()→contracts/warnings.pyare pure Pythonpermea-drylabpermea_ui/static/, already shipped via package-data); temp dirs + user uploads onlyThe many repo-root-relative path constants found by grep (
ROOT = Path(__file__).resolve().parents[3],Path("docs/examples/generated"),Path("benchmarks/*.yaml"),Path("acquisition_manifests"), …) all live in artifact-generation modules driven byscripts/, none of which is reachable from thepermeaCLI. A clean-venv command sweep from outside the repo produced noFileNotFoundError/ModuleNotFoundError/ missing-resource error.What changed
pyproject.toml: added adevoptional-dependency group (build,twine,pytest) so a canonical sdist+wheel can be produced and CI can run the suite. No upload to PyPI in this task.pyproject.toml: registeredslowandpackagingpytest markers.tests/test_packaging_smoke.py: one clean-install smoke test (markedslow+packaging) — builds a real wheel, installs it into a throwaway venv, and drivespermea bench diagnosefrom outside the repo on files produced bypermea_ui.fixtures.write_example, asserting exit 0 and a fired warning code. It gates on a realpython -m buildcapability probe (notimport build), so it skips cleanly without thedevextra and runs in CI where.[dev]is installed. This exists because the existing suite runs against an editable install and structurally cannot catch a packaging regression.src/permea_core/.DS_Store(.DS_Storealready in.gitignore); deleted the finished untracked docdocs/pr/PR-permea-explain-and-packaging.md.Deliberately left alone
package-datarule added — Phase A found no breakage, so moving data would ship files nothing needs. (Explicitly out of scope:schemas/,examples/,benchmarks/,sources/, etc. are used only byscripts/, never by the wheel's CLI.)permea_explain/guardrails.pyverified unmodified (sha25603468738…b2b2732).Test count
build; it was exercised for real in a venv with thedevextra (1 passed in ~89s) and will run in CI.🤖 Generated with Claude Code