docs, test: harden newcomer onboarding (#194, #202, #203, #208, #209, #212)#216
Merged
Conversation
…212) Six tightly-coupled documentation and metadata fixes that share the README, pyproject.toml, and docs/ surface — bundled so the smoke tests (#203) can guard the doc fixes (#194, #202, #209, #212) from regressing the moment they land, and so the new runtime-responsibilities page (#208) is anchored into both the README and the mkdocs nav. #194 README inspect example now explains the in-memory registry seam, links to `docs/cli.md#programmatic-registration`, and shows a working `set_default_registry` entry-script snippet next to the `inspect`/`viz` CLI lines. #202 README opens with a concrete `search → extract → validate → format` before/after narrative (problem in one sentence, naive 4-LLM-call trace, compiled 1-LLM-call trace, copy-paste path) before the prose "Why ChainWeaver?" section. #203 New `tests/test_readme_smoke.py` extracts `<!-- smoke-test: run -->`-marked Python blocks (allowlist by inline marker per the issue's "small allowlist mechanism" criterion) and runs each in a fresh subprocess; covers `chainweaver --help` (asserts every registered command appears) and `chainweaver validate` against the shipped example, and checks the bundled examples listed in the README all exist. New `tests/test_metadata_consistency.py` asserts pyproject.toml ↔ chainweaver.__version__ ↔ latest CHANGELOG release heading agree. #208 New `docs/runtime-responsibilities.md` (5 sections + concrete `handle_request` example) covering host-app responsibilities: when to invoke, trace persistence/redaction, side-effect tools, flows-as-named-operations, MCP parity. Linked from the README MCP section; wired into `mkdocs.yml` nav under "Is this for me?". Anti-drift test in `tests/test_runtime_responsibilities_doc.py`. #209 `pyproject.toml` `[project.urls]` now exposes Documentation, Source, Changelog, and Issues alongside Homepage/Bug Tracker so the PyPI sidebar and `pip show chainweaver` link users to the right places; `keywords` expanded. README §Installation now lists every shipped optional extra (`yaml`, `otel`, `mcp`, `contrib`, `langchain`, `llamaindex`, `test`, `docs`, `dev`) with the transitive deps each pulls. Also bumps `chainweaver.__version__` from "0.9.0" → "0.10.0" to match the release commit in `pyproject.toml` (caught by the new metadata-consistency test). #212 New README "Where ChainWeaver fits in the Weaver Stack" subsection placing this repo on the deterministic-execution seam of the broader stack (weaver-spec / contextweaver / agent-kernel / lessonweaver), with the existing `weaver-stack` extra placeholder called out as the future install seam. All four validation commands clean: - ruff check chainweaver/ tests/ examples/ → All checks passed - ruff format --check chainweaver/ tests/ examples/ → 152 files OK - python -m mypy chainweaver/ tests/ → 125 files, no issues - python -m pytest tests/ -q --no-cov → 1107 passed Closes #194, #202, #203, #208, #209, #212.
Contributor
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'ChainWeaver microbenchmarks'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.25.
| Benchmark suite | Current: 19378b5 | Previous: 530174a | Ratio |
|---|---|---|---|
compiled_total_ms_n2_llm100_tool0 |
0.18995399994992113 ms |
0.14817599998195874 ms |
1.28 |
compiled_overhead_ms_n2_llm100_tool0 |
0.12493199994878523 ms |
0.08674199997926735 ms |
1.44 |
compiled_overhead_ms_n5_llm500_tool50 |
0.36212600002727413 ms |
0.21608299999797964 ms |
1.68 |
This comment was automatically generated by workflow using github-action-benchmark.
CC: @dgenio
There was a problem hiding this comment.
Pull request overview
This PR hardens newcomer onboarding by updating top-level docs and package metadata and adding anti-drift tests that exercise README snippets, verify metadata consistency, and ensure the new runtime-responsibilities doc stays wired into the docs site/navigation.
Changes:
- Add smoke/anti-drift tests for README runnable snippets, CLI help/validate, metadata/version consistency, and the runtime-responsibilities doc presence/structure.
- Expand README onboarding content (30-second narrative, installation extras table, Weaver Stack positioning) and add the new
docs/runtime-responsibilities.mdpage wired intomkdocs.yml. - Update
pyproject.tomlkeywords/URLs and bumpchainweaver.__version__to match the package version.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_runtime_responsibilities_doc.py |
Adds anti-drift assertions for the new runtime-responsibilities doc page and its references. |
tests/test_readme_smoke.py |
Adds opt-in README fenced-python execution + CLI and bundled-example smoke checks. |
tests/test_metadata_consistency.py |
Adds version/URL consistency checks across pyproject.toml, runtime __version__, and CHANGELOG.md. |
README.md |
Updates onboarding narrative, installation extras guidance, Weaver Stack context, and links to runtime responsibilities; adds smoke-test markers. |
pyproject.toml |
Expands keywords and adds [project.urls] entries for user navigation. |
mkdocs.yml |
Adds the runtime responsibilities page to the published nav. |
docs/runtime-responsibilities.md |
Introduces a host-responsibilities guide for embedding ChainWeaver (invocation, traces, side effects, MCP parity). |
chainweaver/__init__.py |
Bumps __version__ to 0.10.0. |
… + smoke contract
- docs/runtime-responsibilities.md:
* Tighten the Redaction bullet: `RedactionPolicy` is dict-only (`redact(dict)`),
hosts must walk `result.execution_log` themselves and apply the policy per
step. Previous wording suggested `RedactionPolicy` could redact
`ExecutionResult` / `StepRecord` directly (Copilot review thread, line 57).
* Rewrite the host example to match real APIs: drop the non-existent
`redactor.redact_execution_result(result)` call (line 182) and the wrong
`redact_fields=...` constructor (line 197). Walk `execution_log`, call
`replace()` on each step with `redact()`-ed `inputs`/`outputs`, instantiate
with `redact_keys=frozenset({...})` — these match `chainweaver.log_utils`.
* Replace "compiling tool chains into flows" → "compiling sequences of tool
calls into flows" so the page does not contradict the enforced domain
vocabulary ("flow", never "chain"/"pipeline" — see review-checklist.md,
line 110).
* Grammar fix: "an `dry_run: bool`" → "a `dry_run: bool`" (line 86).
- pyproject.toml: drop "pipeline" from `keywords` — same domain-vocabulary rule.
PyPI sidebar should not advertise terminology the repo forbids internally
(line 22).
- tests/test_readme_smoke.py: tighten the smoke contract so the module
docstring's "no traceback on stderr" promise is true. The test now also
fails if `stderr` contains "Traceback (most recent call last):" even when
the script exits 0 — catches scripts that swallow exceptions with
`traceback.print_exc()` (line 19).
Validation: ruff lint + format + mypy + pytest (1107 passed) on Python 3.11.
6 tasks
Phase-3 follow-ups from the in-memory audit on #216 (no comments were posted on the PR by Phase 2; these address findings A1, N1, N2): - "See it in 30 seconds" output block (A1, V3-adjacent): the prose output is a stylized condensed view rather than verbatim `python examples/simple_linear_flow.py` stdout (which prints timestamps and the structured step log). Add a one-paragraph caption above the fenced block explaining the relationship, so the values stay accurate and the marketing intent stays intact without overclaiming format. - Installation extras table (N1): add a `chainweaver[weaver-stack]` row so the table matches `pyproject.toml`'s `[project.optional-dependencies]`. The extra is a placeholder today (no transitive dep), but omitting it from the README hides a user-visible seam. - Weaver Stack ecosystem table (N2): drop the `#107` reference from the Routing row so the table cross-references exactly the issues the PR description's "Scope notes (Mode B)" enumerates (#91 / #106 / #89 / #210). Validation: ruff lint + format + mypy + pytest (1107 passed).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Six tightly-coupled documentation and metadata fixes that share the README,
pyproject.toml, anddocs/surface — bundled so the new smoke tests (#203) can guard the doc fixes (#194, #202, #209, #212) from regressing the moment they land, and so the new runtime-responsibilities page (#208) is anchored into both the README and the mkdocs nav.Pure docs + tests + package metadata. No runtime code changes. The single touched non-test source file is
chainweaver/__init__.py, where__version__is bumped from"0.9.0"→"0.10.0"to match the release commit inpyproject.toml— the kind of drift the new metadata-consistency test catches.Changes
pyproject.toml[project.urls]addsDocumentation,Source,Changelog,Issuesso PyPI sidebar /pip showlink users to the right place;keywordsexpanded.chainweaver/__init__.py__version__bumped to"0.10.0"to matchpyproject.toml(now caught bytests/test_metadata_consistency.py).README.mdset_default_registrynext to theinspect/vizCLI examples (#194). Two existing Python blocks marked with<!-- smoke-test: run -->so they get exercised by the new smoke test.docs/runtime-responsibilities.md(new)handle_requestexample.mkdocs.ymltests/test_readme_smoke.py(new)<!-- smoke-test: run -->Python blocks and runs each in a fresh subprocess (with the repo onPYTHONPATH); assertschainweaver --helplists every registered command; assertschainweaver validate examples/double_add_format.flow.yamlexits clean from a fresh checkout; asserts all bundled examples the README points to exist; asserts thechainweaver[yaml]install note appears near the first.flow.yamlmention.tests/test_metadata_consistency.py(new)pyproject.tomlversion ↔chainweaver.__version__↔ latest released## [x.y.z]heading in CHANGELOG.md agree, and that[project.urls]exposes the required user-navigation URLs. Uses stdlibtomllibon 3.11+ with a regex fallback for 3.10.tests/test_runtime_responsibilities_doc.py(new)handle_requestexample, the README links to it, andmkdocs.ymlincludes it in nav.Why
All six issues describe distinct first-30-seconds-of-the-repo failure modes that the maintainer-triage process kept reopening (#190 / #192 / #193 / #195 / #196 / #197 closed earlier today were the concrete instances; #194 and the rest in this PR are the next batch plus the systemic guard). Shipping the README polish (#194 / #202 / #209 / #212) alongside the smoke tests (#203) means the class of drift becomes a CI failure mode from day one rather than something that has to be re-discovered by the next contributor.
The runtime-responsibilities page (#208) is the only sizable new doc; it lands here because the README's MCP-integration section now links to it, so the two issues' acceptance criteria are jointly satisfied without orphaning either page.
Testing
ruff check chainweaver/ tests/ examples/) — All checks passedruff format --check chainweaver/ tests/ examples/) — 152 files OKpython -m mypy chainweaver/) — 125 files, 0 issuespython -m pytest tests/ -v) — 1107 passedReproducible verification:
Targeted run of the new suites:
Related Issues
Closes #194 — README
inspectexample assumes a programmatically-registered registry.Closes #202 — Rewrite README landing path around a concrete before/after use case.
Closes #203 — Add README and docs snippet smoke tests to prevent onboarding drift.
Closes #208 — Add runtime responsibilities page.
Closes #209 — Improve package metadata and install docs.
Closes #212 — docs: link ChainWeaver into the Weaver Stack ecosystem map.
Follows directly from the triage in #214's predecessor sweep (six already-fixed issues closed today: #190, #192, #193, #195, #196, #197).
Checklist
AGENTS.mdanddocs/agent-context/) — anti-drift tests mirrortests/test_cli_docs.py's pattern; smoke-test subprocess plumbing mirrorstests/test_cookbook_examples.py's pattern; commit message follows the repo'sdocs, test: …conventional style.__version__bump is metadata, not API.Scope notes (Mode B)
User explicitly authorized Mode B + "one PR for all" + "I don't care about retro compatibility" + "new dependencies if worthy". I did not add any new runtime dependency (the test deps
pytest/pyyamlare already in[dev]). Mode B latitude used only for: (a) bundling all six issues in one PR per the user's explicit request; (b) fixing the__init__.py↔pyproject.tomlversion drift discovered while writing the metadata consistency test, which is in #209's explicit scope ("Ensure package version metadata remains consistent withchainweaver.__version__"); (c) wiring the new docs page into both the README and the mkdocs nav rather than leaving it floating.Tradeoffs / risks
<!-- smoke-test: run -->) over opt-out (allowlist file) because the README has many illustrative Python blocks (16 today) and the asymmetry is asymmetric: missing one illustrative block from the allowlist breaks CI for innocent reasons, while missing one runnable block from the marker just fails to catch a future regression in that block. Marking two blocks today (Quick Start and@tooldecorator); follow-ups can mark more.PYTHONPATH=<repo_root>into the subprocess so README blocks work from a fresh checkout and from an editable install. This matches what a newcomer runningpython -c "$(cat block.py)"from the repo root would experience.tomllibvstomlifor 3.10. Repo supports Python>=3.10buttomlliblands in stdlib at 3.11. Rather than addtomlito[dev]for the single 3.10 leg, I added a small regex fallback in_pyproject_table()that handles the keys the test reads. If 3.10 ever drops from the matrix, the fallback becomes dead code and is one delete away.agent-context/doc-drift. The README and the new page do not duplicatedocs/agent-context/content; that boundary stays intact (agent-context/remainsnot_in_navper the existing mkdocs convention).weaver-specpublishes the ecosystem map, the placeholder link in the same subsection can be promoted to a real URL in a one-line follow-up.🤖 Generated with Claude Code
Generated by Claude Code