Skip to content

Commit 694de06

Browse files
author
FirstUnicorn
committed
feat(docs): add Sphinx documentation setup plan for 17-package monorepo
Introduced a comprehensive plan for setting up Sphinx documentation using sphinx-autoapi. The plan outlines a multi-stage approach for gathering context from existing code and documentation, scaffolding the necessary infrastructure, and creating both auto-generated API references and hand-written guides. Key stages include context gathering, scaffolding, and guide creation for packages with and without existing documentation.
1 parent 0e54e30 commit 694de06

1 file changed

Lines changed: 254 additions & 0 deletions

File tree

‎.cursor/plans/sphinx_documentation_setup_11ca65fe.plan.md‎

Lines changed: 254 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,254 @@
1+
---
2+
name: Sphinx documentation setup
3+
overview: "Set up Sphinx documentation with sphinx-autoapi for the 17-package monorepo, hosted on Read the Docs. The plan uses a multi-stage approach: first gather full context from all package source files, then generate hand-written guides + auto-generated API reference."
4+
todos:
5+
- id: ctx-batch-1
6+
content: "Stage 1 batch 1: read all __init__.py exports + pyproject.toml metadata (17 packages)"
7+
status: completed
8+
- id: ctx-batch-2
9+
content: "Stage 1 batch 2: read all 134 source files docstrings and signatures (parallel agents by package group)"
10+
status: completed
11+
- id: ctx-batch-3
12+
content: "Stage 1 batch 3: read ALL existing docs -- cqrs-core/docs (4), sqlalchemy-repos (4), outbox-core/detailed-docs (5), EXCEPTION_COMPARISON, all 17 READMEs"
13+
status: completed
14+
- id: ctx-batch-4
15+
content: "Stage 1 batch 4: read root README.md (829 lines) + import-linter contracts in pyproject.toml"
16+
status: completed
17+
- id: scaffold
18+
content: "Stage 2: scaffold -- add docs deps, create docs/conf.py, Makefile, .readthedocs.yaml, update .gitignore"
19+
status: completed
20+
- id: guides-rich
21+
content: "Stage 3a: create thin wrapper pages for 3 rich-docs packages (cqrs-core, sqlalchemy-repos, outbox-core) that toctree-include existing .md"
22+
status: completed
23+
- id: guides-new
24+
content: "Stage 3b: create new guide pages for 14 README-only packages (content from README + docstrings)"
25+
status: completed
26+
- id: guides-top
27+
content: "Stage 3c: create top-level pages -- index.rst, quickstart.md, architecture.md, exception-comparison.md"
28+
status: completed
29+
- id: compat-fixes
30+
content: "Stage 3d: fix existing .md files for myst-parser compatibility (heading levels, emoji, HTML tags)"
31+
status: completed
32+
- id: build-verify
33+
content: "Stage 4: build and verify -- run sphinx build, fix warnings, verify autoapi + toctree rendering"
34+
status: completed
35+
isProject: false
36+
---
37+
38+
# Sphinx documentation for python-web-toolkit monorepo
39+
40+
## Current state
41+
42+
- No Sphinx setup exists anywhere in the repo
43+
- No docs dependencies in [pyproject.toml](pyproject.toml)
44+
- The repo uses Poetry (root) + setuptools (per-package builds)
45+
- 17 packages, 134 source files total, all under `packages/*/src/`
46+
- Inter-package deps: only `python-cqrs-dispatcher` depends on `python-cqrs-core` + `python-mediator`
47+
48+
### Existing markdown documentation (MUST be reused, not duplicated)
49+
50+
**Packages with rich docs (3):**
51+
52+
- **python-cqrs-core** -- `packages/python-cqrs-core/docs/` (4 files):
53+
- `usage-guide.md` (298 lines, full examples for commands/queries/pagination/tracing)
54+
- `api-reference.md` (interfaces, base classes)
55+
- `observability.md` (OTel/Sentry/Prometheus integration)
56+
- `why-use-base-classes.md` (design rationale)
57+
- **sqlalchemy-async-repositories** -- root-level docs (4 files):
58+
- `ARCHITECTURE.md` (design patterns, interactions)
59+
- `HYBRID_IMPLEMENTATION_SUMMARY.md` (hybrid approach details)
60+
- `REFACTORING_SUMMARY_FACTORY_PATTERN.md` (factory pattern)
61+
- `MISSED FROM ARCHITECTURE.md` (supplementary architecture)
62+
- **python-outbox-core** -- `QUICKREF.md` + `detailed-docs/` (5 files):
63+
- `QUICKREF.md` (quick reference card)
64+
- `detailed-docs/API_REFERENCE.md` (full API reference)
65+
- `detailed-docs/IMPLEMENTATION_GUIDE.md` (implementation checklist)
66+
- `detailed-docs/faq/OUTBOX_VS_TASK_QUEUES.md` (comparison)
67+
- `detailed-docs/faq/SERIALIZATION_NOTES.md` (Pydantic/FastStream)
68+
69+
**Cross-package doc (1):**
70+
71+
- `packages/EXCEPTION_LIBRARIES_COMPARISON.md` (python-app-exceptions vs python-infrastructure-exceptions)
72+
73+
**README only (14 packages):** each has a `README.md` with description, installation, basic usage.
74+
75+
### Strategy for existing docs
76+
77+
- myst-parser includes existing `.md` files directly via toctree -- NO copying/duplicating
78+
- Sphinx toctree entries point to relative paths into `packages/` using `../packages/...`
79+
- Each package's guide page in `docs/packages/` will toctree-include its existing docs as sub-pages
80+
- Minor edits to existing `.md` files only if needed for myst compatibility (e.g., adding myst front-matter or fixing heading levels)
81+
82+
## Architecture
83+
84+
```
85+
docs/
86+
├── conf.py # sphinx config (autoapi, myst, furo theme)
87+
├── index.rst # root toctree
88+
├── installation.rst # pip install instructions (TestPyPI/PyPI)
89+
├── quickstart.md # getting started guide (myst)
90+
├── architecture.md # layer diagram, dependency graph
91+
├── exception-comparison.md # symlink/include of packages/EXCEPTION_LIBRARIES_COMPARISON.md
92+
├── packages/ # per-package guide pages (thin wrappers)
93+
│ ├── index.rst # package listing toctree
94+
│ ├── python-app-exceptions.md # new guide (README-only pkg)
95+
│ ├── python-technical-primitives.md # new guide (README-only pkg)
96+
│ ├── python-cqrs-core.md # toctree wrapper -> includes existing docs/
97+
│ ├── sqlalchemy-async-repositories.md # toctree wrapper -> includes existing .md files
98+
│ ├── python-outbox-core.md # toctree wrapper -> includes existing detailed-docs/
99+
│ └── ... (one .md per package)
100+
├── _static/
101+
└── _templates/
102+
```
103+
104+
**Key: packages WITH existing docs get thin wrapper pages that toctree-include their existing .md files.
105+
Packages WITHOUT existing docs get a new guide page with content extracted from README + docstrings.**
106+
107+
Auto-generated API reference lands in `docs/autoapi/` (gitignored, built at build time).
108+
109+
## Key technical decisions
110+
111+
- **sphinx-autoapi** (not sphinx.ext.autodoc): works without importing packages, ideal for monorepo where not all deps are installed. `autoapi_dirs` will point to all 17 `packages/*/src/` directories
112+
- **myst-parser**: allows writing guides in `.md` alongside `.rst` for toctrees
113+
- **furo** theme: clean, modern, RTD-compatible
114+
- **readthedocs.yaml v2**: config for RTD builds with Poetry
115+
116+
## Stage 1: context gathering from code
117+
118+
Before writing any documentation files, systematically grep/read in batches:
119+
120+
**Batch 1 -- public API surface (already partially gathered):**
121+
122+
1. Every `__init__.py` public API exports (17 packages -- done in planning)
123+
2. Each package's `pyproject.toml` for name, version, description, dependencies
124+
125+
**Batch 2 -- docstrings and signatures (134 source files, parallel agents):**
126+
3. All source modules' docstrings and class/function signatures grouped by package
127+
4. Focus on: class docstrings, method signatures with type hints, module-level docstrings
128+
129+
**Batch 3 -- existing documentation (MUST read before writing anything):**
130+
5. All 4 files in `packages/python-cqrs-core/docs/` (full read)
131+
6. All 4 architecture docs in `packages/sqlalchemy-async-repositories/` (full read)
132+
7. All files in `packages/python-outbox-core/detailed-docs/` + `QUICKREF.md` (full read)
133+
8. `packages/EXCEPTION_LIBRARIES_COMPARISON.md` (full read)
134+
9. Every package `README.md` (17 files -- extract usage examples)
135+
136+
**Batch 4 -- architecture context:**
137+
10. The root [README.md](README.md) before/after examples (all 829 lines)
138+
11. The import-linter contracts in [pyproject.toml](pyproject.toml) (layer boundaries)
139+
140+
This context feeds into:
141+
142+
- Stage 2: knowing which existing .md files need myst compatibility fixes
143+
- Stage 3: writing new guide pages for README-only packages
144+
- Stage 3: writing accurate quickstart.md and architecture.md
145+
146+
## Stage 2: scaffold sphinx infrastructure
147+
148+
1. Add docs dependencies to root `pyproject.toml` under `[tool.poetry.group.docs.dependencies]`:
149+
- `sphinx`, `sphinx-autoapi`, `myst-parser`, `furo`, `sphinx-copybutton`
150+
2. Create `docs/conf.py` with:
151+
152+
```python
153+
extensions = ["autoapi.extension", "myst_parser", "sphinx_copybutton"]
154+
autoapi_dirs = [
155+
"../packages/python-technical-primitives/src",
156+
"../packages/python-app-exceptions/src",
157+
# ... all 17 packages
158+
]
159+
autoapi_type = "python"
160+
autoapi_options = ["members", "undoc-members", "show-inheritance", "show-module-summary"]
161+
html_theme = "furo"
162+
myst_enable_extensions = ["colon_fence", "deflist"]
163+
164+
165+
```
166+
167+
1. Create `docs/Makefile` and `docs/make.bat` (standard sphinx)
168+
2. Create `.readthedocs.yaml` at repo root:
169+
170+
```yaml
171+
version: 2
172+
build:
173+
os: "latest"
174+
tools:
175+
python: "3.10"
176+
sphinx:
177+
configuration: docs/conf.py
178+
python:
179+
install:
180+
- method: pip
181+
path: .
182+
extra_requirements:
183+
- docs
184+
185+
186+
```
187+
188+
1. Add `docs/autoapi/` and `docs/_build/` to [.gitignore](.gitignore)
189+
190+
## Stage 3: hand-written guides (context dump)
191+
192+
### For packages WITH existing docs (3 packages) -- thin wrapper approach:
193+
194+
Create `docs/packages/{pkg-name}.md` as a **toctree wrapper** that:
195+
196+
- Has a short intro paragraph (description from pyproject.toml)
197+
- Installation command
198+
- Links to autoapi-generated API reference
199+
- Uses myst `toctree` directive to include existing docs as sub-pages:
200+
201+
```md
202+
# python-cqrs-core
203+
204+
CQRS interfaces and base classes with tracing support.
205+
206+
## Installation
207+
pip install python-cqrs-core
208+
209+
```{toctree}
210+
:maxdepth: 2
211+
212+
../../packages/python-cqrs-core/docs/usage-guide
213+
../../packages/python-cqrs-core/docs/api-reference
214+
../../packages/python-cqrs-core/docs/observability
215+
../../packages/python-cqrs-core/docs/why-use-base-classes
216+
```
217+
218+
Same pattern for `sqlalchemy-async-repositories` (4 docs) and `python-outbox-core` (QUICKREF + detailed-docs/).
219+
220+
### For packages WITHOUT existing docs (14 packages) -- new guide pages:
221+
222+
Create `docs/packages/{pkg-name}.md` containing:
223+
224+
- One-line description (from pyproject.toml)
225+
- Installation command
226+
- Public API summary table (class/function, one-liner)
227+
- Usage examples (extracted from that package's README.md before/after sections + docstrings)
228+
- Cross-links to auto-generated API ref via autoapi roles
229+
230+
### Top-level pages:
231+
232+
- `index.rst`: main toctree linking to quickstart, architecture, packages, exception-comparison, API reference
233+
- `quickstart.md`: install instructions, minimal working example
234+
- `architecture.md`: layer diagram (primitives -> domain -> application -> infrastructure), import-linter contracts, inter-package dependency graph
235+
- `exception-comparison.md`: include of `packages/EXCEPTION_LIBRARIES_COMPARISON.md`
236+
237+
### Existing .md compatibility fixes (if needed):
238+
239+
Some existing docs may need minor adjustments for myst-parser:
240+
241+
- Heading level consistency (ensure no duplicate H1 within a toctree)
242+
- GitHub-flavored markdown features that myst doesn't support (e.g., `<details>` tags may need `{raw} html` directive)
243+
- Remove emoji shortcuts if they break rendering
244+
245+
## Stage 4: build and verify
246+
247+
1. `cd docs && make html` -- verify no warnings
248+
2. Check auto-generated API pages render correctly for all 17 packages
249+
3. Verify cross-references between hand-written guides and autoapi pages
250+
4. Test RTD build locally with `readthedocs-build`
251+
252+
## File size discipline
253+
254+
All files under 75 lines. If a package guide exceeds this, split into `docs/packages/{pkg-name}/index.md` + sub-pages.

0 commit comments

Comments
 (0)