feat(docs): guided adoption IA — theme, hubs, section folders, and redirects (#121)

Switch the docs theme to sphinx-immaterial with Databricks styling, landing/quick-start
IA, and native version dropdown. Restructure sources into section folders
(get-started, architecture, build, deploy, features, contributors, samples) with
redirect stubs so legacy URLs keep working; unify Deploy and refresh Build/Features
navigation (job-based Features hub, Patterns catalog, spec-before-patterns ordering).
Replace static pattern PNGs with lf-arch Mermaid flowcharts and client-side rendering;
standardize feature-page chrome, US English spelling, and guided-adoption content
(Quick Start, CI/CD layout, architecture diagram, config-inheritance callout). Fix
multiversion builds: per-tag conf cutover at v0.21.0, shared versions.json back to
v0.12, immaterial version_json resolution, search on flat builds, CI lockfile/spelling,
header logo, version-selector stacking, and dark-mode link colors.

Author: rederik76

.gitignore

@@ -1,5 +1,6 @@
 .databricks/
-build/
+/build/
+docs/build/
 dist/
 __pycache__/
 *.egg-info

VERSION

@@ -1 +1 @@
-v0.20.0
+v0.21.0

docs/Makefile

@@ -35,7 +35,7 @@ spelling:
 html-multiversion:
 	@./scripts/build_versioned_docs.sh
 
-# Local preview build including all local branches.
+# Local preview build: current branch as local-branch-preview (current stays main).
 html-multiversion-preview:
 	@./scripts/build_versioned_docs.sh --preview
 

docs/README.md

@@ -9,7 +9,7 @@ This folder contains the Sphinx documentation source and build tooling for local
 - [Multiversion Implementation (No sphinx-multiversion)](#multiversion-implementation-no-sphinx-multiversion)
   - [What Gets Built](#what-gets-built)
   - [Version Selection Rules](#version-selection-rules)
-  - [How the Sidebar Switcher Works](#how-the-sidebar-switcher-works)
+  - [How the Version Dropdown Works](#how-the-version-dropdown-works)
 - [Purge and Rebuild Behavior](#purge-and-rebuild-behavior)
 - [Local Build and Testing](#local-build-and-testing)
 - [GitHub Pages Deployment](#github-pages-deployment)
@@ -40,7 +40,7 @@ The `docs/Makefile` supports these explicit targets:
 - `make html-multiversion`
   - Builds versioned docs without `sphinx-multiversion` using `scripts/build_versioned_docs.sh`.
 - `make html-multiversion-preview`
-  - Same as above, plus local branches for preview/testing.
+  - Same as above, plus the current git branch as `local-branch-preview`.
 
 Catch-all behavior is enabled:
 
@@ -59,6 +59,10 @@ Versioned docs are generated by:
 
 - `main` branch is always built and published as `current`.
 - Selected release tags are built from git tags.
+- **Docs IA cutover at v0.21.0:**
+  - **`current` and tags `>= v0.21.0`** use **main's** `docs/conf.py` with that ref's `docs/source` (sphinx-immaterial / section-folder IA).
+  - **Tags `<= v0.20.x`** use **`v0.20.0`'s** `docs/conf.py` (RTD / flat layout) with each tag's own `docs/source`.
+- **`local-branch-preview`** (optional) uses the current branch's own conf + source (pre-merge validation).
 - Output layout:
   - `docs/build/html/current/index.html`
   - `docs/build/html/vX.Y.Z/index.html`
@@ -70,9 +74,10 @@ Versioned docs are generated by:
 `scripts/select_versions.py` applies these rules:
 
 1. Identify the highest major version present in tags.
-2. Include the latest patch release for the last 5 minor series in that major.
-3. Include the latest available release for each of the last 3 majors.
-4. De-duplicate and sort descending (newest first).
+2. For major `0`, include the latest patch release for every minor from the current minor down to `0.12` (inclusive).
+3. For other majors, include the latest patch release for the last 5 minor series in that major.
+4. Include the latest available release for each of the last 3 majors.
+5. De-duplicate and sort descending (newest first).
 
 Example ordering:
 
@@ -82,13 +87,27 @@ Example ordering:
 - `v0.14.0`
 - `v0.13.0`
 - `v0.12.1`
+- … through `v0.12.x` when older minors exist
 
-### How the Sidebar Switcher Works
+### How the Version Dropdown Works
 
-- `scripts/build_versioned_docs.py` writes `versions.json` with `name`, `url`, and `is_latest`.
-- `docs/conf.py` loads this manifest via `DOCS_VERSIONS_FILE`.
-- `docs/_templates/versions.html` renders the version list in the RTD sidebar.
-- Links target `.../index.html` explicitly to avoid directory-listing behavior in local `file://` browsing.
+- `scripts/build_versioned_docs.py` writes a **superset** `versions.json` with:
+  - mike / sphinx-immaterial fields: `version`, `title`, `aliases`
+  - legacy RTD fields: `name`, `display_version`, `url`, `status`, `is_latest`, `release_date`
+- RTD-era refs load the manifest at build time via `DOCS_VERSIONS_FILE` into `versions.html`.
+- Immaterial refs enable `version_dropdown` and fetch the parent `../versions.json` over HTTP (`version_json` in `conf.py`).
+- Use a local HTTP server to test the immaterial switcher (see below).
+
+### Preview builds
+
+```bash
+make -C docs html-multiversion-preview
+```
+
+- Still publishes `current` from `main`.
+- Also builds the **current git branch** once as `local-branch-preview` (skipped if you are on `main` or detached HEAD).
+- To review a feature branch before merge, check it out, run preview, then select `local-branch-preview` in the version menu.
+- Preview builds from the **working tree** (uncommitted conf/theme/source edits are included).
 
 ## Purge and Rebuild Behavior
 
@@ -122,7 +141,7 @@ make -C docs spelling
 make -C docs html-multiversion
 ```
 
-1. If you need to test branch previews in the version menu:
+1. If you need to preview the current branch against `main` / release tags:
 
 ```bash
 make -C docs html-multiversion-preview
@@ -132,7 +151,7 @@ Practical rule of thumb:
 
 - Doc text/layout change only: `html` + `spelling`
 - Versioning/switcher/build-script change: `html` + `spelling` + `html-multiversion`
-- Branch preview verification: add `html-multiversion-preview`
+- Branch theme/IA preview: `html-multiversion-preview`, then select `local-branch-preview`
 
 ### Open docs
 
@@ -145,7 +164,9 @@ cd docs/build/html
 python3 -m http.server 8000
 ```
 
-Then open [http://localhost:8000](http://localhost:8000).
+Then open [http://localhost:8000/current/index.html](http://localhost:8000/current/index.html) to exercise the version dropdown.
+
+Note: `make html` builds a single-version tree without a parent-level `versions.json`; use `html-multiversion` to test version switching.
 
 ## GitHub Pages Deployment