Merge pull request #762 from Pipelex/release/v0.20.12

Release v0.20.12

Author: lchoquel

CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## [v0.20.12] - 2026-03-16
+
+### Fixed
+
+- Fix Google "Sitemap could not be read" error by adding `Allow: /sitemap.xml` to `ROOT_ROBOTS_TXT` — `Disallow: /` was blocking Googlebot from fetching the root sitemap even though `Sitemap:` pointed to it.
+
 ## [v0.20.11] - 2026-03-16
 
 ### Fixed

Makefile

@@ -805,9 +805,16 @@ SITE_DOMAIN := $(shell cat docs/CNAME 2>/dev/null | tr -d '[:space:]')
 # Authoritative robots.txt for the domain root (docs.pipelex.com/robots.txt).
 # Deployed by docs-deploy-root. Note: docs/robots.txt is a no-op — it only
 # lands at /latest/robots.txt, which crawlers ignore per RFC 9309.
+#
+# IMPORTANT: `Disallow: /` blocks ALL root paths not explicitly `Allow`ed —
+# including root-level files like /sitemap.xml. The `Sitemap:` directive tells
+# crawlers *where* the sitemap is, but does NOT override Disallow rules —
+# crawlers still need permission to fetch the URL. Any root-level file that
+# crawlers must access needs its own `Allow:` line.
 define ROOT_ROBOTS_TXT
 User-agent: *
 Allow: /latest/
+Allow: /sitemap.xml
 Disallow: /
 Sitemap: https://$(SITE_DOMAIN)/sitemap.xml
 endef

docs/CLAUDE.md

@@ -38,7 +38,7 @@ There must be **NO** `docs/404.md` — a Markdown 404 would be treated as conten
 ## robots.txt: Two Files, Different Purposes
 
 - **`docs/robots.txt`** — Lands at `/latest/robots.txt` inside the versioned site. Crawlers ignore it (RFC 9309: only domain-root robots.txt is authoritative). Exists as a courtesy/fallback.
-- **`Makefile` `ROOT_ROBOTS_TXT`** — The authoritative robots.txt deployed to domain root by `docs-deploy-root`. Allows only `/latest/`, disallows everything else, points sitemap to root `/sitemap.xml` (which contains `/latest/` URLs, generated by `docs-deploy-root` from the `latest/` alias sitemap via sed version-to-`latest` rewrite).
+- **`Makefile` `ROOT_ROBOTS_TXT`** — The authoritative robots.txt deployed to domain root by `docs-deploy-root`. Allows only `/latest/` and `/sitemap.xml`, disallows everything else, points sitemap to root `/sitemap.xml` (which contains `/latest/` URLs, generated by `docs-deploy-root` from the `latest/` alias sitemap via sed version-to-`latest` rewrite). **Critical:** `Disallow: /` blocks ALL root paths not explicitly `Allow`ed. The `Sitemap:` directive tells crawlers where the sitemap is but does NOT override `Disallow` — crawlers still need an `Allow:` line to fetch it.
 
 ## Deployment (`docs-deploy-root`)
 
@@ -85,6 +85,7 @@ Mike (`mike deploy X.Y.Z latest`) creates two directories on gh-pages: `X.Y.Z/`
 - Do NOT set `site_url` to include `/latest/` — mike will create double paths like `/latest/X.Y.Z/page/`
 - Do NOT assume `latest/sitemap.xml` has `/latest/` URLs — mike copies sitemaps verbatim from the version directory without rewriting
 - Do NOT source the root sitemap from `$(DOCS_VERSION)/` — use `latest/` so pre-release deploys don't overwrite it with pages not served at `/latest/`
+- Do NOT add root-level files to `docs-deploy-root` without also adding a corresponding `Allow:` line in `ROOT_ROBOTS_TXT` — `Disallow: /` blocks them from crawlers even if a `Sitemap:` directive references them
 
 ## Spec vs Blueprint in Documentation
 

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "pipelex"
-version = "0.20.11"
+version = "0.20.12"
 description = "Execute composable AI methods declared in the MTHDS open standard"
 authors = [{ name = "Evotis S.A.S.", email = "oss@pipelex.com" }]
 maintainers = [{ name = "Pipelex staff", email = "oss@pipelex.com" }]