Skip to content

Make docs discoverable by LLMs and coding agents (sphinx-llm)#451

Merged
aaronspring merged 5 commits into
mainfrom
llms-txt-docs
Jun 18, 2026
Merged

Make docs discoverable by LLMs and coding agents (sphinx-llm)#451
aaronspring merged 5 commits into
mainfrom
llms-txt-docs

Conversation

@aaronspring

@aaronspring aaronspring commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

What & why

As coding agents become a primary way developers discover and use libraries, they read our documentation and write code based on what they find. This PR makes xskillscore's docs easy for LLMs/agents to discover and parse, following the recommendations from Jacob Tomlinson's PyData talk "Marketing your open source project to LLMs" (https://youtu.be/bRDtplyhRYw).

Changes

  • Generate llms.txt, llms-full.txt and per-page .md during the Sphinx HTML build via NVIDIA's sphinx-llm extension (sphinx_llm.txt). It runs the markdown builder under the hood to emit true markdown following the llms.txt standard:
    • llms.txt — a sitemap with per-page descriptions
    • llms-full.txt — the entire docs concatenated as one context-efficient markdown file
    • index.html.md, etc. — a markdown version of every page at the same URL + .md
    • llms_txt_description sets the project summary; the title comes from the project name.
  • Documentation breadcrumbs in xskillscore/__init__ — a module docstring pointing at the docs, the llms.txt/llms-full.txt files, and the source repo, so an agent reading the installed source is guided to the right places.
  • Build deps: add sphinx-llm to ci/doc.yml and the complete optional-dependencies extra (it pulls in sphinx-markdown-builder).

After this is merged and built on Read the Docs, the files will be available at:

Verification

  • Confirmed sphinx-llm (0.4.1) generates llms.txt, llms-full.txt and per-page .md with the exact config used here (sphinx_llm.txt + llms_txt_description) in an isolated Sphinx build.
  • import xskillscore works with the new module docstring.
  • ruff check / ruff format --check pass; conf.py compiles.

Note: a full local make html currently fails on an unrelated, pre-existing incompatibility between the newest Sphinx and sphinx-autosummary-accessors; the conda-pinned RTD/CI environment is unaffected.

🤖 Generated with Claude Code

Following Jacob Tomlinson's PyData talk on marketing open source projects
to LLMs (https://youtu.be/bRDtplyhRYw), make xskillscore documentation
easy for coding agents to discover and parse:

- Generate llms.txt (summary) and llms-full.txt (full docs as a single
  context-efficient text file) during the Sphinx HTML build via the
  sphinx-llms-txt extension, with a project title/summary and absolute
  links driven by html_baseurl.
- Add documentation breadcrumbs to the package __init__ so agents reading
  the source are pointed at the docs and the llms.txt files.
- Add sphinx-llms-txt to the docs build environments (ci/doc.yml and the
  `complete` optional-dependencies extra).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.50%. Comparing base (e457985) to head (9ae20d2).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #451      +/-   ##
==========================================
- Coverage   94.57%   94.50%   -0.08%     
==========================================
  Files          27       27              
  Lines        2841     2841              
==========================================
- Hits         2687     2685       -2     
- Misses        154      156       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Switch from sphinx-llms-txt to NVIDIA's sphinx-llm package (the
`sphinx_llm.txt` extension). It uses the markdown builder to emit true
markdown: llms.txt (sitemap), llms-full.txt (full docs) and per-page .md
files, following the llms.txt standard.

- conf.py: enable `sphinx_llm.txt`; set `llms_txt_description` (title comes
  from the project name). Drop the html_baseurl and llms_txt_title/summary
  keys that were specific to sphinx-llms-txt.
- ci/doc.yml and the `complete` extra: depend on sphinx-llm.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@aaronspring aaronspring changed the title Make docs discoverable by LLMs and coding agents (llms.txt) Make docs discoverable by LLMs and coding agents (sphinx-llm) Jun 17, 2026
@aaronspring aaronspring requested a review from Zeitsperre June 17, 2026 08:01
Link to https://context7.com/xarray-contrib/xskillscore so users and
coding agents can find the LLM-indexed documentation. Uses a shields.io
static badge in Context7 brand colour.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@Zeitsperre Zeitsperre left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread docs/source/index.rst Outdated
aaronspring and others added 2 commits June 18, 2026 09:02
Addresses review feedback to copy the README badge table into the docs
landing page. Replaces the flat list of image directives with the same
categorized grid table and substitution definitions used in README.rst.

Merge main to drop the removed geophysical-data toctree entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@aaronspring aaronspring merged commit acf57ba into main Jun 18, 2026
17 checks passed
@aaronspring aaronspring deleted the llms-txt-docs branch June 18, 2026 07:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants