AFDocs fixes: improve llms-txt-coverage with complete customSets (#41)

* AFDocs fixes: improve llms-txt-coverage with complete customSets

- Add root pages (index, quickstart) to the Getting Started custom set
- Simplify Support custom set to use support-and-community/** glob
  instead of manually listing individual pages, which missed
  warp-preview-and-alpha-program
- Fix indentation inconsistency in customSets config

Co-Authored-By: Oz <oz-agent@warp.dev>

* Add /api to markdown-url-support allowlist in known-exceptions

/api is a custom Astro page, not a Starlight doc page, so it
won't have a .md variant. This is expected and intentional.

Co-Authored-By: Oz <oz-agent@warp.dev>

* Extend starlight-llms-txt patch to exclude pages from custom sets

The upstream plugin only applied the `exclude` option to llms-small.txt.
Our existing patch already extended it to llms-full.txt; this adds the
same exclude pass to llms-custom.txt.ts so custom sets (e.g. the Support
set) also skip pages that cause a stack overflow in hast-util-to-text.

Fixes the build failure caused by `support-and-community/**` including
the ~25k-line open-source-licenses page in the Support custom set.

Co-Authored-By: Oz <oz-agent@warp.dev>

---------

Co-authored-by: Oz <oz-agent@warp.dev>

Author: rachaelrenk

.agents/skills/afdocs-audit/references/known-exceptions.md

@@ -46,6 +46,12 @@ This file lists checks from the afdocs-audit skill that may flag as warnings or
 **Reason**: Only evaluated when tab panels contain section headers. Most sampled pages with tabs don't have headers inside the tab panels, so the check is skipped.
 **Action**: None needed.
 
+## markdown-url-support
+
+**Expected status**: fail (for `/api`)
+**Reason**: `/api` is a custom Astro page (`src/pages/api.astro`), not a Starlight content page. The docs-markdown integration only generates `.md` variants for Starlight doc pages under `src/content/docs/`, so `/api.md` does not exist.
+**Action**: No fix needed. This page is intentionally outside the Starlight content pipeline.
+
 ## auth-alternative-access
 
 **Expected status**: skip

astro.config.mjs

@@ -139,25 +139,27 @@ export default defineConfig({
 				// widely consumed by AI agents.
 				starlightLlmsTxt({
 					projectName: 'Warp',
-					// Excludes pages from llms-small.txt that cause a stack overflow
-					// in hast-util-to-text due to their size. Note: the `exclude`
-					// option only applies to llms-small.txt, not llms-full.txt.
-					exclude: ['support-and-community/community/open-source-licenses'],
+				// Excludes pages that cause a stack overflow in hast-util-to-text
+				// due to their size. The upstream plugin only applies `exclude` to
+				// llms-small.txt; our patch (patches/starlight-llms-txt+0.8.1.patch)
+				// extends it to llms-full.txt and custom sets as well.
+				exclude: ['support-and-community/community/open-source-licenses'],
 					description:
 						'Documentation for Warp, the agentic development environment, and Oz, Warp\'s programmable agent for running and coordinating agents at scale.',
 					customSets: [
 						{ label: 'Terminal', description: 'Warp Terminal features and configuration.', paths: ['terminal/**'] },
 						{ label: 'Agent Platform', description: 'Warp\'s Agent Platform: capabilities, local agents, CLI agents, cloud agents.', paths: ['agent-platform/**'] },
 						{ label: 'Code', description: 'Code editor, code review, and Git worktrees.', paths: ['code/**'] },
 						{ label: 'Enterprise', description: 'Enterprise features, SSO, team management, and security.', paths: ['enterprise/**'] },
-						{ label: 'Getting Started', description: 'Installation, quickstart, and migration guides.', paths: ['getting-started/**'] },
+						{ label: 'Getting Started', description: 'Installation, quickstart, and migration guides.', paths: ['index', 'quickstart', 'getting-started/**'] },
 						{ label: 'Knowledge and Collaboration', description: 'Warp Drive, teams, and the Admin Panel.', paths: ['knowledge-and-collaboration/**'] },
 						{ label: 'Reference', description: 'CLI and API reference.', paths: ['reference/**'] },
-				// Includes all support-and-community/ pages except open-source-licenses.mdx
-				// (excluded globally above — ~25k lines causes a stack overflow in hast-util-to-text).
-				{ label: 'Support', description: 'Troubleshooting, billing, and privacy.', paths: ['support-and-community/index', 'support-and-community/plans-and-billing/**', 'support-and-community/privacy-and-security/**', 'support-and-community/troubleshooting-and-support/**', 'support-and-community/community/contributing', 'support-and-community/community/open-source-partnership', 'support-and-community/community/refer-a-friend'] },
-					{ label: 'Guides', description: 'Task-oriented walkthroughs and tutorials.', paths: ['guides/**'] },
-					{ label: 'Changelog', description: 'Warp release notes by year.', paths: ['changelog/**'] },
+						// All support-and-community/ pages. open-source-licenses.mdx is excluded
+						// globally above (stack overflow in hast-util-to-text); the patch ensures
+						// it's excluded from this custom set as well.
+						{ label: 'Support', description: 'Troubleshooting, billing, and privacy.', paths: ['support-and-community/**'] },
+						{ label: 'Guides', description: 'Task-oriented walkthroughs and tutorials.', paths: ['guides/**'] },
+						{ label: 'Changelog', description: 'Warp release notes by year.', paths: ['changelog/**'] },
 					],
 				}),
 			],

patches/starlight-llms-txt+0.8.1.patch

@@ -1,3 +1,15 @@
+diff --git a/node_modules/starlight-llms-txt/llms-custom.txt.ts b/node_modules/starlight-llms-txt/llms-custom.txt.ts
+index f96dbbd..7f19ab5 100644
+--- a/node_modules/starlight-llms-txt/llms-custom.txt.ts
++++ b/node_modules/starlight-llms-txt/llms-custom.txt.ts
+@@ -30,6 +30,7 @@ export const GET: APIRoute<Props, Params> = async (context) => {
+ 		minify: true,
+ 		description,
+ 		include: context.props.paths,
++		exclude: starlightLllmsTxtContext.exclude,
+ 	});
+ 	return new Response(body);
+ };
 diff --git a/node_modules/starlight-llms-txt/llms-full.txt.ts b/node_modules/starlight-llms-txt/llms-full.txt.ts
 index 03f408b..9907aed 100644
 --- a/node_modules/starlight-llms-txt/llms-full.txt.ts