Fix AgentGrade content negotiation

[Th0rgal]

Summary

• Add an RFC-style Accept negotiator for the docs proxy.
• Return root markdown, text, JSON, or HTML according to client preference, with Vary: Accept.
• Add JSON 404 responses for JSON-preferring agents and link llms.txt from the HTML head.

Verification

• npm run build
• Local probes confirmed the four preferred Content-Type cases from the AgentGrade report.
• Local probes confirmed Accept: application/json root JSON and JSON 404 behavior.

---

Note

Low Risk
Public docs routing and response headers only; no auth or data changes, with behavior gated on Accept and existing proxy paths.

Overview
Adds RFC-style Accept negotiation for the docs proxy and a dedicated root API so agents get the right representation instead of always HTML.

The new negotiateContentType helper picks among text/html, text/markdown, text/plain, and application/json. / rewrites to /api/agentgrade/root when negotiation is not HTML, returning markdown (from readDoc("index") or fallbacks), plain text, JSON metadata, or minimal HTML. Non-root paths still rewrite to raw markdown when cues or negotiation favor markdown/plain; JSON-preferring clients on non-root paths get a structured 404 JSON instead of HTML.

Proxy responses and agent discovery now set Vary: Accept. The site layout adds an llms.txt alternate link in the document head.

^{Reviewed by Cursor Bugbot for commit 69684ae. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
verity	Ready	Preview, Comment	Jun 4, 2026 8:28am

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit bb74195. Configure here.}