Add LLM-friendly documentation (llms.txt)

[Igor Rodionov (goruha)]

what

• Add support for llms.txt standard using docusaurus-plugin-llms
• Generate two LLM-friendly documentation files:
  • /llms.txt - Compact link list
  • /llms-full.txt - Full documentation content
• Configure plugin to prioritize core documentation sections
• Include blog content in generated files

why

• LLMs need structured documentation: The llms.txt standard (https://llmstxt.org/) provides a standardized way for LLMs to efficiently understand website documentation
• Address context window limitations: Instead of crawling entire websites, LLMs can access curated, structured content optimized for their understanding
• Improve AI assistance: Tools like Claude, ChatGPT, and other AI assistants can provide better answers about Atmos by referencing these files
• Follow emerging standards: Similar to robots.txt for search engines, llms.txt is becoming a standard for AI-friendly documentation
• Better developer experience: Users asking AI assistants will get more accurate, up-to-date responses

references

• https://llmstxt.org/ - llms.txt standard specification
• https://github.com/rachfop/docusaurus-plugin-llms - Docusaurus plugin used
• Live files will be available at:
  • https://docs.cloudposse.com/llms.txt
  • https://docs.cloudposse.com/llms-full.txt