Skip to content

AGENTS.md

Latest commit

 

History

History
203 lines (152 loc) · 10.4 KB

AGENTS.md

File metadata and controls

203 lines (152 loc) · 10.4 KB

Apify Documentation Standards

Project overview

Docusaurus-based documentation site combining multiple content types:

  • Platform docs (sources/platform/) - Product documentation
  • Academy (sources/academy/) - Educational courses and tutorials
  • API reference (apify-api/) - Generated from OpenAPI specs
  • Multi-repo architecture - SDK/Client docs from separate repos served via nginx

One unified site built from multiple repositories. See CONTRIBUTING.md for multi-repo setup.

Commands

npm install              # Install dependencies (runs patch-package via postinstall)
npm start                # Dev server (rebuilds API docs, port 3000)
npm run build            # Production build (catches broken links, bad frontmatter)
npm run lint             # Run all linters (markdownlint + ESLint)
npm run lint:md          # Markdownlint only
npm run lint:code        # ESLint only
npm run lint:fix         # Auto-fix both linters
vale sync                # Download Vale styles (first time only)
vale "path/to/file.md" --minAlertLevel=error  # Prose style check
npm run api:rebuild      # Regenerate API docs from OpenAPI specs
npm run openapi:lint     # Validate OpenAPI spec (Redocly + Spectral + YAML)

Architecture

OpenAPI documentation pipeline

API docs are generated, NOT hand-written. The workflow:

  1. Source of truth: apify-api/openapi/openapi.yaml (splits into /paths and /components)
  2. Redocly plugins (apify-api/plugins/apify.mjs) inject custom behavior:
    • code-samples-decorator.mjs - Auto-adds code samples if files exist in /code_samples/{js,curl}/
    • legacy-doc-url-decorator.mjs - Adds backward-compatible URLs
    • client-references-links-decorator.mjs - Links to client library docs
  3. Build command: npm run api:rebuild = clean + bundle with Redocly + generate with Docusaurus
  4. Output: Markdown files in apify-api/docs/ (gitignored, regenerated on each build)

Never edit generated API docs directly. Edit the OpenAPI YAML source or add code samples.

OpenAPI code samples

Add code samples by creating files in apify-api/openapi/code_samples/{javascript,curl}/:

  • Filename must match operationId from OpenAPI spec (e.g., actorRun_get.js)
  • Decorator auto-detects and adds x-codeSamples property
  • Missing samples are logged during build

OpenAPI specification changes

  • Target OpenAPI specification version should be extracted from /openapi/openapi.yaml. All specification changes should be compliant with syntax for that specific OpenAPI specification version.
  • Prefer re-use of existing objects via $ref over duplication. Reusable components can be found in /openapi/components.
  • Components most suitable for re-use are:
    • Request parameters and path parameters defined in /openapi/components/parameters
    • Request/response schemas defined in /openapi/components/schemas
    • Explicit non-automatic examples defined in /openapi/components/examples
    • Objects that are not standardized in the OpenAPI specification defined in /openapi/components/objects
  • Objects that exist in several variants with only minor differences across different files can be extracted into file in /openapi/components/objects. Within one file the YAML anchor syntax can be used to define a shared and unique portion of such objects and avoid some code duplication. These objects will be used only during bundling process through references, but they will not be standalone entities in the generated specification.
  • When changing files in /openapi/paths look for opportunities to extract shared duplicate objects into re-usable components saved in /openapi/components.
  • When adding new endpoints, check first if any existing path is similar and if yes, try to re-use same components. If by adding new paths you create new duplication, try to extract it into a new components and reference it instead.
  • Prefer automatically generated examples from schema over explicit examples.

Error responses

  • Re-use schemas for error responses defined in /apify-api/openapi/components/responses
  • Each endpoint should have at least following error responses: 400 (Bad Request), 405 (Method Not Allowed), 429 (Too Many Requests).
  • Endpoints that define security: [] do not use any authentication.
  • Each endpoint that uses authentication should have at least following error responses: 401 (Unauthorized), 403 (Forbidden).
  • Each endpoint that has runs/last in its path or that has any ID related parameter (for example actorId, buildId, runId, datasetId and so on) should have at least one 404 (Not Found) error.
  • Each endpoint that has requestBody should have at least following error responses: 413 (Payload Too Large), 415 (Unsupported Media Type).

Syntax hints

  • Instead of using one item enum, use const: Avoid this:
schema:
  type: string
  enum:
    - "constantValue"

Use this:

schema:
  type: string
  const: "constantValue"

Python API client model generation

OpenAPI spec changes in this repo automatically trigger Pydantic model regeneration in apify-client-python. The pipeline:

  1. This repo (.github/workflows/openapi-ci.yaml):

    • On PR with changes to apify-api/openapi/**: lint, build, and validate the bundled spec
    • Upload static/api/openapi.{json,yaml} as artifacts
    • trigger-client-model-regeneration job calls gh workflow run regenerate_models.yaml in apify/apify-client-python, passing docs_pr_number and docs_workflow_run_id
    • On PR close: cleanup-client-model-pr job closes the corresponding PR in apify-client-python and deletes its branch

  2. apify-client-python (.github/workflows/manual_regenerate_models.yaml):

    • Triggered via workflow_dispatch (automatically from this repo's CI or manually from GitHub UI)
    • Downloads the OpenAPI spec artifact from this repo's workflow run (or fetches from https://docs.apify.com/api/openapi.json for manual runs)
    • Runs datamodel-codegen to generate Pydantic models into src/apify_client/_models.py
    • Runs scripts/postprocess_generated_models.py to fix known codegen issues (e.g. camelCase discriminator fields)
    • Commits to branch update-models-docs-pr-{PR_NUMBER}, creates/updates a PR
    • Posts a cross-repo comment on the original docs PR linking to the generated client PR

Branch naming convention update-models-docs-pr-{N} links the two PRs.

Theme system

Uses @apify/docs-theme package - a shared theme across all 6+ documentation repos. Don't modify theme files directly. Changes to the theme propagate via CI to all projects.

LLMs.txt generation

Post-build scripts (scripts/joinLlmsFiles.mjs + indentLlmsFile.mjs) combine llms.txt files from all sections into root. Edit source llms.txt files in content directories, not the generated root one.

Slug collection

tools/utils/collectSlugs.js scans directories for slug: in frontmatter, used in docusaurus.config.js for dynamic navigation with activeBaseRegex. Ensure slugs are collected when adding new sections.

Multi-repo ecosystem

Repository Port Content
apify-docs (this repo) 3000 Platform, Academy, OpenAPI
apify-client-js 3001 JavaScript client docs
apify-client-python 3002 Python client docs
apify-sdk-js 3003 JavaScript SDK docs
apify-sdk-python 3004 Python SDK docs
apify-cli 3005 CLI documentation

Use npm run start:dev + nginx to serve all repos together locally. See CONTRIBUTING.md for setup.

Deployment

  • Auto-deploy on merge to master
  • Preview builds on pull requests
  • PR titles must use Conventional Commits format (docs:, fix:, feat:, etc.) - enforced by CI

Common pitfalls

  1. Editing generated API docs - Always edit OpenAPI YAML source, never generated markdown in apify-api/docs/
  2. Broken links on build - onBrokenLinks: 'throw' fails CI. Check slugs match file paths
  3. Missing frontmatter - Description or slug errors break SEO and navigation
  4. Missing code block language - Always specify language for syntax highlighting
  5. Stale API docs locally - Run npm run api:rebuild after changing OpenAPI specs

Quick reference

  • Add new doc: Create .md in sources/{platform,academy}/, add frontmatter with title/description/slug
  • Add API endpoint: Edit apify-api/openapi/paths/**/*.yaml, add code samples, run npm run api:rebuild
  • Fix broken build: Check onBrokenLinks errors, verify slugs match file paths, validate frontmatter

Standards

Detailed writing and formatting standards are in standards/:

  • standards/writing-style.md - Prose voice, tone, headings, links, numbers
  • standards/content-standards.md - Front matter, admonitions, code blocks, images
  • standards/terminology.md - Product names, capitalization, article usage
  • standards/grammar-rules.md - Hyphenation, punctuation, numbers, brand spelling
  • standards/file-organization.md - File naming and directory structure
  • standards/quality-standards.md - Complete quality checklist before submitting

Key rules at a glance:

  • US English, active voice, imperative tone, no sales language
  • Sentence case headings, no gerunds
  • Bold for UI elements only; code for filenames, commands, variables
  • All admonitions require titles
  • 140-160 character descriptions in front matter
  • See standards/terminology.md for Apify product name capitalization
  • Don't use em dashes (—) - use hyphen with spaces ( - ) instead

Skills

Documentation skills live in .agents/skills/ (AgentSkills spec), each with its own references/ and scripts/:

  • .agents/skills/review-docs/ - Documentation review process and output format
  • .agents/skills/doc-write/ - Writing and editing documentation pages
  • .agents/skills/tutorial/ - Creating structured tutorials
  • .agents/skills/api-doc/ - OpenAPI specification and API documentation

Review checklist

When creating or reviewing documentation, verify:

  • Sentence case headings, no gerunds, proper hierarchy
  • Front matter complete (title, description 140-160 chars, sidebar_position, slug)
  • Bold used only for UI elements
  • All admonitions have titles
  • Code examples are complete with syntax highlighting
  • Links use descriptive text, internal links use relative paths
  • Images have alt text, use light theme
  • Terminology matches rules above
  • US English, active voice, no sales language
  • npm run lint passes