Document Elasticsearch exporters

[KOTungseth]

Description

Elasticsearch exporters (es and semantic) are now supported in v3, but they are not documented anywhere. This makes it difficult for contributors to understand what they are, why they exist, and how to use them.

Currently, the exporters:

• Can be specified as options to docs-builder and docs-assembler.
• Handle indexing documentation into Elasticsearch (with or without semantic fields).
• Automatically bootstrap inference endpoints, settings, and mappings (no manual setup required).

Requested documentation

• Overview:
  • What exporters are, why they exist, and when to use them.
  • Difference between es and semantic.
• Usage examples
  • Using exporters with docs-builder to index docs from a single repo.
  • Using exporters with docs-assembler to index all published docs.
  • Example environment variable setup (DOCUMENTATION_ELASTIC_URL, DOCUMENTATION_ELASTIC_APIKEY).
• Details + behavior
  • What happens under the hood (inference endpoint, settings, mappings bootstrapped automatically).
  • Limitations or caveats (if any).
• Troubleshooting
  • Common errors and how to resolve them (e.g., API key issues, missing environment variables).

Acceptance criteria

• Documentation page(s) exist that explain Elasticsearch exporters in plain language.
• Writers and contributors can successfully use exporters with both docs-builder and docs-assembler without developer intervention.
• Examples provided are copy-pastable and verified to work.

Impact

• Reduces confusion among contributors.
• Makes semantic search testing/adoption across /docs straightforward.
• Removes dependency on engineering for setup.

[cotti]

AI triage

Automated investigation against the current codebase. A human still needs to confirm before any code change.

Original concern

The Elasticsearch exporters (es and semantic) shipped in v3 but have no contributor-facing documentation explaining what they are, how to run them, the required environment variables, and how to troubleshoot them.

Findings

• src/Elastic.Documentation.Tooling/Exporter.cs:L9-L25 — the Exporter enum defines Elasticsearch as one of the export targets; ExporterExtensions.CreateMarkdownExporters (src/Elastic.Markdown/Exporters/ExporterExtensions.cs:L26-L27) instantiates ElasticsearchMarkdownExporter when it's enabled. The --exporters flag accepts es/elasticsearch (src/services/Elastic.Documentation.Assembler/Building/ExporterParser.cs:L33-L35).
• The CLI surface the issue asks to document already exists: single-repo docs-builder index (src/tooling/docs-builder/Commands/IndexCommand.cs:L25-L49) and full-site assembler index (src/tooling/docs-builder/Commands/Assembler/AssemblerIndexCommand.cs:L27-L52), plus codex index. None of these have a narrative how-to page.
• The env vars the issue calls out are documented — but only as XML doc-comments on ElasticsearchIndexOptions (src/Elastic.Documentation.Configuration/ElasticsearchEndpointConfigurator.cs:L17-L96, e.g. DOCUMENTATION_ELASTIC_URL at L21-L23, DOCUMENTATION_ELASTIC_APIKEY at L25-L26). They surface in docs/cli-schema.json and --help, not in prose.
• The auto-bootstrap behavior (inference endpoint, settings, mappings — "no manual setup") is implemented and described mechanically in the one existing narrative doc, docs/development/ingest/index.md (see "Phase 2: Semantic Index Bootstrap", L52-L64).
• That doc is the only prose, and it's an internal architecture deep-dive (lexical vs semantic mechanics, hash-based change detection, reindex/sync phases). It does not cover the issue's acceptance criteria: plain-language what/why/when, the es vs semantic distinction for users, copy-pastable usage examples, an env-var setup walkthrough, or troubleshooting. It and the exporter both landed in PR #2055 (commit a6ed98a4, 2025-10-16).

Verdict

Partially done

The exporter feature and its options are fully implemented, and the connection/index options (including the two env vars) are documented in code as CLI-schema/--help reference. An internal architecture page (docs/development/ingest/index.md) also exists. But the user-facing contributor guide this issue requests — overview, es vs semantic in plain language, copy-pastable docs-builder index / assembler index examples, env-var setup, and troubleshooting — does not exist. This is a documentation-authoring task on top of a stable, knowable feature surface.

Proposal

Writer-led page (the technical material is all available from the citations above). Open questions to resolve first:

• Writer: Where should the page live — under docs/development/ next to the existing architecture-only ingest/index.md, or as public-facing docs? Who's the audience (writers/contributors vs external)? Should it complement or partly supersede ingest/index.md?
• Eng (for accuracy of examples): The CLI just migrated to Nullean.Argh (PR #3202, plus flag renames in e0d8641e), so flag names/help are recent — which flags/env vars are "public/supported" vs internal, and what's the recommended quickstart (which command, EIS on/off) before examples are written and verified copy-pastable?

Related issues

• [Feature Request]: Add search functionality to the docs-builder docs site #1566 (closed) — "Add search functionality to the docs-builder docs site", the feature these exporters power. Useful context, not a duplicate.

Notes

• PR #2055 added both the exporter and the internal architecture doc — a writer can lift the "under the hood" section from there.
• A second ai:eng-question label is defensible (CLI-surface stability), but the bulk of the work is writing docs against a knowable surface, so this is routed primarily as ai:writer-question.