Skip to content

docs(fern): add SEO meta descriptions + headlines to nightly docs (#1589)#1600

Open
lbliii wants to merge 1 commit into
mainfrom
lbliii/fern-seo-metadata-sweep
Open

docs(fern): add SEO meta descriptions + headlines to nightly docs (#1589)#1600
lbliii wants to merge 1 commit into
mainfrom
lbliii/fern-seo-metadata-sweep

Conversation

@lbliii

@lbliii lbliii commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

What

Fills in per-page SEO description frontmatter for the main/latest (nightly) docs and adds headline SEO-title overrides for generic titles. Scoped to fern/versions/latest/pages only.

Closes #1589.

Why

Investigating the live site (docs.nvidia.com/nemo/gym) showed the issue's suspected problems are already handled by Fern's defaults:

  • sitemap.xml published (108 URLs) and listed in robots.txt
  • ✅ Googlebot/Bingbot allowed; only tracking-param URLs disallowed
  • ✅ Cross-version duplicate content collapsed via auto rel=canonical (/main/… and /v0.3.0/… both canonical to the un-prefixed URL)
  • ✅ Old version /v0.2.1/* already noindex

The actual gap was per-page metadata: 57 of 71 indexed pages had description: "". With no meta description, search engines fall back to generic body snippets, so deep pages have weak SERP relevance and the landing page outranks them for specific queries — exactly the user-reported "I only land on the main page."

Changes

  • 63 pages updated under fern/versions/latest/pages (the indexed default "Main"/nightly version).
  • Descriptions: 74–165 chars, unique across all pages, each adversarially verified against the page body (no fabricated features, flags, or numbers).
  • 28 headline overrides give query-matching <title> tags to generic titles (Overview, FAQ, Setup, …) without changing the visible H1.
  • The v0.3.0 mirror and noindex'd v0.2.1 are intentionally not touched.

Verification

  • fern check → 0 errors.
  • Diff is frontmatter-only (only description/headline lines added; only empty/thin description lines removed); no body edits.
  • Independent re-scan: 0 empty, 0 thin (<70 chars), 0 duplicate descriptions.

Follow-ups (not in this PR)

  • Add a site-level metadata: block in docs.yml (og:image, og:site_name, twitter:site) for richer social-share cards.
  • Confirm the docs property is verified in Google Search Console and the Fern sitemap is submitted.

🤖 Generated with Claude Code

Fill in per-page `description` frontmatter for the main/latest (nightly)
docs and add `headline` SEO-title overrides for terse titles.

Addresses #1589. Investigating the live site showed Fern already handles
sitemap, robots, cross-version rel=canonical, and noindex of old versions
— so crawl/indexing was healthy. The real gap was per-page metadata: 57 of
71 indexed pages had `description: ""`, so search engines fell back to
generic body snippets and the landing page outranked deep pages for
specific queries (the user-reported "I only land on the main page").

- 63 pages updated under fern/versions/latest/pages (the indexed default
  "Main"/nightly version). The v0.3.0 mirror and noindex'd v0.2.1 are
  intentionally untouched.
- descriptions: 74-165 chars, unique across pages, adversarially verified
  against each page body (no fabricated features/flags/numbers).
- 28 `headline` overrides give query-matching <title> tags to generic
  titles (Overview, FAQ, Setup, ...) without changing the visible H1.
- `fern check` passes (0 errors).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Lawrence Lane <llane@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jun 15, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@lbliii lbliii self-assigned this Jun 15, 2026
title: "Release Notes"
description: ""
headline: "NeMo Gym Release Notes & Changelog by Version"
description: "Browse NeMo Gym release notes by version, from v0.3.0 back to v0.1.1, covering new environments, agent harnesses, integrations, bug fixes, and deprecations."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
description: "Browse NeMo Gym release notes by version, from v0.3.0 back to v0.1.1, covering new environments, agent harnesses, integrations, bug fixes, and deprecations."
description: "Browse NeMo Gym release notes by version, covering new environments, agent harnesses, integrations, bug fixes, and deprecations."

---
title: "Agent Server"
description: ""
description: "The NeMo Gym Agent server orchestrates rollouts: it runs the agent loop, calls the Model server, routes tool calls to Resources, and collects the final reward."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
description: "The NeMo Gym Agent server orchestrates rollouts: it runs the agent loop, calls the Model server, routes tool calls to Resources, and collects the final reward."
description: "The NeMo Gym Agent server houses the agent harness, which wraps a model with an agent loop."

@sephmard sephmard added the documentation Improvements to documentation label Jun 26, 2026
@sephmard sephmard self-requested a review June 29, 2026 13:57

@sephmard sephmard left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work on the metadata sweep — the bulk of this PR (55+ pages) is accurate and ready. A rebase is needed before merge due to conflicts and stale CLI references from PRs that landed after this was opened.

Merge conflicts — 3 files need manual resolution:

  • resources-server/index.mdx — deleted by #1781. The proposed description ("The NeMo Gym Resources server defines an agent's task, tools, and per-session state...") should move to build-verifiers/index.mdx, which currently has no description.
  • environment-tutorials/verification-patterns/index.mdx — deleted by #1771. The proposed description ("Choose a verification pattern to score agent behavior...") should move to build-verifiers/verification-patterns/index.mdx, which also has no description.
  • reference/cli-commands.mdx — main already has description: "Reference for the unified gym CLI: command groups, flags, and migration from the legacy ng_* commands." from the CLI rewrite. The proposed description ("Reference for every NeMo Gym CLI command (ng_ short and nemo_gym_ full forms)...") is also stale — ng_* is now a deprecated alias, not the primary form.

Stale CLI references in descriptions — 4 files will merge cleanly but put incorrect text in the docs:

File Current description says Should be
data/index.mdx ng_prepare_data gym dataset collate
data/download-huggingface.mdx ng_download_dataset_from_hf gym dataset download
data/prepare-validate.mdx ng_prepare_data gym dataset collate
infrastructure/engineering-notes/system-design.mdx "starts up on ng_run" "starts up on gym env start"

Happy to approve once rebased and those 7 files are updated.

@sephmard sephmard left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work on the metadata sweep — the bulk of this PR (55+ pages) is accurate and ready. A rebase is needed before merge due to conflicts and stale CLI references from PRs that landed after this was opened.

Merge conflicts — 3 files need manual resolution:

  • resources-server/index.mdx — deleted by #1781. The proposed description ("The NeMo Gym Resources server defines an agent's task, tools, and per-session state...") should move to build-verifiers/index.mdx, which currently has no description.
  • environment-tutorials/verification-patterns/index.mdx — deleted by #1771. The proposed description ("Choose a verification pattern to score agent behavior...") should move to build-verifiers/verification-patterns/index.mdx, which also has no description.
  • reference/cli-commands.mdx — main already has description: "Reference for the unified gym CLI: command groups, flags, and migration from the legacy ng_* commands." from the CLI rewrite. The proposed description ("Reference for every NeMo Gym CLI command (ng_ short and nemo_gym_ full forms)...") is also stale — ng_* is now a deprecated alias, not the primary form.

Stale CLI references in descriptions — 4 files will merge cleanly but put incorrect text in the docs:

File Current description says Should be
data/index.mdx ng_prepare_data gym dataset collate
data/download-huggingface.mdx ng_download_dataset_from_hf gym dataset download
data/prepare-validate.mdx ng_prepare_data gym dataset collate
infrastructure/engineering-notes/system-design.mdx "starts up on ng_run" "starts up on gym env start"

Happy to approve once rebased and those 7 files are updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve docs discoverability / SEO based on user feedback

3 participants