docs: switch over to fern (#123)

#### Overview

Switch the documentation site from Sphinx to Fern and migrate the docs content, navigation, and API reference generation to the Fern structure.

- [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license.
- [x] I searched existing issues and open pull requests, and this does not duplicate existing work.

#### Details

- Adds Fern site configuration, NVIDIA assets, shared Fern components, and site styling.
- Migrates docs pages from the previous Markdown/Sphinx layout to MDX/Fern paths, including getting started, concepts, integrations, plugin docs, reference, resources, and troubleshooting content.
- Replaces Sphinx-oriented API reference generation with Fern-friendly Python, Node.js, and Rust reference generation scripts.
- Updates docs build scripts, CI docs workflows, path filters, pre-commit hooks, package metadata, lockfiles, attributions, and contribution/release guidance for the Fern workflow.
- Validated with `just docs`, `just docs-linkcheck`, and `uv run pre-commit run --all-files`; the Fern checks pass with one light-mode accent contrast warning reported by Fern.

#### Where should the reviewer start?

Start with `fern/docs.yml` and `scripts/build-docs.sh` to understand the new docs entry points and navigation, then sample the migration pattern in `docs/index.yml` and `docs/getting-started/quick-start/index.mdx`.

#### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

- Relates to #71



## Summary by CodeRabbit

* **New Features**
  * Added Fern-based documentation site configuration with custom React components
  * Added API reference generation scripts for Python, Node.js, and Rust libraries

* **Documentation**
  * Migrated documentation from Sphinx to Fern using MDX format with updated structure and navigation
  * Converted all pages to use MDX frontmatter, updated internal links to absolute routes, and standardized SPDX headers

* **Refactor**
  * Removed Sphinx build infrastructure and replaced with Fern validation workflow
  * Updated CI/CD to publish docs via Fern instead of GitHub Pages

* **Chores**
  * Updated .gitignore, pre-commit hooks, and CI path filters for Fern docs structure
  * Updated copyright checking script to support MDX files



[![Review Change Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/NVIDIA/NeMo-Relay/pull/123?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

Authors:
  - Will Killian (https://github.com/willkill07)

Approvers:
  - https://github.com/Salonijain27

URL: #123

Author: willkill07

.agents/skills/contribute-docs/SKILL.md

@@ -26,13 +26,17 @@ Use this skill for docs-only or example-heavy changes.
 - Keep stable user-facing wrappers at `scripts/` root in docs and examples;
   only point at namespaced helper paths when documenting internal maintenance
   work
+- In MDX files, top-of-file comments must use JSX comment delimiters:
+  `{/*` to open and `*/}` to close. Do not use HTML comments for MDX SPDX
+  headers.
 
 ## Checklist
 
 - [ ] `README.md` or `docs/index.md` updated when entry points changed
 - [ ] Relevant getting-started or reference docs updated
 - [ ] Example commands still match current package names and paths
 - [ ] Relevant package or crate `README.md` files updated when examples or binding guidance changed
+- [ ] New or regenerated MDX files use `{/* ... */}` for top-of-file SPDX comments
 - [ ] Release-policy docs still point to GitHub Releases as the only release-history source of truth
 - [ ] Run `just docs` when the docs site changed; `./scripts/build-docs.sh html` remains the compatibility wrapper
 

.agents/skills/review-doc-style/SKILL.md

@@ -47,6 +47,8 @@ Treat these as blocking issues:
 - A doc claims support for a binding, feature, or workflow that the repo no longer provides
 - Examples or procedures are likely to fail as written
 - User-facing naming is inconsistent with current repo terminology
+- MDX top-of-file SPDX comments use HTML comment delimiters instead of
+  `{/* ... */}`
 - NVIDIA is not capitalized correctly
 - Code, commands, paths, or filenames are not formatted as inline code where needed
 
@@ -71,6 +73,7 @@ Flag these when they materially improve clarity or consistency:
 - **Voice**: Prefer active voice, present tense, short sentences, and plain English.
 - **Links**: Use descriptive anchor text, not bare URLs or weak labels.
 - **Formatting**: Commands, code elements, expressions, file names, and paths are monospace.
+- **MDX headers**: Top-of-file MDX SPDX comments use `{/*` and `*/}` delimiters.
 - **Procedures**: Steps are easy to scan, imperative, and split into smaller tasks when long.
 - **Examples**: Code blocks are introduced by full sentences and match current APIs and build commands.
 - **Terminology**: Use consistent terms throughout the document.

.github/ISSUE_TEMPLATE/03-documentation.yml

@@ -40,7 +40,7 @@ body:
       description: Link to pages or list file paths. Enter N/A if you are not sure.
       placeholder: |
         - https://...
-        - docs/...
+        - fern/... or docs/...
     validations: {required: true}
   - type: textarea
     id: proposed_change

.github/ci-path-filters.yml

@@ -111,12 +111,18 @@ dependencies:
 docs:
   - 'CONTRIBUTING.md'
   - 'README.md'
+  - 'docs/**'
+  - 'Cargo.lock'
+  - 'Cargo.toml'
   - 'crates/adaptive/src/**'
   - 'crates/core/src/**'
+  - 'crates/ffi/src/**'
   - 'crates/node/*.d.ts'
   - 'crates/node/*.js'
+  - 'crates/node/package.json'
+  - 'crates/*/Cargo.toml'
   - 'crates/**/*.md'
-  - 'docs/**'
+  - 'fern/**'
   - 'integrations/coding-agents/**'
   - 'python/nemo_relay/**'
   - 'scripts/build-docs.sh'

.github/workflows/ci.yaml

@@ -29,7 +29,6 @@ jobs:
         && !cancelled()
         && needs.prepare.result == 'success'
         && needs.ci_required.result == 'success'
-        && needs.prepare.outputs.publish_docs != 'true'
       }}
     permissions:
       contents: read
@@ -74,22 +73,17 @@ jobs:
         run: |
           set -euo pipefail
           full_ci=false
-          publish_docs=false
           publish_packages=false
 
           if [[ "$REF_TYPE" == "tag" || "$REF_NAME" == "$DEFAULT_BRANCH" ]]; then
             full_ci=true
           fi
-          if [[ "$REF_TYPE" == "tag" ]]; then
-            publish_docs=true
-          fi
           if [[ "$REF_TYPE" == "tag" && ! "$REF_NAME" =~ -alpha\.[0-9]+$ ]]; then
             publish_packages=true
           fi
 
           {
             printf 'full_ci=%s\n' "$full_ci"
-            printf 'publish_docs=%s\n' "$publish_docs"
             printf 'publish_packages=%s\n' "$publish_packages"
           } >> "$GITHUB_OUTPUT"
 
@@ -98,7 +92,6 @@ jobs:
       is_pr: ${{ startsWith(github.ref_name, 'pull-request/') }}
       is_main_branch: ${{ github.ref_name == 'main' }}
       has_skip_ci_label: ${{ steps.get-pr-info.outcome == 'success' && contains(fromJSON(steps.get-pr-info.outputs.pr-info).labels.*.name, 'skip-ci') || false }}
-      publish_docs: ${{ steps.policy.outputs.publish_docs }}
       publish_packages: ${{ steps.policy.outputs.publish_packages }}
       pr_info: ${{ steps.get-pr-info.outcome == 'success' && steps.get-pr-info.outputs.pr-info || '' }}
 
@@ -158,10 +151,8 @@ jobs:
       }}
     permissions:
       contents: read
-    with:
-      ref_type: ${{ github.ref_type }}
-      ref_name: ${{ github.ref_name }}
-      publish_docs: ${{ needs.prepare.outputs.publish_docs == 'true' }}
+    secrets:
+      DOCS_FERN_TOKEN: ${{ secrets.DOCS_FERN_TOKEN }}
 
   ci_rust:
     name: Rust
@@ -256,7 +247,6 @@ jobs:
           NODE_RESULT: ${{ needs.ci_node.result }}
           PYTHON_RESULT: ${{ needs.ci_python.result }}
           WEBASSEMBLY_RESULT: ${{ needs.ci_wasm.result }}
-          publish_docs: ${{ needs.prepare.outputs.publish_docs }}
           publish_packages: ${{ needs.prepare.outputs.publish_packages }}
         run: |
           set -euo pipefail
@@ -283,11 +273,7 @@ jobs:
           require_success "Changes" "$CHANGES_RESULT"
           require_success "Check" "$CHECK_RESULT"
 
-          if [[ "$publish_docs" == "true" ]]; then
-            require_success "Documentation" "$DOCS_RESULT"
-          else
-            allow_success_or_skipped "Documentation" "$DOCS_RESULT"
-          fi
+          allow_success_or_skipped "Documentation" "$DOCS_RESULT"
 
           if [[ "$publish_packages" == "true" ]]; then
             require_success "Rust" "$RUST_RESULT"
@@ -307,22 +293,6 @@ jobs:
             exit 1
           fi
 
-  deploy-docs:
-    name: Deploy Documentation
-    needs: [prepare, ci_required]
-    if: ${{ needs.prepare.outputs.publish_docs == 'true' && needs.ci_required.result == 'success' }}
-    runs-on: ubuntu-latest
-    permissions:
-      pages: write
-      id-token: write
-    environment:
-      name: github-pages
-      url: ${{ steps.deployment.outputs.page_url }}
-    steps:
-      - name: Deploy GitHub Pages Site
-        id: deployment
-        uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5
-
   release-cli-artifacts:
     name: Release CLI Artifacts
     needs: [prepare, ci_required]

.github/workflows/ci_docs.yml

@@ -5,29 +5,15 @@ name: Documentation
 
 on:
   workflow_call:
-    inputs:
-      ref_type:
-        description: 'The ref type from the triggering workflow'
-        required: true
-        type: string
-      ref_name:
-        description: 'The ref name from the triggering workflow'
-        required: true
-        type: string
-      publish_docs:
-        description: 'Whether to upload a GitHub Pages artifact for deployment'
+    secrets:
+      DOCS_FERN_TOKEN:
         required: false
-        default: false
-        type: boolean
 defaults:
   run:
     shell: bash
 
 env:
-  GH_TOKEN: "${{ github.token }}"
-  GIT_COMMIT: "${{ github.sha }}"
   NEMO_RELAY_CI_WORKSPACE: "${{ github.workspace }}"
-  NEMO_RELAY_CI_WORKSPACE_TMP: "${{ github.workspace }}/tmp"
   UV_PYTHON_DOWNLOADS: never
 
 jobs:
@@ -41,8 +27,6 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
-        with:
-          fetch-depth: 0
 
       - name: Load CI tool versions
         id: ci-config
@@ -66,7 +50,7 @@ jobs:
 
       - uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
         with:
-          shared-key: nemo-relay-rust-${{ runner.os }}-${{ runner.arch }}-${{ steps.ci-config.outputs.rust_version }}
+          shared-key: nemo-relay-rust-docs-${{ runner.os }}-${{ runner.arch }}-${{ steps.ci-config.outputs.rust_version }}
           workspaces: . -> target
           cache-all-crates: true
           cache-bin: false
@@ -79,43 +63,17 @@ jobs:
 
       - uses: taiki-e/install-action@c070f87102a1c75b3183910f391c1cb887fe13c8 # v2.77.6
         with:
-          tool: just@${{ steps.ci-config.outputs.just_version }},sphinx-rustdocgen@1.0.1
+          tool: just@${{ steps.ci-config.outputs.just_version }}
 
       - name: Install Documentation Dependencies
         working-directory: ${{ env.NEMO_RELAY_CI_WORKSPACE }}
-        run: uv sync --no-default-groups --group docs --no-install-project
-
-      - name: Install Node.js Documentation Dependencies
-        working-directory: ${{ env.NEMO_RELAY_CI_WORKSPACE }}
-        run: npm ci --ignore-scripts
-
-      - name: Materialize Main Branch For Versioned Docs
-        if: ${{ inputs.ref_type == 'tag' }}
-        working-directory: ${{ env.NEMO_RELAY_CI_WORKSPACE }}
-        run: git fetch --force origin +refs/heads/main:refs/heads/main
-
-      - name: Check Documentation Links
-        working-directory: ${{ env.NEMO_RELAY_CI_WORKSPACE }}
-        env:
-          NEMO_RELAY_DOCS_DEPS_READY: "1"
-        run: just docs-linkcheck
+        run: |
+          uv sync --no-default-groups --group dev --no-install-project
+          npm ci --ignore-scripts
 
-      - name: Build Documentation Site
-        if: ${{ startsWith(inputs.ref_name, 'pull-request/') || inputs.ref_name == 'main' }}
+      - name: Check Documentation
         working-directory: ${{ env.NEMO_RELAY_CI_WORKSPACE }}
         env:
           NEMO_RELAY_DOCS_DEPS_READY: "1"
+          FERN_TOKEN: ${{ secrets.DOCS_FERN_TOKEN }}
         run: just docs
-
-      - name: Build Versioned Documentation Site
-        if: ${{ inputs.publish_docs }}
-        working-directory: ${{ env.NEMO_RELAY_CI_WORKSPACE }}
-        env:
-          NEMO_RELAY_DOCS_DEPS_READY: "1"
-        run: just docs-github-pages
-
-      - name: Upload GitHub Pages Artifact
-        if: ${{ inputs.publish_docs }}
-        uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5
-        with:
-          path: ${{ env.NEMO_RELAY_CI_WORKSPACE }}/docs/_build/pages