fix(docs): use scalar docusaurus_tag "default" for api-nr records (#23097)

critesjosh · web-flow · commit 8b76042ae681 · 2026-05-08T17:07:33.000Z
## Summary Fourth in the series fixing search after #22861. After #23058 merged, the production index still has **0 records under `aztec-nr-api/mainnet/...`**. Confirmed by querying the live Typesense collection directly (`filter_by:url:=https://docs.aztec.network/aztec-nr-api/mainnet/*` returns `found: 0`) and by inspecting the most recent scraper run logs. ## Root cause The schema override added by #23058 doesn't take effect. Every api-nr document import is rejected by Typesense with HTTP 400 `'Field \`docusaurus_tag\` must be a string.'`, even though `custom_settings.field_definitions` lists an explicit `{ \"name\": \"docusaurus_tag\", \"type\": \"string*\" }` ahead of the wildcard `.*_tag: string`. Per Typesense docs an explicit field should win over a regex pattern field, but in practice the wildcard's `string` type appears to be what's enforced. The CI guard from #23042 (`MIN_HITS=5000`) didn't trip because the ~12k non-api docs still passed. ## Fix The PR over-engineered the solution. Reading the docusaurus theme: ```ts // docusaurus-theme-common/src/utils/searchUtils.ts export const DEFAULT_SEARCH_TAG = 'default'; ``` ```ts // docusaurus-theme-common/src/index.ts const tags = [DEFAULT_SEARCH_TAG, ...docsTags]; return {locale: i18n.currentLocale, tags}; ``` …the theme unconditionally prepends `'default'` to the `docusaurus_tag` filter on every dropdown query, in every plugin context. So api-nr records only need the single scalar value `\"default\"` to satisfy the filter from anywhere on the docs site. No array, no schema surgery, no version-specific tag derivation. Three changes: ### 1. `docs/typesense.config.json` Drop the `custom_settings.field_definitions` override entirely (the scraper's default schema with `.*_tag: string` accepts scalar string values cleanly), and collapse the api-nr `extra_attributes.docusaurus_tag` to scalar `\"default\"`. ### 2. `.github/workflows/docs-typesense.yml` — remove jq mutation The jq block that derived versioned tags is no longer needed. The scraper now reads `docs/typesense.config.json` verbatim. ### 3. `.github/workflows/docs-typesense.yml` — log api-nr visibility post-index After the scraper completes its alias swap, curl the live `aztec-docs` alias for `docusaurus_tag:=[default]&&language:=en` and log the count. No existing docusaurus page carries the `\"default\"` tag (each is stamped with its plugin-context tag, e.g. `docs-developer-v4.2.0`, from the `<meta name=\"docsearch:docusaurus_tag\">` tag), so this count is effectively the count of indexed api-nr records — and the filter mirrors what the theme actually sends. Informational only; not gated by a threshold. ## Behavior change api-nr records will now appear in the search dropdown from every plugin context (developer, network, root, participate) and every doc version (mainnet, testnet, nightly), because they're stamped with the always-prepended `\"default\"` tag rather than version-specific tags. Today we only generate `aztec-nr-api/mainnet/`, so a user browsing testnet developer docs would see mainnet aztec-nr API links in their dropdown. Probably desirable (an aztec-nr API symbol is the same regardless of which doc version you're reading), but a behavior change vs the (non-functional) #23058 attempt. ## Caveat api-nr visibility now depends on the docusaurus theme's `DEFAULT_SEARCH_TAG = 'default'` invariant. If a future caller ever issues a search query that doesn't include `'default'` in the tag list (e.g. a custom search page bypassing `useContextualSearchFilters`), api-nr records would silently disappear from that surface. ## Test plan - [ ] Manually dispatch `Docs Scraper` workflow via `workflow_dispatch` on this branch. - [ ] Confirm the run logs `Indexed N records (threshold: 5000)` with N >> 5000. - [ ] Confirm the run logs `api-nr records visible under docusaurus_tag:=[default]: M` with M well above zero (#23049 indexed 14,773 api-nr records before the schema rejection started silently dropping them, so we expect a similar count). - [ ] Confirm no `'Field \`docusaurus_tag\` must be a string.'` 400s in the scraper output. - [ ] After merge, search docs.aztec.network from the homepage, /developers/, /network/, and /participate/ for an Aztec.nr identifier (e.g. `ContractClassId`, `balance_set`, `compute_log_tag`, `address_note`) and confirm API reference pages appear in the dropdown in all four contexts.
diff --git a/.github/workflows/docs-typesense.yml b/.github/workflows/docs-typesense.yml
@@ -34,51 +34,43 @@ jobs:
           # regression (which happened with #22861 dropping the index from
           # ~12k to 48 records) into a loud CI failure.
           MIN_HITS: "5000"
+          TYPESENSE_API_KEY: ${{ secrets.TYPESENSE_API_KEY }}
+          TYPESENSE_HOST: ${{ secrets.TYPESENSE_HOST }}
         run: |
           set -euo pipefail
 
-          # Derive the version-specific docusaurus_tag values from the docs version
-          # configs and append them to the api-nr start_url's docusaurus_tag array.
-          # Each plugin instance produces a tag of the form `docs-${pluginId}-${versionName}`.
-          # The unversioned tags (participate, root, default) are already in the static
-          # config; this step adds entries for `developer` and `network` which bump on
-          # release. Without this, the api-nr records would lose contextual search
-          # visibility every time mainnet/testnet versions change.
-          extra_tags=$(jq -nc \
-            --slurpfile dev docs/developer_version_config.json \
-            --slurpfile net docs/network_version_config.json \
-            '[
-              ("docs-developer-" + ($dev[0].mainnet // "")),
-              ("docs-developer-" + ($dev[0].testnet // "")),
-              ("docs-network-"   + ($net[0].mainnet // "")),
-              ("docs-network-"   + ($net[0].testnet // ""))
-             ] | map(select(. != "docs-developer-" and . != "docs-network-")) | unique')
-          echo "Derived docusaurus_tag values: $extra_tags"
-
-          config_json=$(jq -c --argjson extra "$extra_tags" '
-            .start_urls |= map(
-              if .selectors_key == "api-nr"
-              then .extra_attributes.docusaurus_tag = ((.extra_attributes.docusaurus_tag // []) + $extra | unique)
-              else .
-              end
-            )
-          ' docs/typesense.config.json)
-
           docker run \
-            -e "TYPESENSE_API_KEY=${{ secrets.TYPESENSE_API_KEY }}" \
-            -e "TYPESENSE_HOST=${{ secrets.TYPESENSE_HOST }}" \
+            -e "TYPESENSE_API_KEY=$TYPESENSE_API_KEY" \
+            -e "TYPESENSE_HOST=$TYPESENSE_HOST" \
             -e "TYPESENSE_PORT=443" \
             -e "TYPESENSE_PROTOCOL=https" \
-            -e "CONFIG=$config_json" \
+            -e "CONFIG=$(cat docs/typesense.config.json)" \
             typesense/docsearch-scraper:0.11.0 2>&1 | tee scraper.log
 
           nb_hits=$(grep -oE 'Nb hits: *[0-9]+' scraper.log | tail -1 | grep -oE '[0-9]+' || true)
           if [ -z "$nb_hits" ]; then
-            echo "::error::Could not parse 'Nb hits' from scraper output — assuming index is broken."
+            echo "::error::Could not parse 'Nb hits' from scraper output, assuming index is broken."
             exit 1
           fi
           echo "Indexed $nb_hits records (threshold: $MIN_HITS)"
           if [ "$nb_hits" -lt "$MIN_HITS" ]; then
             echo "::error::Indexed only $nb_hits records (expected at least $MIN_HITS). Search index is likely broken."
             exit 1
           fi
+
+          # Log how many api-nr records are visible in the live index. The
+          # docusaurus theme always prepends `default` to its contextual
+          # docusaurus_tag filter, and no docusaurus page is stamped with
+          # `default` (each carries its plugin-context tag instead), so this
+          # facet count is effectively the count of indexed api-nr records.
+          # Informational only: the count varies with aztec-nr content size.
+          api_hits=$(curl -fsS \
+            "https://$TYPESENSE_HOST/collections/aztec-docs/documents/search" \
+            -H "X-TYPESENSE-API-KEY: $TYPESENSE_API_KEY" \
+            -G \
+            --data-urlencode "q=*" \
+            --data-urlencode "query_by=hierarchy.lvl0" \
+            --data-urlencode "filter_by=docusaurus_tag:=[default]&&language:=en" \
+            --data-urlencode "per_page=1" \
+            | jq -r '.found')
+          echo "api-nr records visible under docusaurus_tag:=[default]: $api_hits"
diff --git a/docs/typesense.config.json b/docs/typesense.config.json
@@ -7,11 +7,7 @@
       "page_rank": 2,
       "extra_attributes": {
         "language": "en",
-        "docusaurus_tag": [
-          "docs-participate-current",
-          "docs-root-current",
-          "default"
-        ]
+        "docusaurus_tag": "default"
       }
     },
     {
@@ -63,76 +59,6 @@
       "url",
       "url_without_anchor",
       "type"
-    ],
-    "field_definitions": [
-      { "name": "anchor", "type": "string", "optional": true },
-      { "name": "content", "type": "string", "optional": true },
-      { "name": "url", "type": "string", "facet": true },
-      {
-        "name": "url_without_anchor",
-        "type": "string",
-        "facet": true,
-        "optional": true
-      },
-      {
-        "name": "version",
-        "type": "string[]",
-        "facet": true,
-        "optional": true
-      },
-      {
-        "name": "hierarchy.lvl0",
-        "type": "string",
-        "facet": true,
-        "optional": true
-      },
-      {
-        "name": "hierarchy.lvl1",
-        "type": "string",
-        "facet": true,
-        "optional": true
-      },
-      {
-        "name": "hierarchy.lvl2",
-        "type": "string",
-        "facet": true,
-        "optional": true
-      },
-      {
-        "name": "hierarchy.lvl3",
-        "type": "string",
-        "facet": true,
-        "optional": true
-      },
-      {
-        "name": "hierarchy.lvl4",
-        "type": "string",
-        "facet": true,
-        "optional": true
-      },
-      {
-        "name": "hierarchy.lvl5",
-        "type": "string",
-        "facet": true,
-        "optional": true
-      },
-      {
-        "name": "hierarchy.lvl6",
-        "type": "string",
-        "facet": true,
-        "optional": true
-      },
-      { "name": "type", "type": "string", "facet": true, "optional": true },
-      {
-        "name": "docusaurus_tag",
-        "type": "string*",
-        "facet": true,
-        "optional": true
-      },
-      { "name": ".*_tag", "type": "string", "facet": true, "optional": true },
-      { "name": "language", "type": "string", "facet": true, "optional": true },
-      { "name": "tags", "type": "string[]", "facet": true, "optional": true },
-      { "name": "item_priority", "type": "int64" }
     ]
   },
   "conversation_id": ["833762294"],
