Merge pull request #139 from agentgateway/adb-api-docs

[docs] update ref docs workflow

Author: artberger

.github/workflows/reference-docs.yaml

@@ -118,6 +118,7 @@ jobs:
             --source-path="$PWD/../kgateway/api/v1alpha1/agentgateway/" \
             --renderer=markdown \
             --output-path ./ \
+            --max-depth=100 \
             --config=scripts/crd-ref-docs-config.yaml
 
           if [ ! -f "./out.md" ]; then
@@ -147,13 +148,31 @@ jobs:
           
           popd || exit 1
 
+      - name: Generate shared types documentation
+        run: |
+          API_FILE="${{ needs.setup.outputs.api_file }}"
+          TARGET_FILE="website/assets/agw-docs/pages/reference/api/${API_FILE}"
+          SHARED_DIR="kgateway/api/v1alpha1/shared"
+          AGENTGATEWAY_SOURCE_DIR="kgateway/api/v1alpha1/agentgateway"
+          if [ -f "$TARGET_FILE" ]; then
+            echo "Generating shared types documentation for ${API_FILE}..."
+            python3 website/scripts/generate-shared-types.py \
+              "$SHARED_DIR" \
+              "$TARGET_FILE" \
+              "$AGENTGATEWAY_SOURCE_DIR"
+          else
+            echo "API doc file not found - skipping shared types"
+          fi
+
       - name: Generate Helm and metrics docs
         run: |
           DOC_VERSION="${{ needs.setup.outputs.doc_version }}" \
           WEBSITE_DIR="website" \
           KGATEWAY_DIR="kgateway" \
           python3 website/scripts/generate-ref-docs.py
 
+      # Create PR: needs "Allow GitHub Actions to create and approve pull requests" (Settings → Actions)
+      # or a repo secret CREATE_PR_TOKEN (PAT with pull_requests: write).
       - name: Create PR for website repo
         uses: peter-evans/create-pull-request@v7
         id: create-pr
@@ -164,17 +183,17 @@ jobs:
             Generate reference documentation for agentgateway (${{ needs.setup.outputs.api_file }}).
             
             Source: kgateway-dev/kgateway at `${{ needs.setup.outputs.ref }}`.
-            Updates API reference, Helm (agentgateway + agentgateway-crds), and control plane metrics for the ${{ needs.setup.outputs.directory }} version.
+            Updates API reference (including shared types such as CEL expression), Helm (agentgateway + agentgateway-crds), and control plane metrics for the ${{ needs.setup.outputs.directory }} version.
             
             This PR was created automatically by the reference-docs workflow.
             
             Workflow run: https://github.com/agentgateway/website/actions/runs/${{ github.run_id }}
           branch: ref-docs-${{ needs.setup.outputs.doc_version }}
-          commit-message: 'docs: update API, Helm, and metrics docs from kgateway ${{ needs.setup.outputs.ref }}'
+          commit-message: 'docs: update API (incl. shared types), Helm, and metrics docs from kgateway ${{ needs.setup.outputs.ref }}'
           committer: GitHub Action <action@github.com>
           delete-branch: true
           title: '[Automated] Update reference docs (${{ needs.setup.outputs.doc_version }}) from kgateway ${{ needs.setup.outputs.ref }}'
-          token: ${{ secrets.GITHUB_TOKEN }}
+          token: ${{ secrets.CREATE_PR_TOKEN || secrets.GITHUB_TOKEN }}
 
       - name: Output PR URL
         run: |

scripts/README.md

@@ -21,6 +21,17 @@ The workflow and `generate-ref-docs.py` read this file to decide which ref to us
 
 Configuration for [crd-ref-docs](https://github.com/elastic/crd-ref-docs). Used when generating Kubernetes API reference docs from kgateway’s `api/v1alpha1/agentgateway` types. The workflow passes this as `--config` to `go run github.com/elastic/crd-ref-docs@latest`.
 
+### `generate-shared-types.py`
+
+Python script that appends **shared types** documentation (e.g. CEL expression) to the generated API reference markdown. It parses Go source in kgateway’s shared and agentgateway packages, finds broken type links in the API doc, and appends markdown for those types so crd-ref-docs output is complete. The workflow runs it after writing the API doc:
+
+- **Arguments:** `generate-shared-types.py <shared_dir> <doc_file> [source_dir...]`
+- **shared_dir** – e.g. `kgateway/api/v1alpha1/shared`
+- **doc_file** – e.g. `website/assets/agw-docs/pages/reference/api/api-22x.md`
+- **source_dir** (optional) – e.g. `kgateway/api/v1alpha1/agentgateway`
+
+If the doc file is missing, the script is skipped. If `shared_dir` does not exist, the script still runs and uses any provided source dirs.
+
 ### `generate-ref-docs.py`
 
 Python script that generates **Helm** and **metrics** reference docs only (API docs are generated in the workflow). It expects:
@@ -40,7 +51,8 @@ Requires Go (for helm-docs and findmetrics). No cloning; the workflow checks out
 
 1. **Setup job** – Checks out the website, reads `scripts/versions.json`, resolves the kgateway ref for the chosen doc version, and verifies that branch exists.
 2. **API docs step** – Checks out kgateway at that ref, runs crd-ref-docs with `scripts/crd-ref-docs-config.yaml`, and writes API markdown to `assets/agw-docs/pages/reference/api/{apiFile}`.
-3. **Helm and metrics step** – Runs `scripts/generate-ref-docs.py` with `DOC_VERSION`, `WEBSITE_DIR=website`, and `KGATEWAY_DIR=kgateway` so the script generates Helm and metrics into the website tree.
-4. **Create PR** – Commits all changes under the website path and opens a PR.
+3. **Shared types step** – Runs `scripts/generate-shared-types.py` with kgateway’s shared and agentgateway API dirs and the API doc file, appending documentation for shared types (e.g. CEL) that crd-ref-docs does not emit.
+4. **Helm and metrics step** – Runs `scripts/generate-ref-docs.py` with `DOC_VERSION`, `WEBSITE_DIR=website`, and `KGATEWAY_DIR=kgateway` so the script generates Helm and metrics into the website tree.
+5. **Create PR** – Commits all changes under the website path and opens a PR.
 
 To add a new doc version (e.g. 2.4.x), add an entry to `versions.json` with the appropriate `apiFile`, `kgatewayRef`, and `metricsSnippet`, then run the workflow with that version.