Skip to content

Commit 291102d

Browse files
ben-fornefeldben-fornefeld
authored
Feature: SDK Reference generation & hosting (#78)
* wip: sdk reference generation workflow * Refactor SDK reference generation workflow - Updated the SDK generation workflow to load SDK configurations from `sdks.json`, improving maintainability. - Removed individual SDK generator scripts for CLI, Code Interpreter (JS and Python), and Desktop SDKs, consolidating functionality into a single generator script. - Enhanced the `generate-sdk-reference.sh` script to dynamically handle SDK types and versions. - Cleaned up the workflow YAML file for better readability and consistency in string formatting. * Enhance SDK navigation and styling - Updated `.gitignore` to include `sdk_navigation.json` for generated SDK navigation. - Added new styles for version switcher in `style.css`, improving UI for SDK and version selection. - Modified `generate-sdk-nav.js` to group SDKs by family, enhancing navigation structure. - Updated SDK configurations in `sdks.json` to include family and language attributes for better organization. - Improved CLI and Python documentation generation scripts for consistency and added frontmatter to markdown files. * cleanup * fix: sdks.json * Enhance SDK reference generation and configuration - Updated `sdk-reference-sync.yml` to allow version generation for "all" versions by default. - Modified `generate-sdk-reference.sh` to support a new `--limit` option for controlling the number of versions processed. - Improved `generate-sdk.sh` to filter versions based on a minimum version requirement and to handle version discovery more effectively. - Updated `sdks.json` to include `minVersion` attributes for SDKs, ensuring only compatible versions are processed. - Enhanced common library functions for version management and caching during dependency installation. * improve: better idempotency comparison * cleanup * refactor: bash to typescript generator * cleanup: - Added verification functions to ensure generated documentation meets quality standards, including checks for empty files and missing frontmatter. - Implemented a logging utility for better output management during the generation process. - Created a `.mintignore` file to exclude specific directories from version control. - Enhanced CLI commands for improved user experience and added options for forced regeneration of existing versions. * refactor: migrate SDK configuration from JSON to TypeScript - Introduced a new `sdks.config.ts` file to define SDK configurations in TypeScript, enhancing type safety and maintainability. - Removed the outdated `sdks.json` file to streamline the configuration process. - Updated relevant code to utilize the new configuration structure, including adjustments in the CLI and navigation logic. - Improved type definitions for SDK configurations to support additional properties and ensure consistency across the codebase. * refactor: improve SDK reference generation and navigation - Updated CSS styles for version switcher to enhance UI consistency and usability. - Modified GitHub Actions workflow to streamline SDK generation input descriptions and improve summary output. - Refactored context handling in SDK generation functions for better clarity and maintainability. - Introduced new utility functions for version validation and normalization, enhancing version management. - Added tests for cache handling and file processing to ensure robustness in SDK reference generation. * chore: remove package.json and pnpm-lock.yaml - Deleted `package.json` and `pnpm-lock.yaml` files to clean up the project structure as they are no longer needed for SDK reference generation. * fix: config * refactor: update SDK configuration and improve generation logic - Replaced `basePackage` and `fallbackPackages` with `allowedPackages` in SDK configurations for better clarity and usage. - Refactored the `generateVersion` function to utilize a new `CheckoutManager` for handling repository checkouts and version switching. - Simplified the `generatePydoc` function to focus on allowed packages, removing the need for fallback and submodule handling. - Introduced a new `checkout.ts` file to manage SDK checkouts, enhancing code organization and maintainability. - Removed obsolete cache handling code and related tests to streamline the project structure. * chore * cleanup * refactor: improve error handling and logging in CLI and generator - Enhanced error handling in the CLI to ensure proper exit on validation failures. - Improved logging for workflow abort scenarios in the generator, ensuring clearer output. - Introduced a new function to find the CLI output directory, enhancing the generation process and error reporting for missing markdown files. * add: last 3 versions of all sdks to docs * reactive TOC * refactor: enhance SDK reference generation workflow * fix: pip install fallback * fix: update SDK documentation for clarity and accuracy * remove: generated sdk reference * add: limit as input * fix: infer limit input correctly in workflow * fix: limit * fix limit * chore: force color * cleanup: action * refactor: replace process.exit with error throwing for better error handling * add: tests * feat: enhance SDK configuration with version-specific overrides and add versioning documentation * fix: versioning config * fix: silent fs move issues * fix: pip installs * cleanup files before force regeneration * fix: normailze versioning * improve: pr test workflow + chore: cleanup docs.json * restore: docs.json * Apply suggestion from @ben-fornefeld
1 parent 96e46c9 commit 291102d

44 files changed

Lines changed: 6693 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/sdk-reference-generator-test.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
name: Test SDK Reference Generator
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- "sdk-reference-generator/**"
7+
8+
jobs:
9+
test:
10+
runs-on: ubuntu-latest
11+
12+
steps:
13+
- name: Checkout
14+
uses: actions/checkout@v4
15+
16+
- name: Setup Node.js
17+
uses: actions/setup-node@v4
18+
with:
19+
node-version: "20"
20+
21+
- name: Install pnpm
22+
uses: pnpm/action-setup@v4
23+
with:
24+
version: 9
25+
26+
- name: Install dependencies
27+
working-directory: sdk-reference-generator
28+
run: pnpm install --frozen-lockfile
29+
30+
- name: Run tests
31+
working-directory: sdk-reference-generator
32+
run: pnpm test
33+
34+
- name: TypeScript check
35+
working-directory: sdk-reference-generator
36+
run: npx tsc --noEmit

‎.github/workflows/sdk-reference-sync.yml‎

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
name: Sync SDK Reference Documentation
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
sdk:
7+
description: "SDK to generate (see sdks.config.ts for available SDKs, or use 'all')"
8+
required: true
9+
default: "all"
10+
type: string
11+
version:
12+
description: "Version to generate (all, latest, or specific like v2.9.0)"
13+
required: true
14+
default: "latest"
15+
type: string
16+
limit:
17+
description: "Limit number of versions to generate (default: 5)"
18+
required: false
19+
default: 5
20+
type: number
21+
force:
22+
description: "Force regeneration of existing versions"
23+
required: false
24+
default: false
25+
type: boolean
26+
27+
repository_dispatch:
28+
types: [sdk-release]
29+
30+
# prevent concurrent runs that could conflict
31+
concurrency:
32+
group: sdk-reference-${{ github.ref }}
33+
cancel-in-progress: false
34+
35+
jobs:
36+
generate:
37+
runs-on: ubuntu-latest
38+
timeout-minutes: 30
39+
permissions:
40+
contents: write
41+
42+
steps:
43+
- name: Checkout docs repo
44+
uses: actions/checkout@v4
45+
with:
46+
fetch-depth: 0
47+
48+
- name: Setup Node.js
49+
uses: actions/setup-node@v4
50+
with:
51+
node-version: "20"
52+
53+
- name: Install pnpm
54+
uses: pnpm/action-setup@v4
55+
with:
56+
version: 9
57+
58+
- name: Cache pnpm dependencies
59+
uses: actions/cache@v4
60+
with:
61+
path: |
62+
~/.pnpm-store
63+
sdk-reference-generator/node_modules
64+
key: ${{ runner.os }}-pnpm-${{ hashFiles('sdk-reference-generator/pnpm-lock.yaml') }}
65+
restore-keys: |
66+
${{ runner.os }}-pnpm-
67+
68+
- name: Setup Python
69+
uses: actions/setup-python@v5
70+
with:
71+
python-version: "3.11"
72+
cache: "pip"
73+
74+
- name: Install generator dependencies
75+
working-directory: sdk-reference-generator
76+
run: pnpm install --frozen-lockfile
77+
78+
- name: Install Python dependencies
79+
run: pip install -r requirements.txt
80+
81+
- name: Determine SDK and Version
82+
id: params
83+
env:
84+
EVENT_NAME: ${{ github.event_name }}
85+
INPUT_SDK: ${{ github.event.inputs.sdk }}
86+
INPUT_VERSION: ${{ github.event.inputs.version }}
87+
INPUT_LIMIT: ${{ github.event.inputs.limit }}
88+
INPUT_FORCE: ${{ github.event.inputs.force }}
89+
PAYLOAD_SDK: ${{ github.event.client_payload.sdk }}
90+
PAYLOAD_VERSION: ${{ github.event.client_payload.version }}
91+
PAYLOAD_LIMIT: ${{ github.event.client_payload.limit }}
92+
run: |
93+
if [[ "$EVENT_NAME" == "workflow_dispatch" ]]; then
94+
SDK="$INPUT_SDK"
95+
VERSION="$INPUT_VERSION"
96+
LIMIT="$INPUT_LIMIT"
97+
FORCE="$INPUT_FORCE"
98+
elif [[ "$EVENT_NAME" == "repository_dispatch" ]]; then
99+
SDK="$PAYLOAD_SDK"
100+
VERSION="${PAYLOAD_VERSION:-latest}"
101+
LIMIT="${PAYLOAD_LIMIT:-5}"
102+
FORCE="false"
103+
fi
104+
105+
echo "sdk=${SDK:-all}" >> $GITHUB_OUTPUT
106+
echo "version=${VERSION:-latest}" >> $GITHUB_OUTPUT
107+
echo "limit=${LIMIT:-5}" >> $GITHUB_OUTPUT
108+
echo "force=${FORCE:-false}" >> $GITHUB_OUTPUT
109+
110+
- name: Generate SDK Reference
111+
working-directory: sdk-reference-generator
112+
env:
113+
SDK_NAME: ${{ steps.params.outputs.sdk }}
114+
SDK_VERSION: ${{ steps.params.outputs.version }}
115+
LIMIT: ${{ steps.params.outputs.limit }}
116+
FORCE: ${{ steps.params.outputs.force }}
117+
FORCE_COLOR: "2"
118+
run: |
119+
ARGS="--sdk $SDK_NAME --version $SDK_VERSION"
120+
121+
if [[ -n "$LIMIT" && "$LIMIT" != "0" ]]; then
122+
ARGS="$ARGS --limit $LIMIT"
123+
fi
124+
125+
if [[ "$FORCE" == "true" ]]; then
126+
ARGS="$ARGS --force"
127+
fi
128+
129+
pnpm run generate $ARGS
130+
131+
- name: Commit and push changes
132+
id: commit
133+
env:
134+
SDK_NAME: ${{ steps.params.outputs.sdk }}
135+
SDK_VERSION: ${{ steps.params.outputs.version }}
136+
run: |
137+
git config user.name "github-actions[bot]"
138+
git config user.email "github-actions[bot]@users.noreply.github.com"
139+
140+
git add docs/sdk-reference/
141+
git add docs.json
142+
143+
if git diff --staged --quiet; then
144+
echo "No changes to commit"
145+
echo "changes=false" >> $GITHUB_OUTPUT
146+
else
147+
git commit -m "docs: update SDK reference for $SDK_NAME $SDK_VERSION"
148+
git push
149+
echo "changes=true" >> $GITHUB_OUTPUT
150+
fi
151+
152+
- name: Summary
153+
env:
154+
SDK_NAME: ${{ steps.params.outputs.sdk }}
155+
SDK_VERSION: ${{ steps.params.outputs.version }}
156+
LIMIT: ${{ steps.params.outputs.limit }}
157+
CHANGES: ${{ steps.commit.outputs.changes }}
158+
run: |
159+
echo "## SDK Reference Generation Complete" >> $GITHUB_STEP_SUMMARY
160+
echo "" >> $GITHUB_STEP_SUMMARY
161+
echo "| Parameter | Value |" >> $GITHUB_STEP_SUMMARY
162+
echo "|-----------|-------|" >> $GITHUB_STEP_SUMMARY
163+
echo "| SDK | $SDK_NAME |" >> $GITHUB_STEP_SUMMARY
164+
echo "| Version | $SDK_VERSION |" >> $GITHUB_STEP_SUMMARY
165+
echo "| Limit | ${LIMIT:-No limit} |" >> $GITHUB_STEP_SUMMARY
166+
echo "| Changes committed | $CHANGES |" >> $GITHUB_STEP_SUMMARY
167+
echo "" >> $GITHUB_STEP_SUMMARY
168+
169+
if [[ "$CHANGES" == "true" ]]; then
170+
echo "### Generated Files" >> $GITHUB_STEP_SUMMARY
171+
echo '```' >> $GITHUB_STEP_SUMMARY
172+
echo "Total MDX files: $(find docs/sdk-reference -type f -name '*.mdx' | wc -l)" >> $GITHUB_STEP_SUMMARY
173+
echo '```' >> $GITHUB_STEP_SUMMARY
174+
fi

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,7 @@
11
.idea
22
.DS_Store
3+
4+
node_modules
5+
6+
# Generated SDK navigation (intermediate file)
7+
sdk_navigation.json

‎.mintignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
sdk-reference-generator/
2+
3+
scripts/

‎requirements.txt‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Python dependencies for SDK reference generation
2+
pydoc-markdown>=4.8.2
3+
poetry>=1.8.0
4+

‎scripts/generate-mcp-servers.sh‎

100644100755
File mode changed.

‎sdk-reference-generator/.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
node_modules/
2+
dist/
3+
*.log
4+

0 commit comments

Comments
 (0)