[typespec-python] Add generated test regeneration workflow (#46414)

Author: iscai-msft

.github/CODEOWNERS

@@ -862,5 +862,8 @@
 /eng/emitter-package.json                                            @mccoyp @catalinaperalta @iscai-msft @msyyc @ChenxiJiang333
 /eng/emitter-package-lock.json                                       @mccoyp @catalinaperalta @iscai-msft @msyyc @ChenxiJiang333
 
+# TypeSpec Python generated tests and regeneration workflow
+/eng/tools/emitter/                                                  @tadelesh @msyyc @iscai-msft @lmazuel @lirenhe
+
 /pylintrc                                                            @l0lawrence @scbedd @mccoyp
 /sdk/**/ci.yml                                                       @msyyc @lmazuel @scbedd

.github/skills/emitter-package-update/SKILL.md

@@ -7,6 +7,14 @@ description: Automate bumping typespec-python version in emitter-package.json fo
 
 Bump `@azure-tools/typespec-python` to the latest version in `emitter-package.json` and create a PR.
 
+## Background
+
+The Python emitter ecosystem consists of two packages:
+- **Branded emitter** (`@azure-tools/typespec-python`): Lives in [Azure/typespec-azure](https://github.com/Azure/typespec-azure/tree/main/packages/typespec-python). This is the emitter used for Azure SDK generation.
+- **Unbranded emitter** (`@typespec/http-client-python`): Lives in [microsoft/typespec](https://github.com/microsoft/typespec/tree/main/packages/http-client-python). The branded emitter wraps this package.
+
+When `eng/emitter-package.json` is updated on `main`, the [TypeSpec Python Regenerate Tests](../../workflows/typespec-python-regenerate.yml) workflow triggers automatically and regenerates test code at `eng/emitter/gen/`.
+
 ## Prerequisites
 
 Before running this workflow, verify the following tools are installed:
@@ -45,15 +53,15 @@ Run npm-check-updates to find the latest `@azure-tools/typespec-python` version:
 npx npm-check-updates --packageFile eng/emitter-package.json
 ```
 
-Extract the target version from the output (e.g., `0.46.4`).
+Extract the target version from the output (e.g., `0.61.3`).
 
 ### 3. Create Feature Branch
 
 ```bash
 git checkout -b bump-typespec-python-{version}
 ```
 
-Replace `{version}` with the actual version number (e.g., `bump-typespec-python-0.46.4`).
+Replace `{version}` with the actual version number (e.g., `bump-typespec-python-0.61.3`).
 
 ### 4. Update Dependencies
 
@@ -65,24 +73,41 @@ npx npm-check-updates --packageFile eng/emitter-package.json -u
 
 Align `@azure-tools/openai-typespec` and `@typespec/openapi3` with the versions pinned in [azure-rest-api-specs/package.json](https://github.com/Azure/azure-rest-api-specs/blob/main/package.json) to ensure consistency between the emitter and the spec repo. Check the spec repo's versions and update `eng/emitter-package.json` accordingly (e.g., set `"@azure-tools/openai-typespec": "1.8.0"` and `"@typespec/openapi3": "1.9.0"` to match).
 
-Regenerate the lock file:
+### 5. Regenerate Config Files
+
+If you have a local clone of `typespec-azure`, regenerate the lock file using the branded emitter's `package.json` to ensure `devDependencies` are aligned:
+
+```bash
+tsp-client generate-config-files \
+  --package-json=<path-to-local-typespec-azure>/packages/typespec-python/package.json
+```
+
+Otherwise, regenerate the lock file only:
 
 ```bash
 tsp-client generate-lock-file
 ```
 
-### 5. Commit Changes
+### 6. Commit Changes
 
 ```bash
 git add eng/emitter-package.json eng/emitter-package-lock.json
 git commit -m "bump typespec-python {version}"
 ```
 
-### 6. Create Pull Request
+### 7. Create Pull Request
 
 Push branch and create PR:
 
 ```bash
 git push -u origin bump-typespec-python-{version}
 gh pr create --title "bump typespec-python {version}" --body "Bump @azure-tools/typespec-python to version {version}"
 ```
+
+### 8. After Merge
+
+Once the PR merges to `main`, the [TypeSpec Python Regenerate Tests](../../workflows/typespec-python-regenerate.yml) workflow triggers automatically because `eng/emitter-package.json` was modified. It will:
+1. Install the branded emitter at the version specified in `eng/emitter-package.json`
+2. Regenerate all test code
+3. Create a follow-up PR with the updated generated files at `eng/emitter/gen/`
+4. Tag @iscai-msft and @msyyc as reviewers

.github/workflows/typespec-python-regenerate.yml

@@ -0,0 +1,228 @@
+name: TypeSpec Python Regenerate Tests
+
+on:
+  # Trigger when eng/emitter-package.json is updated on main (branded emitter version bump)
+  push:
+    branches: [main]
+    paths:
+      - "eng/emitter-package.json"
+  # Allow manual triggering with emitter choice
+  workflow_dispatch:
+    inputs:
+      emitter:
+        description: "Which emitter to regenerate with"
+        required: true
+        type: choice
+        options:
+          - "branded (@azure-tools/typespec-python)"
+          - "unbranded (@typespec/http-client-python)"
+        default: "branded (@azure-tools/typespec-python)"
+      version:
+        description: "Emitter version (leave empty to use version from emitter-package.json for branded, or latest for unbranded)"
+        required: false
+      typespec_ref:
+        description: "Git ref (branch, tag, or SHA) of typespec repo to build regeneration infrastructure from"
+        required: false
+        default: "main"
+      typespec_repo:
+        description: "TypeSpec repository (owner/repo) to checkout from (default: microsoft/typespec)"
+        required: false
+        default: "microsoft/typespec"
+
+permissions:
+  contents: write
+  pull-requests: write
+  issues: write
+
+concurrency:
+  group: ${{ github.workflow }}
+  cancel-in-progress: true
+
+jobs:
+  regenerate:
+    name: "Regenerate TypeSpec Python tests"
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout azure-sdk-for-python
+        # SHA corresponds to actions/checkout@v6
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
+        with:
+          fetch-depth: 0
+
+      - name: Determine emitter to use
+        id: emitter-info
+        run: |
+          set -euo pipefail
+          if [ "${{ github.event_name }}" = "push" ]; then
+            # Push trigger: always use branded emitter at version from emitter-package.json
+            EMITTER_NAME="@azure-tools/typespec-python"
+            VERSION=$(jq -r '.dependencies["@azure-tools/typespec-python"]' eng/emitter-package.json)
+            echo "Push trigger: using branded emitter @ $VERSION"
+          elif [[ "${{ github.event.inputs.emitter }}" == branded* ]]; then
+            EMITTER_NAME="@azure-tools/typespec-python"
+            VERSION="${{ github.event.inputs.version }}"
+            if [ -z "$VERSION" ]; then
+              VERSION=$(jq -r '.dependencies["@azure-tools/typespec-python"]' eng/emitter-package.json)
+              echo "No version specified, using emitter-package.json version: $VERSION"
+            fi
+          else
+            EMITTER_NAME="@typespec/http-client-python"
+            VERSION="${{ github.event.inputs.version }}"
+            if [ -z "$VERSION" ]; then
+              VERSION=$(npm view @typespec/http-client-python version 2>/dev/null)
+              echo "No version specified, using latest npm version: $VERSION"
+            fi
+          fi
+          echo "emitter_name=$EMITTER_NAME" >> $GITHUB_OUTPUT
+          echo "emitter_version=$VERSION" >> $GITHUB_OUTPUT
+          echo "::notice::Regenerating with $EMITTER_NAME@$VERSION"
+
+      - name: Checkout microsoft/typespec
+        # SHA corresponds to actions/checkout@v6
+        # Checkout to "_typespec" (not "typespec") to avoid the workspace path
+        # "azure-sdk-for-python" causing spec.includes("azure") to match all specs
+        # in regenerate.ts, which breaks unbranded package name detection
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
+        with:
+          repository: ${{ github.event.inputs.typespec_repo || 'microsoft/typespec' }}
+          ref: ${{ github.event.inputs.typespec_ref || 'main' }}
+          path: _typespec
+          fetch-depth: 0
+
+      - name: Setup Node.js
+        # SHA corresponds to actions/setup-node@v6
+        uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e
+        with:
+          node-version: lts/*
+
+      - name: Setup Python
+        # SHA corresponds to actions/setup-python@v5
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
+        with:
+          python-version: "3.12"
+
+      - name: Build http-client-python
+        working-directory: _typespec/packages/http-client-python
+        run: |
+          npm install --ignore-scripts
+          npm run build
+
+      - name: Install target emitter
+        working-directory: _typespec/packages/http-client-python
+        run: |
+          EMITTER="${{ steps.emitter-info.outputs.emitter_name }}"
+          VERSION="${{ steps.emitter-info.outputs.emitter_version }}"
+          echo "Installing $EMITTER@$VERSION"
+          npm install "${EMITTER}@${VERSION}"
+
+      - name: Prepare Python environment
+        working-directory: _typespec/packages/http-client-python
+        run: |
+          npm run install
+          npm run prepare
+
+      - name: Regenerate tests
+        working-directory: _typespec/packages/http-client-python
+        run: |
+          EMITTER="${{ steps.emitter-info.outputs.emitter_name }}"
+          npm run regenerate -- --emitterName "$EMITTER"
+
+      - name: Copy regenerated tests
+        run: |
+          set -euo pipefail
+          TARGET="eng/tools/emitter/gen"
+          rm -rf "$TARGET/azure" "$TARGET/unbranded"
+          mkdir -p "$TARGET"
+          cp -r "_typespec/packages/http-client-python/tests/generated/azure" "$TARGET/azure"
+          cp -r "_typespec/packages/http-client-python/tests/generated/unbranded" "$TARGET/unbranded"
+
+      - name: Clean up typespec checkout
+        run: rm -rf "_typespec"
+
+      - name: Commit and push changes
+        id: push-changes
+        run: |
+          set -euo pipefail
+          EMITTER="${{ steps.emitter-info.outputs.emitter_name }}"
+          VERSION="${{ steps.emitter-info.outputs.emitter_version }}"
+          BRANCH="auto/typespec-python-regenerate"
+          git config user.name "github-actions[bot]"
+          git config user.email "github-actions[bot]@users.noreply.github.com"
+
+          git add eng/tools/emitter/gen/
+          if git diff --cached --quiet; then
+            echo "No changes to commit"
+            echo "has_changes=false" >> $GITHUB_OUTPUT
+            exit 0
+          fi
+
+          git checkout -B "$BRANCH"
+          git commit -m "[typespec-python] Regenerate tests with ${EMITTER}@${VERSION}"
+          git push origin "$BRANCH" --force
+          echo "has_changes=true" >> $GITHUB_OUTPUT
+          echo "branch=$BRANCH" >> $GITHUB_OUTPUT
+
+      - name: Create or update Pull Request
+        id: create-pr
+        if: steps.push-changes.outputs.has_changes == 'true'
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          set -euo pipefail
+          EMITTER="${{ steps.emitter-info.outputs.emitter_name }}"
+          VERSION="${{ steps.emitter-info.outputs.emitter_version }}"
+          BRANCH="${{ steps.push-changes.outputs.branch }}"
+          TITLE="[typespec-python] Regenerate tests with ${EMITTER}@${VERSION}"
+          BODY="Automated regeneration of TypeSpec Python generated tests.
+
+          - Emitter: \`${EMITTER}@${VERSION}\`
+          - Workflow run: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+
+          This PR was auto-generated."
+
+          # Check if a PR already exists for this branch
+          EXISTING_PR=$(gh pr list --head "$BRANCH" --json number --jq '.[0].number' || echo "")
+          if [ -n "$EXISTING_PR" ]; then
+            echo "Updating existing PR #$EXISTING_PR"
+            gh pr edit "$EXISTING_PR" --title "$TITLE" --body "$BODY" --add-reviewer iscai-msft,msyyc
+            PR_NUMBER="$EXISTING_PR"
+          else
+            echo "Creating new PR"
+            PR_NUMBER=$(gh pr create --head "$BRANCH" --base main \
+              --title "$TITLE" --body "$BODY" \
+              --reviewer iscai-msft,msyyc | grep -oP '\d+$')
+            echo "Created PR #$PR_NUMBER"
+          fi
+          echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT
+
+      - name: Enable auto-merge
+        if: steps.push-changes.outputs.has_changes == 'true'
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          gh pr merge "${{ steps.create-pr.outputs.pr_number }}" --auto --squash || \
+            echo "::warning::Auto-merge could not be enabled (may require branch protection rules)"
+
+  notify-on-failure:
+    name: "Notify on failure"
+    needs: regenerate
+    if: failure()
+    runs-on: ubuntu-latest
+    steps:
+      - name: Send failure notification
+        # SHA corresponds to actions/github-script@v7
+        uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b
+        with:
+          script: |
+            const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
+            await github.rest.issues.create({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              title: '[typespec-python] Regeneration workflow failed',
+              body: `The TypeSpec Python test regeneration workflow failed.\n\n` +
+                    `- **Run:** ${runUrl}\n` +
+                    `- **Trigger:** ${context.eventName}\n\n` +
+                    `cc @iscai-msft @msyyc`,
+              labels: ['typespec-python'],
+              assignees: ['iscai-msft', 'msyyc']
+            });

.vscode/cspell.json

@@ -295,6 +295,7 @@
     "inotify",
     "ints",
     "iohttp",
+    "iscai",
     "IOHTTP",
     "IOLRO",
     "incompat",
@@ -348,6 +349,7 @@
     "msauth",
     "msrest",
     "msrestazure",
+    "msyyc",
     "MSSQL",
     "MSRC",
     "mutex",

doc/dev/static_type_checking.md

@@ -5,7 +5,7 @@ It also walks through the setup necessary to run mypy and pyright, static type c
 
 For the TL;DR version, please see the [Static Type Checking Cheat Sheet](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/static_type_checking_cheat_sheet.md).
 
-> Note: If you are seeing typing errors in **generated code**, please first try to regenerate with the latest version of the generator. If typing errors persist, open an issue in the [generator repo](https://github.com/Azure/autorest.python).
+> Note: If you are seeing typing errors in **generated code**, please first try to regenerate with the latest version of the generator. If typing errors persist, open an issue in the [TypeSpec repo](https://github.com/microsoft/typespec) for TypeSpec-based SDKs or the [autorest.python repo](https://github.com/Azure/autorest.python) for swagger-based SDKs.
 
 ## Table of contents
   - [Intro to typing in Python](#intro-to-typing-in-python)

eng/tools/emitter/README.md

@@ -0,0 +1,10 @@
+# TypeSpec Python Emitter
+
+This directory holds files related to the TypeSpec Python emitter (`@azure-tools/typespec-python` and `@typespec/http-client-python`).
+
+## Contents
+
+- **`gen/`** — Auto-generated Python SDK test code produced by the emitter.
+  These files are regenerated automatically by the [typespec-python-regenerate](https://github.com/Azure/azure-sdk-for-python/blob/main/.github/workflows/typespec-python-regenerate.yml) workflow whenever the emitter version in `eng/emitter-package.json` is updated.
+
+> **Do not edit files in `gen/` by hand.** They will be overwritten on the next regeneration.

eng/tools/emitter/gen/README.md

@@ -0,0 +1,30 @@
+# TypeSpec Python Generated Tests
+
+This folder contains the generated Python SDK test code produced by
+`@typespec/http-client-python` against the [http-specs](https://github.com/microsoft/typespec/tree/main/packages/http-specs)
+and [azure-http-specs](https://github.com/Azure/typespec-azure/tree/main/packages/azure-http-specs) test suites.
+
+## Structure
+
+```
+azure/       # Generated from azure-http-specs (branded/Azure flavor)
+unbranded/   # Generated from http-specs (unbranded flavor)
+```
+
+## Regeneration
+
+These files are automatically regenerated by the
+[`typespec-python-regenerate.yml`](https://github.com/Azure/azure-sdk-for-python/blob/main/.github/workflows/typespec-python-regenerate.yml) workflow.
+
+To regenerate manually, trigger the workflow via GitHub Actions or run locally:
+
+```bash
+# From the microsoft/typespec repo (packages/http-client-python):
+npm install --ignore-scripts
+npm run build
+npm run install
+npm run prepare
+npm run regenerate
+
+# Then copy tests/generated/{azure,unbranded} to this folder
+```

eng/tools/emitter/gen/azure/authentication-api-key/CHANGELOG.md

@@ -0,0 +1,7 @@
+# Release History
+
+## 1.0.0b1 (1970-01-01)
+
+### Other Changes
+
+  - Initial version