Python: Split type checkers by target (pyright source, 5 checkers on tests/samples) (#6443)

* Python: Split type checkers by target (pyright source, 5 checkers on tests/samples)

Rework the typing setup along the lines of the 'too many type checkers'
approach:

- Pyright (strict) is now the sole source-code type checker; mypy is
  removed from source and its [tool.mypy] block becomes a relaxed profile
  used only for tests/samples.
- Tests are checked by all five checkers (pyright relaxed, mypy, pyrefly,
  ty, zuban); samples by pyright, pyrefly, and ty. All run in a relaxed/
  basic profile so authors aren't forced into over-annotation.
- Add pyrightconfig.tests.json and bump sample pyright configs to basic.
- Unify test/sample typing onto the same parallel fan-out used by source
  pyright via run_command_items in task_runner.py.
- Make version-conditional imports symmetric: keep or drop the
  '# type: ignore' on both branches so results match across interpreter
  versions (local vs CI).
- Update SKILL.md, DEV_SETUP.md, and CODING_STANDARD.md for the five
  gating checkers and pyright on source+tests+samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix merge regressions from main (typing + runtime)

Merging main into the type-checker split branch surfaced regressions that
the new five-checker test suite and unit tests caught:

Runtime fixes:
- anthropic: restore the dropped `cache_read_input_token_count` mapping in
  _parse_usage_from_anthropic (lost during merge conflict resolution).
- gemini: _get_function_calling_mode test helper returned str(enum)
  ('FunctionCallingConfigMode.AUTO') instead of the enum value ('AUTO').
- openai: _response_id_from_token test helper was an infinite self-recursion;
  return token['response_id'].
- orchestrations: reset output_events per approval iteration so the terminal
  output assertion counts only the final run.
- core: drop a stale duplicate harness test whose message ('non-negative')
  contradicted the source ('positive').
- purview: import PolicyLocation/PolicyScope/ProtectionScopeActivities/
  ExecutionMode used by the processor tests.

Type-checker fixes (tests, relaxed profile):
- core: pyright/mypy/pyrefly/ty/zuban green-ups across the harness, MCP,
  observability and types tests.
- anthropic/openai: route provider-namespaced UsageDetails keys through a
  dict cast (extra_items TypedDict unsupported by mypy/ty).
- purview: typed model constructors and cache-mock casts.
- ag-ui: annotate WorkflowContext[Any, Any] so yield_output accepts test
  payloads, guard Optional forwarded_props, and ty-ignore intentional bad args.

Source pyright (sole source checker) flagged unnecessary ignores newly
introduced by merged code in core _tools.py and declarative _declarative_base.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Isolate per-package mypy cache in test-typing fan-out

The parallel test-typing fan-out runs many mypy processes concurrently,
all defaulting to a single shared ./.mypy_cache. Concurrent writes corrupt
the cache and mypy aborts with INTERNAL ERROR (intermittently, depending on
worker timing) -- which is why CI's Test Typing job failed on a shifting set
of packages while a single-package run was fine.

Give each mypy invocation an isolated cache dir keyed by its target paths so
incremental caching still works per package without races. Other checkers
(zuban/pyrefly/ty/pyright) maintain their own caches and are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Make lab pyright-only on source (drop source mypy)

Lab was the last package still running mypy on its source code, requiring
mypy-only `# type: ignore` comments that pyright (the sole source checker
everywhere else) flags as unnecessary. Align lab with the rest of the
monorepo:

- Remove the lab source mypy poe tasks (mypy-gaia/lightning/tau2) and the
  now-dead strict [tool.mypy] config block.
- Drop the 'Run lab mypy' CI step; lab source is type-checked by pyright only.

Lab tests remain covered by the workspace test-typing fan-out (mypy, pyrefly,
ty, zuban, pyright over tests using the relaxed root config).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix test-typing regressions from latest main merge

A fresh merge from main brought in new test code never run under the
five-checker test-typing suite. Green up across the affected packages:

- core: narrow Optional span.attributes with 'and' guards in span filters
  and assert+cast the json.loads(...attributes[...]) reads (test_observability);
  match the existing as_agent ignore on the protocol-typed fixture (test_clients).
- openai: align new streaming tests with the established chat_options dict
  pattern (ChatOptions TypedDict isn't assignable to dict), route Optional
  .annotations[0] access through a small _first_annotation helper (mirrors the
  file's assert-not-None convention), and annotate a mapped ResponseStream.
- foundry_hosting: annotate error: dict[str, Any] = body.get(...) or {}
  (zuban needs the annotation).
- foundry: narrow ignores for the live AIProjectClient credential arg (pyrefly)
  and connections.get_default (zuban) SDK type gaps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* updated pyright version

* pyright fix

* Python: Fix source typing for pyright 1.1.410

Pyright 1.1.410 tightened several checks. Apply the same source fixes as
upstream PR #6275:

- anthropic: import AsyncAnthropicBedrock from anthropic.lib.bedrock and
  AsyncAnthropicVertex from anthropic.lib.vertex (no longer re-exported from
  the anthropic top-level package -> reportPrivateImportUsage).
- core _types.py: cast the transform-hook result to UpdateT (reportAssignmentType).
- core _workflows/_events.py: annotate the @contextmanager helper as
  Generator[None] instead of Iterator[None] (reportDeprecated).
- redis: build the combined filter expression with an explicit loop instead of
  reduce(and_, ...), which pyright could no longer fully type (drops the now
  unused functools.reduce / operator.and_ imports).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Accept plain-text body in Azure Functions workflow/run endpoint

The workflow_orchestrator already accepts plain strings as well as JSON
objects via context.get_input(), but the start_workflow_orchestration HTTP
handler only accepted JSON and returned 400 for any non-JSON body. This made
the functions integration tests that POST text/plain to /api/workflow/run
(e.g. test_09_workflow_shared_state) fail consistently with 400 != 202.

Fall back to the raw request body (decoded as UTF-8) when the body is not
JSON, rejecting only a truly empty body. The JSON path is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: eavanvalkenburg

.github/workflows/python-code-quality.yml

@@ -113,8 +113,8 @@ jobs:
       - name: Run markdown code lint
         run: uv run poe markdown-code-lint
 
-  mypy:
-    name: Mypy Checks
+  test-typing:
+    name: Test Typing Checks
     if: "!cancelled()"
     strategy:
       fail-fast: false
@@ -139,7 +139,5 @@ jobs:
           os: ${{ runner.os }}
         env:
           UV_CACHE_DIR: /tmp/.uv-cache
-      - name: Run Mypy
-        env:
-          GITHUB_BASE_REF: ${{ github.event.pull_request.base.ref || github.base_ref || 'main' }}
-        run: uv run python scripts/workspace_poe_tasks.py ci-mypy
+      - name: Run tests/samples type checkers (mypy, pyrefly, ty)
+        run: uv run python scripts/workspace_poe_tasks.py ci-test-typing

.github/workflows/python-lab-tests.yml

@@ -92,9 +92,6 @@ jobs:
       - name: Run lab type checking
         run: cd packages/lab && uv run poe pyright
 
-      - name: Run lab mypy
-        run: cd packages/lab && uv run poe mypy
-
       # Surface failing tests
       - name: Surface failing tests
         if: always()

python/.github/skills/python-code-quality/SKILL.md

@@ -21,13 +21,22 @@ uv run poe syntax -C    # Check only
 uv run poe syntax -S    # Samples only
 
 # Type checking
-uv run poe pyright       # Pyright fan-out across packages
+#
+# Division of labor (see "Type checking architecture" below):
+#   - Pyright (strict) is the source-code type checker.
+#   - Pyright (relaxed `basic`), mypy, pyrefly, ty, zuban all check the TESTS;
+#     pyright/pyrefly/ty also check the SAMPLES (mypy/zuban skip script-style samples).
+uv run poe pyright       # Pyright (strict) over SOURCE, fan-out across packages
 uv run poe pyright -P core
 uv run poe pyright -A
-uv run poe mypy          # MyPy fan-out across packages
+uv run poe test-typing   # mypy + pyrefly + ty + zuban + pyright over each package's TESTS
+uv run poe test-typing -P core
+uv run poe test-typing -S                       # samples (pyrefly + ty + pyright)
+uv run poe test-typing -P core --checker mypy   # narrow to one checker (repeatable)
+uv run poe test-typing -P core --checker pyright # relaxed pyright over the tests
+uv run poe mypy          # alias: MyPy over the tests only
 uv run poe mypy -P core
-uv run poe mypy -A
-uv run poe typing        # Both pyright and mypy
+uv run poe typing        # Pyright (source) + the tests checkers
 uv run poe typing -P core
 uv run poe typing -A
 
@@ -67,6 +76,33 @@ when markdown files change, and sample syntax lint/pyright only when files
 under `samples/` change.
 They intentionally do not run workspace `pyright` or `mypy` by default.
 
+## Type checking architecture
+
+Following the "too many type checkers" approach, type checkers are split by target:
+
+| Target | Checker(s) | Mode | Config |
+|--------|-----------|------|--------|
+| Source (`agent_framework*`) | **pyright** | strict | `[tool.pyright]` in `pyproject.toml` |
+| Tests | pyright, mypy, pyrefly, ty, zuban | relaxed/basic | `pyrightconfig.tests.json`, `[tool.mypy]`, `pyrefly.toml`, `ty` rules |
+| Samples | pyright, pyrefly, ty | basic | `pyrightconfig.samples.json`, `pyrefly.samples.toml`, `ty.samples.toml` |
+
+- **Pyright is the only *strict* source-code checker**, and it ALSO runs in a relaxed
+  `basic` profile over the tests and samples (so the surfaces customers copy from are
+  validated by every checker, including pyright). MyPy was removed from source; its
+  `[tool.mypy]` block is now a *relaxed* profile used only for tests/samples.
+- The extra checkers run over tests/samples because those exercise the public API the way
+  users do. The profile is intentionally relaxed (private access allowed, untyped test
+  bodies allowed) so authors aren't forced into ugly over-annotation.
+- **Gating checkers** are `pyright`, `mypy`, `pyrefly`, `ty`, and `zuban` — all five run by
+  default and gate CI. `zuban` is the strictest of the mypy-compatible pair, so the same
+  `[tool.mypy]` config yields more findings; suppress zuban-only friction with shared
+  `# type: ignore[code]`. Suppress relaxed-pyright friction with `# pyright: ignore[rule]`.
+- **Samples** add `pyright` to `pyrefly` + `ty` — mypy/zuban can't resolve script-style
+  sample layouts (numeric-prefixed dirs, duplicate `main.py`), but pyright handles them.
+- The strict source-pyright (`[tool.pyright]`) enforces `reportUnnecessaryTypeIgnoreComment`
+  and excludes tests/samples; the relaxed test/sample pyright configs do not flag unnecessary
+  ignores.
+
 ## Ruff Configuration
 
 - Line length: 120
@@ -77,8 +113,12 @@ They intentionally do not run workspace `pyright` or `mypy` by default.
 
 ## Pyright Configuration
 
-- Strict mode enabled
-- Excludes: tests, .venv, packages/devui/frontend
+- **Source**: strict mode (`[tool.pyright]`), `reportUnnecessaryTypeIgnoreComment = "error"`,
+  excludes tests, samples, .venv, packages/devui/frontend.
+- **Tests**: relaxed `basic` profile (`pyrightconfig.tests.json`) — private import/usage and
+  not-required TypedDict access allowed; runs as the `pyright` checker in `test-typing`.
+- **Samples**: relaxed `basic` profile (`pyrightconfig.samples.json`, with a py310 variant) —
+  runs as the `pyright` checker in `test-typing -S`.
 
 ## Parallel Execution
 
@@ -90,6 +130,6 @@ in-process with streaming output.
 
 CI splits into 4 parallel jobs:
 1. **Pre-commit hooks** — lightweight hooks (SKIP=poe-check)
-2. **Package checks** — syntax/pyright via check-packages
+2. **Package checks** — syntax/pyright (source) via check-packages
 3. **Samples & markdown** — `check -S` plus `markdown-code-lint`
-4. **Mypy** — change-detected mypy checks
+4. **Test Typing** — change-detected mypy/pyrefly/ty over tests (`ci-test-typing`)

python/CODING_STANDARD.md

@@ -92,11 +92,18 @@ Use typing as a helper first and suppressions as a last resort:
 - **Prefer explicit typing before suppression**: Start with clearer type annotations, helper types, overloads,
   protocols, or refactoring dynamic code into typed helpers. Prioritize performance over completeness of typing, but make a good-faith effort to reduce uncertainty with typing before ignoring. Prefer to use a cast over a typeguard function since that does add overhead.
 - **Avoid redundant casts**: Do not add `cast(...)` if the type already matches; casts should be reserved for
-  unavoidable narrowing where the runtime contract is known, we will use mypy's check on redundant casts to enforce this.
+  unavoidable narrowing where the runtime contract is known.
 - **Avoid multiple assignments**: Avoid assigning multiple variables just to get typing to pass, that has performance impact while typing should not have that.
-- **Line-level pyright ignores only**: If suppression is still required, use a line-level rule-specific ignore
+- **Source vs tests/samples**: Source code (`agent_framework*`) is checked **by pyright in strict mode** — use
+  `# pyright: ignore[...]` there, never `# type: ignore` (strict pyright flags unnecessary ignores as errors). Tests
+  and samples are checked by pyright (relaxed `basic`), mypy, pyrefly, ty (and zuban on tests) in a relaxed/basic
+  profile; prefer real fixes (`isinstance`, `cast`, annotations, asserts for Optional access) over per-line ignores,
+  and keep test/sample bodies readable rather than over-annotated. When a relaxed-pyright suppression is genuinely
+  needed in tests/samples, use `# pyright: ignore[rule]`; the relaxed test/sample configs do not flag unnecessary
+  ignores, so combine with a mypy/zuban `# type: ignore[code]` on the same line only where both are required.
+- **Line-level pyright ignores only**: If suppression is still required in source, use a line-level rule-specific ignore
   (`# pyright: ignore[reportGeneralTypeIssues]`), file-level is allowed if there is a compelling reason for it, that should be documented right beneath the ignore.
-  Never change the global suppression flags for mypy and pyright unless the dev team okays it.
+  Never change the global suppression flags unless the dev team okays it.
 - **Private usage boundary**: Accessing private members across `agent_framework*` packages can be acceptable for this
   codebase, but private member usage for non-Agent Framework dependencies should remain flagged.
 

python/DEV_SETUP.md

@@ -289,23 +289,36 @@ uv run poe <command> -A           # aggregate sweep where supported
 ```
 
 #### `pyright`
-Run Pyright type checking:
+Run Pyright type checking. Pyright is the **strict source-code type checker**, and also runs
+in a relaxed `basic` profile over the tests + samples (as one of the `test-typing` checkers):
 ```bash
 uv run poe pyright
 uv run poe pyright -P core
 uv run poe pyright -A
 ```
 
+#### `test-typing`
+Run the **tests + samples** type checkers. Source code is owned by strict Pyright; the tests
+and samples are checked by `pyright` (relaxed), `mypy`, `pyrefly`, `ty`, and `zuban` in a
+deliberately relaxed/basic profile so real public-API type errors surface without forcing
+test/sample authors to fully annotate their code. All five gate CI:
+```bash
+uv run poe test-typing            # all checkers over every package's tests
+uv run poe test-typing -P core    # one package
+uv run poe test-typing -S         # samples (pyright + pyrefly + ty; mypy/zuban skip script-style samples)
+uv run poe test-typing -P core --checker mypy     # narrow to one checker (repeatable)
+uv run poe test-typing -P core --checker pyright  # relaxed pyright over the tests
+```
+
 #### `mypy`
-Run MyPy type checking:
+Convenience alias that runs MyPy over the test suite (MyPy no longer runs on source):
 ```bash
 uv run poe mypy
 uv run poe mypy -P core
-uv run poe mypy -A
 ```
 
 #### `typing`
-Run both Pyright and MyPy:
+Run Pyright over source **and** the tests/samples checkers:
 ```bash
 uv run poe typing
 uv run poe typing -P core

python/packages/a2a/agent_framework_a2a/_agent.py

@@ -344,7 +344,7 @@ def run(
         **kwargs: Any,
     ) -> ResponseStream[AgentResponseUpdate, AgentResponse[Any]]: ...
 
-    def run(  # pyright: ignore[reportIncompatibleMethodOverride]
+    def run(
         self,
         messages: AgentRunInputs | None = None,
         *,
@@ -464,7 +464,7 @@ async def _map_a2a_stream(
             if session is None:
                 raise RuntimeError("Provider session must be available when context providers are configured.")
             await provider.before_run(
-                agent=self,  # type: ignore[arg-type]
+                agent=self,
                 session=session,
                 context=session_context,
                 state=session.state.setdefault(provider.source_id, {}),