Skip to content

typing: graduate tests.test_agent* chunk out of the mypy baseline#1877

Merged
JSv4 merged 3 commits into
mainfrom
claude/determined-faraday-pjUsP
Jun 1, 2026
Merged

typing: graduate tests.test_agent* chunk out of the mypy baseline#1877
JSv4 merged 3 commits into
mainfrom
claude/determined-faraday-pjUsP

Conversation

@JSv4

@JSv4 JSv4 commented Jun 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Continues the mypy baseline drain (issue #1738, the #1331 → #1335 → #1447 cadence), graduating the agent test domain — the next cohesive chunk in the issue's tests.* leaf-files batching plan (corpus / document / annotation / agent / extract / misc), following the corpus chunk in #1866, document in #1851, and annotation in #1776.

  • Removed all 9 [mypy-opencontractserver.tests.test_agent*] ignore_errors blocks from mypy.ini (test_agent_action_result, test_agent_action_result_admin, test_agent_api, test_agent_factory, test_agent_framework_api, test_agent_memory, test_agent_tasks, test_agentic_highlighter_task, test_agents).
  • Pruned the corresponding 364 lines from docs/typing/mypy_baseline.txt (4012 → 3648).
  • Fixed the 338 errors that surfaced across all 9 files.

Full surface passes under both pinned toolchains:

  • mypy==2.0.0 / django-stubs==6.0.5 (pre-commit .pre-commit-config.yaml) → Success: no issues found in 1247 source files
  • mypy==2.1.0 / django-stubs==6.0.5 (CI requirements/local.txt) → Success

The 338 is below the 364-line baseline snapshot because the newer stubs already resolve several historical errors — notably the set_permissions_for_obj_to_user arg-type family in test_agentic_highlighter_task.py (the same observation the document/annotation chunks made), which is now superseded by a different surfaced error (assign-to-ClassVar).

How

All changes are type-level and behaviour-neutral or behaviour-equivalent. Patterns are the established ones from prior chunks:

  • Class-level annotations for setUpTestData/setUpClass attributes mypy can't follow into classmethod bodies (the typing: graduate opencontractserver.discovery.tests.test_discovery_views from mypy baseline #1479 fix) — the bulk of the 338. test_agent_framework_api.py has a single shared TestAPISetup base (11 fixtures) inherited by 3 subclasses, so one annotation block covers all four.
  • User = get_user_model()from opencontractserver.users.models import User so the name is usable as a type (and the now-unused get_user_model import removed).
  • Graphene-client rename (test_agents.py): a graphene.test.Client assigned to self.client shadows the inherited django.test.TestCase.client, which has no .execute(). (test_agent_action_result_admin.py was deliberately not renamed — it uses the real django.test.Client for admin GET requests.)
  • list[ToolType] annotations (test_agent_api.py, test_agent_factory.py): list[str] / mixed tool literals passed to _resolve_tools / create_document_agent (both take list[ToolType]) annotated so list invariance is satisfied.
  • Optional narrowing: assertIsNotNone(x)assert x is not None (preserves the assertion while narrowing) for ToolRegistryEntry/CoreTool registry returns, _FakeConfig.system_prompt (Optional[str]), and AgentActionResult.duration_seconds (float | None).
  • A handful of error-code-scoped # type: ignores (no blanket ignores; warn_unused_ignores=True keeps them honest): a deliberately-None AdminSite; the deliberately-invalid framework="invalid_framework_name" string that exercises the ValueError path; a deliberately-invalid ["summarize", 123, None] tool list that exercises the skip-and-warn path; the per-instance override of the base ClassVar corpus/docs fixtures in test_agentic_highlighter_task.py; and the legacy stream_chat wrapper (see below).

⚠️ Test-quality findings surfaced behind the mocks

The issue frames graduation as a chance to audit for bugs hidden behind auto-attribute mocks. Two surfaced:

  1. test_agent_api.py::test_contextual_embedding_generation passed a non-existent embedder_path= kwarg to embeddings.generate. The public EmbeddingAPI.generate parameter is embedder (no embedder_path, no **kwargs); the call only "worked" because generate was mocked, hiding the wrong kwarg name. Corrected the call (and its assert_called_once_with) to embedder= — keeps the test green and self-consistent while aligning it with the real public API (mypy now confirms embedder is the correct param).

  2. CoreAgent.stream_chat is a legacy compatibility wrapper that lives on the concrete CoreAgentBase but is deliberately absent from the modern CoreAgent protocol (which exposes stream). It is never called in production — only test_agent_api.py and test_agent_framework_api.py exercise it, via mocks that replace it wholesale. Left the protocol as-is (the omission looks intentional, unlike the config attribute completed in typing: graduate tests.permissioning.* subpackage out of the mypy baseline #1768) and scoped the two test calls with # type: ignore[attr-defined]. Follow-up candidate: either fold stream_chat into the protocol or remove the now-effectively-dead wrapper.

Acceptance (per issue #1738)

  • mypy.ini agent blocks removed (9).
  • docs/typing/mypy_baseline.txt pruned (364 lines).
  • Pre-commit mypy passes on the full project surface.
  • No production code modified; no test assertions/logic changed (the embedder_pathembedder rename keeps the mocked test green and self-consistent).
  • black==26.1.0 / isort==6.0.1 / flake8==7.2.0 (+flake8-isort) / pyupgrade --py39-plus clean on all changed files; py_compile clean.

Test plan

  • mypy --config-file mypy.ini opencontractserver configSuccess: no issues found in 1247 source files, under a venv mirroring the .pre-commit-config.yaml pin set (mypy==2.0.0, django-stubs==6.0.5, plus the Django runtime stack), and again under CI's mypy==2.1.0.
  • black / isort / flake8 / pyupgrade clean; whitespace/EOF clean.
  • Backend CI (backend.yml linter + test jobs) — the authoritative runtime check.

Note on local testing: the Dockerized backend suite could not be exercised in the authoring container (docker compose -f test.yml is unavailable in this sandbox). The changes are behaviour-neutral (see above), so CI's backend job is relied on for the runtime check.

Refs #1738

claude and others added 3 commits June 1, 2026 05:34
Continues the mypy baseline drain (issue #1738), following the corpus
chunk (PR #1866). Removes the 9 [mypy-opencontractserver.tests.test_agent*]
ignore_errors blocks from mypy.ini, prunes the corresponding 364 lines from
docs/typing/mypy_baseline.txt (4012 -> 3648), and fixes the 338 errors that
surface under the pinned mypy==2.0.0 / django-stubs==6.0.5 (also verified
clean under CI's mypy==2.1.0).

Fixes use the established patterns from prior chunks: class-level
annotations for setUpTestData/setUpClass attributes, User = get_user_model()
-> direct import, the graphene self.client -> self.graphene_client rename
(test_agents.py), list[ToolType] annotations to satisfy list invariance,
assert ... is not None narrowing of Optional returns, and error-code-scoped
type: ignore for deliberately-invalid test inputs. All changes are
type-level and behaviour-neutral or behaviour-equivalent.

Two test-quality findings surfaced behind the mocks:
- test_agent_api.py passed a non-existent embedder_path= kwarg to
  embeddings.generate (the public param is embedder); corrected to embedder=,
  keeping the mocked test green while aligning it with the real API.
- CoreAgent.stream_chat is a legacy wrapper on the concrete base, absent from
  the modern CoreAgent protocol and never called in production; the two test
  call sites are scoped with type: ignore[attr-defined] rather than
  re-blessing it into the protocol.

Full surface: mypy --config-file mypy.ini opencontractserver config ->
"Success: no issues found in 1247 source files".

Refs #1738
The mypy-baseline graduation carried a blanket embedder_path -> embedder
rename into TestVectorStoreAPI::test_vector_store_with_all_options, but
VectorStoreAPI.create's public parameter is genuinely named embedder_path
(forwarded straight to UnifiedVectorStoreFactory.create_vector_store).
Because create has a **kwargs catch-all, the wrong embedder= kwarg was
silently absorbed while embedder_path defaulted to None, so the
assert_called_once_with(embedder=...) no longer matched the actual
create_vector_store(embedder_path=None, ..., embedder=...) call.

Restore embedder_path on both the call and the assertion. Document the
correction in CHANGELOG.
@claude

claude Bot commented Jun 1, 2026

Copy link
Copy Markdown

Code Review

This is a clean, well-executed typing graduation PR. All changes are confined to test files, config (mypy.ini), and documentation — no production code is touched. The patterns are consistent with the prior chunks in the #1738 cadence.

Positives

  • Genuine bug fix surfaced by mypy — the embedder_path=embedder= correction in test_agent_api.py::test_contextual_embedding_generation is a real improvement. The wrong kwarg was silently swallowed by the mock; mypy caught what the test couldn't. The companion note that test_vector_store_with_all_options correctly retains embedder_path= shows careful analysis rather than a blanket rename.

  • warn_unused_ignores = True discipline — every # type: ignore added in this PR is scoped to a specific error code and is load-bearing. No blanket ignores, no stale suppressions.

  • Shared base class annotation in test_agent_framework_api.py — a single TestAPISetup annotation block covering 11 fixtures inherited by 3 subclasses is the right architectural choice; avoids duplicating declarations across each subclass.

  • CHANGELOG entry — thorough and well-structured, consistent with project conventions.


Minor Concerns / Follow-up Candidates

stream_chat legacy wrapper (test_agent_api.py:497, test_agent_framework_api.py:235)

The PR correctly scopes these as # type: ignore[attr-defined] rather than adding stream_chat to the CoreAgent protocol, and the description explains the reasoning. Worth filing a follow-up: either fold stream_chat into the protocol (making it part of the public contract) or remove it as effectively-dead code. Leaving it in an ambiguous state accumulates design debt.

ClassVar overrides in test_agentic_highlighter_task.py (lines 1041, 1050)

The # type: ignore[misc] approach is correct for now, and the inline comment explains the intent clearly. Longer-term, the setUp override of base ClassVar fixtures is a structural awkwardness — if this test class is ever refactored, it would be cleaner to give it its own fixture names rather than shadow the inherited ones. Not blocking, just worth noting.

list[ToolType] annotation style

The inline tools_list: list[ToolType] = ["summarize", ...] approach in test_agent_api.py is correct and idiomatic. One very minor nit: several of these annotated locals are only used once immediately after declaration. This is fine for clarity at the call site, but the annotation could move to a cast() at the call site if the test methods get refactored.


Summary

No blocking issues. The real-value find is the embedder_pathembedder bug fix. The stream_chat dead-code question is the only item worth tracking as a follow-up (a new issue pointing at the wrapper). Everything else is correct, minimal, and consistent with the established cadence.

@JSv4 JSv4 merged commit 93ab361 into main Jun 1, 2026
10 checks passed
@JSv4 JSv4 deleted the claude/determined-faraday-pjUsP branch June 1, 2026 06:58
pull Bot pushed a commit to osamakaram/OpenContracts that referenced this pull request Jun 5, 2026
Continues the mypy baseline drain (issue Open-Source-Legal#1738, the Open-Source-Legal#1331 -> Open-Source-Legal#1335 -> Open-Source-Legal#1447
cadence). After the config tail (Open-Source-Legal#1748), tests.permissioning.* (Open-Source-Legal#1768), and
the annotation (Open-Source-Legal#1776) / document (Open-Source-Legal#1851) / corpus (Open-Source-Legal#1866) / agent (Open-Source-Legal#1877)
leaf-file chunks, this is the next cohesive batch from the issue's tests.*
batching plan: the extract domain.

Removes the 6 [mypy-opencontractserver.tests.test_*] ignore_errors blocks for
the structured-data-extraction feature -- Extracts plus their Fieldsets/Columns,
Datacells, and metadata columns (all in opencontractserver/extracts/models.py):
test_extract_mutations, test_extract_queries, test_extract_tasks,
test_datacell_mutations, test_column_mutations, test_metadata_columns_graphql.
Prunes the corresponding 57 lines from docs/typing/mypy_baseline.txt
(3648 -> 3591) and fixes the 42 errors that surface.

Every surfaced error was the same: a graphene.test.Client assigned to
self.client shadows django.test.TestCase.client (which has no .execute()).
Renamed self.client -> self.graphene_client in the 5 GraphQL test modules -- a
behaviour-equivalent rename, since the assignment already shadowed the Django
client at runtime and every read was .execute(). test_extract_tasks needed no
code change: its historical set_permissions_for_obj_to_user arg-type errors no
longer reproduce under django-stubs==6.0.5.

Full project surface (mypy --config-file mypy.ini opencontractserver config)
stays clean under both the pre-commit pin (mypy==2.0.0) and CI's pin
(mypy==2.1.0).

Refs Open-Source-Legal#1738
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants