Skip to content

claude-code-chat-browser: Adversarial predicate-overlap fixtures for tool dispatch classification #116

Description

@clean6378-max-it

Calendar Day

Wednesday, July 8, 2026 (PR 2 of 2)

Planned Effort

3 story points (Medium–High) — sprint item #4

Companion PR: Wednesday PR 1 (renderer tests #3) — independent files.

Blocks: Thursday PR 2 (dispatch simplification #6) — adversarial fixtures must land before refactor.

Problem

utils/tool_dispatch.py classifies tool results through _parse_tool_result using _TOOL_RESULT_DISPATCH — a 15-entry ordered (predicate, builder) tuple where first matching predicate wins and order is load-bearing.

tests/test_tool_dispatch_ordering.py (#95 / PR #98) only checks ordering structurally via _predicate_index for 4 declared ORDERING_INVARIANTS. It does not construct JSON blobs that satisfy multiple predicates simultaneously and assert the actual classified winner.

Week 1 PR #104 added KNOWN_TOOL_TYPES and test_tool_dispatch_sync.py but not behavioral overlap fixtures. A new upstream tool shape matching a broad predicate before a narrower handler would misclassify silently while existing tests still pass.

Relationship to prior work (not a duplicate)

Prior Status What it covers What it does not cover
#95 Predicate ordering structural enforcement Closed Declarative ORDERING_INVARIANTS; index(A) < index(B) in test_tool_dispatch_ordering.py Adversarial JSON; _parse_tool_result winner; result_type assertions
PR #98 test: enforce tool_dispatch predicate ordering invariants Merged Implementation of #95 — structural tuple-position guards Behavioral overlap fixtures
PR #104 KNOWN_TOOL_TYPES registry + sync test Merged Tool-name set parity across backend/frontend (test_tool_dispatch_sync.py) Predicate overlap classification

This issue is the behavioral half that planning explicitly marked as residual after #95/#98 and #104. Keep test_tool_dispatch_ordering.py unchanged; add test_tool_dispatch_adversarial.py (or a behavioral section) alongside it.

Goal

One merged PR that adds adversarial toolUseResult fixtures asserting documented classification winners via _parse_tool_result, complementing (not replacing) the structural ordering tests.

Scope

Touch points

  • tests/test_tool_dispatch_adversarial.py (new) — or extend tests/test_tool_dispatch_ordering.py with a behavioral section.
  • Build adversarial JSON shapes satisfying multiple predicates; assert result_type via _parse_tool_result.
  • Cover every pair in ORDERING_INVARIANTS:
    • is_plan_tool_result vs is_file_write_tool_result (filePath + content).
    • is_task_message_tool_result vs narrower is_task_retrieval_tool_result / is_task_completed_tool_result / is_task_async_tool_result.
  • Regression case: demonstrate reordering a predicate flips classification and fails the new test.
  • Optional: hypothesis-based generation (pattern from tests/test_parser_fuzz.py).

Out of scope

Acceptance Criteria

  • Adversarial fixtures for every ORDERING_INVARIANTS pair in test_tool_dispatch_ordering.py.
  • Behavioral classification via _parse_tool_result, not tuple index comparison.
  • Boundary shapes distinguish broad is_task_message_tool_result from narrower task predicates.
  • At least one regression case proves reordering a predicate causes test failure.
  • Existing structural tests in test_tool_dispatch_ordering.py preserved.
  • Full pytest, mypy --strict, and ruff pass.
  • PR approved by at least 1 reviewer.

Verification

cd C:\Users\Jasen\CppAliance\claude-code-chat-browser
.\.venv\Scripts\Activate.ps1
pytest tests/test_tool_dispatch_adversarial.py tests/test_tool_dispatch_ordering.py -v
pytest tests/test_tool_dispatch_sync.py tests/test_real_session_fixtures.py -q
pytest -q
mypy .
ruff check .

Manual: temporarily swap two predicates in _TOOL_RESULT_DISPATCH — confirm adversarial test fails with a clear message, then revert.

References

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions