fix: comprehensive quality audit — 30+ production bugs, test cleanup, lint

Production bugs fixed:
- 3 missing await on state.get/delete in Teams adapter
- _from_full_stream missing finish-step handling and text-delta field extraction
- _fallback_stream not using StreamingMarkdownRenderer (raw text instead)
- _fallback_stream edit loop 500ms cancellation latency (Event instead of bool)
- OnLockConflict return type bool vs "force"/"drop" string mismatch
- MemoryStateAdapter.disconnect() clearing cache (TS preserves it)
- options.limit or N truthiness bug across 15 adapter locations
- Teams status code string/int comparison + error dict missing statusCode
- Slack Retry-After header int() crash on non-integer values
- Token refresh race conditions in GChat, Teams, GitHub (added asyncio.Lock)
- _remend bold repair producing invalid markdown (**wor* instead of **wor**)
- _extract_message_content: PostableCard "[card]", PostableAst str(dict), PostableMarkdown no AST
- Slack empty table cell fallback, Teams table cell escaping incomplete
- Teams cards missing task/fetch for modal buttons, ButtonElement missing action_type
- AI module media_type snake_case keys (should be camelCase for SDK compat)
- PermissionError shadowing Python builtin (renamed to AdapterPermissionError)
- Slack installation camelCase/snake_case key incompatibility with TS
- GChat subscription pending-waiters re-raising errors (TS silently returns)
- Telegram or vs is not None for message_thread_id
- Linear TypedDict field names not matching actual JSON keys
- Teams _handle_teams_error return type None vs NoReturn
- from_full_stream silently dropping non-dict StreamChunk objects
- stringify_markdown bullet default "-" vs TS "*"

Infrastructure improvements:
- Session reuse: all 7 adapters share aiohttp.ClientSession per adapter
- Task GC pinning: fire-and-forget tasks held in _background_tasks set
- Task cancellation on shutdown: Chat.shutdown() cancels active handler tasks
- Unbounded cache eviction: Discord, GChat, GitHub caches now bounded
- Memory state periodic expired entry purge

Test quality:
- 76 exact duplicate tests removed
- 59 phantom absorbers converted to real tests or removed
- 47 all-weak-assertion tests strengthened with specific value checks
- 15 new tests for production fixes (race conditions, session reuse, etc.)
- 37 warnings eliminated (AsyncMock for sync methods, unclosed sessions, mark.anyio)
- 97 unused imports removed, 70 import sorting fixes, 21 manual lint fixes
- Fidelity script regex hardened (word boundary, empty-name guard)

API and packaging:
- Serialization uses camelCase keys matching TS SDK (with snake_case compat)
- StateNotConnectedError replaces bare RuntimeError in state adapters
- Chat(**kwargs) validates against ChatConfig fields
- AdapterError and Thread exported from top-level __init__.py
- Adapters use lazy __getattr__ imports (no optional dep failures)
- Event handler fields typed (was Any) for IDE autocomplete
- ChatConfig field docstrings added
- pyproject.toml duplicate deps removed
- ruff check src/ tests/ passes clean, ruff format passes clean
- audit_test_quality.py added to CI pipeline

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: patrick-chinchill

.github/workflows/test.yml

@@ -23,5 +23,7 @@ jobs:
         run: uv run ruff check src/
       - name: Format check
         run: uv run ruff format --check src/
+      - name: Test quality audit
+        run: uv run python scripts/audit_test_quality.py
       - name: Run tests
         run: uv run pytest tests/ -q --tb=short --cov=chat_sdk --cov-fail-under=70

CLAUDE.md

@@ -17,19 +17,33 @@ Python port of Vercel Chat SDK. Multi-platform async chat framework.
 - `src/chat_sdk/adapters/` -- 8 platform adapters
 - `src/chat_sdk/shared/` -- Markdown parser, format converter, streaming renderer
 - `src/chat_sdk/state/` -- Memory, Redis, Postgres backends
-- `tests/` -- 2,477+ tests
-
-## Critical Rules
-1. **Never use `datetime.utcnow()`** -- use `datetime.now(tz=timezone.utc)`
-2. **Never use `asyncio.ensure_future`** -- use `asyncio.get_running_loop().create_task()`
-3. **Never pass raw dicts to `self._chat.process_*`** -- use typed dataclasses (ActionEvent, ReactionEvent, etc.)
-4. **Never use camelCase keys in dispatch dicts** -- always snake_case
-5. **Never use `random.choices` for security tokens** -- use `secrets.token_hex`
-6. **Never import optional deps at module level** -- lazy import inside functions
-7. **Always use `hmac.compare_digest` for signature verification** -- never `==`
-8. **Always use `is not None` for empty-string-valid fields** -- never `or`
-9. **Always validate external URLs before HTTP requests** (SSRF prevention)
-10. **Always check `extend_lock` return value** in processing loops
+- `tests/` -- 3,267 tests
+
+## Principles
+
+1. **Every test must fail when the code is wrong.** No `assert True` stubs, no
+   bare truthiness checks when specific values are available, no MagicMock where
+   AsyncMock is needed. If a test can't catch a regression, it's not a test.
+2. **Every async call must be awaited.** Unawaited coroutines silently return
+   truthy objects. Use AsyncMock (not MagicMock) in tests to surface these.
+3. **No two tests should verify the same thing.** Duplicates inflate counts
+   without catching more bugs.
+
+## Port Rules (TS → Python)
+
+These are specific patterns that broke during the port. The principles above
+explain *why*; these explain *what to watch for*.
+
+- `datetime.utcnow()` → `datetime.now(tz=UTC)` (deprecated, naive)
+- `asyncio.ensure_future` → `loop.create_task()` (deprecated)
+- Raw dicts to `process_*` → typed dataclasses (ActionEvent, etc.)
+- camelCase dispatch keys → snake_case
+- `random.choices` for tokens → `secrets.token_hex`
+- Optional deps at module level → lazy import
+- `==` for signatures → `hmac.compare_digest`
+- `or` for empty-string-valid fields → `is not None`
+- Validate external URLs before requests (SSRF)
+- Check `extend_lock` return value in loops
 
 ## Adding a New Adapter
 See docs/ARCHITECTURE.md and CONTRIBUTING.md.
@@ -42,17 +56,12 @@ See docs/UPSTREAM_SYNC.md for TS->Python translation patterns.
 - StreamingMarkdownRenderer's _remend is simplified vs the npm `remend` library
 - No setext headings, no footnotes, no HTML nodes in the parser
 
-## Test Fidelity Verification
+## Test Quality
 
-After modifying or adding tests, run:
-```bash
-python3 scripts/verify_test_fidelity.py
-```
-This verifies every TS `it("...")` test has a matching Python `def test_...()`.
-The script must show `0 missing` before committing test changes.
+**CI runs `scripts/audit_test_quality.py` before tests.** It catches phantoms,
+async mock bugs, and cross-file duplicates. PRs that introduce hard failures
+will not pass CI.
 
-When porting a new TS test file:
-1. Add the mapping to `scripts/verify_test_fidelity.py` MAPPING dict
-2. Run with `--fix` to generate stubs
-3. Translate each stub by reading the TS test body line-by-line
-4. Verify with the script before committing
+**Fidelity check** (`scripts/verify_test_fidelity.py`) verifies every TS
+`it("...")` has a matching Python `def test_*()`. Name match ≠ faithful port —
+the audit script catches the quality side.

README.md

@@ -2,7 +2,7 @@
 
 Multi-platform async chat SDK for Python. Port of [Vercel Chat](https://github.com/vercel/chat).
 
-> **Status: Alpha (0.0.1a10)** — API may change. Not yet tested in production.
+> **Status: Alpha (0.0.1a11)** — API may change. Not yet tested in production.
 
 ## Why chat-sdk?
 

docs/TESTING.md

@@ -161,6 +161,84 @@ The `asyncio_mode = "auto"` setting means all `async def test_*` functions are a
        assert result.status == 401
    ```
 
+## Test Quality Invariants
+
+Rules learned from past bugs in the TS→Python port process:
+
+### 1. Name match ≠ faithful port
+
+The fidelity script (`scripts/verify_test_fidelity.py`) only checks that a matching
+`def test_*` exists for each TS `it("...")`. It does **not** check assertion quality.
+A test with `assert True` satisfies the fidelity checker but tests nothing.
+
+**Rule:** After porting, every test MUST have real assertions. The only acceptable
+`assert True` tests are JSX-specific ones that have no Python equivalent (currently 3).
+
+### 2. Never use `MagicMock` for async methods
+
+If the real method is `async def`, the mock **must** be `AsyncMock`. `MagicMock`
+returns a truthy Mock object instead of a coroutine. This hides missing `await` in
+production code — the test passes, but production silently gets coroutine objects
+instead of values.
+
+**Example of the bug we found:**
+```python
+# BAD — state.get is async, MagicMock returns Mock not coroutine
+state.get = MagicMock(side_effect=lambda k: cache.get(k))
+
+# GOOD
+state.get = AsyncMock(side_effect=lambda k: cache.get(k))
+```
+
+### 3. Check for duplicates before adding tests
+
+When agents work in parallel on overlapping scopes, they may write the same test
+in different files. Before committing new test files, scan for identical test names
+and bodies across the suite:
+
+```bash
+# Quick check for cross-file exact duplicates
+uv run python -c "
+import ast, os, collections
+bodies = collections.defaultdict(list)
+for root, _, files in os.walk('tests'):
+    for f in sorted(files):
+        if not f.startswith('test_') or not f.endswith('.py'): continue
+        path = os.path.join(root, f)
+        src = open(path).read()
+        for node in ast.walk(ast.parse(src)):
+            if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): continue
+            if not node.name.startswith('test_'): continue
+            body = '\n'.join(src.split('\n')[node.lineno-1:node.end_lineno]).strip()
+            if len(body) > 50: bodies[body].append(f'{path}:{node.lineno} {node.name}')
+for body, locs in bodies.items():
+    files = set(l.split(':')[0] for l in locs)
+    if len(files) > 1:
+        print(f'DUPLICATE: {locs[0].split(\" \")[1]}')
+        for l in locs: print(f'  {l}')
+"
+```
+
+### 4. Phantom absorber audit
+
+After any test changes, run this to catch `assert True`-only tests:
+
+```bash
+uv run python -c "
+import ast, os
+for root, _, files in os.walk('tests'):
+    for f in sorted(files):
+        if not f.startswith('test_') or not f.endswith('.py'): continue
+        path = os.path.join(root, f)
+        for node in ast.walk(ast.parse(open(path).read())):
+            if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): continue
+            if not node.name.startswith('test_'): continue
+            stmts = [s for s in node.body if not (isinstance(s, ast.Expr) and isinstance(s.value, ast.Constant))]
+            if len(stmts) == 1 and isinstance(stmts[0], ast.Assert) and isinstance(stmts[0].test, ast.Constant) and stmts[0].test.value is True:
+                print(f'PHANTOM: {path}:{node.lineno} {node.name}')
+"
+```
+
 ## Known Coverage Gaps
 
 The following modules are under 60% coverage as of the initial alpha release. These are tracked for improvement:

docs/UPSTREAM_SYNC.md

@@ -75,7 +75,7 @@ These are intentionally different from TS:
 
 These exist only in the Python port and have no TS equivalent:
 
-- `shared/errors.py`: Typed adapter error hierarchy (`AdapterRateLimitError`, `AuthenticationError`, `ValidationError`, `NetworkError`, `ResourceNotFoundError`, `PermissionError`). TS throws plain `Error` objects.
+- `shared/errors.py`: Typed adapter error hierarchy (`AdapterRateLimitError`, `AuthenticationError`, `ValidationError`, `NetworkError`, `ResourceNotFoundError`, `AdapterPermissionError`). TS throws plain `Error` objects.
 - `testing/__init__.py` + `shared/mock_adapter.py`: Test utilities with `MockAdapter`, `MockStateAdapter`, `create_test_message()`.
 - `from __future__ import annotations` everywhere: Enables PEP 604 union syntax (`X | Y`) on Python 3.10.
 - Input validation on adapter config dataclasses (e.g., rejecting empty `signing_secret`).

pyproject.toml

@@ -34,8 +34,7 @@ discord = ["pynacl>=1.5", "aiohttp>=3.9"]
 teams = ["aiohttp>=3.9"]
 telegram = ["aiohttp>=3.9"]
 whatsapp = ["aiohttp>=3.9"]
-google-chat = ["aiohttp>=3.9",
-    "pyjwt[crypto]>=2.8", "google-auth>=2.0", "pyjwt>=2.8"]
+google-chat = ["aiohttp>=3.9", "pyjwt[crypto]>=2.8", "google-auth>=2.0"]
 linear = ["aiohttp>=3.9"]
 all = [
     "slack-sdk>=3.27.0",
@@ -45,7 +44,6 @@ all = [
     "cryptography>=42.0",
     "pynacl>=1.5",
     "aiohttp>=3.9",
-    "pyjwt[crypto]>=2.8",
     "google-auth>=2.0",
 ]