Skip to content

Commit a44d5b4

Browse files
LEANDERANTONYclaude
andcommitted
test: fix 16 stale-test failures from the token-meter migration
CI was red with 16 failures across three test files — all test-harness staleness from the token-meter work (T4/T5), not production bugs. No production code changed. - test_rate_limit.py: T5's login gate 401'd anonymous uploads before the rate-limiter ran. Apply the get_required_auth_tokens -> get_optional_auth_tokens override on the freshly-reloaded app inside _reload_app() (a module-level fixture would patch the stale pre-reload app object). - test_error_handling_scenarios.py: same T5 gate 401'd LLM-route scenarios. Add the standard _satisfy_llm_route_auth autouse override; no scenario expects a 401 so nothing is masked. - test_resume_quota_enforcement.py: T4 loosened resume_parses / resume_builder_sessions to UNLIMITED. Pin the pre-migration finite caps in _fresh_quota_backend, matching the cap-pinning in the sibling workspace/assistant quota-enforcement files. The dedicated 401-gate coverage in test_llm_route_login_required.py is untouched — no suite-wide override was added. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 988c807 commit a44d5b4

4 files changed

Lines changed: 110 additions & 0 deletions

File tree

docs/DEVLOG.md

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3675,3 +3675,56 @@ sections still dropped cleanly. `test_exporters.py` 37 passed;
36753675
full suite 865 passed with the same 16 pre-existing failures in
36763676
the rate-limit / quota / error-handling suites (confirmed
36773677
identical on a clean baseline — unrelated to rendering).
3678+
3679+
## Day 73: Fix 16 stale-test failures from the token-meter migration
3680+
3681+
CI was red with 16 failures across three test files — all
3682+
test-harness staleness from the token-meter work (T4/T5), not
3683+
production bugs. No production code changed; the fixes are scoped
3684+
to the three test files plus this entry.
3685+
3686+
**`tests/test_rate_limit.py` (5 failures).** T5 made the
3687+
resume-upload route require login via `get_required_auth_tokens`,
3688+
so the anonymous uploads these tests fire returned 401 before the
3689+
rate-limiter ran (expected 200/429). This file's `_reload_app()`
3690+
helper reloads `backend.app` and binds the client to the
3691+
freshly-built app object, so a module-level autouse fixture would
3692+
patch the stale pre-reload app. Fixed by installing the
3693+
`get_required_auth_tokens → get_optional_auth_tokens` override on
3694+
`backend.app.app` inside `_reload_app()`, right before the
3695+
`TestClient` is returned — so the rate-limiter decides the status.
3696+
3697+
**`tests/test_error_handling_scenarios.py` (4 failures).** Same
3698+
T5 gate: scenarios hitting LLM routes anonymously got 401 instead
3699+
of 400/422, and one failed `KeyError: 'session_id'` because a 401
3700+
body has no `session_id`. Scanned `_SCENARIOS` first — no scenario
3701+
expects a 401 (the admin-refresh scenarios accept 401/403 but that
3702+
route uses `get_optional_auth_tokens`, not the gated dependency),
3703+
so a file-wide autouse override masks nothing. Added the standard
3704+
`_satisfy_llm_route_auth` override fixture.
3705+
3706+
**`tests/backend/test_resume_quota_enforcement.py` (7 failures).**
3707+
All `DID NOT RAISE QuotaExceededError`. The gate is driven through
3708+
the service layer directly here, so the 401 gate is not the cause —
3709+
the real cause is T4 loosening `resume_parses` and
3710+
`resume_builder_sessions` to UNLIMITED (`-1`) in `TIER_CAPS`, and
3711+
this file lacked the cap-pinning autouse fixture its siblings have.
3712+
Pinned the pre-migration finite caps in `_fresh_quota_backend`
3713+
(`resume_parses` free 3 / pro 25 / business 100;
3714+
`resume_builder_sessions` free 1 / pro 3 / business 15), matching
3715+
the `tailored_applications` / `assistant_turns` pinning in
3716+
`test_workspace_quota_enforcement` / `test_assistant_quota_enforcement`.
3717+
3718+
The T5 401 gate's own coverage
3719+
(`tests/backend/test_llm_route_login_required.py`) is untouched —
3720+
no suite-wide conftest override was added, so its anonymous-401
3721+
assertions still hold.
3722+
3723+
Verification: the three files green (30 passed). Full suite in a
3724+
CI-like environment (no `.env`) — 881 passed, 0 failed. With a
3725+
populated local `.env`, two unrelated tests
3726+
(`test_cached_jobs_store::test_search_calls_rpc_with_normalized_args`,
3727+
`test_workspace_retention::test_sweep_with_no_service_role_client_logs_and_returns_zero`)
3728+
fail because the dev `.env` sets `JOB_SEARCH_HYBRID_ENABLED=true`
3729+
and `SUPABASE_SERVICE_ROLE_KEY`; both are green on CI (which ships
3730+
no `.env`) and out of scope for this fix.

tests/backend/test_resume_quota_enforcement.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,20 @@ def is_configured(self) -> bool:
6363
return False
6464

6565
monkeypatch.setattr(quota, "_SUPABASE_BACKEND", _NeverConfigured())
66+
# T4 of the token-meter migration loosened resume_parses and
67+
# resume_builder_sessions to UNLIMITED in the production TIER_CAPS.
68+
# The gate CODE in parse_resume_upload / start_resume_builder_session
69+
# is still present and revertable ("not a hard swap"), so this file
70+
# pins the pre-migration finite caps to verify that gate machinery
71+
# still fires ("Nth call -> 429"). This mirrors the autouse cap-
72+
# pinning in test_workspace_quota_enforcement (tailored_applications)
73+
# and test_assistant_quota_enforcement (assistant_turns). The
74+
# production cap POLICY (UNLIMITED) is asserted in test_tiers.py; the
75+
# LIVE LLM gate for these routes is the unified llm_tokens meter.
76+
for _tier, _cap in (("free", 3), ("pro", 25), ("business", 100)):
77+
monkeypatch.setitem(quota.TIER_CAPS[_tier], "resume_parses", _cap)
78+
for _tier, _cap in (("free", 1), ("pro", 3), ("business", 15)):
79+
monkeypatch.setitem(quota.TIER_CAPS[_tier], "resume_builder_sessions", _cap)
6680
reset_in_memory_backend()
6781
yield
6882
reset_in_memory_backend()

tests/test_error_handling_scenarios.py

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,12 +22,31 @@
2222
from fastapi.testclient import TestClient
2323

2424
from backend.app import app
25+
from backend.request_auth import get_optional_auth_tokens, get_required_auth_tokens
2526
from src.errors import AgentExecutionError, AppError
2627

2728

2829
client = TestClient(app)
2930

3031

32+
@pytest.fixture(autouse=True)
33+
def _satisfy_llm_route_auth():
34+
"""The token-meter migration (T5) made every LLM route require
35+
login. These error-handling scenarios predate that gate and
36+
exercise route ERROR behaviour (bad payloads, unknown sessions,
37+
LLM failures), not the auth gate itself — most fire anonymously.
38+
Overriding ``get_required_auth_tokens`` with
39+
``get_optional_auth_tokens`` restores pre-T5 behaviour: real
40+
tokens when auth headers are present, ``(None, None)`` when
41+
absent, never a 401 — so the validation/error paths under test
42+
are reached. None of the scenarios in ``_SCENARIOS`` expects a
43+
401, so a file-wide override masks nothing. The 401 gate itself
44+
is covered by ``tests/backend/test_llm_route_login_required.py``."""
45+
app.dependency_overrides[get_required_auth_tokens] = get_optional_auth_tokens
46+
yield
47+
app.dependency_overrides.pop(get_required_auth_tokens, None)
48+
49+
3150
# ---------------------------------------------------------------------------
3251
# Friendliness assertions — what counts as "leaky"?
3352
# ---------------------------------------------------------------------------

tests/test_rate_limit.py

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,20 +28,44 @@ def _reload_app(monkeypatch: pytest.MonkeyPatch, override: str = "2/minute"):
2828
Returns a fresh TestClient bound to a freshly-decorated app.
2929
Reloads in dependency order so the decorators see the new
3030
LIMIT_* constants.
31+
32+
The token-meter migration (T5) made every LLM route — including
33+
the resume-upload endpoint these rate-limit tests fire — require
34+
login via the ``get_required_auth_tokens`` dependency. Anonymous
35+
requests now 401 BEFORE the rate-limiter runs, which would mask
36+
the 200/429 behaviour under test. Because this helper RELOADS
37+
``backend.app`` and returns a client bound to the freshly-built
38+
app object, the auth-override has to be installed on THAT object
39+
(a module-level autouse fixture would patch the stale pre-reload
40+
app). Overriding ``get_required_auth_tokens`` with
41+
``get_optional_auth_tokens`` restores pre-T5 behaviour — real
42+
tokens when auth headers/cookies are present, ``(None, None)``
43+
when absent, never a 401 — so the rate-limiter is what decides
44+
the status code. The 401 gate itself is covered by
45+
``tests/backend/test_llm_route_login_required.py``.
3146
"""
3247
monkeypatch.setenv("RATE_LIMIT_OVERRIDE", override)
3348

3449
import backend.rate_limit
3550
import backend.routers.workspace
3651
import backend.routers.jobs
3752
import backend.app
53+
from backend.request_auth import (
54+
get_optional_auth_tokens,
55+
get_required_auth_tokens,
56+
)
3857

3958
# Reload bottom-up so decorators on routes pick up new limits.
4059
importlib.reload(backend.rate_limit)
4160
importlib.reload(backend.routers.workspace)
4261
importlib.reload(backend.routers.jobs)
4362
importlib.reload(backend.app)
4463

64+
# Apply the auth-override to the FRESHLY-reloaded app object.
65+
backend.app.app.dependency_overrides[get_required_auth_tokens] = (
66+
get_optional_auth_tokens
67+
)
68+
4569
return TestClient(backend.app.app)
4670

4771

0 commit comments

Comments
 (0)