Skip to content

Commit aecdd4e

Browse files
test(conftest): isolate polish cache per test, harden API-key delete (#32)
Two related test-isolation gaps that surfaced as snapshot flakes: 1. The autouse polish fixture set ``ATTUNE_AUTHOR_STRICT_POLISH=false`` and deleted ``ANTHROPIC_API_KEY``, but did not redirect ``ATTUNE_AUTHOR_POLISH_CACHE``. Every test shared the dev machine's real ``~/.attune/polish_cache``. A prior live ``regenerate`` run would populate the cache with polished output; subsequent golden-snapshot tests would then observe LLM-rewritten content instead of the deterministic Jinja fallback, depending on which tests had run before — flaky between machines and between sessions on the same machine. Point the cache at a per-session tmp directory via ``tmp_path_factory``. 2. ``monkeypatch.delenv`` raised ``KeyError`` when the var was not already set — fragile across environments. Use ``raising=False``. Repro of the flake (on a dev machine with ``.env`` carrying a live key and ``~/.attune/polish_cache`` populated from a real run): pytest tests/test_generated_templates_golden.py # FAILED test_task_template_matches_snapshot — snapshot shows raw # Jinja output, observed value is the LLM-polished rewrite After this commit, the test suite is hermetic w.r.t. polish state regardless of the host's cache or env. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 3de9323 commit aecdd4e

1 file changed

Lines changed: 19 additions & 4 deletions

File tree

tests/conftest.py

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,17 @@
22

33
from __future__ import annotations
44

5-
import os
65
from collections.abc import Iterator
76
from pathlib import Path
87

98
import pytest
109

1110

1211
@pytest.fixture(autouse=True)
13-
def _lenient_polish_by_default(monkeypatch: pytest.MonkeyPatch) -> Iterator[None]:
12+
def _lenient_polish_by_default(
13+
monkeypatch: pytest.MonkeyPatch,
14+
tmp_path_factory: pytest.TempPathFactory,
15+
) -> Iterator[None]:
1416
"""Disable strict polish mode for every test by default.
1517
1618
Polish is strict in production (a missing API key raises),
@@ -21,13 +23,26 @@ def _lenient_polish_by_default(monkeypatch: pytest.MonkeyPatch) -> Iterator[None
2123
test has mocked ``_call_llm``. Tests that specifically
2224
exercise strict-mode behavior override it with their own
2325
``patch.dict`` block.
26+
27+
The polish cache directory is also pointed at a per-test
28+
tmp directory. Without this, every test would share the
29+
dev machine's real ``~/.attune/polish_cache``, so a
30+
previous live ``regenerate`` run can populate the cache
31+
with polished output and cause golden-snapshot tests to
32+
silently observe LLM-rewritten content instead of the
33+
deterministic Jinja-only fallback. The behavior is
34+
environment-dependent and surfaces as flakes between
35+
machines (or between sessions on the same machine).
2436
"""
2537
monkeypatch.setenv("ATTUNE_AUTHOR_STRICT_POLISH", "false")
38+
monkeypatch.setenv(
39+
"ATTUNE_AUTHOR_POLISH_CACHE",
40+
str(tmp_path_factory.mktemp("polish_cache")),
41+
)
2642
# Also make sure the tests never accidentally call the
2743
# real Anthropic API with a key picked up from the dev
2844
# machine's environment — a huge cost/latency hazard.
29-
if "ANTHROPIC_API_KEY" in os.environ:
30-
monkeypatch.delenv("ANTHROPIC_API_KEY")
45+
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
3146
yield
3247

3348

0 commit comments

Comments
 (0)