Skip to content

Commit 01b0eb8

Browse files
committed
Live smoke: 22 rigorous integration tests + 2 more bug findings
Expanded tests/test_live_smoke.py from 4 to 22 live tests against the real Internxt backend. All 22 pass on 3 consecutive runs (with auto- rerun for transient API flakiness). New live coverage: - Read-only: login, list root, storage usage, /users/me known-404 marker - Upload variations: full cycle, unicode filenames, extensionless files, 2 MB file (multipart-threshold path) - Path: resolve_path against live tree, FileNotFoundError on missing, list_folder_with_paths enrichment, recursive 3-level nesting (the cache-coherency regression) - File ops: rename in-place, move between folders, copy preserves content (separate uuid + identical bytes), update_file replaces content (WebDAV PUT path) - Folder ops: rename, move to another parent - Trash: trash_file removes from listing - Search: server-side fuzzy finds uniquely-named upload (with retry for index latency); bogus query returns low-similarity safely - Find: client-side wildcard returns exact match set Two more bug findings during this round: 1. /drive/users/me returns 404 on the live backend. api.get_user_info() is therefore dead code. New regression test pins this down — if the endpoint comes back later, the test fails and tells us to wire it back into the CLI. 2. Internxt's fuzzy search is *very* fuzzy: a 32-char random hex query returns ~10 substring/Levenshtein matches at 1-2% similarity. Test assertion was wrong — relaxed to "no result has similarity > 10%" since none of the bogus matches are real. Robustness improvements: - Every file/folder name in tests now includes a per-call UUID suffix via _unique_name() helper. Critical for pytest-rerunfailures: failed attempts may have already created entries with the same plainName, causing 'already exists' on retry. Per-call UUIDs make reruns idempotent. - Per-call probe extension in find_files test for the same reason. - Module-scope authed_session fixture: log in once, share across all 22 tests. Cleanup in teardown clears cache before trashing sentinel folder. - Auto-rerun (reruns=2, delay=2s) via pytest-rerunfailures for transient API failures (rate-limit, eventual consistency). requirements-dev.txt: added pytest-rerunfailures. CHANGELOG.md and readme.md updated with the expanded coverage. Stats: 557 unit + 22 live = 579 tests. Unit suite still 90% coverage, 3-second runtime. Live suite 60-90 seconds, runs only when creds present.
1 parent 18194e8 commit 01b0eb8

4 files changed

Lines changed: 663 additions & 69 deletions

File tree

CHANGELOG.md

Lines changed: 45 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -271,40 +271,70 @@ a live Internxt backend, not more unit mocks.
271271

272272
#### Live integration smoke test
273273

274-
`tests/test_live_smoke.py` — opt-in test that runs against the real
274+
`tests/test_live_smoke.py`**22 opt-in tests** that run against the real
275275
Internxt backend. Auto-skipped unless `IXT_ACCOUNT` and `IXT_PWD` are
276-
set in env (or in a gitignored `.env` file). Safety properties:
276+
set in env (or in a gitignored `.env` file).
277+
278+
Safety properties:
277279

278280
- All operations happen under a **sentinel folder**
279281
`/__pytest_internxt_cli_smoke__/<run-uuid>/` with a fresh UUID per
280282
run. Nothing outside that prefix is touched.
283+
- Every file/folder name within a test includes a per-call UUID suffix
284+
so reruns are idempotent and never collide with prior attempt's
285+
leftovers in the shared sentinel.
281286
- A try/finally cleanup trashes the entire sentinel folder at module
282287
teardown, even on test failure.
288+
- Auto-rerun on transient failures (rate-limit / eventual-consistency)
289+
via `pytest-rerunfailures`, with up to 2 retries and a 2s delay.
283290
- **No cassette recording** — bytes and responses live only in memory;
284291
nothing about the user's account is written to the repo.
285292

286-
Tests covered:
287-
1. Login + whoami (read-only)
288-
2. List root folder (read-only)
289-
3. End-to-end cycle: create folder → upload → list → download → assert
290-
byte-for-byte recovery
291-
4. Path resolution against the live folder tree
293+
Coverage:
294+
295+
| Category | Tests |
296+
|---|---|
297+
| Read-only smoke | login + whoami; list root; storage usage; user_info-known-404 |
298+
| Upload variations | full cycle (round-trip integrity); unicode filenames; extensionless files; 2 MB file (multipart-threshold path) |
299+
| Path operations | resolve_path; missing-path FileNotFoundError; list_folder_with_paths enrichment; recursive folder creation across 3 nesting levels (the cache-coherency regression) |
300+
| File operations | rename in place; move between folders; copy preserves content; update_file replaces content (WebDAV PUT path) |
301+
| Folder operations | rename; move to another parent |
302+
| Trash | trash_file removes from listing |
303+
| Search | server-side fuzzy finds uniquely-named upload (with retry for index latency); bogus query returns low-similarity results without crashing |
304+
| Find | client-side wildcard search returns exact match set |
292305

293306
To run:
294307

295308
```bash
296-
# Once: put creds in .env (gitignored)
309+
# Once: put creds in .env (gitignored, never committed)
297310
echo 'IXT_ACCOUNT=you@example.com' > .env
298311
echo 'IXT_PWD=your-password' >> .env
299312

300-
# Run the live smoke
301-
pytest tests/test_live_smoke.py -v -s
313+
# Run the live smoke (~60-90s)
314+
pytest tests/test_live_smoke.py -v
302315

303316
# Force-skip (e.g., in CI)
304317
PYTEST_SKIP_LIVE=1 pytest
305318
```
306319

307-
This test surfaced the `create_folder_recursive` cache-coherency bug
308-
listed above (item 3 in the Critical bug-fix section), which all the
309-
unit-mocked tests missed because they stubbed `get_folder_content`
310-
directly.
320+
Live test results across 3 consecutive runs against a live account:
321+
22/22 → 22/22 → 22/22 (zero retries on the third run).
322+
323+
These tests surfaced two real bugs that all 557 unit-mocked tests had
324+
missed:
325+
326+
1. The `create_folder_recursive` **cache-coherency bug** (item 3 in the
327+
Critical bug-fix section). Intermediate folders bypassed the parent
328+
cache update; subsequent `resolve_path` walks fell through with
329+
`FileNotFoundError` from the stale root cache.
330+
331+
2. **`/drive/users/me` returns 404** on the live backend. The
332+
`api.get_user_info()` helper is therefore dead code from the CLI's
333+
perspective. A regression test now pins this down so we'll know if
334+
the endpoint becomes available later.
335+
336+
The bogus-search test also revealed that the Internxt fuzzy search is
337+
*very* fuzzy: even a 32-char random hex string returns ~10 substring/
338+
Levenshtein matches with 1-2% similarity scores. The test now asserts
339+
the response shape and that all returned matches are below a 10%
340+
similarity threshold, rather than incorrectly expecting an empty list.

readme.md

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -332,21 +332,28 @@ list of bugs found and fixed during the test build-out.
332332

333333
#### Live integration smoke (optional)
334334

335-
`tests/test_live_smoke.py` runs an end-to-end cycle (login → create
336-
folder → upload → list → download → verify bytes → trash) against the
337-
real Internxt backend. Auto-skipped unless credentials are present.
335+
`tests/test_live_smoke.py` is a 22-test suite that runs end-to-end
336+
against the real Internxt backend, covering: login, list, upload (small
337+
+ unicode + extensionless + 2 MB), download with byte-for-byte
338+
verification, recursive folder creation, file rename/move/copy/update,
339+
folder rename/move, trash, server-side fuzzy search, and client-side
340+
wildcard find. Auto-skipped unless credentials are present.
338341

339342
```bash
340343
# Put creds in a .env file (gitignored — never committed)
341344
echo 'IXT_ACCOUNT=you@example.com' > .env
342345
echo 'IXT_PWD=your-password' >> .env
343346

344-
pytest tests/test_live_smoke.py -v -s
347+
# Runs in ~60-90 seconds, always cleans up
348+
pytest tests/test_live_smoke.py -v
345349
```
346350

347351
All operations happen inside a unique sentinel folder
348352
(`/__pytest_internxt_cli_smoke__/<run-uuid>/`) which is always trashed
349-
at teardown — your real files are never touched.
353+
at teardown — your real files are never touched. Every file/folder name
354+
within a test includes a UUID suffix so transient retries never
355+
collide. Transient API failures (rate-limit, eventual-consistency) are
356+
auto-retried via `pytest-rerunfailures`.
350357

351358
### Quality Gates
352359

requirements-dev.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,8 @@ bandit>=1.7.0
99
# Optional: load IXT_ACCOUNT/IXT_PWD from .env for the live smoke test
1010
# (tests fall back to a built-in parser if not installed)
1111
python-dotenv>=1.0.0
12+
13+
# Auto-rerun flaky live integration tests on transient API failures
14+
# (rate limiting / eventual consistency). Required for tests/test_live_smoke.py
15+
# to be reliable end-to-end; not used by the main unit suite.
16+
pytest-rerunfailures>=12.0

0 commit comments

Comments
 (0)