You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: python/CHANGELOG.md
+60Lines changed: 60 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,6 +3,66 @@ All notable changes to this project will be documented in this file.
3
3
4
4
This project adheres to [Semantic Versioning](http://semver.org/).
5
5
6
+
## [Unreleased]
7
+
8
+
This release is a major overhaul of the pytest plugin.
9
+
10
+
### What's New
11
+
12
+
#### Single-import plugin
13
+
The plugin is now `sift_client.pytest_plugin`, wired in with one line:
14
+
15
+
```python
16
+
pytest_plugins = ["sift_client.pytest_plugin"]
17
+
```
18
+
19
+
A default `sift_client` fixture reads `SIFT_*` env vars with `sift_grpc_uri` / `sift_rest_uri` ini fallback, so per-project config lives in `pyproject.toml`. `sift_include` / `sift_exclude` markers and the `sift_test_results_autouse` ini key scope which tests produce Sift reports, and autouse fixtures are gated per test so mixed unit and integration suites share one `pytest` invocation without unit tests paying a backend round-trip.
20
+
21
+
#### Hierarchical report tree
22
+
Python packages, test modules, test classes (including nested), and `@pytest.mark.parametrize` axes each open a nested parent step automatically, so the report mirrors the test layout. Parents are keyed by identity rather than held on a shared stack, so the tree is order-independent: it works with `pytest-randomly`, `pytest-ordering`, and fixture-scope reordering. Each parent closes as soon as the last test in its subtree finishes, so containers resolve progressively in incrementally uploaded reports instead of all at session end.
23
+
24
+
Parametrize placement is scope-aware: function-scoped params nest under the test, while a class, module, or session-scoped parametrized fixture sits at that level, matching where pytest re-runs the work. Four ini keys gate the layers individually: `sift_package_step`, `sift_module_step`, `sift_class_step`, and `sift_parametrize_nesting` (all default `true`).
25
+
26
+
#### Offline and disabled modes
27
+
Two named connection modes replace the previous check-connection toggle:
28
+
29
+
-`--sift-offline` skips the connectivity ping and routes create/update calls to a JSONL log for later replay via `import-test-result-log`, tolerating missing `SIFT_*` env vars.
30
+
-`--sift-disabled` (also `SIFT_DISABLED=1`) contacts nothing and writes no log, but `step.measure(...)` still evaluates bounds and returns a real pass/fail result. No credentials required.
31
+
32
+
`is_simulated` is exposed on `TestReport`, `TestStep`, `TestMeasurement`, and `ReportContext` so callers can branch on provenance. In online mode a failed ping now aborts the session once via `pytest.exit`, before any test body runs and with the full error untruncated.
33
+
34
+
#### Pass/fail reporting
35
+
Assertion failures record the concise assertion message on `error_info` (the step stays `FAILED` with no traceback) so the detail surfaces in the report UI. `step.pytest_fail_if_step_failed()` fails the pytest test when the step or any descendant failed, closing the gap where a failed substep or `report_outcome()` marked a step `FAILED` in the report but left pytest green. The failure message names each out-of-bounds measurement by name, value, and bounds, plus any failed substep.
36
+
37
+
#### Terminal summary
38
+
A session-start header shows the SDK version and active mode (e.g. `Sift: sift-stack-py 0.17.0, online mode`). An end-of-run Sift report panel reports status, step and measurement counts (color-coded), test case, system, log file, and a clickable report link (online) or upload command (offline). New `--sift-open-report` / `sift_open_report` opens the report in a browser at session end on interactive terminals. `--sift-report-url-base` / `sift_report_url_base` / `SIFT_APP_URL` sets the web-app origin for on-prem and custom deployments; `SiftClient.app_url` exposes the resolved origin to non-plugin consumers.
39
+
40
+
#### Configurable report content
41
+
Report fields are configurable from `[tool.sift.pytest.report]` in `pyproject.toml` (`name`, `test_case`, `test_system_name`, `system_operator`, `serial_number`, `part_number`, and a `metadata` table) or from `SIFT_REPORT_*` env vars for per-run values, with env winning over TOML. `name` and `test_case` accept placeholders including `{target}`, `{command}`, `{timestamp}`, `{count}`, `{git_repo}`, `{git_branch}`, and `{git_commit}`. When unset, `name` defaults to `{target} {timestamp}` and `test_case` to `{target}`, where `target` is derived from the collected tests so it stays stable across flag order and filters. The full invocation is always recorded under the `pytest_command` metadata key.
42
+
43
+
Unknown `SIFT_*` env vars and `[tool.sift.pytest.*]` keys produce a warning with a closest-match suggestion, so typos surface immediately.
44
+
45
+
#### Audit logging
46
+
A DEBUG audit trace lets field failures be diagnosed from one file without a rerun. `--sift-audit-log` / `sift_audit_log` accepts `path | true | false` and is on by default (DEBUG trace to a temp file, warnings echoed to stdout). The trace records config typos, connection state, the resolved-settings snapshot with provenance (API key redacted), status transitions, step open/close, and the replay lifecycle. Per-session temp artifacts group under `<tmpdir>/sift_test_results/<random>/`, and the terminal summary gains an audit-log section.
47
+
48
+
### Bugfixes
49
+
- Fix incremental upload of test-result logs failing on any resume tick after the first, which left the report stuck in progress. Resume ticks now read the report ID from the tracking map instead of requiring an in-memory report, and the recovery hint points at `import-test-result-log --incremental <log>` so a manual retry resumes from the cursor rather than creating a duplicate report. ([#611](https://github.com/sift-stack/sift/pull/611))
0 commit comments