Skip to content

Commit adfe0a3

Browse files
Adding proposal for test-mode retrospectively
As the proposal docs are are useful to understand the architecture behind the big features we are doing. So having a doc which explain the motivation and architecture behind the feature would be useful for future contributors. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Lalatendu Mohanty <lmohanty@redhat.com>
1 parent 648a35f commit adfe0a3

1 file changed

Lines changed: 113 additions & 0 deletions

File tree

docs/proposals/test-mode.md

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
# Bootstrap `--test-mode`: resilient source builds with pre-built fallback
2+
3+
- Author: Lalatendu Mohanty, Doug Hellmann
4+
- Created: 2026-06-01 (retrospective; feature merged 2025-12-19)
5+
- Status: Implemented
6+
- GitHub issue: [#713](https://github.com/python-wheel-build/fromager/issues/713)
7+
- Pull request: [#865](https://github.com/python-wheel-build/fromager/pull/865)
8+
9+
## What
10+
11+
Serial `fromager bootstrap --test-mode` continues after failures instead of
12+
aborting. On source-build failure, fromager downloads a pre-built wheel for
13+
the same package so traversal can proceed; dependents are still discovered and
14+
built. The run ends with `test-mode-failures-<timestamp>.json` and exit code 1
15+
if any failure was recorded.
16+
17+
Primary outputs of a degraded run are `graph.json`, `build-order.json`, and the
18+
failure report—not a full set of source-built wheels.
19+
20+
## Why
21+
22+
Large source bootstraps often fail one package at a time. Without fallback,
23+
dependents fail too, hiding which packages have real build problems. Operators
24+
work around this by manually marking failures as `pre_built` and re-running
25+
bootstrap repeatedly ([#713](https://github.com/python-wheel-build/fromager/issues/713)).
26+
27+
Test mode automates that loop: continue traversal, substitute pre-built wheels
28+
on source failure, surface many build gaps in one run, and emit a structured
29+
failure list for automation.
30+
31+
## Goals
32+
33+
- Continue serial `bootstrap` after failures; fallback via
34+
`BootstrapRequirementResolver.resolve(..., pre_built=True)`.
35+
- Classify failures: `resolution`, `bootstrap` (fatal, error log) vs `hook`,
36+
`dependency_extraction` (non-fatal, warning log). All are recorded; any
37+
failure yields exit code 1.
38+
- Preserve graph traversal and record `source_url_type` in `build-order.json`
39+
(`"prebuilt"` marks fallback wheels).
40+
41+
## Non-goals
42+
43+
- `bootstrap-parallel` (serial only); incompatible with `--sdist-only`.
44+
- Replacing fail-fast default bootstrap or treating fallback wheels as the
45+
shipped product.
46+
- Recording source-build failures when fallback succeeds ([#1166](https://github.com/python-wheel-build/fromager/issues/1166)).
47+
48+
## How
49+
50+
`commands/bootstrap.py` enables the flag; logic lives in `Bootstrapper`
51+
([iterative-bootstrap.rst](iterative-bootstrap.rst)).
52+
53+
### Architecture
54+
55+
**Run flow** — serial `bootstrap` with `--test-mode` never aborts the whole
56+
command on a single package failure; it finishes the run and reports at the end.
57+
58+
Resolve top-level requirements → Bootstrap dependency tree → finalize →
59+
graph.json + build-order.json + failures JSON
60+
61+
**On error**`_handle_phase_error()` in `bootstrapper.py` decides whether to
62+
skip the package, substitute a pre-built wheel, or continue with a warning.
63+
64+
On phase error, branch by context:
65+
66+
- **Resolution** → Record failure, skip package
67+
- **Source build** → Is a pre-built wheel available?
68+
- Yes → Use wheel, continue traversal
69+
- No → Record failure, skip package
70+
- **Hook or dep extraction** → Record warning, continue
71+
72+
**Phase errors** (`_handle_phase_error()`):
73+
74+
| Context | Behavior |
75+
| -- | -- |
76+
| `RESOLVE` | Record `resolution`; skip |
77+
| `PREPARE_SOURCE` / `PREPARE_BUILD` / `BUILD` | `_handle_test_mode_failure()`: resolve+download pre-built wheel; on success advance to `PROCESS_INSTALL_DEPS` with `SourceType.PREBUILT`; else record `bootstrap` |
78+
| Post-bootstrap hooks | Record `hook` (warning); continue |
79+
| Install dep extraction | Record `dependency_extraction` (warning); empty deps |
80+
81+
Settings `pre_built` packages skip the fallback path. Failure records include
82+
package, version, exception type/message, and `failure_type`; see
83+
`FailureRecord` in `bootstrapper.py`.
84+
85+
**Artifacts:** `build-order.json` `source_url_type` reflects runtime fallback;
86+
graph node `pre_built` reflects settings only (not updated after fallback).
87+
88+
| | `--test-mode` | `--multiple-versions` |
89+
| -- | -- | -- |
90+
| Purpose | Source-build gap analysis | All matching versions |
91+
| On build failure | Pre-built fallback, keep traversing | Remove version from graph |
92+
| Output | `test-mode-failures-*.json` | Logs only |
93+
94+
## Usage
95+
96+
```bash
97+
fromager bootstrap --test-mode -r requirements.txt
98+
```
99+
100+
Review `test-mode-failures-*.json`, `build-order.json` (`source_url_type: "prebuilt"`), and `graph.json`. Tests: `tests/test_bootstrap_test_mode.py`,
101+
`e2e/test_mode_*.sh`.
102+
103+
## Limitations
104+
105+
1. Source-build failure not recorded when fallback succeeds ([#1166](https://github.com/python-wheel-build/fromager/issues/1166)).
106+
2. Audit fallback via `build-order.json`, not graph `pre_built`.
107+
3. Serial bootstrap only.
108+
4. Pre-built version may differ from requested source version (warning logged).
109+
110+
## Key source files
111+
112+
`commands/bootstrap.py`, `bootstrapper.py`, `bootstrap_requirement_resolver.py`,
113+
`dependency_graph.py`

0 commit comments

Comments
 (0)