Skip to content

Commit 2a56862

Browse files
alanpeixinhoalanpeixinho
committed
docs: add high-level overview of regzbot
1 parent a9ec99e commit 2a56862

1 file changed

Lines changed: 223 additions & 0 deletions

File tree

‎docs/overview.md‎

Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,223 @@
1+
# Regzbot: high-level overview
2+
3+
This document introduces **regzbot**: what it is, why it exists, how it fits
4+
into the Linux kernel development process, and how the codebase is organized.
5+
Regzbot is **now part of [KernelCI](https://kernelci.org/)**, where it contributes
6+
to coordinated regression tracking across the kernel ecosystem.
7+
For usage instructions see [getting\_started.md](getting_started.md) and
8+
[reference.md](reference.md). For setup, see
9+
[installation.md](installation.md).
10+
11+
***
12+
13+
## What is regzbot?
14+
15+
**Regzbot** is a Linux kernel **regression tracking bot**. It monitors mailing
16+
lists, bug trackers, and Git repositories so reported regressions do not slip
17+
between first report and a fix in the appropriate kernel tree.
18+
19+
It is designed around the kernel's **email-centric** workflow: reporters and
20+
developers interact with it through special lines in emails (`#regzbot`
21+
commands) or existing conventions (`Link:` / `Closes:` tags in commits). The
22+
goal is **low overhead**: no new tooling, no mandatory accounts, and no mandatory
23+
web interfaces.
24+
25+
The project **runs in production** for public Linux kernel regression tracking
26+
(see [Public dashboards](#public-dashboards-and-artifacts)) and is authored by
27+
**Thorsten Leemhuis**.
28+
29+
***
30+
31+
## Key concepts for newcomers
32+
33+
### What is a regression?
34+
35+
A **regression** is a change in the kernel that breaks something that previously
36+
worked. It does not have to be a crash: it can be degraded performance, a
37+
feature that stops working, or hardware that is no longer recognized. The
38+
critical distinction is that **something used to work and now does not**, because of a
39+
specific change in the code.
40+
41+
Regressions are treated with higher urgency than ordinary bugs because they
42+
violate user trust: someone upgraded their kernel and something broke that was
43+
fine before. Linus Torvalds has always made clear that fixing regressions takes
44+
priority over adding new features.
45+
46+
> WE DO NOT BREAK USER SPACE!
47+
48+
### The kernel's email-based workflow
49+
50+
Unlike many projects that rely on web-based platforms (GitHub, GitLab, and the
51+
like) for code review and bug tracking, Linux kernel development happens
52+
primarily over **email**. Understanding a few pieces of this infrastructure
53+
helps explain how regzbot operates:
54+
55+
* **Mailing lists.** Kernel developers communicate through dozens of mailing
56+
lists organized by subsystem. Bug reports, patches, code reviews, and
57+
discussions all happen as email threads. The list
58+
`regressions@lists.linux.dev` is specifically for regression reports.
59+
60+
* **lore.kernel.org.** The public archive of all kernel mailing lists. Every
61+
email sent to a kernel list is stored here with a permanent URL. When someone
62+
says "link to the report on lore," they mean a URL such as
63+
`https://lore.kernel.org/r/<message-id>/`.
64+
65+
* **Message-ID, In-Reply-To, References.** Standard email headers that define
66+
threading. Each email has a unique `Message-ID`; replies carry `In-Reply-To`
67+
and `References` headers pointing at earlier messages. Regzbot uses this to tie
68+
replies, patches, and fixes to the original report without a centralized
69+
database.
70+
71+
* **`Link:` and `Closes:` tags.** Conventions used in Git commit messages to
72+
reference related mailing list discussions. When a developer writes `Link:
73+
https://lore.kernel.org/r/...` in a fix commit, it yields a traceable link
74+
between the fix and the report. Regzbot watches these tags to detect when a
75+
regression has been fixed.
76+
77+
### Mainline, linux-next, and stable
78+
79+
Regzbot tracks three trees because fixes arrive at different times:
80+
81+
* **Mainline** (`torvalds/linux.git`) is where official releases appear.
82+
`#regzbot introduced:` ranges (such as `v6.8..v6.9-rc1`) use tags from here.
83+
84+
* **linux-next** (`linux-next.git`) merges subsystem work before mainline.
85+
A fixing commit may appear here first; until mainline has it, regzbot can show **fix incoming**.
86+
87+
* **Stable** (`stable/linux.git`) holds **`linux-X.Y.y`** branches for kernels already shipped to users.
88+
89+
Reporting targets mainline; linux-next and stable explain whether the fix has landed yet.
90+
91+
***
92+
93+
## Why regzbot exists
94+
95+
### The "no regressions" rule
96+
97+
Kernel development rests on the idea of avoiding regressions, and that
98+
regressions take priority over new features. Official kernel documentation
99+
lays out expectations:
100+
101+
| Audience | Document | Key points |
102+
|----------|----------|------------|
103+
| Reporters | [Reporting regressions](https://docs.kernel.org/admin-guide/reporting-regressions.html) | How to report; CC `regressions@lists.linux.dev`; `#regzbot introduced:` recommended |
104+
| Developers | [Handling regressions](https://docs.kernel.org/process/handling-regressions.html) | Use `Link:`/`Closes:` in commits; timing expectations; `#regzbot ^introduced` for missed reports |
105+
106+
### The problem regzbot solves
107+
108+
Before automated tracking, regressions reported on lists could disappear in the
109+
traffic. Regzbot automates the bookkeeping:
110+
111+
* **Visibility:** open regressions appear on a public web dashboard.
112+
113+
* **Accountability:** periodic mail summaries list outstanding regressions ahead
114+
of releases.
115+
116+
* **Automatic correlation:** `Link:` tags in fixes tie into tracked regressions
117+
without an extra manual step.
118+
119+
***
120+
121+
## Core concepts
122+
123+
### 1. Starting tracking: `#regzbot` commands
124+
125+
A regression enters tracking when an email contains a line like:
126+
127+
```text
128+
#regzbot introduced: v5.13..v5.14-rc1
129+
```
130+
131+
If they are kind enough, one might include naming a direct *culprit* commit
132+
(`#regzbot introduced: 1f2e3d4c5d`).
133+
134+
### 2. Connecting fixes without extra bot commands
135+
136+
Patches and commits that carry `Link:` or `Closes:` tags targeting the report URL
137+
let regzbot associate fixes automatically. Developers already follow **this same
138+
convention**, so nothing new is required there.
139+
140+
### 3. Tree awareness: "where is it fixed?"
141+
142+
Regzbot distinguishes **mainline**, **linux-next**, and **stable** trees. A fix
143+
present only in linux-next surfaces as **"fix incoming"** until it reaches the
144+
tree that matters for that regression. That is enforced using local Git clones
145+
configured during setup.
146+
147+
### 4. Report sources (pluggable backends)
148+
149+
Not every report originates on mailing lists. Regzbot supports several backends:
150+
151+
| Source | Implementation | Notes |
152+
|--------|----------------|-------|
153+
| **lore** (NNTP/HTTPS) | `_repsources/_lore.py` | Primary source; kernel mailing list archives |
154+
| **bugzilla.kernel.org** | `_repsources/_bugzilla.py` | REST API with API key |
155+
| **GitLab** | `_repsources/_gitlab.py` | Issue tracker integration |
156+
| **GitHub** | `_repsources/_github.py` | Issue tracker integration |
157+
158+
Tracker polling logic lives in `_repsources/_trackers.py`.
159+
160+
### 5. Persistence
161+
162+
State is stored in **SQLite** (`~/.local/share/regzbot/database.db`): tracked
163+
regressions, processed message IDs, repository metadata, and report history.
164+
165+
***
166+
167+
### Available CLI commands
168+
169+
| Command | Purpose |
170+
|---------|---------|
171+
| `setup` | Initialize database, register sources, first Git tree sync |
172+
| `run` | Full update cycle (sources → git → web) |
173+
| `pages` | Regenerate web output only |
174+
| `report` | Build interactive mail reports (operator sends manually) |
175+
| `recheck` | Reprocess specific message IDs |
176+
| `test` | Run offline/online test suites |
177+
178+
## Codebase organization
179+
180+
| Area | Files | Role |
181+
|------|-------|------|
182+
| Core + DB | `__init__.py` | Regression model, `GitTree`/`GitBranch`, `ReportThread`, `run()`/`generate_web()`/`report()` |
183+
| Bot commands | `_rbcmd.py` | Parse and execute `#regzbot` subcommands |
184+
| CLI | `commandl.py` | argparse-based subcommands |
185+
| Web export | `export_web.py` | Static HTML generation |
186+
| Mail reports | `export_mail.py` | Weekly text/mail report layout |
187+
| CSV export | `export_csv.py` | CSV-oriented export (tests) |
188+
| Lore ingestion | `_repsources/_lore.py` | NNTP and HTTPS access to lore archives |
189+
| Tracker sources | `_bugzilla.py`, `_gitlab.py`, `_github.py`, `_generic.py` | Tracker-specific API integrations |
190+
| Tests | `testing_online.py`, `testing_offline.py`, `testing_trackers.py`, `testdata/*` | Offline/online/tracker tests/expected results |
191+
192+
### Dependencies
193+
194+
From `requirements.txt`: **GitPython**, **requests**, **yattag**,
195+
**python-bugzilla**, **python-gitlab**, **PyGithub**. Standard library modules
196+
used include `sqlite3`, `email`, and `nntplib`.
197+
198+
***
199+
200+
## Public dashboards and artifacts
201+
202+
| Resource | URL |
203+
|----------|-----|
204+
| Tracked regressions (mainline) | https://linux-regtracking.leemhuis.info/regzbot/mainline/ |
205+
| All views (index) | https://linux-regtracking.leemhuis.info/regzbot/ |
206+
| About the effort | https://linux-regtracking.leemhuis.info/about/ |
207+
| Weekly reports (lore search) | [lore.kernel.org](https://lore.kernel.org/lkml/?q=%22Linux+regressions+report%22+f%3Aregzbot) |
208+
209+
***
210+
211+
## Further reading
212+
213+
* [Getting started with regzbot](getting_started.md) (reporter and developer quick-start).
214+
215+
* [Reference documentation](reference.md) (full command syntax and behavior).
216+
217+
* [Installation](installation.md) (local instance).
218+
219+
* [About Linux kernel regression tracking](https://linux-regtracking.leemhuis.info/about/) (broader context and aims).
220+
221+
* [Reporting regressions (kernel.org)](https://docs.kernel.org/admin-guide/reporting-regressions.html).
222+
223+
* [Handling regressions (kernel.org)](https://docs.kernel.org/process/handling-regressions.html).

0 commit comments

Comments
 (0)