Skip to content

Commit 848313b

Browse files
committed
docs(bench): motivate the suite by the hardware verification gap
The README opened by framing the whole suite around one merged PR series. The value is general: CI never boots NuttX and SITL runs on a host OS, so defects in boot ordering, link lifecycle, storage, loop rates, and RTOS-only concurrency are invisible until a board is on a bench. Reword the introduction around that gap and the timeout-first design rule instead of the originating incident. Signed-off-by: Ramon Roche <mrpollo@gmail.com>
1 parent ac6f03d commit 848313b

1 file changed

Lines changed: 16 additions & 8 deletions

File tree

Tools/bench_test/README.md

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,22 @@ Semi-automated release qualification for PX4 on real NuttX hardware
44
(Pixhawk-class boards on a bench, connected over USB and optionally a
55
telemetry radio).
66

7-
PX4 v1.18 merged a large TSAN/concurrency series (PR #27606 plus follow-ups
8-
#27809 and #27813) that reworked mavlink locking, uORB callbacks, WorkQueue
9-
lifetime, parameters, and dataman. CI builds NuttX firmware but never boots
10-
it. The characteristic failure mode of this class of bug on NuttX is a
11-
silent hang (for example a mutex that was previously zero-initialized and
12-
locked from the wrong context), not a crash. This suite makes hangs visible:
13-
every operation has a timeout, and a timeout is reported as FAIL naming
14-
exactly what stalled. A hang is the finding, not a nuisance.
7+
CI builds every NuttX target but never boots one, and SITL runs the same
8+
code on a host OS whose threading, scheduling, and libc behave differently
9+
from the RTOS on the flight controller. That gap hides an entire class of
10+
defects that only exist on real hardware: boot-time initialization ordering,
11+
link lifecycle across USB re-enumeration, storage and filesystem behavior,
12+
loop-rate drift, and concurrency bugs that are invisible on glibc but fatal
13+
on NuttX. A board can pass every SITL test and still ship with a dead
14+
telemetry link.
15+
16+
px4bench closes that gap with repeatable, scriptable verification on the
17+
bench: qualify a release candidate before cutting a branch, baseline a board
18+
before and after a firmware upgrade, or hand a beta to a test team knowing
19+
the basics hold. Failures on embedded targets are often silent hangs rather
20+
than crashes, so the suite is built around one rule: every operation has a
21+
timeout, and a timeout is reported as FAIL naming exactly what stalled. A
22+
hang is the finding, not a nuisance.
1523

1624
The bench tests never arm the vehicle; logging is triggered with
1725
`logger on` / `logger off`. The one test that flies does so in simulation on

0 commit comments

Comments
 (0)