Skip to content

Commit fee2da3

Browse files
authored
feat(coverage): branch coverage MVP via static branch-point detection (#661)
2 parents 3b026de + 06eceba commit fee2da3

6 files changed

Lines changed: 808 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
- npm registry distribution: `npm install -g bashunit` (#244)
1111
- `bashunit::env::supports_color` and `bashunit::io::clear_screen` helpers (#247)
1212
- LCOV reports now include `FN`, `FNDA`, `FNF` and `FNH` function records, consumed by `genhtml`, Codecov and Coveralls
13+
- LCOV reports now include `BRDA`, `BRF` and `BRH` branch records for `if`/`elif`/`else` chains and `case` patterns (see `adrs/adr-007-branch-coverage-mvp.md`)
1314
- `BASHUNIT_COVERAGE_SHOW_FUNCTIONS=true` adds a per-function coverage block to the text report
1415
- `BASHUNIT_COVERAGE_SHOW_UNCOVERED=true` adds an "Uncovered Lines" block to the text report, with consecutive line numbers compressed into ranges
1516

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Branch Coverage MVP via Static Branch-Point Detection
2+
3+
* Status: accepted
4+
* Date: 2026-05-04
5+
6+
## Context and Problem Statement
7+
8+
Coverage today reports line-level execution only. Standard tooling (genhtml, Codecov, Coveralls) consumes branch records via the LCOV `BRDA`/`BRF`/`BRH` fields, which let reviewers see whether `else`/`elif` arms and individual `case` patterns were exercised. Adding true branch coverage to a Bash framework is non-trivial because:
9+
10+
1. Bash exposes no native instrumentation comparable to gcov branch counters.
11+
2. The DEBUG trap fires on commands, not on branch decisions.
12+
3. `BASH_COMMAND` reflects the *next* command, not the boolean outcome of a conditional.
13+
14+
We need a path that yields useful, mostly-correct branch metrics in LCOV reports without breaking Bash 3.0+ compatibility or the cost profile of the existing line tracker.
15+
16+
## Decision Drivers
17+
18+
* Bash 3.0+ compatibility (no associative arrays, no `[[`, no Bash 4-only features).
19+
* Reuse existing line-hit data; do not double the runtime cost of coverage.
20+
* LCOV output must be consumable by genhtml, Codecov and Coveralls without custom processing.
21+
* Implementation must fit in `src/coverage.sh` and remain testable with the existing unit-test patterns.
22+
* Behavior must be predictable enough to pin in tests; "best-effort heuristic" outputs are not acceptable.
23+
24+
## Considered Options
25+
26+
1. **Static branch-point detection plus line-hit inference** — parse the source file for branch-introducing constructs (`if`/`elif`/`else`, `case` patterns), compute the line range owned by each outcome, then mark the outcome as "taken" iff any line inside its range was hit.
27+
2. **Runtime decision tracing via `BASH_COMMAND`** — record the actual command being executed in the DEBUG trap and reconstruct decisions taken (`if X` followed by execution of either then-block or else-block).
28+
3. **Patch-based instrumentation** — preprocess source files to insert hit recorders inside each branch arm, run tests against the instrumented copy, post-process the data file.
29+
30+
## Decision Outcome
31+
32+
Chosen option: **Option 1 (static branch-point detection plus line-hit inference)**.
33+
34+
It reuses the existing line-hit data file with no DEBUG-trap changes. Bash 3.0+ compatibility is preserved because the parser is a single pass over the source with brace counting, identical in shape to the existing `extract_functions` walker. The output maps cleanly to LCOV `BRDA` records, and the contract ("an arm is taken iff any executable line inside it was hit") is precise enough to write unit tests against.
35+
36+
### Positive Consequences
37+
38+
* Zero runtime cost beyond the existing line tracker. Branch records are computed during report generation, not during test execution.
39+
* Reuses `is_executable_line` and `get_all_line_hits`, which already tolerate Bash 3.0 limitations.
40+
* LCOV output remains a single file, consumed unchanged by downstream tools.
41+
42+
### Negative Consequences
43+
44+
* Branch detection is line-presence based, not outcome based. A `then` arm whose only statement is a comment-line will register as `not taken` even if the conditional fired (because there are no executable lines inside). This is documented as a known limitation.
45+
* Implicit `else` (when an `if/elif` chain has no explicit `else`) is reported only when at least one explicit arm exists; the synthetic "fall-through" outcome is omitted from this MVP and may be added in a follow-up.
46+
* Compound conditionals (`if A && B`) are reported as a single binary decision, not per sub-expression.
47+
48+
## Pros and Cons of the Options
49+
50+
### Option 1: Static + line-hit inference (chosen)
51+
52+
* Good, because reuses existing data and code paths.
53+
* Good, because matches the implementation pattern of `extract_functions` already shipping in the codebase.
54+
* Good, because output is deterministic and easy to test.
55+
* Bad, because cannot distinguish "arm executed but produced no executable lines" from "arm not executed".
56+
57+
### Option 2: Runtime DEBUG-trap decision tracing
58+
59+
* Good, because reflects actual runtime behavior.
60+
* Bad, because `BASH_COMMAND` semantics across Bash 3.x and 5.x diverge for `((...))`, `[[...]]` and pipelines, requiring per-version logic.
61+
* Bad, because increases per-line overhead; the existing tracker already has measurable cost.
62+
* Bad, because subshell context loss (already documented for line coverage) extends to branches taken inside `$(...)`.
63+
64+
### Option 3: Source-rewrite instrumentation
65+
66+
* Good, because most accurate signal possible.
67+
* Bad, because requires either running tests against a rewritten source tree or hooking `source` to redirect to instrumented copies — both invasive and brittle.
68+
* Bad, because debugging stack traces and line numbers no longer match the user's source.
69+
* Bad, because doubles the code surface and breaks the "DEBUG-trap only" simplicity model.
70+
71+
## Scope of MVP
72+
73+
Included:
74+
75+
* `if`/`elif`/`else` chains: each arm is one outcome.
76+
* `case` statements: each pattern is one outcome.
77+
* LCOV `BRDA:<line>,<block>,<branch>,<taken>` lines.
78+
* `BRF:<count>` and `BRH:<count>` per file.
79+
80+
Deferred (potential follow-ups):
81+
82+
* Synthetic "implicit-else" outcomes for `if/elif` chains without an explicit `else`.
83+
* Per-sub-expression decisions inside `if A && B`.
84+
* `&&` / `||` short-circuit branches outside `if`.
85+
* Loop-entry decisions (`while`/`until`).
86+
87+
## Links
88+
89+
* Builds on the function extractor introduced in `src/coverage.sh` (see `bashunit::coverage::extract_functions`).
90+
* LCOV format reference: <https://manpages.debian.org/unstable/lcov/geninfo.1.en.html>

docs/coverage.md

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -278,6 +278,13 @@ end_of_record
278278
|-------|-------------|---------|
279279
| `TN:` | Test Name (usually empty) | `TN:` |
280280
| `SF:` | Source File path | `SF:/home/user/project/src/math.sh` |
281+
| `FN:` | Function: `start_line,name` | `FN:5,multiply` |
282+
| `FNDA:` | Function call data: `count,name` (1 if any line in body was hit, else 0) | `FNDA:1,add` |
283+
| `FNF:` | Functions Found | `FNF:2` |
284+
| `FNH:` | Functions Hit | `FNH:1` |
285+
| `BRDA:` | Branch data: `decision_line,block,arm,taken` | `BRDA:12,0,1,1` |
286+
| `BRF:` | Branches Found | `BRF:6` |
287+
| `BRH:` | Branches Hit | `BRH:4` |
281288
| `DA:` | Line Data: `line_number,hit_count` | `DA:15,3` (line 15 hit 3 times) |
282289
| `LF:` | Lines Found (total executable lines) | `LF:25` |
283290
| `LH:` | Lines Hit (lines with hits > 0) | `LH:20` |
@@ -351,6 +358,131 @@ These lines are not counted toward coverage:
351358
- Control flow keywords (`then`, `else`, `fi`, `do`, `done`, `esac`, `in`)
352359
- Case statement patterns (`--option)`, `*)`) and terminators (`;;`, `;&`, `;;&`)
353360

361+
## Branch Coverage
362+
363+
Beyond line and function coverage, bashunit emits **branch coverage** records in the LCOV report so reviewers can see whether each `else`/`elif` arm and each `case` pattern was exercised. Branch records are produced automatically; no extra flags are needed.
364+
365+
### What Counts as a Branch
366+
367+
| Construct | Arms |
368+
|-----------|------|
369+
| `if X; then ... fi` | 1 (the `then` body) |
370+
| `if X; then ... else ... fi` | 2 (`then` + `else`) |
371+
| `if X; then ... elif Y; then ... else ... fi` | 3 (one per arm) |
372+
| `case X in a) ... ;; b) ... ;; *) ... ;; esac` | one per pattern |
373+
374+
An arm is reported as **taken** iff at least one executable line inside its range was hit by tests.
375+
376+
### Verbose Output Helpers
377+
378+
Two opt-in environment variables enrich the text report when investigating coverage gaps:
379+
380+
::: code-group
381+
```bash [Per-function block]
382+
BASHUNIT_COVERAGE_SHOW_FUNCTIONS=true bashunit tests/ --coverage
383+
```
384+
```bash [Uncovered lines block]
385+
BASHUNIT_COVERAGE_SHOW_UNCOVERED=true bashunit tests/ --coverage
386+
```
387+
```bash [Both]
388+
BASHUNIT_COVERAGE_SHOW_FUNCTIONS=true \
389+
BASHUNIT_COVERAGE_SHOW_UNCOVERED=true \
390+
bashunit tests/ --coverage
391+
```
392+
:::
393+
394+
The default text report stays compact; opt in only when triaging.
395+
396+
### Worked Example
397+
398+
Given `src/route.sh`:
399+
400+
```bash
401+
#!/usr/bin/env bash
402+
function route() {
403+
if [ "$1" = "GET" ]; then
404+
echo "fetch"
405+
elif [ "$1" = "POST" ]; then
406+
echo "create"
407+
else
408+
echo "405"
409+
fi
410+
}
411+
```
412+
413+
If tests only call `route GET`, the LCOV record looks like:
414+
415+
```
416+
TN:
417+
SF:/path/to/src/route.sh
418+
FN:2,route
419+
FNDA:1,route
420+
FNF:1
421+
FNH:1
422+
BRDA:3,0,0,1
423+
BRDA:3,0,1,0
424+
BRDA:3,0,2,0
425+
BRF:3
426+
BRH:1
427+
DA:3,1
428+
DA:4,1
429+
DA:5,0
430+
DA:6,0
431+
DA:7,0
432+
DA:8,0
433+
LF:6
434+
LH:2
435+
end_of_record
436+
```
437+
438+
**Reading the branch records:**
439+
- `BRDA:3,0,0,1`: decision on line 3, block 0, arm 0 (`then`/GET), taken.
440+
- `BRDA:3,0,1,0`: same decision, arm 1 (`elif`/POST), not taken.
441+
- `BRDA:3,0,2,0`: same decision, arm 2 (`else`/405), not taken.
442+
- `BRF:3` `BRH:1`: 3 branches found, 1 taken.
443+
444+
### Visualizing with genhtml
445+
446+
LCOV's `genhtml` renders branch coverage alongside line and function coverage:
447+
448+
::: code-group
449+
```bash [Generate]
450+
bashunit tests/ --coverage
451+
genhtml --branch-coverage coverage/lcov.info -o coverage/html
452+
```
453+
:::
454+
455+
The resulting site shows a red/green diamond next to each branch decision, mirroring `gcov`'s C/C++ output.
456+
457+
### CI Integration
458+
459+
Codecov and Coveralls pick up the new records without configuration. To require branch coverage in PR gates:
460+
461+
::: code-group
462+
```yaml [Codecov]
463+
coverage:
464+
status:
465+
project:
466+
default:
467+
target: 80%
468+
patch:
469+
default:
470+
target: 80%
471+
threshold: 0%
472+
flags:
473+
- branch
474+
```
475+
:::
476+
477+
### Limitations
478+
479+
- An arm whose body has no executable lines (only comments or braces) registers as not-taken even when the conditional fired.
480+
- Implicit `else` (an `if`/`elif` chain without an explicit `else`) reports only the explicit arms; the synthetic fall-through outcome is omitted.
481+
- Compound conditionals (`if A && B`) are reported as a single binary decision, not per sub-expression.
482+
- `&&`/`||` short-circuit branches outside `if` and loop-entry decisions (`while`/`until`) are not tracked.
483+
484+
See `adrs/adr-007-branch-coverage-mvp.md` for the design rationale and the rejected alternatives.
485+
354486
## Limitations
355487

356488
### External Commands

0 commit comments

Comments
 (0)