Skip to content

Commit 66b46e9

Browse files
dblnzCopilot
dblnz
and
Copilot
committed
feat: add coverage report generation with cargo-llvm-cov
Add coverage infrastructure for the Hyperlight project: - Justfile: add coverage, coverage-html, coverage-lcov, and coverage-ci recipes using cargo-llvm-cov for LLVM source-based code coverage - CI: add Coverage.yml weekly workflow (Monday 06:00 UTC, manual trigger) running on kvm/amd with self-built guest binaries - coverage-ci mirrors test-like-ci by running multiple test phases with different feature combinations (default, single-driver, crashdump, tracing) and merging profdata into a single unified report - Coverage summary is displayed in the GitHub Actions Job Summary for quick viewing; full HTML report is downloadable as an artifact - docs: add how-to-run-coverage.md with local and CI usage instructions Guest/no_std crates are excluded from coverage because they define coverage instrumentation. Coverage targets host-side crates only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Doru Blânzeanu <dblnz@pm.me>
1 parent 564aacb commit 66b46e9

File tree

3 files changed

+225
-0
lines changed

3 files changed

+225
-0
lines changed

‎.github/workflows/Coverage.yml‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json
2+
3+
name: Weekly Coverage
4+
5+
on:
6+
schedule:
7+
# Runs every Monday at 06:00 UTC
8+
- cron: '0 6 * * 1'
9+
workflow_dispatch: # Allow manual trigger
10+
11+
env:
12+
CARGO_TERM_COLOR: always
13+
RUST_BACKTRACE: full
14+
15+
permissions:
16+
contents: read
17+
18+
defaults:
19+
run:
20+
shell: bash
21+
22+
jobs:
23+
coverage:
24+
timeout-minutes: 90
25+
strategy:
26+
fail-fast: false
27+
matrix:
28+
hypervisor: [kvm]
29+
cpu: [amd]
30+
runs-on: ${{ fromJson(
31+
format('["self-hosted", "Linux", "X64", "1ES.Pool=hld-{0}-{1}"]',
32+
matrix.hypervisor,
33+
matrix.cpu)) }}
34+
steps:
35+
- uses: actions/checkout@v6
36+
37+
- uses: hyperlight-dev/ci-setup-workflow@v1.8.0
38+
with:
39+
rust-toolchain: "1.89"
40+
env:
41+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
42+
43+
- name: Fix cargo home permissions
44+
run: |
45+
sudo chown -R $(id -u):$(id -g) /opt/cargo || true
46+
47+
- name: Rust cache
48+
uses: Swatinem/rust-cache@v2
49+
with:
50+
shared-key: "${{ runner.os }}-debug"
51+
cache-on-failure: "true"
52+
53+
- name: Build guest binaries
54+
run: just guests
55+
56+
- name: Install cargo-llvm-cov
57+
run: |
58+
cargo install cargo-llvm-cov
59+
rustup component add llvm-tools
60+
61+
- name: Generate coverage report
62+
run: just coverage-ci ${{ matrix.hypervisor }}
63+
64+
- name: Coverage summary
65+
if: always()
66+
run: |
67+
echo '## 📊 Code Coverage Report' >> $GITHUB_STEP_SUMMARY
68+
echo '' >> $GITHUB_STEP_SUMMARY
69+
if [ -f target/coverage/summary.txt ]; then
70+
echo '```' >> $GITHUB_STEP_SUMMARY
71+
cat target/coverage/summary.txt >> $GITHUB_STEP_SUMMARY
72+
echo '```' >> $GITHUB_STEP_SUMMARY
73+
else
74+
echo '⚠️ Coverage report was not generated.' >> $GITHUB_STEP_SUMMARY
75+
fi
76+
echo '' >> $GITHUB_STEP_SUMMARY
77+
echo '> 📥 For a detailed per-file breakdown, download the **HTML coverage report** from the Artifacts section below.' >> $GITHUB_STEP_SUMMARY
78+
79+
- name: Upload HTML coverage report
80+
if: always()
81+
uses: actions/upload-artifact@v7
82+
with:
83+
name: coverage-html-${{ matrix.hypervisor }}-${{ matrix.cpu }}
84+
path: target/coverage/html/
85+
86+
- name: Upload LCOV coverage report
87+
if: always()
88+
uses: actions/upload-artifact@v7
89+
with:
90+
name: coverage-lcov-${{ matrix.hypervisor }}-${{ matrix.cpu }}
91+
path: target/coverage/lcov.info

‎Justfile‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -446,6 +446,57 @@ fuzz-trace-timed max_time fuzz-target="fuzz_guest_trace":
446446
build-trace-fuzzers:
447447
cargo +nightly fuzz build fuzz_guest_trace --features trace
448448

449+
####################
450+
### COVERAGE #######
451+
####################
452+
453+
# install cargo-llvm-cov if not already installed and ensure llvm-tools component is available
454+
ensure-cargo-llvm-cov:
455+
command -v cargo-llvm-cov >/dev/null 2>&1 || cargo install cargo-llvm-cov
456+
rustup component add llvm-tools 2>/dev/null || true
457+
458+
# host-side packages to collect coverage for (guest/no_std crates are excluded because they
459+
# define #[panic_handler] and cannot be compiled for the host target under coverage instrumentation)
460+
coverage-packages := "-p hyperlight-common -p hyperlight-host -p hyperlight-testing -p hyperlight-component-util -p hyperlight-component-macro"
461+
462+
# generate a text coverage summary to stdout (run `just guests` first to build guest binaries)
463+
coverage: ensure-cargo-llvm-cov
464+
cargo llvm-cov {{ coverage-packages }}
465+
466+
# generate an HTML coverage report to target/coverage/html/ (run `just guests` first to build guest binaries)
467+
coverage-html: ensure-cargo-llvm-cov
468+
cargo llvm-cov {{ coverage-packages }} --html --output-dir target/coverage/html
469+
470+
# generate LCOV coverage output to target/coverage/lcov.info (run `just guests` first to build guest binaries)
471+
coverage-lcov: ensure-cargo-llvm-cov
472+
mkdir -p target/coverage
473+
cargo llvm-cov {{ coverage-packages }} --lcov --output-path target/coverage/lcov.info
474+
475+
# generate coverage for CI: mirrors test-like-ci by running all feature combinations,
476+
# accumulating profdata across runs, then generating LCOV + HTML + text reports.
477+
# (run `just guests` first to build guest binaries)
478+
coverage-ci hypervisor="kvm": ensure-cargo-llvm-cov
479+
mkdir -p target/coverage
480+
cargo llvm-cov clean --workspace
481+
482+
@# Phase 1: default features (all drivers)
483+
cargo llvm-cov {{ coverage-packages }} --no-report
484+
485+
@# Phase 2: single driver + build-metadata (matches test-like-ci single-driver pass)
486+
cargo llvm-cov {{ coverage-packages }} --no-default-features --features build-metadata,{{ if hypervisor == "mshv3" { "mshv3" } else { "kvm" } }} --no-report
487+
488+
@# Phase 3: crashdump feature tests
489+
cargo llvm-cov {{ coverage-packages }} --no-default-features --features crashdump,{{ if hypervisor == "mshv3" { "mshv3" } else { "kvm" } }} --no-report -- test_crashdump
490+
491+
@# Phase 4: tracing feature tests (host-side only; hyperlight-guest-tracing is no_std)
492+
cargo llvm-cov -p hyperlight-common --no-default-features --features trace_guest --no-report
493+
cargo llvm-cov -p hyperlight-host --no-default-features --features trace_guest,{{ if hypervisor == "mshv3" { "mshv3" } else { "kvm" } }} --no-report
494+
495+
@# Generate merged reports from all accumulated profile data
496+
cargo llvm-cov report --html --output-dir target/coverage/html
497+
cargo llvm-cov report --lcov --output-path target/coverage/lcov.info
498+
cargo llvm-cov report | tee target/coverage/summary.txt
499+
449500
###################
450501
### FLATBUFFERS ###
451502
###################

‎docs/how-to-run-coverage.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# How to Run Coverage
2+
3+
This guide explains how to generate code coverage reports for Hyperlight.
4+
5+
## Prerequisites
6+
7+
- A working Rust toolchain
8+
- [cargo-llvm-cov](https://github.com/taiki-e/cargo-llvm-cov) (installed automatically by the `just` recipes)
9+
- Guest binaries must be built first: `just guests`
10+
11+
## Local Usage
12+
13+
Build guest binaries (required before running coverage):
14+
15+
```sh
16+
just guests
17+
```
18+
19+
### Text Summary
20+
21+
Print a coverage summary to the terminal:
22+
23+
```sh
24+
just coverage
25+
```
26+
27+
### HTML Report
28+
29+
Generate a browsable HTML report in `target/coverage/html/`:
30+
31+
```sh
32+
just coverage-html
33+
```
34+
35+
Open `target/coverage/html/index.html` in a browser to explore per-file and per-line coverage.
36+
37+
### LCOV Output
38+
39+
Generate an LCOV file at `target/coverage/lcov.info` for use with external tools or CI integrations:
40+
41+
```sh
42+
just coverage-lcov
43+
```
44+
45+
## Available Recipes
46+
47+
| Recipe | Output | Description |
48+
|---|---|---|
49+
| `just coverage` | stdout | Text summary of line coverage |
50+
| `just coverage-html` | `target/coverage/html/` | HTML report for browsing |
51+
| `just coverage-lcov` | `target/coverage/lcov.info` | LCOV format for tooling |
52+
| `just coverage-ci <hypervisor>` | All of the above | CI recipe: HTML + LCOV + text summary |
53+
54+
## CI Integration
55+
56+
Coverage runs automatically on a **weekly schedule** (every Monday at 06:00 UTC) via the `Coverage.yml` workflow. It can also be triggered manually from the Actions tab using `workflow_dispatch`. The workflow runs on a single configuration (kvm/amd) to keep resource usage reasonable. It:
57+
58+
1. Builds guest binaries (`just guests`)
59+
2. Installs `cargo-llvm-cov`
60+
3. Runs `just coverage-ci kvm` — this mirrors `test-like-ci` by running multiple test phases with different feature combinations and merging the results into a single coverage report
61+
4. Displays a coverage summary directly in the **GitHub Actions Job Summary** (visible on the workflow run page)
62+
5. Uploads the full HTML report and LCOV file as downloadable build artifacts
63+
64+
### Viewing Coverage Results
65+
66+
- **Quick view**: Open the workflow run in the Actions tab — the coverage table is displayed in the **Job Summary** section at the bottom of the run page.
67+
- **Detailed view**: Download the `coverage-html-*` artifact from the Artifacts section, extract the ZIP, and open `index.html` in a browser for per-file, per-line drill-down.
68+
- **Tooling integration**: Download the `coverage-lcov-*` artifact for use with IDE plugins, Codecov, Coveralls, or other coverage services.
69+
70+
## How It Works
71+
72+
`cargo-llvm-cov` instruments Rust code using LLVM's source-based code coverage. It replaces `cargo test` — when you run `cargo llvm-cov`, it compiles the project with coverage instrumentation, runs the test suite, and then merges the raw profiling data into a human-readable report.
73+
74+
The CI recipe (`coverage-ci`) mirrors the `test-like-ci` workflow by running multiple test phases with different feature combinations:
75+
76+
1. **Default features** — all drivers enabled (kvm + mshv3 + build-metadata)
77+
2. **Single driver** — only one hypervisor driver + build-metadata
78+
3. **Crashdump** — tests with the `crashdump` feature enabled
79+
4. **Tracing** — tests with `trace_guest` feature (host-side crates only)
80+
81+
Each phase uses `--no-report` to accumulate raw profiling data, then a single `report` step merges everything into unified HTML, LCOV, and text reports. This ensures coverage reflects all exercised code paths across all feature combinations.
82+
83+
Coverage is collected for the host-side workspace crates (`hyperlight_common`, `hyperlight_host`, `hyperlight_testing`, `hyperlight_component_util`, `hyperlight_component_macro`). Guest crates (`hyperlight-guest`, `hyperlight-guest-bin`, `hyperlight-guest-capi`, `hyperlight-guest-tracing`) and the `fuzz` crate are excluded because guest crates are `no_std` and cannot be compiled for the host target under coverage instrumentation.

0 commit comments

Comments
 (0)