feat(kao): add support for bao kao testing framework by danielRep · Pull Request #363 · bao-project/bao-hypervisor

danielRep · 2026-05-15T10:01:46Z

Overview

This PR integrates Bao Kao (abbreviated as bkao, "to put to the test"), an end-to-end testing (and benchmarking) framework for the Bao Hypervisor. Bao Kao automates the full pipeline from source build to on-target execution: it fetches toolchains, builds guest workloads, generates Bao configuration files based on targeted setup from YAML descriptors, assembles a bootable image (with optional firmware), launches it on the target platform (emulated or physical), and captures serial output to determine pass/fail.

Two new submodules are introduced:

tests/bkao: submodule containing the Bao Kao framework
tests/benchs: OSYX close-source benchmarks

Platforms supported:

Virtual:
- qemu-aarch64-virt
- qemu-riscv64-virt
- fvp-a
- fvp-r
Physical
- zcu104
- s32z270
- tc4dx
- rh850 (partially)

Note

This PR supersedes #154. We have refactored completely the framework, removing nix support and leveraging only python to implement bao kao.

Source Tree Structure

  tests/
  ├── bkao/                   # Bao Kao framework (submodule)
  │   └── src/
  │       ├── bkao.py         # Main runner and orchestration entry point
  │       ├── platforms/      # Per-platform build and launch logic
  │       ├── firmware/       # ATF, OpenSBI, U-Boot constructors
  │       ├── guests/         # Guest workload builders (baremetal, ...)
  │       ├── hypervisor/     # Hypervisor builders including Bao and a config generator
  │       └── utils/          # Helpers
  ├── benchs/                 # Benchmarks workload repository (submodule)
  │   ├── src/benchmarks/
  │   │   ├── ctx-switch/     # Context-switch benchmark
  │   │   └── irq-lat/        # IRQ latency benchmark
  │   └── configs/            # Per-benchmark YAML platform configs
  └── tests/
      ├── configs/
      │   ├── baremetal/      # Baremetal guest configs (YAML + platform BSPs)
      │   ├── freertos/       # FreeRTOS guest configs
      │   └── linux/          # Linux guest configs
      └── src/
          ├── inc/            # Test framework headers (testf.h, asserts, commands)
          ├── boot.c          # Boot test
          ├── irq.c           # IRQ test 
          └── bao-test.mk     # Test build integration makefile

Tests Available

┌─────┬────────────┬──────────┬───────────┬──────────────────────────────────────────────┐
│ ID  │   Suite    │   Test   │   Setup   │                 Description                  │
├─────┼────────────┼──────────┼───────────┼──────────────────────────────────────────────┤
│ 100 │ BOOT_CHECK │ VM_BOOT  │ baremetal │ Check that baremetal guest boots             │
│     │            │          │           │ successfully                                 │
├─────┼────────────┼──────────┼───────────┼──────────────────────────────────────────────┤
│ 101 │ BOOT_CHECK │ CPU_BOOT │ baremetal │ Check that all CPUs on the baremetal guest   │
│     │            │          │           │ boot successfully                            │
├─────┼────────────┼──────────┼───────────┼──────────────────────────────────────────────┤
│ 200 │ IRQ_CHECK  │ TIMER    │ baremetal │ Check that timer interrupt is triggered and  │
│     │            │          │           │ handled successfully                         │
├─────┼────────────┼──────────┼───────────┼──────────────────────────────────────────────┤
│ 201 │ IRQ_CHECK  │ UART     │ baremetal │ Check that UART interrupt is triggered and   │
│     │            │          │           │ handled successfully                         │
└─────┴────────────┴──────────┴───────────┴──────────────────────────────────────────────┘

Tests are defined using the BAO_TEST(suite, test, setup, description) macro and discovered automatically at runtime by scanning tests/tests/src/*.c. Each test is tagged with a suite, a setup (which selects the VM configuration to use), and an ID used for selective execution.

IDs are computed as (file_index × 100) + test_index_within_file, where:

file_index — 1-based position of the .c file in alphabetical order within
tests/tests/src/
test_index — 0-based order of BAO_TEST(...) appearances within that file

So with the current files (boot.c, irq.c):

┌────────┬────────────┬─────────┬─────┐
│  File  │ file_index │ test_nr │ ID  │
├────────┼────────────┼─────────┼─────┤
│ boot.c │ 1          │ 0       │ 100 │
├────────┼────────────┼─────────┼─────┤
│ boot.c │ 1          │ 1       │ 101 │
├────────┼────────────┼─────────┼─────┤
│ irq.c  │ 2          │ 0       │ 200 │
├────────┼────────────┼─────────┼─────┤
│ irq.c  │ 2          │ 1       │ 201 │
└────────┴────────────┴─────────┴─────┘

Guest types supported in test configs: baremetal, freertos, linux. For now, we only target a baremetal setup.

How to Run (TLDR)

Via bao Makefile targets:
Run all tests for a platform:
make tests PLATFORM=<platform>

Run all benchmarks for a platform:
make benchs PLATFORM=<platform>

Other options only via calling bkao-py directly:

Run specific tests or benchmarks by ID:
python3 tests/bkao/src/bkao.py -t 100 101 -p <platform>
python3 tests/bkao/src/bkao.py -b 100 -p <platform>

Run all tests except some:
python3 bkao.py -t -x 200,201 -p <platform>

Skip firmware and toolchain rebuild (faster iteration):
python3 bkao.py -t -p <platform> --no-firmware-build --no-toolchain-build

Use custom hypervisor sources:
python3 bkao.py -t -p <platform> --hyp-srcs /path/to/bao

Set verbosity (0=final report only, 1=failures, 2=all):
python3 bkao.py -t -p <platform> -l 2

Pass platform-specific args - only virtual platforms (e.g. GIC version):
python3 bkao.py -t -p qemu-aarch64-virt --plat-virt-args "GICV3"

Bao Kao Process Pipeline

For each test or benchmark, bkao executes the following pipeline:

Discover: scans tests/tests/src/*.c for BAO_TEST macros, benchs/src/benchmarks/ for benchmark dirs, and platforms/*.py for platform builders.
Parse args: reads CLI arguments, resolves which tests/benchmarks to run, and builds the runtime config.
Platform setup: instantiates the requested platform class and calls setup_platform() (emulator paths, serial config, etc.).
Toolchain: builds or fetches the cross-compilation toolchain; skipped with --no-toolchain-build, expecting the prefix to be in PATH.
launch_tests(): for each group of tests/benchmarks sharing the same setup:

Renders config.c from the YAML VM descriptor via Jinja2 templates.
Builds each guest workload (baremetal, FreeRTOS, Linux, or benchmark binary).
Builds the Bao hypervisor against the generated config and guest images.
Builds firmware (ATF/OpenSBI/U-Boot); skipped with --no-firmware-build.
Launches the platform and captures serial output, parsing [TESTF-C] tokens for pass/fail.

Final cleanup: removes wrkdir/ build artifacts.

Signed-off-by: Daniel Oliveira <drawnpoetry@gmail.com> Signed-off-by: Diogo Costa <diogoandreveigacosta@gmail.com> Signed-off-by: Miguel Silva <miguelafsilva5@gmail.com>

miguelafsilva5 · 2026-05-19T13:37:39Z

+        irq_set_handler(TIMER_IRQ_ID, test_interrupt_timer_callback);
+        timer_set(TIMER_INTERVAL);
+        irq_enable(TIMER_IRQ_ID);
+        irq_set_prio(TIMER_IRQ_ID, IRQ_MAX_PRIO);


Suggested change

irq_set_prio(TIMER_IRQ_ID, IRQ_MAX_PRIO);

irq_set_prio(TIMER_IRQ_ID, TIMER_IRQ_PRIO);

should follow the baremetal nomenclature

miguelafsilva5 · 2026-05-19T13:38:06Z

+        irq_set_handler(UART_IRQ_ID, uart_rx_handler);
+        uart_enable_rxirq();
+        irq_enable(UART_IRQ_ID);
+        irq_set_prio(UART_IRQ_ID, IRQ_MAX_PRIO);


Suggested change

irq_set_prio(UART_IRQ_ID, IRQ_MAX_PRIO);

irq_set_prio(UART_IRQ_ID, UART_IRQ_PRIO);

should follow the baremetal nomenclature

miguelafsilva5 · 2026-05-19T13:38:27Z

+        cpu_num: 6
+        regions:
+          - base: 0x90100000
+            size: 0x100000


Suggested change

size: 0x100000

size: 0x80000

correct memory region size

miguelafsilva5 · 2026-05-19T13:38:52Z

+            interrupts:
+              - 173
+        arch:
+          gspr_num: 1


Suggested change

gspr_num: 1

gpsr_num: 1

typo

miguelafsilva5 · 2026-05-19T13:40:05Z

+              - 173
+        arch:
+          gspr_num: 1
+          gspr_groups:


Suggested change

gspr_groups:

gpsr_groups:

typo.
Also, the config generator needs to generate a cast. the output should be

.arch = { .gpsr_num = 1, .gpsr_groups = (unsigned long int[]){1}, }

danielRep requested review from DavidMCerdeira, josecm, miguelafsilva5 and sandro2pinto as code owners May 15, 2026 10:01

danielRep mentioned this pull request May 15, 2026

feat(test-framework): Add a test-framework to bao #154

Closed

danielRep force-pushed the feat/bao-kao branch 8 times, most recently from 6f43b3a to 2ef3a2a Compare May 15, 2026 16:17

danielRep changed the title ~~feat(bkao): add support for bao kao testing framework~~ feat(kao): add support for bao kao testing framework May 15, 2026

danielRep force-pushed the feat/bao-kao branch from 2ef3a2a to 010c9b8 Compare May 15, 2026 16:23

feat(kao): add support for bao kao testing framework

fe6b4ea

Signed-off-by: Daniel Oliveira <drawnpoetry@gmail.com> Signed-off-by: Diogo Costa <diogoandreveigacosta@gmail.com> Signed-off-by: Miguel Silva <miguelafsilva5@gmail.com>

danielRep force-pushed the feat/bao-kao branch from 010c9b8 to fe6b4ea Compare May 18, 2026 09:11

miguelafsilva5 reviewed May 19, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(kao): add support for bao kao testing framework#363

feat(kao): add support for bao kao testing framework#363
danielRep wants to merge 1 commit into
mainfrom
feat/bao-kao

danielRep commented May 15, 2026

Uh oh!

miguelafsilva5 May 19, 2026

Uh oh!

miguelafsilva5 May 19, 2026

Uh oh!

miguelafsilva5 May 19, 2026

Uh oh!

miguelafsilva5 May 19, 2026

Uh oh!

miguelafsilva5 May 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	irq_set_prio(TIMER_IRQ_ID, IRQ_MAX_PRIO);
	irq_set_prio(TIMER_IRQ_ID, TIMER_IRQ_PRIO);

	irq_set_prio(UART_IRQ_ID, IRQ_MAX_PRIO);
	irq_set_prio(UART_IRQ_ID, UART_IRQ_PRIO);

Conversation

danielRep commented May 15, 2026

Overview

Source Tree Structure

Tests Available

How to Run (TLDR)

Bao Kao Process Pipeline

Uh oh!

miguelafsilva5 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

miguelafsilva5 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

miguelafsilva5 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

miguelafsilva5 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

miguelafsilva5 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants