Skip to content

feat(mbr): add PMU-based memory bandwidth throttling with timer support#373

Draft
Diogo21Costa wants to merge 3 commits into
mainfrom
feat/memort_throttling
Draft

feat(mbr): add PMU-based memory bandwidth throttling with timer support#373
Diogo21Costa wants to merge 3 commits into
mainfrom
feat/memort_throttling

Conversation

@Diogo21Costa

Copy link
Copy Markdown
Member

Summary

This PR adds PMU-based memory bandwidth throttling for Armv8 platforms, together with the timer and PMU infrastructure required to support it.

At a high level, a VM can now be configured with:

  • A PMU event budget.
  • A refresh period for that budget.

When throttling is enabled, the hypervisor reserves PMU event counters, monitors the configured event, stops a CPU when its budget overflows, and periodically restores the budget using the EL2 physical timer.

This PR also includes ZCU104-specific PMU event and interrupt metadata used to validate the feature with a cache-refill-based workload.

Main changes

1. PMU event counter support

Adds an event-counter abstraction backed by the Arm PMU:

  • Partitions PMU counters between guests and the hypervisor.
  • Configures overflow callbacks.
  • Routes the platform PMU interrupt on each CPU.

Relevant commit:

  • feat(armv8/pmu): add event counter support

2. EL2 timer support

Adds support for the hypervisor physical timer through a core timer abstraction:

  • Exposes the EL2 timer to the core layer.
  • Supports per-CPU callbacks.
  • Reserves and routes the platform timer interrupt.
  • Supports periodic rescheduling in counter ticks or microseconds.

Relevant commit:

  • feat(armv8/timer): add hypervisor timer support

3. Memory bandwidth throttling

Adds the memory throttling mechanism itself:

  • Allows VM configurations to define a PMU event budget and refresh period.
  • Reserves hypervisor PMU counters when throttling is enabled.
  • Stops a CPU after budget overflow.
  • Periodically replenishes the budget through the EL2 timer.
  • Adds ZCU104 PMU event and interrupt metadata for validation.

Relevant commit:

  • feat(core): add memory bandwidth throttling

Validation artifacts

Validation artifacts and test instructions are available in the following repository:

  • git@github.com:Diogo21Costa/bao-demos.git

This repository is based on bao-demos, which is the Bao demo guide repository used to build and run guest configurations across supported platforms. It extends that flow with support for a baremetal guest that benchmarks memory bandwidth, which makes it possible to observe and analyse the throttling effect directly. The output is something similar to:

buffer=2097152 bytes cache_line=64 bytes duration=10 s sample=100 ms
sample[00] accesses=360448 touched_bytes=23068672 bw=192 MiB/s
sample[01] accesses=327680 touched_bytes=20971520 bw=192 MiB/s
sample[02] accesses=327680 touched_bytes=20971520 bw=182 MiB/s
sample[03] accesses=327680 touched_bytes=20971520 bw=182 MiB/s
...

Validation setup

The current validation targets ZCU104.

For this setup, the throttling event is configured as L1D_CACHE_REFILL. The baremetal benchmark performs cache-line-sized pointer-chase reads and invalidates the buffer between passes, so each counted refill corresponds approximately to one 64-byte cache line.

The expected bandwidth is derived from the configured PMU event budget and throttle window:

$$\text{expected\_bw\_MiB\_s} = \frac{\text{budget} \times 64 \times 100}{1{,}048{,}576}$$

Where:

$$\text{bytes\_per\_event} = 64$$ $$\text{periods\_per\_second} = \frac{1{,}000{,}000}{\text{period\_us}}$$

Therefore, the expected bandwidth in MiB/s is:

$$\text{expected\_bw\_MiB\_s} = \frac{\text{budget} \times 64 \times \left(\frac{1{,}000{,}000}{\text{period\_us}}\right)}{1{,}048{,}576}$$

For these tests, period_us = 10000, so there are 100 throttle windows per second:

$$\text{expected\_bw\_MiB\_s} = \frac{\text{budget} \times 64 \times 100}{1{,}048{,}576}$$

Examples:

  • T1: 50000 * 64 * 100 / 1048576 = 305 MiB/s
  • T2: 30000 * 64 * 100 / 1048576 = 183 MiB/s
  • T3: 10000 * 64 * 100 / 1048576 = 61 MiB/s

The measured results closely follow the expected linear scaling, which validates that:

  • The PMU budget is enforced.
  • The CPU is throttled after budget exhaustion.
  • The budget is periodically replenished by the EL2 timer.

Notes

This validation uses L1D_CACHE_REFILL as the accounting event because it maps well to the benchmark used in the artifacts and provides an intuitive approximation of bandwidth on ZCU104.

However, this is only one possible configuration. Other PMU events can also be used depending on the platform and on the type of memory activity that should be controlled or observed.

Testing

Please refer to the validation repository for:

  • Build instructions.
  • Deployment steps.
  • Benchmark execution.
  • Expected output and measured bandwidth results.

For the base platform bring-up flow on ZCU104, the upstream bao-demos repository already documents the general image preparation, SD card setup, UART connection, and boot procedure.

Signed-off-by: Diogo Costa <diogoandreveigacosta@gmail.com>
Signed-off-by: Diogo Costa <diogoandreveigacosta@gmail.com>
Signed-off-by: Diogo Costa <diogoandreveigacosta@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants