Skip to content

libbpf-tools/softirqslower: Add libbpf CO-RE port of softirqslower#5516

Open
ismhong wants to merge 1 commit into
iovisor:masterfrom
ismhong:master
Open

libbpf-tools/softirqslower: Add libbpf CO-RE port of softirqslower#5516
ismhong wants to merge 1 commit into
iovisor:masterfrom
ismhong:master

Conversation

@ismhong
Copy link
Copy Markdown
Contributor

@ismhong ismhong commented May 29, 2026

Description

Convert from legacy Python/bcc to modern libbpf-based C implementation. This tool traces slow softirq events by measuring two critical latency dimensions of softirq handling.

Features:

  • Trace irq(hard)-to-softirq latency: delay from raise_softirq() until the softirq handler starts executing
  • Trace softirq runtime: actual handler execution duration (entry to exit)
  • Filter by minimum latency threshold (default 10000 us)
  • Filter by CPU with -c/--cpu option
  • Output columns: TIME, STAGE, SOFTIRQ, LAT(us), CPU, COMM
  • Supports both tp_btf (preferred) and raw_tp fallback tracepoints
  • Uses PERCPU_ARRAY maps indexed by softirq vector for lock-free tracking

Test on x86_64/Manjaro/5.15.60

❯ sudo ./softirqslower 100
Tracing softirq latency higher than 100 us... Hit Ctrl-C to end.
TIME     STAGE                SOFTIRQ  LAT(us)        CPU    COMM
18:43:53 irq(hard) to softirq rcu      116            0      ksoftirqd/0
18:43:53 irq(hard) to softirq net_rx   100            3      ksoftirqd/3
18:43:53 softirq runtime      net_rx   143            3      ksoftirqd/3
18:43:53 irq(hard) to softirq sched    387            3      ksoftirqd/3

❯ uname -a
Linux ism-manjaro 5.15.60-1-MANJARO #1 SMP PREEMPT Thu Aug 11 13:14:05 UTC 2022 x86_64 GNU/Linux

Test on arm64/Android/6.12.38

RTKRTD1325GTV_36:/ # /data/softirqslower 100
Tracing softirq latency higher than 100 us... Hit Ctrl-C to end.
TIME     STAGE                SOFTIRQ  LAT(us)        CPU    COMM
10:37:09 softirq runtime      net_rx   113            0      swapper/0
10:37:09 irq(hard) to softirq sched    104            3      swapper/3
10:37:09 softirq runtime      net_rx   101            0      swapper/0

RTKRTD1325GTV_36:/ # uname -a
Linux localhost 6.12.38-android16-5-gb9e1e77593f9-ab14266828-4k #1 SMP PREEMPT Tue Oct 14 10:02:12 UTC 2025 armv8l Toybox

Why this approach

Just port softirqslower from BCC to libbpf


Checklist

  • Commit prefix matches changed area (e.g., tools/toolname:, libbpf-tools/toolname:, src/cc:, docs:, build:, tests/python:)
  • Commit body explains why this change is needed

About AI Code Review: This project uses GitHub Copilot to assist with code review.
If a Copilot review is added, treat its feedback as you would any reviewer comment — you can
agree, disagree (with explanation), or ask questions. The maintainer makes all final decisions.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Ports the BCC Python softirqslower tool to a libbpf CO-RE implementation under libbpf-tools/. The new tool traces two latency dimensions of softirq processing — raise→entry and entry→exit — emitting events via a perf buffer when latency exceeds a configurable threshold, with optional per-CPU filtering and tp_btf/raw_tp fallback paths.

Changes:

  • New softirqslower.bpf.c with per-CPU arrays indexed by vector and matched tp_btf/raw_tp programs for raise/entry/exit.
  • New softirqslower.c userspace driver (argp, rodata configuration, BTF probe, perf buffer polling).
  • Shared softirqslower.h defining event layout and stage enum; Makefile APPS updated.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
libbpf-tools/softirqslower.h Shared event struct, stage enum, NR_SOFTIRQS constant.
libbpf-tools/softirqslower.bpf.c BPF programs and helpers for raise/entry/exit tracking.
libbpf-tools/softirqslower.c Userspace argp, BTF/raw_tp selection, perf buffer loop.
libbpf-tools/Makefile Adds softirqslower to APPS.

Comment thread libbpf-tools/softirqslower.c Outdated

printf("%-8s %-20s %-8s %-14llu %-6u %-16s\n",
ts,
e.stage < 2 ? stage_names[e.stage] : "unknown",
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Added ARRAY_SIZE(stage_names) macro and replaced the magic number with it, so the bound stays in sync automatically when new stages are added.

Comment thread libbpf-tools/Makefile
sigsnoop \
slabratetop \
softirqs \
softirqslower \
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the existing libbpf-tools ports (runqslower, hardirqs, softirqs, etc.) — none of them add a separate entry to the top-level README.md. That file lists BCC Python tools only; libbpf-tools/README.md covers the libbpf side but does not enumerate individual tools either.

The man/man8/softirqslower.8 SYNOPSIS has been updated from softirqslower.py to softirqslower to reflect the libbpf binary.

Comment on lines +6 to +8
/* Matches NR_SOFTIRQS in linux/interrupt.h
* Update if kernel adds new vectors */
#define NR_SOFTIRQS 10
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The BPF programs already guard against out-of-bounds access with vec >= NR_SOFTIRQS before any map lookup (softirqslower.bpf.c lines 77, 104, 135), so there is no silent data corruption.

The concern about future kernel vectors is valid, but increasing the map size would only defer the problem — the vec_names[] table in userspace would still need a matching update. Keeping NR_SOFTIRQS = 10 makes the two tables explicitly coupled: when the kernel adds a new vector, both the map size and the name table must be updated together. Happy to increase it to a safe upper bound if the maintainers prefer that style.

Port softirqslower from BCC Python to libbpf CO-RE.

The tool traces two latency dimensions of softirq processing:
- raise->entry: delay from raise_softirq() until handler starts
- entry->exit:  actual handler execution duration

Uses PERCPU_ARRAY maps indexed by softirq vector for lock-free
per-CPU tracking, with tp_btf (preferred) and raw_tp fallback.

Change-Id: Ie1d6eae35397a3181e9432d24137e92d66f815af
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants