Skip to content

[Bug] eBPF program df_T_exit_write exceeds BPF verifier instruction limit (1,000,000) on Linux 5.2+ nodes, and bpf_jit_enable inaccessible in container #11807

Description

@wolfeagle6611

Environment

  • DeepFlow Agent version: v7.1
  • Deployment: Kubernetes DaemonSet
  • Image: registry.cn-beijing.aliyuncs.com/deepflow-ce/deepflow-agent:v7.1
  • linux:Amazon Linux 2023
  • kernel version: 6.18

Issue 1: BPF program too large — df_T_exit_write exceeds verifier instruction limit

Symptoms

On several nodes, agent WARN logs show:

WARN bcc_prog_load() failed. name: df_T_exit_write, Argument list too long errno: 7
WARN BPF program is too large. Processed 1000001 insn
WARN processed 1000001 insns (limit 1000000) max_states_per_insn 99 total_states 60904 peak_states 10613
WARN The number of EBPF instructions (23455) exceeded the maximum limit (4096).
WARN bcc_prog_load() still failed. name: df_T_exit_write, errno after retry: 7 (orig 7)
WARN eBPF load programs failed. (errno 7)
WARN bpf load 'socket-trace-bpf-linux-5.2_plus' failed, error: Argument list too long (7)

Analysis

The BPF verifier on these nodes hits the 1,000,000 instruction processing limit when verifying df_T_exit_write. This causes the entire socket-trace-bpf-linux-5.2_plus program to fail loading, and the agent falls back to cBPF (AF_PACKET), which significantly increases CPU usage under high traffic.

Expected Behavior

The BPF program should either:

  1. Be split into smaller sub-programs to stay within verifier limits, or
  2. Provide a graceful per-program fallback without disabling the entire socket-trace

Issue 2: Cannot open /proc/sys/net/core/bpf_jit_enable inside container

Symptoms

WARN Open debug file("/proc/sys/net/core/bpf_jit_enable") open failed.
WARN "/proc/sys/net/core/bpf_jit_enable value is invalid

Analysis

The agent attempts to read /proc/sys/net/core/bpf_jit_enable as a diagnostic check inside a Kubernetes container, but the file is not accessible due to container permission restrictions (no CAP_SYS_ADMIN or read access to this sysctl path).

Expected Behavior

This should be a best-effort diagnostic check. The agent should gracefully skip this check (log at DEBUG level or suppress entirely) rather than emitting WARN, to avoid log noise in standard container deployments.


Additional Context

  • The fallback from eBPF to cBPF when df_T_exit_write fails causes a measurable CPU load increase on busy nodes, as cBPF requires copying every packet to userspace.
  • Nodes affected are running in AWS cn-north-1 region (Kubernetes worker nodes).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions