Environment
- DeepFlow Agent version: v7.1
- Deployment: Kubernetes DaemonSet
- Image:
registry.cn-beijing.aliyuncs.com/deepflow-ce/deepflow-agent:v7.1
- linux:Amazon Linux 2023
- kernel version: 6.18
Issue 1: BPF program too large — df_T_exit_write exceeds verifier instruction limit
Symptoms
On several nodes, agent WARN logs show:
WARN bcc_prog_load() failed. name: df_T_exit_write, Argument list too long errno: 7
WARN BPF program is too large. Processed 1000001 insn
WARN processed 1000001 insns (limit 1000000) max_states_per_insn 99 total_states 60904 peak_states 10613
WARN The number of EBPF instructions (23455) exceeded the maximum limit (4096).
WARN bcc_prog_load() still failed. name: df_T_exit_write, errno after retry: 7 (orig 7)
WARN eBPF load programs failed. (errno 7)
WARN bpf load 'socket-trace-bpf-linux-5.2_plus' failed, error: Argument list too long (7)
Analysis
The BPF verifier on these nodes hits the 1,000,000 instruction processing limit when verifying df_T_exit_write. This causes the entire socket-trace-bpf-linux-5.2_plus program to fail loading, and the agent falls back to cBPF (AF_PACKET), which significantly increases CPU usage under high traffic.
Expected Behavior
The BPF program should either:
- Be split into smaller sub-programs to stay within verifier limits, or
- Provide a graceful per-program fallback without disabling the entire socket-trace
Issue 2: Cannot open /proc/sys/net/core/bpf_jit_enable inside container
Symptoms
WARN Open debug file("/proc/sys/net/core/bpf_jit_enable") open failed.
WARN "/proc/sys/net/core/bpf_jit_enable value is invalid
Analysis
The agent attempts to read /proc/sys/net/core/bpf_jit_enable as a diagnostic check inside a Kubernetes container, but the file is not accessible due to container permission restrictions (no CAP_SYS_ADMIN or read access to this sysctl path).
Expected Behavior
This should be a best-effort diagnostic check. The agent should gracefully skip this check (log at DEBUG level or suppress entirely) rather than emitting WARN, to avoid log noise in standard container deployments.
Additional Context
- The fallback from eBPF to cBPF when
df_T_exit_write fails causes a measurable CPU load increase on busy nodes, as cBPF requires copying every packet to userspace.
- Nodes affected are running in AWS cn-north-1 region (Kubernetes worker nodes).
Environment
registry.cn-beijing.aliyuncs.com/deepflow-ce/deepflow-agent:v7.1Issue 1: BPF program too large — df_T_exit_write exceeds verifier instruction limit
Symptoms
On several nodes, agent WARN logs show:
WARN bcc_prog_load() failed. name: df_T_exit_write, Argument list too long errno: 7
WARN BPF program is too large. Processed 1000001 insn
WARN processed 1000001 insns (limit 1000000) max_states_per_insn 99 total_states 60904 peak_states 10613
WARN The number of EBPF instructions (23455) exceeded the maximum limit (4096).
WARN bcc_prog_load() still failed. name: df_T_exit_write, errno after retry: 7 (orig 7)
WARN eBPF load programs failed. (errno 7)
WARN bpf load 'socket-trace-bpf-linux-5.2_plus' failed, error: Argument list too long (7)
Analysis
The BPF verifier on these nodes hits the 1,000,000 instruction processing limit when verifying
df_T_exit_write. This causes the entiresocket-trace-bpf-linux-5.2_plusprogram to fail loading, and the agent falls back to cBPF (AF_PACKET), which significantly increases CPU usage under high traffic.Expected Behavior
The BPF program should either:
Issue 2: Cannot open /proc/sys/net/core/bpf_jit_enable inside container
Symptoms
WARN Open debug file("/proc/sys/net/core/bpf_jit_enable") open failed.
WARN "/proc/sys/net/core/bpf_jit_enable value is invalid
Analysis
The agent attempts to read
/proc/sys/net/core/bpf_jit_enableas a diagnostic check inside a Kubernetes container, but the file is not accessible due to container permission restrictions (noCAP_SYS_ADMINor read access to this sysctl path).Expected Behavior
This should be a best-effort diagnostic check. The agent should gracefully skip this check (log at DEBUG level or suppress entirely) rather than emitting WARN, to avoid log noise in standard container deployments.
Additional Context
df_T_exit_writefails causes a measurable CPU load increase on busy nodes, as cBPF requires copying every packet to userspace.