Add sample usage for BPF_PROG_TYPE_NETFILTER by zq-david-wang · Pull Request #98 · xdp-project/bpf-examples

zq-david-wang · 2023-09-05T13:26:58Z

BPF_PROG_TYPE_NETFILTER was introduced in 6.4, now with a new kernel, a bpf program could attach to netfilter hooks and handles package in a similiar way as iptables/nftables. By now, 6.5.0, there is no bpf kfunc implemented yet for DNAT/SNAT, and the only thing a bpf program can do is to decide whether to DROP the package or not.

netfilter_ip4_blocklist.c/netfilter_ip4_blocklist.bpf.c

This sample code implements a simple ipv4 blacklist.
The bpf program drops package if destination ip address hits a match in the map of type BPF_MAP_TYPE_LPM_TRIE,
The userspace code would load the bpf program, attach it to netfilter's FORWARD/OUTPUT hook, and then write ip patterns into the bpf map.

tohojo · 2023-09-05T14:15:47Z

Instead of directing people to copy the files to the kernel tree, could you please add a Makefile to build the sample as part of this repo? You can copy the Makefile from one of the other directories and adjust appropriately :)

zq-david-wang · 2023-09-05T16:55:31Z

Instead of directing people to copy the files to the kernel tree, could you please add a Makefile to build the sample as part of this repo? You can copy the Makefile from one of the other directories and adjust appropriately :)

The problem is with those new enum defined in kernel headers and the "new" vmlinux.h file.
Is it acceptable if I made a copy of vmlinux.h and redefine those enums, I think this can make it build within this repo.

tohojo · 2023-09-06T13:01:14Z

The problem is with those new enum defined in kernel headers and the "new" vmlinux.h file. Is it acceptable if I made a copy of vmlinux.h and redefine those enums, I think this can make it build within this repo.

Sure. Have a look at the existing header files in headers/vmlinux in the project root dir. You can just copy the enum you need from a locally generated vmlinux.h into one of those files; vmlinux_net.h is probably appropriate for the netfilter definition :)

zq-david-wang · 2023-09-07T16:56:22Z

New types can be easily dealt with, but the "changed" type bpf_attr in libbpf is hard to make a workaround....

netfilter_ip4_blocklist.c: In function ‘main’:
netfilter_ip4_blocklist.c:58:25: error: ‘struct <anonymous>’ has no member named ‘netfilter’
   58 |         attr.link_create.netfilter.pf = NFPROTO_IPV4;
      |                         ^
netfilter_ip4_blocklist.c:59:25: error: ‘struct <anonymous>’ has no member named ‘netfilter’
   59 |         attr.link_create.netfilter.hooknum = NF_INET_FORWARD;
      |                         ^
netfilter_ip4_blocklist.c:60:25: error: ‘struct <anonymous>’ has no member named ‘netfilter’
   60 |         attr.link_create.netfilter.priority = -128;
      |                         ^
netfilter_ip4_blocklist.c:67:25: error: ‘struct <anonymous>’ has no member named ‘netfilter’
   67 |         attr.link_create.netfilter.hooknum = NF_INET_LOCAL_OUT;

zq-david-wang · 2023-09-07T17:11:01Z

I turns out that lastet https://github.com/xdp-project/libbpf has netfilter added to link_create, any chance to make a update to the libbpf submodule?

tohojo · 2023-09-07T21:22:50Z

David Wang ***@***.***> writes:

I turns out that lastet https://github.com/xdp-project/libbpf has netfilter added to link_create, any chance to make a update to the libbpf submodule?

Sure, feel free to include a libbpf version bump as part of the PR (just make it a separate commit) :)

zq-david-wang · 2023-09-08T13:47:15Z

Sad story.... Even I git submodule update --remote lib/libbpf to latest commit, replace headers/linux/bpf.h with lib/libbpf/include/uapi/linux/bpf.h and fix the missing defination for enum BPF_NETFILTER, I still could not get it through and got the following runtime error:

$ sudo ./netfilter_ip4_blocklist 
libbpf: prog 'netfilter_ip4block': BPF program load failed: Invalid argument
libbpf: prog 'netfilter_ip4block': failed to load: -22
libbpf: failed to load object './netfilter_ip4_blocklist.bpf.o'
loading BPF object file failed

The bpf program works when I replace lib/install/lib/libbpf.a with 6.5.0 in-tree version.....

zq-david-wang · 2023-09-08T13:58:33Z

Could https://github.com/xdp-project/libbpf.git resync with in-tree version?....

tohojo · 2023-09-08T14:05:44Z

David Wang ***@***.***> writes:

Could `https://github.com/xdp-project/libbpf.git` resync with in-tree version?....

Sure, fast-forwarded it to the latest upstream :)

Signed-off-by: David Wang <00107082@163.com>

zq-david-wang · 2023-09-09T11:59:21Z

nat64-bpf has build error, but it failed the same way before this commit

$ make
sh configure
clang: 18.0.0 (https://github.com/llvm/llvm-project.git 8dce4c56dd760b34b222d3188a0fe26232928406)
libmnl support: yes
libbpf support: submodule
Submodule 'lib/libbpf' (https://github.com/xdp-project/libbpf.git) registered for path 'lib/libbpf'
Submodule 'lib/xdp-tools' (https://github.com/xdp-project/xdp-tools) registered for path 'lib/xdp-tools'
Cloning into '/home/linan/codes/git/bpf-examples/bpf-examples/lib/libbpf'...
Cloning into '/home/linan/codes/git/bpf-examples/bpf-examples/lib/xdp-tools'...
Submodule path 'lib/libbpf': checked out 'a6d7530cb7dff87ac1e64a540e63b67ddde2e0f9'
Submodule path 'lib/xdp-tools': checked out '344b241da22a5358c714d6db1ea6f225f951dbdb'
ELF support: yes
zlib support: yes
libxdp support: submodule
Configuring libxdp to use our libbpf submodule
Found clang binary 'clang' with version 18 (from 'clang version 18.0.0 (https://github.com/llvm/llvm-project.git 8dce4c56dd760b34b222d3188a0fe26232928406)')
using emacs: GNU Emacs 26.1
using bpftool v5.8.0
libbpf support: custom v1.0.0
  perf_buffer__consume support: yes
  btf__load_from_kernel_by_id support: yes
  btf__type_cnt support: yes
  bpf_object__next_map support: yes
  bpf_object__next_program support: yes
  bpf_program__insn_cnt support: yes
  bpf_map_create support: yes
  perf_buffer__new_raw support: yes
  bpf_xdp_attach support: yes
  bpf_map__set_autocreate support: yes
  bpf_prog_test_run_opts support: yes
zlib support: yes
ELF support: yes
pcap support: yes
secure_getenv support: yes

lib

  libbpf
    CC       install/lib/libbpf.a
    INSTALL  install/lib/libbpf.a

  libxdp
    CC       install/lib/libxdp.a
    INSTALL  install/lib/libxdp.a

  util
    CC       json_writer.o
    CC       logging.o

encap-forward
    CLANG    xdp_encap.o
    LLC      xdp_encap.o
    CLANG    tc_bpf_encap.o
    LLC      tc_bpf_encap.o

ktrace-CO-RE
    CC       ktrace01
    CLANG    ktrace01_kern.o
    LLC      ktrace01_kern.o

lsm-nobpf
    CC       lsm-nobpf
    CLANG    lsm-nobpf-kern.o
    LLC      lsm-nobpf-kern.o

nat64-bpf
    CLANG    nat64_kern.o
    LLC      nat64_kern.o
    GEN      nat64_kern.skel.h
libbpf: bad map relo against section 2
Error: failed to open BPF object file: Relocation failed
make[1]: *** [../lib/common.mk:99: nat64_kern.skel.h] Error 255
make: *** [Makefile:37: nat64-bpf] Error 2

And second make yield following error:

$ make

lib

  util
make[2]: Nothing to be done for 'all'.

encap-forward
make[1]: Nothing to be done for 'all'.

ktrace-CO-RE
make[1]: Nothing to be done for 'all'.

lsm-nobpf
make[1]: Nothing to be done for 'all'.

nat64-bpf
    CC       nat64
nat64.c: In function ‘main’:
nat64.c:381:15: warning: implicit declaration of function ‘nat64_kern__open’ [-Wimplicit-function-declaration]
  381 |         obj = nat64_kern__open();
      |               ^~~~~~~~~~~~~~~~
nat64.c:381:13: warning: assignment to ‘struct nat64_kern *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
  381 |         obj = nat64_kern__open();
      |             ^
nat64.c:391:12: error: invalid use of undefined type ‘struct nat64_kern’
  391 |         obj->bss->config = cfg.c;
      |            ^~
nat64.c:392:37: error: invalid use of undefined type ‘struct nat64_kern’
  392 |         bpf_map__set_max_entries(obj->maps.v6_state_map, num_addr);
      |                                     ^~
nat64.c:393:37: error: invalid use of undefined type ‘struct nat64_kern’
  393 |         bpf_map__set_max_entries(obj->maps.v4_reversemap, num_addr);
      |                                     ^~
nat64.c:394:37: error: invalid use of undefined type ‘struct nat64_kern’
  394 |         bpf_map__set_max_entries(obj->maps.reclaimed_addrs, num_addr);
      |                                     ^~
nat64.c:396:15: warning: implicit declaration of function ‘nat64_kern__load’ [-Wimplicit-function-declaration]
  396 |         err = nat64_kern__load(obj);
      |               ^~~~~~~~~~~~~~~~
nat64.c:408:58: error: invalid use of undefined type ‘struct nat64_kern’
  408 |                 err = bpf_map_update_elem(bpf_map__fd(obj->maps.allowed_v6_src),
      |                                                          ^~
nat64.c:417:53: error: invalid use of undefined type ‘struct nat64_kern’
  417 |         attach_ingress.prog_fd = bpf_program__fd(obj->progs.nat64_ingress);
      |                                                     ^~
nat64.c:424:52: error: invalid use of undefined type ‘struct nat64_kern’
  424 |         attach_egress.prog_fd = bpf_program__fd(obj->progs.nat64_egress);
      |                                                    ^~
nat64.c:460:9: warning: implicit declaration of function ‘nat64_kern__destroy’ [-Wimplicit-function-declaration]
  460 |         nat64_kern__destroy(obj);
      |         ^~~~~~~~~~~~~~~~~~~
make[1]: *** [../lib/common.mk:83: nat64] Error 1
make: *** [Makefile:37: nat64-bpf] Error 2

tohojo

A bunch of minor-ish things, mostly on the userspace code.

Also, please add the netfilter-bpf directory to the toplevel Makefile.

tohojo · 2023-09-14T15:41:06Z

+	struct bpf_dynptr ptr;
+	struct iphdr *p, iph = {};
+	struct ipv4_lpm_key key;
+	__u32 *pvalue;


Please follow the kernel networking style of declaring variables in reverse x-mas tree order

tohojo · 2023-09-14T15:42:09Z

+	__u32 *pvalue;
+
+	if (skb->len <= 20 || bpf_dynptr_from_skb(skb, 0, &ptr))
+		return NF_ACCEPT;


For consistency, let's have an empty line after this early return as well as the other ones.

tohojo · 2023-09-14T15:42:24Z

+	int err;
+	struct bpf_object *obj;
+	struct bpf_program *prog;
+	union bpf_attr attr = { };


Reverse x-mas tree here as well, please.

tohojo · 2023-09-14T15:47:53Z

+	{
+		struct ipv4_lpm_key key;
+		__u32 value = 0;
+		__u8 *p = (__u8 *) &key.data;


Instead of this casting to an array of u8, just do a htonl(0xc0a80b6b) with a comment explaining what it means.

Also, drop the block and just move the variables to the top of the function definition.

tohojo · 2023-09-14T15:48:09Z

+	}
+	/* attach to netfilter output handler */
+	attr.link_create.netfilter.hooknum = NF_INET_LOCAL_OUT;
+	err = sys_bpf(BPF_LINK_CREATE, &attr, sizeof(attr));


Shouldn't the map be populated before attaching the program?

tohojo · 2023-09-14T15:53:24Z

+	if (libbpf_get_error(obj)) {
+		printf("fail to open bpf file\n");
+		return 1;
+	}


From here on down, there should be proper error handling on the error paths. I.e., close the bpf object on exit, close the link fd, etc

tohojo · 2023-09-14T15:54:23Z

+		printf("fail to find bpf program\n");
+		return 1;
+	}
+	bpf_program__set_type(prog, BPF_PROG_TYPE_NETFILTER);


Why is this needed, the bpf code already has a SEC("netfilter") declaration?

tohojo · 2023-09-14T15:56:33Z

+		printf("loading BPF object file failed\n");
+		return 1;
+	}
+	map_fd = bpf_object__find_map_fd_by_name(obj, "ipv4_lpm_map");


This should use the high-level libbpf API; i.e. bpf_object__find_map_by_name() and bpf_map__update_elem().

tohojo · 2023-09-14T15:56:56Z

+		return 1;
+	}
+	/* attach to netfilter forward handler */
+	prog_fd = bpf_program__fd(prog);


Same here, this should use bpf_program__attach_netfilter()

tohojo · 2023-09-14T15:57:41Z

+static inline int sys_bpf(enum bpf_cmd cmd, union bpf_attr *attr, unsigned int size)
+{
+	return syscall(__NR_bpf, cmd, attr, size);
+}


This is not needed when using the highlevel libbpf APIs, see below.

zq-david-wang added 2 commits September 8, 2023 22:36

bump submodule libbpf and bpf.h header to libbpf head 5fc0677

aca10fb

Signed-off-by: David Wang <00107082@163.com>

Add sample usage for BPF_PROG_TYPE_NETFILTER

40bbc1d

Signed-off-by: David Wang <00107082@163.com>

zq-david-wang force-pushed the master branch from 28c42fe to 40bbc1d Compare September 8, 2023 14:46

tohojo requested changes Sep 14, 2023

View reviewed changes

Conversation

zq-david-wang commented Sep 5, 2023

Uh oh!

tohojo commented Sep 5, 2023

Uh oh!

zq-david-wang commented Sep 5, 2023

Uh oh!

tohojo commented Sep 6, 2023

Uh oh!

zq-david-wang commented Sep 7, 2023

Uh oh!

zq-david-wang commented Sep 7, 2023

Uh oh!

tohojo commented Sep 7, 2023 via email

Uh oh!

zq-david-wang commented Sep 8, 2023

Uh oh!

zq-david-wang commented Sep 8, 2023

Uh oh!

tohojo commented Sep 8, 2023 via email

Uh oh!

zq-david-wang commented Sep 9, 2023

Uh oh!

tohojo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants