krun_add_net_unixgram broken when using fd

[nirs]

#367 introduced the option to add a virtio-net device using a connected file descriptor. Testing with krunkit and vment-helper shows that this option does not work.

How to reproduce

1. Build krunkit with libkrun main

git clone https://github.com/containers/krunkit.git
cd krunkit
gh pr checkout 63
make LIBKRUN_EFI=/usr/local/lib/libkrun-efi.dylib
sudo make install

2. Build vment-helpler

Checkout PR supporting new krunkit API:

git clone https://github.com/nirs/vmnet-helper.git
cd vmnet-helper
gh pr checkout 92

Build and install:

meson setup build
meson compile -C build
meson install -C build
sudo install -m 0640 sudoers.d/vmnet-helper /etc/sudoers.d/

Create virtual environment for testing:

python3 -m venv .venv
source .venv/bin/activate
pip install pyyaml

3. Start a vm using fd

% ./example server --driver krunkit --connection fd
Starting vmnet-helper for 'server' with interface id 'bdc7b3e0-8594-4814-aa25-05187ad2a36e'
Creating cloud-init iso '/Users/nir/.vmnet-helper/vms/server/cidata.iso'
Starting 'krunkit' virtual machine 'server' with mac address '92:c9:52:b7:6c:08'
Timeout looking up ip address

The vm does not get an ip address and is not accessible.

The krunkit command:

% ps | grep krunkit | grep fd=         
 3406 ttys007    0:03.30 krunkit --memory=1024 --cpus=1 --restful-uri=none:// --device=virtio-blk,path=/Users/nir/.vmnet-helper/vms/server/disk.img --device=virtio-blk,path=/Users/nir/.vmnet-helper/vms/server/cidata.iso --device=virtio-serial,logFilePath=/Users/nir/.vmnet-helper/vms/server/serial.log --krun-log-level=3 --device=virtio-net,type=unixgram,fd=3,mac=92:c9:52:b7:6c:08,offloading=off

The socket inherited by krunkit is created like this:

def socketpair():
    pair = socket.socketpair(socket.AF_UNIX, socket.SOCK_DGRAM, 0)
    for sock in pair:
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, SEND_BUFFER_SIZE)
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, RECV_BUFFER_SIZE)
        os.set_inheritable(sock.fileno(), True)
    return pair

Why it fails

In libkrun/src/devices/src/virtio/net/unixgram.rs we gave 2 code paths:

• new(fd) - create a backend from existing socket
• open(path) - open a unix socket and create a backend from the fd

In the open() we make the socket non-blocking and the backend code depends on having a non-blocking socket. In new we access a file descriptor create by the user which is likely in blocking mode.

The docs do not mention that non-blocking socket is a requirement:

 *  "fd"       - a file descriptor for an already open unixstream
 *               connection to the userspace network proxy. Must
 *               be -1 if "c_path" is not NULL.

(This seems to be copied from krun_add_net_unixstream and says unixstream instead of unixgram)

Workaround

Make the krunkit file descriptor non-blocking. This can be done in the program running krunkit, or in krunkit.

Fix

Make the socket non-blocking in libkrun so the user does not have to care about this implementation detail, allowing libkrun to change the implementation.

[slp]

Fixed via #402