What is the startup and stop latency of a runsc container on a single core?

[simon28li]

Description

Hello, I recently have a project that requires researching the startup and shutdown times of gVisor. I came across this resource: https://onidel.com/blog/gvisor-kata-firecracker-2025, where he mentions this.

Startup Time Comparison
gVisor: 50-100ms typical startup latency, as it initializes the user-space kernel and establishes system call interception mechanisms.
Kata Containers: 150-300ms startup time due to full VM initialization, kernel boot, and container runtime setup within the guest.
Firecracker: 100-200ms cold start, optimized through pre-warming techniques and minimal kernel configurations.

However, in my tests, I cannot achieve such standards. It takes more than 300ms to fully start and stop a container. The testing method I used was to run runsc run with a single-core limit and 2GB of memory. Could the issue be with my testing method?

1. Preparation phase (Prepare)
   Export the rootfs from the Docker image, and write the Python payload script (test.py) and test data (test.txt) into it.
   Use runsc spec to generate an OCI configuration file, and modify it to make the rootfs read-only. Set the entry command to python3 /test.py, and set the memory and CPU limits.
   Perform a self-test to confirm that runsc run is basically functional.

2. Test phase (Run)
   Obtain the list of available CPU cores from the NUMA node, and select cores according to the configured gradient (e.g., 1 core).
   For each core, repeatedly execute the following loop concurrently within a 1-second time window:
   Create separate bundle and state directories.
   Start the container using runsc run (full lifecycle: create → start → run → delete).
   Record the total container execution time (from the start of runsc run to the return of the command) and the Elapsed time reported by the Python script.
   Each time a loop is successfully completed, increment the count by one and return the time data.

Steps to reproduce

No response

runsc version

runsc -version
runsc version release-20260504.0
spec: 1.1.0-rc.1

docker version (if using docker)

uname

uname -a Linux localhost.localdomain 6.6.0-95.0.0.99.oe2403sp2.aarch64 #1 SMP Thu Jun 5 23:21:14 CST 2025 aarch64 aarch64 aarch64 GNU/Linux

kubectl (if using Kubernetes)

repo state (if built from source)

No response

runsc debug logs (if available)

[zkoopmans]

Thanks for posting your methodology. Internally, we have macro-benchmarks for gVisor w/ OCI (runsc), so our numbers also contain things like API calls to Containerd / the k8s stack. We mostly care about preventing regressions.

However, from your methodology, you're measuring start + run + stop vs the claim of start times listed. Are the other runtimes around the claim (gVisor < firecracker < Kata)? Maybe isolate the actual start / stop time vs the run?

CCing @EtiennePerot because he may be able to add to this.

[simon28li]

Thank you for your reply. In fact, the runtime load I have is extremely short and can almost be ignored. Here are my execution results,

===== Calculate the average result =====
Number of cores: 1
Average throughput: 3.25 ± 0.10 cycles/s
Average total time: 308.010 ± 9.542 ms
Average Python time: 0.148 ± 0.004 ms

as you can see from the above, the total time is around 308ms (the entire lifecycle of runsc run).Python time, that is, the running time of my workload is only 0.148 ms.
I have attached my execution script. Could you please take a look?
https://github.com/simon28li/li/blob/master/run.sh

[EtiennePerot]

Your benchmark seems fine to me, though it doesn't really distinguish between the overhead of runsc's start time vs stop time. I'd also say that gVisor isn't well tested when bound to a specific core or NUMA node (your script calls it under numactl --cpunodebind=$NUMA_NODE --membind=$NUMA_NODE), and that the default platform (Systrap) may not make the best decisions in this scenario.

All that said, startup time on the order of 300ms seems like a reasonable approximation of the current state of gVisor startup performance at the moment. I am not sure how the author in the blog post quoted in the OP gets the 50-100ms figure from, I'm not aware of gVisor ever hitting that. Startup performance is something that hasn't been looked at or optimized that much in gVisor (other than in the checkpoint/restore case), because usually gVisor is used in a Kubernetes or Docker-type environment, and the majority of the overhead in that setting is already coming from pod scheduling and higher-up components like containerd, relative to gVisor's own overhead. Patches welcome to improve it though :) Structurally speaking, gVisor has fewer operations to perform than a full VM does, so the best speed it could get in theory should be able to be better.

As a side-note, whenever comparing gVisor with hypervisors like Firecracker, it's important to distinguish between benchmarks that only measure the hypervisor and VM boot time (which is where the "100ms-200ms" figure usually comes from for Firecracker). To compare apples-to-apples, the benchmarks must also include the preparation steps that need to happen before VM creation (OCI bundle preparation, block device creation) and after VM boot (i.e kata-agent initialization, in-VM cgroups and namespace creation).