Skip to content

[Bug] Go binaries may crash on aarch64 with 64KB page size kernel #853

@richenlin

Description

@richenlin

Description

OpenSandbox's Go-compiled components (execd, ingress, egress) are likely incompatible with aarch64 systems running 64KB page size kernels. This affects modern ARM platforms such as NVIDIA Grace Hopper (GH200), ARM Neoverse-V2 based HPC clusters, and any Linux kernel configured with CONFIG_ARM64_64K_PAGES=y.

Background

Go's default linker produces ELF binaries with 4KB-aligned PT_LOAD segments. On kernels with 64KB page granularity, the dynamic linker or kernel may refuse to load these binaries (or they crash with SIGSEGV/SIGABRT) because differing page protections cannot be assigned within the same 64KB page boundary.
This is a known class of issue affecting multiple projects:

Environment

  • Architecture: aarch64 / ARM64
  • Kernel page size: 65536 (64KB)
  • Affected platforms:
    • NVIDIA Grace Hopper (GH200)
    • NVIDIA Grace CPU (Neoverse-V2)
    • Any aarch64 kernel with CONFIG_ARM64_64K_PAGES=y
    • RHEL/CentOS aarch64+64k kernel variants (e.g., 5.14.0-xxx.el9.aarch64+64k)

Steps to Reproduce

  1. Deploy OpenSandbox on an aarch64 node with 64KB page size kernel
  2. Create a sandbox — the execd process inside the container crashes immediately
    Verify page size:
getconf PAGESIZE
# Output on affected systems: 65536
Check ELF alignment of Go binaries:
readelf -l /path/to/execd | grep -A1 LOAD
# Alignment shows 0x1000 (4KB), incompatible with 64KB page systems
Expected Behavior
All Go binaries should run correctly on aarch64 systems regardless of whether the kernel uses 4KB or 64KB page sizes.
Additional Context
- PR #330 already builds linux/arm64 ingress images, but without 64KB page alignment consideration.
- Issue #634 proposes adding platform (os/arch) to the sandbox spec — this page size issue is a concrete runtime concern for heterogeneous ARM scheduling.
- This does NOT affect the Python server component or the SDKs — only compiled Go binaries.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions