Summary
On 7.1, the server fails to start on hosts where the Graylog data directory (<data_dir>, e.g. /var/lib/graylog-server) lives on a noexec-mounted filesystem. Startup aborts during Guice injector creation with:
NoClassDefFoundError: Could not initialize class oshi.software.os.linux.LinuxOperatingSystemJNA
...
Caused by: UnsatisfiedLinkError: /var/lib/graylog-server/libnative/jna/jnaXXXX.tmp: failed to map
segment from shared object
This is a regression introduced in 7.1.0 — the same host configuration starts fine on 7.0.
Root cause
NodeMetricPeriodical (new in 7.1.0, #24493) eagerly creates a CpuLoadGauge in a field initializer:
// NodeMetricPeriodical.java
private final CpuLoadGauge cpuLoadGauge = new CpuLoadGauge();
CpuLoadGauge in turn touches OSHI native code in its own field initializer:
// CpuLoadGauge.java
private long[] lastTicks = processor().getSystemCpuLoadTicks(); // -> new
SystemInfo().getHardware().getProcessor()
Because NodeMetricPeriodical is a @Singleton Periodical (eagerly instantiated when Guice builds the injector), this forces OSHI to load its native library at startup. OSHI/JNA unpacks the native .so into <data_dir>/libnative/jna (we deliberately moved jna.tmpdir there in #21223) and dlopen()s it. On a noexec mount the kernel refuses to map the executable segments -> UnsatisfiedLinkError -> injector creation fails -> the JVM never starts.
7.0 had no boot-time consumer that forced the OSHI native load, so the same noexec data dir was tolerated. 7.1 turns a latent, harmless misconfiguration into a hard boot failure.
noexec on /var, /var/lib, /tmp, etc. is common on CIS-benchmarked / STIG-hardened systems, so this is likely to hit security-conscious customers specifically.
Impact
- Server will not start at all (not a degraded-metrics situation — total boot failure).
- Affects hardened/
noexec deployments upgrading 7.0 -> 7.1.
Workaround
Point the native lib dir at an exec-capable location via JVM options:
-Djna.tmpdir=/opt/graylog/libnative/jna -Dio.netty.native.workdir=/opt/graylog/libnative
(directory must be writable by the graylog user and on a non-noexec mount).
Proposed fix
Node metrics are a nice-to-have; their unavailability must not prevent startup.
- Make OSHI/
CpuLoadGauge initialization lazy and guarded — wrap the native access in try/catch, log a single warning, and disable the CPU-load metric (return null/NaN) instead of propagating the error.
- Remove the eager field initializers (
new CpuLoadGauge() in NodeMetricPeriodical, and lastTicks = processor()... in CpuLoadGauge) so nothing forces the native load during injector creation; defer to first periodical run.
- Consider applying the same defensive guard to
OshiService so any OSHI-backed startup path degrades gracefully on noexec/restricted hosts.
Affected versions
7.1.0 and later. Not present in 7.0.x.
References
Summary
On 7.1, the server fails to start on hosts where the Graylog data directory (
<data_dir>, e.g./var/lib/graylog-server) lives on anoexec-mounted filesystem. Startup aborts during Guice injector creation with:This is a regression introduced in 7.1.0 — the same host configuration starts fine on 7.0.
Root cause
NodeMetricPeriodical(new in 7.1.0, #24493) eagerly creates aCpuLoadGaugein a field initializer:CpuLoadGaugein turn touches OSHI native code in its own field initializer:Because
NodeMetricPeriodicalis a@SingletonPeriodical(eagerly instantiated when Guice builds the injector), this forces OSHI to load its native library at startup. OSHI/JNA unpacks the native.sointo<data_dir>/libnative/jna(we deliberately movedjna.tmpdirthere in #21223) anddlopen()s it. On anoexecmount the kernel refuses to map the executable segments ->UnsatisfiedLinkError-> injector creation fails -> the JVM never starts.7.0 had no boot-time consumer that forced the OSHI native load, so the same
noexecdata dir was tolerated. 7.1 turns a latent, harmless misconfiguration into a hard boot failure.noexecon/var,/var/lib,/tmp, etc. is common on CIS-benchmarked / STIG-hardened systems, so this is likely to hit security-conscious customers specifically.Impact
noexecdeployments upgrading 7.0 -> 7.1.Workaround
Point the native lib dir at an
exec-capable location via JVM options:(directory must be writable by the graylog user and on a non-
noexecmount).Proposed fix
Node metrics are a nice-to-have; their unavailability must not prevent startup.
CpuLoadGaugeinitialization lazy and guarded — wrap the native access in try/catch, log a single warning, and disable the CPU-load metric (returnnull/NaN) instead of propagating the error.new CpuLoadGauge()inNodeMetricPeriodical, andlastTicks = processor()...inCpuLoadGauge) so nothing forces the native load during injector creation; defer to first periodical run.OshiServiceso any OSHI-backed startup path degrades gracefully onnoexec/restricted hosts.Affected versions
7.1.0 and later. Not present in 7.0.x.
References