process: fix execve()/fork() new-namespace exec hanging OSv shutdown

execve() (CONF_fork) launches the target as a fresh OSv application in its own
ELF namespace via application::run(new_program=true) and, matching Linux, does
not return to the caller.  A successful exec DID launch the program (the
elf::program new-namespace construction path is fine), but after the exec'd
program finished the unikernel would hang at shutdown instead of powering off:
the loader's application::join() blocked forever on the top-level app's
_terminated flag.

Root cause is the fork child thread's lifecycle, not execve or elf::program.
fork_thread() (arch/{x64,aarch64}/fork.cc) creates the child as a normal
*attached* sched::thread.  At construction the thread captures a shared_ptr to
the current application's application_runtime (sched.cc sets _app_runtime =
app->runtime()).  Nothing ever join()s the fork child -- the parent reaps it
through the fork pid registry / waitpid(), not sched::thread::join() -- so the
thread object is never destroyed and its _app_runtime shared_ptr is never
released.  With that reference outstanding the application_runtime's use count
never reaches zero, ~application_runtime never runs, the app's _terminated is
never set, and application::join() waits forever.  (This bit every fork(), and
was most visible after fork()+execve() where the child adopts the caller app's
runtime.)

Fix: create the fork child detached and dispose it in its cleanup.
- arch/{x64,aarch64}/fork.cc: mark the child attr().detached(), so on
  completion it is handed to the thread reaper (which runs its set_cleanup()),
  rather than sitting forever waiting to be joined.
- libc/process/fork.cc: the child's cleanup now also calls
  sched::thread::dispose(child) after the existing bookkeeping (record exit
  status if it fell off the end, free the copied user stack).  Disposing the
  thread releases its _app_runtime reference, letting the owning application's
  runtime drop to zero so join() completes and OSv powers off.  This mirrors
  the default detached-thread cleanup ([this]{ dispose(this); }).

Both files are compiled only under CONF_fork, so default OSv builds are
unchanged.

Tests: tst-fork test 2 now execs a real payload (/tests/payload-exit7.so, which
prints a marker and exit(7)s) instead of the previous _exit(7) sentinel that
masked whether execve actually launched anything; a return from execve() is now
treated as a failure.  Added tst-execve.so (fork+execve launches the payload and
its exit code is reaped; missing path returns -1/ENOENT) and payload-exit7.so.

Validated on x86-64 (KVM disk boot): tst-fork 10/10 and tst-execve 3/3 pass with
the real exec payload and OSv shuts down cleanly (5/5 repeat runs, no hang).

Copyright (C) 2026 Greg Burd

Author: gburd

arch/aarch64/fork.cc

@@ -69,7 +69,15 @@ sched::thread *fork_thread(void *caller_ret, void *caller_sp,
            "br %1 \n\t"         // resume in fork()'s caller
            : : "r"(resume_sp), "r"(resume_pc) : "x0", "memory");
     }, sched::thread::attr().
-        stack(4096 * 4),
+        stack(4096 * 4).
+        // Detached: nobody join()s the fork child (the parent reaps it via the
+        // pid registry / waitpid, not sched::thread::join).  A detached thread
+        // is handed to the reaper on completion, which runs our set_cleanup()
+        // (freeing the copied stack and disposing the thread object, releasing
+        // its application_runtime reference).  Without this the thread object
+        // (and its app_runtime shared_ptr) would leak and OSv would hang at
+        // shutdown -- see the cleanup comment in libc/process/fork.cc.
+        detached(),
         false,
         true);
     t->set_app_tcb(parent->get_app_tcb());

arch/x64/fork.cc

@@ -91,7 +91,15 @@ sched::thread *fork_thread(void *caller_ret, void *caller_sp, void **out_stack_t
            "jmpq *%1 \n\t"          // resume in fork()'s caller
            : : "r"(resume_sp), "r"(resume_pc) : "memory");
     }, sched::thread::attr().
-        stack(4096 * 4),
+        stack(4096 * 4).
+        // Detached: nobody join()s the fork child (the parent reaps it via the
+        // pid registry / waitpid, not sched::thread::join).  A detached thread
+        // is handed to the reaper on completion, which runs our set_cleanup()
+        // (freeing the copied stack and disposing the thread object, releasing
+        // its application_runtime reference).  Without this the thread object
+        // (and its app_runtime shared_ptr) would leak and OSv would hang at
+        // shutdown -- see the cleanup comment in libc/process/fork.cc.
+        detached(),
         false,
         true);
     t->set_app_tcb(parent->get_app_tcb());

libc/process/fork.cc

@@ -184,14 +184,22 @@ pid_t fork(void)
     // ahead of the parent's bookkeeping.
     fork::register_child(cpid, parent);
 
-    // Single cleanup: free the copied user stack and, if the child fell off the
-    // end without exit(), record a default status.  Real exit codes are
-    // recorded by exit()/execve() via fork::child_exited() before this runs.
-    child->set_cleanup([cpid, stack_to_free] {
+    // Single cleanup, run by the thread reaper once the (detached) child has
+    // fully terminated: free the copied user stack, record a default status if
+    // the child fell off the end without exit() (real codes are recorded by
+    // exit()/execve() via fork::child_exited() before this runs), and finally
+    // dispose the child thread object itself.  Disposing is essential: the
+    // child thread holds a shared_ptr to its application_runtime (set at thread
+    // construction), and only destroying the thread releases that reference.
+    // Without it the top-level application's runtime never drops to zero, its
+    // ~application_runtime never fires, application::join() blocks forever on
+    // _terminated, and OSv hangs at shutdown instead of powering off.
+    child->set_cleanup([cpid, stack_to_free, child] {
         fork::child_exited(cpid, 0);
         if (stack_to_free) {
             free(stack_to_free);
         }
+        sched::thread::dispose(child);
     });
 
     child->start();

modules/tests/Makefile

@@ -130,7 +130,7 @@ tests := tst-pthread.so misc-ramdisk.so tst-vblk.so tst-mq-smoke.so tst-bsd-evh.
 	tst-queue-mpsc.so tst-af-local.so tst-pipe.so tst-yield.so \
 	misc-ctxsw.so tst-read.so tst-symlink.so tst-openat.so \
 	tst-eventfd.so tst-remove.so misc-wake.so tst-epoll.so misc-lfring.so \
-	tst-fork.so \
+	tst-fork.so tst-execve.so payload-exit7.so \
 	misc-fsx.so tst-sleep.so tst-resolve.so tst-except.so \
 	misc-tcp-sendonly.so tst-tcp-nbwrite.so misc-tcp-hash-srv.so \
 	misc-loadbalance.so misc-scheduler.so tst-console.so tst-app.so \

tests/payload-exit7.cc

@@ -0,0 +1,16 @@
+/*
+ * Copyright (C) 2026 Greg Burd
+ *
+ * This work is open source software, licensed under the terms of the
+ * BSD license as described in the LICENSE file in the top-level directory.
+ *
+ * Trivial exec payload used by tst-fork / tst-execve: prints a marker so the
+ * test can confirm the exec'd program actually ran, then exit(7)s so the
+ * parent can verify the exit code was reaped.
+ */
+#include <cstdio>
+int main() {
+    printf("payload-exit7: running, will exit(7)\n");
+    fflush(stdout);
+    return 7;
+}

tests/tst-execve.cc

@@ -0,0 +1,56 @@
+/*
+ * Copyright (C) 2026 Greg Burd
+ *
+ * This work is open source software, licensed under the terms of the
+ * BSD license as described in the LICENSE file in the top-level directory.
+ *
+ * Regression test for execve() launching a program in a fresh ELF namespace
+ * (see documentation/fork.md).  Covers the two independent facts that the
+ * original execve() new-namespace path broke:
+ *   1. a successful execve() actually LAUNCHES the target program, and
+ *   2. after the exec'd program exits, control returns/reaps cleanly and OSv
+ *      is able to shut down (the fork-child app_runtime leak used to hang it).
+ * Requires CONF_fork (execve is a stub otherwise).
+ */
+#include <unistd.h>
+#include <sys/wait.h>
+#include <cstdio>
+#include <cerrno>
+#include <cstring>
+
+static int failures = 0;
+#define CHECK(cond, msg) do { \
+    if (!(cond)) { printf("FAIL: %s\n", msg); failures++; } \
+    else { printf("PASS: %s\n", msg); } \
+} while (0)
+
+int main()
+{
+    printf("=== tst-execve ===\n"); fflush(stdout);
+
+    // execve() a real payload from a fork child; a successful exec never
+    // returns, so if it returns the exec FAILED (child exits 99).  The payload
+    // (/tests/payload-exit7.so) prints a marker and exit(7)s.
+    pid_t pid = fork();
+    if (pid == 0) {
+        char *const argv[] = { (char*)"/tests/payload-exit7.so", nullptr };
+        char *const envp[] = { nullptr };
+        execve(argv[0], argv, envp);
+        _exit(99);   // execve returned => it failed to launch the program
+    }
+    int status = 0;
+    pid_t w = waitpid(pid, &status, 0);
+    CHECK(w == pid, "waitpid() reaps the exec'd child");
+    CHECK(WIFEXITED(status) && WEXITSTATUS(status) == 7,
+          "execve launched the payload and its exit code (7) was reaped");
+
+    // execve() of a missing path must fail cleanly with ENOENT (not crash).
+    char *const bad[] = { (char*)"/tests/does-not-exist.so", nullptr };
+    errno = 0;
+    int rc = execve(bad[0], bad, nullptr);
+    CHECK(rc == -1 && errno == ENOENT,
+          "execve of a missing path returns -1/ENOENT");
+
+    printf("=== tst-execve done: %d failures ===\n", failures); fflush(stdout);
+    return failures == 0 ? 0 : 1;
+}

tests/tst-fork.cc

@@ -49,27 +49,28 @@ static void test_fork_return()
           "parent's stack local intact after child mutated its own copy");
 }
 
-// 2. fork() + execve(): child execs a trivial program; parent waits.
+// 2. fork() + execve(): the child really execs a program in a fresh ELF
+// namespace and the parent reaps it.  The payload (/tests/payload-exit7.so,
+// built by modules/tests) prints a marker and exit(7)s; the child never
+// returns from a successful execve(), so a return means exec failed and we
+// signal that with a distinct code (99) that the parent flags as a failure.
 static void test_fork_exec()
 {
     pid_t pid = fork();
     if (pid == 0) {
-        // Exec /libtrue.so or fall back to a program that exits 7.  On OSv the
-        // test image includes a simple echo/true-like helper; if none exists
-        // execve returns and we _exit a sentinel the parent recognizes.
         char *const argv[] = { (char*)"/tests/payload-exit7.so", nullptr };
         char *const envp[] = { nullptr };
         execve(argv[0], argv, envp);
-        // execve failed (no such payload in this image) -> sentinel
-        _exit(7);
+        _exit(99);   // only reached if execve() FAILED to launch the payload
     }
     CHECK(pid > 0, "fork() before execve returns child pid");
     int status = 0;
     pid_t w = waitpid(pid, &status, 0);
     CHECK(w == pid, "waitpid() reaps the fork+exec child");
-    // Whether the exec payload ran or the sentinel fired, exit status is 7.
+    // 7 means the exec'd payload actually ran and exited; 99 means execve()
+    // returned (failed to launch) -- that is now a real failure.
     CHECK(WIFEXITED(status) && WEXITSTATUS(status) == 7,
-          "fork+exec child exit status observed (7)");
+          "fork+exec: exec'd payload ran and its exit code (7) was reaped");
 }
 
 // 3. vfork() maps to fork(); same contract.