feat: cross-platform force-kill primitive for stuck PHP threads

Introduces a self-contained primitive that unblocks a PHP thread stuck
in a blocking call (sleep, synchronous I/O, etc.) so the graceful drain
used by RestartWorkers, DrainWorkers, and Shutdown makes progress
instead of hanging for the duration of the block. The primitive is
useful on its own and gives follow-up graceful-shutdown work a reviewed
foundation to build on.

Design: each PHP thread, at boot from its own TSRM context, hands a
force_kill_slot (pointers to its own EG(vm_interrupt) and EG(timed_out)
atomic bools, plus pthread_t / Windows HANDLE for the wake-up) back to
Go via go_frankenphp_store_force_kill_slot. The slot lives on phpThread
and is protected by a per-thread RWMutex so the zero-and-release path
at thread exit cannot race an in-flight kill. From any goroutine, Go
passes the slot back to frankenphp_force_kill_thread, which stores true
into both bools (waking the VM at the next opcode boundary and routing
through zend_timeout -> "Maximum execution time exceeded") and then
delivers a platform-specific wake-up:

- Linux/FreeBSD: pthread_kill(SIGRTMIN+3) with a no-op handler installed
  via pthread_once, SA_ONSTACK, no SA_RESTART. Signal delivery causes
  any in-flight blocking syscall to return EINTR.
- Windows: CancelSynchronousIo + QueueUserAPC covers alertable I/O and
  SleepEx. Non-alertable Sleep (including PHP's usleep) stays
  uninterruptible.
- macOS: atomic-bool-only path. Threads stuck in blocking syscalls wait
  to return on their own.

JIT caveat: under the OPcache JIT some hot code paths skip vm_interrupt
checks (see php-src#21267), so a pure-PHP busy loop under JIT may not
observe the store and will fall through to the abandon path below.

Drain flow:

- worker.go: drainWorkerThreads waits drainGracePeriod (5s) for each
  drained thread to reach Yielding; then arms force-kill on stragglers
  and waits forceKillDeadline (5s) more. Threads still stuck past that
  are abandoned rather than hanging the drain forever.
- drainWorkerThreads returns (drained, abandoned). RestartWorkers puts
  drained threads back to Ready and abandoned ones into the new
  state.Abandoned (handlers treat it like ShuttingDown on next
  callback) so an abandoned thread that finally unwinds exits instead
  of re-entering the serve loop under stale request state. Abandoned
  threads are also filtered out of worker.threads immediately so
  isAtThreadLimit and the scaler see accurate capacity; the matching
  deactivateThreads cleanup drops Abandoned/Done entries from the
  auto-scaling slice so they do not permanently consume a global
  scaling slot. If any were abandoned, RestartWorkers returns
  errIncompleteRestart wrapped with abandoned/restarted counts -
  admin endpoint and watcher surface it.
- phpthread.go: phpThread.shutdown mirrors the same grace + force-kill
  + abandon pattern so Shutdown cannot hang on an uninterruptible
  blocking call either. RequestSafeStateChange and shutdown's fast-fail
  WaitFor both accept Abandoned so Shutdown racing a RestartWorkers
  that marks a thread Abandoned does not park on a transition that
  will never come.

Lifecycle hardening: Shutdown intentionally leaves phpThreads allocated
and thread_metrics alive - an abandoned thread that eventually unwinds
still calls through the SAPI and lifecycle callbacks which index those
structures. initPHPThreads blocks on a package-level sync.WaitGroup
(Add on every php_thread entry, Done on every exit path, routed
through a single goto exit: label in php_thread so future return paths
cannot silently leak an Add) so the next Init cycle cannot reassign
them out from under a lingering abandoned thread, then frees the
previous allocation inside frankenphp_init_thread_metrics before
allocating fresh. A dedicated C-side atomic (shutdown_in_progress,
toggled by frankenphp_set_shutdown_in_progress) is the signal the
unhealthy-thread restart path uses to refuse respawning past Shutdown.

- go_frankenphp_store_force_kill_slot / clear_force_kill_slot /
  on_thread_shutdown: take the per-thread write lock; clear runs before
  ts_free_thread on both healthy and unhealthy exit paths so the
  captured &EG() pointers are zeroed before their backing storage is
  freed.
- php_thread unblocks FRANKENPHP_KILL_SIGNAL with pthread_sigmask at
  startup so Go's runtime signal mask cannot silently drop deliveries.

Teardown after abandonment: php_main runs sapi_shutdown / tsrm_shutdown
unconditionally once mainThread.state.Done is observed. By definition
we already gave up on abandoned threads in drainPHPThreads, so we tear
down rather than try to outlive them. If an abandoned thread ever does
unwind after teardown it would touch torn-down state; embedders that
observe errIncompleteRestart and care about cleanliness should
terminate rather than re-Init.

- worker_test.go + testdata/worker-sleep.php: the regression test
  drives the full path via a request marker file so it only arms
  RestartWorkers once the worker is proven parked in sleep(), then
  asserts both the bounded elapsed time and that the "should not
  reach" line after sleep never runs (which would indicate the VM
  interrupt was never observed).

RestartWorkers now returns an error - a source-compatible Go API change
(callers that ignored it still compile) but worth noting for embedders.

Author: nicolas-grekas

caddy/admin.go

@@ -39,7 +39,13 @@ func (admin *FrankenPHPAdmin) restartWorkers(w http.ResponseWriter, r *http.Requ
 		return admin.error(http.StatusMethodNotAllowed, fmt.Errorf("method not allowed"))
 	}
 
-	frankenphp.RestartWorkers()
+	if err := frankenphp.RestartWorkers(); err != nil {
+		// Restart is incomplete: at least one worker thread was stuck in
+		// an uninterruptible blocking call and did not reload code. Do
+		// not let the admin endpoint lie to automation with a 200.
+		caddy.Log().Sugar().Errorf("workers restart incomplete: %v", err)
+		return admin.error(http.StatusInternalServerError, err)
+	}
 	caddy.Log().Info("workers restarted from admin api")
 	admin.success(w, "workers restarted successfully\n")
 

frankenphp.c

@@ -92,6 +92,107 @@ static bool is_forked_child = false;
 static void frankenphp_fork_child(void) { is_forked_child = true; }
 #endif
 
+/* Best-effort force-kill for stuck PHP threads.
+ *
+ * Each thread captures &EG(vm_interrupt) / &EG(timed_out) at boot and
+ * hands them to Go via go_frankenphp_store_force_kill_slot. To kill,
+ * Go passes the slot back to frankenphp_force_kill_thread, which stores
+ * true into both bools (the VM bails through zend_timeout() at the next
+ * opcode boundary) and then wakes any in-flight syscall:
+ *   - Linux/FreeBSD: pthread_kill(SIGRTMIN+3) -> EINTR.
+ *   - Windows: CancelSynchronousIo + QueueUserAPC for alertable I/O +
+ *     SleepEx. Non-alertable Sleep (including PHP's usleep) stays stuck.
+ *   - macOS: atomic-bool only; busy loops bail, blocking syscalls don't.
+ *
+ * Reserved signal: SIGRTMIN+3. PHP's pcntl_signal(SIGRTMIN+3, ...)
+ * clobbers it. glibc NPTL reserves SIGRTMIN..SIGRTMIN+2; embedders with
+ * their own Go signal usage may need to patch this constant.
+ *
+ * The slot lives Go-side on phpThread; the C side has no global table.
+ * The signal handler is installed once via pthread_once. */
+#ifdef PHP_WIN32
+static void CALLBACK frankenphp_noop_apc(ULONG_PTR param) { (void)param; }
+#endif
+
+#ifdef FRANKENPHP_HAS_KILL_SIGNAL
+/* No-op: delivery itself is what unblocks the syscall via EINTR. */
+static void frankenphp_kill_signal_handler(int sig) { (void)sig; }
+
+static pthread_once_t kill_signal_handler_installed = PTHREAD_ONCE_INIT;
+static void install_kill_signal_handler(void) {
+  /* No SA_RESTART so syscalls return EINTR rather than being restarted.
+   * SA_ONSTACK guards against an accidental process-level delivery to a
+   * Go-managed thread, where Go requires the alternate signal stack. */
+  struct sigaction sa;
+  memset(&sa, 0, sizeof(sa));
+  sa.sa_handler = frankenphp_kill_signal_handler;
+  sigemptyset(&sa.sa_mask);
+  sa.sa_flags = SA_ONSTACK;
+  sigaction(FRANKENPHP_KILL_SIGNAL, &sa, NULL);
+}
+#endif
+
+/* Set by frankenphp_set_shutdown_in_progress to gate the unhealthy-thread
+ * respawn loop off once Shutdown begins. */
+static zend_atomic_bool shutdown_in_progress;
+
+void frankenphp_set_shutdown_in_progress(bool v) {
+  zend_atomic_bool_store(&shutdown_in_progress, v);
+}
+
+/* Must run on the PHP thread itself: EG() resolves to its own TSRM
+ * context and pthread_self() captures the right tid. */
+void frankenphp_register_thread_for_kill(uintptr_t idx) {
+  force_kill_slot slot;
+  memset(&slot, 0, sizeof(slot));
+  slot.vm_interrupt = &EG(vm_interrupt);
+  slot.timed_out = &EG(timed_out);
+#ifdef FRANKENPHP_HAS_KILL_SIGNAL
+  slot.tid = pthread_self();
+  pthread_once(&kill_signal_handler_installed, install_kill_signal_handler);
+#elif defined(PHP_WIN32)
+  if (!DuplicateHandle(GetCurrentProcess(), GetCurrentThread(),
+                       GetCurrentProcess(), &slot.thread_handle, 0, FALSE,
+                       DUPLICATE_SAME_ACCESS)) {
+    /* On failure, force_kill falls back to atomic-bool only. */
+    slot.thread_handle = NULL;
+  }
+#endif
+  go_frankenphp_store_force_kill_slot(idx, slot);
+}
+
+void frankenphp_force_kill_thread(force_kill_slot slot) {
+  if (slot.vm_interrupt == NULL) {
+    /* Boot aborted before register_thread_for_kill. */
+    return;
+  }
+  /* Atomic stores first: by the time the thread wakes (signal-driven or
+   * natural) the VM sees them and bails through zend_timeout(). */
+  zend_atomic_bool_store(slot.timed_out, true);
+  zend_atomic_bool_store(slot.vm_interrupt, true);
+
+#ifdef FRANKENPHP_HAS_KILL_SIGNAL
+  /* ESRCH (thread already exited) / EINVAL are both benign here. */
+  pthread_kill(slot.tid, FRANKENPHP_KILL_SIGNAL);
+#elif defined(PHP_WIN32)
+  if (slot.thread_handle != NULL) {
+    CancelSynchronousIo(slot.thread_handle);
+    QueueUserAPC((PAPCFUNC)frankenphp_noop_apc, slot.thread_handle, 0);
+  }
+#endif
+}
+
+/* CloseHandle on Windows; no-op on POSIX. */
+void frankenphp_release_thread_for_kill(force_kill_slot slot) {
+#ifdef PHP_WIN32
+  if (slot.thread_handle != NULL) {
+    CloseHandle(slot.thread_handle);
+  }
+#else
+  (void)slot;
+#endif
+}
+
 void frankenphp_update_local_thread_context(bool is_worker) {
   is_worker_thread = is_worker;
 
@@ -1065,6 +1166,20 @@ static void *php_thread(void *arg) {
   snprintf(thread_name, 16, "php-%" PRIxPTR, thread_index);
   set_thread_name(thread_name);
 
+  /* Paired with go_frankenphp_thread_exited at the exit: label below;
+   * lets initPHPThreads wait for prior-generation threads. */
+  go_frankenphp_thread_spawned();
+
+#ifdef FRANKENPHP_HAS_KILL_SIGNAL
+  /* The spawning Go-managed M may block realtime signals, which the
+   * new pthread inherits. Unblock FRANKENPHP_KILL_SIGNAL here so
+   * force-kill deliveries are not silently dropped. */
+  sigset_t unblock;
+  sigemptyset(&unblock);
+  sigaddset(&unblock, FRANKENPHP_KILL_SIGNAL);
+  pthread_sigmask(SIG_UNBLOCK, &unblock, NULL);
+#endif
+
   /* Initial allocation of all global PHP memory for this thread */
 #ifdef ZTS
   (void)ts_resource(0);
@@ -1073,6 +1188,11 @@ static void *php_thread(void *arg) {
 #endif
 #endif
 
+  /* Register this thread's vm_interrupt/timed_out addresses so the Go side
+   * can force-kill it after the graceful-drain grace period if it gets stuck
+   * in a busy PHP loop. */
+  frankenphp_register_thread_for_kill(thread_index);
+
   bool thread_is_healthy = true;
   bool has_attempted_shutdown = false;
 
@@ -1150,6 +1270,11 @@ static void *php_thread(void *arg) {
   }
   zend_end_try();
 
+  /* Must precede ts_free_thread: that frees the TSRM storage backing
+   * the slot's &EG() pointers. Clearing first means any concurrent
+   * force-kill either ran before us or sees a zero slot. */
+  go_frankenphp_clear_force_kill_slot(thread_index);
+
   /* free all global PHP memory reserved for this thread */
 #ifdef ZTS
   ts_free_thread();
@@ -1158,19 +1283,28 @@ static void *php_thread(void *arg) {
   /* Thread is healthy, signal to Go that the thread has shut down */
   if (thread_is_healthy) {
     go_frankenphp_on_thread_shutdown(thread_index);
-
-    return NULL;
+    goto exit;
   }
 
-  /* Thread is unhealthy, PHP globals might be in a bad state after a bailout,
-   * restart the entire thread */
+  /* Unhealthy: respawn unless Shutdown is in progress; respawning then
+   * would hand a fresh pthread a phpThreads slice already untracked. */
+  if (zend_atomic_bool_load(&shutdown_in_progress)) {
+    frankenphp_log_message(
+        "Unhealthy thread unwinding after Shutdown; not restarting",
+        LOG_WARNING);
+    goto exit;
+  }
   frankenphp_log_message("Restarting unhealthy thread", LOG_WARNING);
 
   if (!frankenphp_new_php_thread(thread_index)) {
     /* probably unreachable */
     frankenphp_log_message("Failed to restart an unhealthy thread", LOG_ERR);
   }
 
+exit:
+  /* Single exit so spawned()/exited() pairing can never drift if a new
+   * return is added above. */
+  go_frankenphp_thread_exited();
   return NULL;
 }
 
@@ -1265,7 +1399,14 @@ static void *php_main(void *arg) {
 
   go_frankenphp_main_thread_is_ready();
 
-  /* channel closed, shutdown gracefully */
+  /* channel closed, shutdown gracefully. Abandoned threads (force-kill
+   * could not interrupt them within forceKillDeadline) may still unwind
+   * later when the syscall returns naturally - sleep finishes, select
+   * times out, etc. Waiting for them here would mean blocking Shutdown
+   * for an unbounded duration, so we run SAPI/TSRM teardown anyway and
+   * accept the use-after-free risk if one of them resumes afterwards.
+   * Embedders that observe errIncompleteRestart should terminate the
+   * process rather than re-Init - see RestartWorkers' doc. */
   frankenphp_sapi_module.shutdown(&frankenphp_sapi_module);
 
   sapi_shutdown();
@@ -1470,6 +1611,10 @@ int frankenphp_reset_opcache(void) {
 int frankenphp_get_current_memory_limit() { return PG(memory_limit); }
 
 void frankenphp_init_thread_metrics(int max_threads) {
+  /* Frees any prior generation's allocation; Shutdown leaves it alive
+   * for abandoned threads. lingeringThreads.Wait() upstream guarantees
+   * those have all exited before we get here. free(NULL) is a no-op. */
+  free(thread_metrics);
   thread_metrics = calloc(max_threads, sizeof(frankenphp_thread_metrics));
 }
 

frankenphp.h

@@ -46,6 +46,28 @@ static inline HRESULT LongLongSub(LONGLONG llMinuend, LONGLONG llSubtrahend,
 #include <stdbool.h>
 #include <stdint.h>
 
+#ifndef PHP_WIN32
+#include <pthread.h>
+#include <signal.h>
+#endif
+
+/* Platform capabilities for the force-kill primitive; declared in the
+ * header so Go (via CGo) gets the correct struct layout too. */
+#if !defined(PHP_WIN32) && defined(SIGRTMIN)
+#define FRANKENPHP_HAS_KILL_SIGNAL 1
+#define FRANKENPHP_KILL_SIGNAL (SIGRTMIN + 3)
+#endif
+
+typedef struct {
+  zend_atomic_bool *vm_interrupt;
+  zend_atomic_bool *timed_out;
+#ifdef FRANKENPHP_HAS_KILL_SIGNAL
+  pthread_t tid;
+#elif defined(PHP_WIN32)
+  HANDLE thread_handle;
+#endif
+} force_kill_slot;
+
 #ifndef FRANKENPHP_VERSION
 #define FRANKENPHP_VERSION dev
 #endif
@@ -193,6 +215,19 @@ void frankenphp_init_thread_metrics(int max_threads);
 void frankenphp_destroy_thread_metrics(void);
 size_t frankenphp_get_thread_memory_usage(uintptr_t thread_index);
 
+/* Best-effort force-kill primitives. The slot is populated by each PHP
+ * thread at boot (frankenphp_register_thread_for_kill calls back into Go
+ * via go_frankenphp_store_force_kill_slot) and lives in the Go-side
+ * phpThread. force_kill_thread interrupts the Zend VM at the next opcode
+ * boundary; on POSIX it also delivers SIGRTMIN+3 to the target thread,
+ * on Windows it calls CancelSynchronousIo + QueueUserAPC. release_thread
+ * drops any OS-owned resource tied to the slot (currently the Windows
+ * thread handle). */
+void frankenphp_set_shutdown_in_progress(bool v);
+void frankenphp_register_thread_for_kill(uintptr_t thread_index);
+void frankenphp_force_kill_thread(force_kill_slot slot);
+void frankenphp_release_thread_for_kill(force_kill_slot slot);
+
 void register_extensions(zend_module_entry **m, int len);
 
 #endif

internal/state/state.go

@@ -35,6 +35,13 @@ const (
 	Rebooting
 	// C thread has exited and ZTS state is cleaned up, ready to spawn a new C thread
 	RebootReady
+
+	// Abandoned is set by RestartWorkers on threads that did not yield
+	// within the force-kill deadline. Handlers treat it like ShuttingDown
+	// on the next callback, so an abandoned thread that eventually
+	// unwinds exits cleanly instead of re-entering the serve loop with
+	// stale request state.
+	Abandoned
 )
 
 func (s State) String() string {
@@ -67,6 +74,8 @@ func (s State) String() string {
 		return "rebooting"
 	case RebootReady:
 		return "reboot ready"
+	case Abandoned:
+		return "abandoned"
 	default:
 		return "unknown"
 	}
@@ -172,12 +181,12 @@ func (ts *ThreadState) WaitFor(states ...State) {
 func (ts *ThreadState) RequestSafeStateChange(nextState State) bool {
 	ts.mu.Lock()
 	switch ts.currentState {
-	// disallow state changes if shutting down or done
-	case ShuttingDown, Done, Reserved:
+	// Terminal: Abandoned never transitions to Ready/Inactive/ShuttingDown,
+	// so waiting would park forever.
+	case ShuttingDown, Done, Reserved, Abandoned:
 		ts.mu.Unlock()
 
 		return false
-	// ready and inactive are safe states to transition from
 	case Ready, Inactive:
 		ts.currentState = nextState
 		ts.notifySubscribers(nextState)
@@ -187,8 +196,9 @@ func (ts *ThreadState) RequestSafeStateChange(nextState State) bool {
 	}
 	ts.mu.Unlock()
 
-	// wait for the state to change to a safe state
-	ts.WaitFor(Ready, Inactive, ShuttingDown)
+	// Done and Abandoned in the set so a concurrent terminal transition
+	// wakes us; the recursive call below then hits the reject branch.
+	ts.WaitFor(Ready, Inactive, ShuttingDown, Done, Abandoned)
 
 	return ts.RequestSafeStateChange(nextState)
 }