Skip to content

ltc: budget Phase-1 think() verify + compute-thread watchdog (.157 wedge fix) - #246

Merged
frstrtr merged 2 commits into
masterfrom
ltc-doge/think-p1-budget-watchdog
Jun 20, 2026
Merged

ltc: budget Phase-1 think() verify + compute-thread watchdog (.157 wedge fix)#246
frstrtr merged 2 commits into
masterfrom
ltc-doge/think-p1-budget-watchdog

Conversation

@frstrtr

@frstrtr frstrtr commented Jun 19, 2026

Copy link
Copy Markdown
Owner

Fix-bearing SHA 974c2e2 (GPG-signed)

Root cause (.157 wedge 2026-06-19 19:53Z)

Phase-1 head-verification in think() was unbudgeted. A backward reorg that makes every head tip fail attempt_verify walks the FULL rooted chain x PPLNS under the exclusive tracker lock -> multi-minute wedge. The pre-existing io_context-liveness watchdog is blind to this: the IO thread only ever try_to_locks, so io_context stays responsive while the compute thread spins under the exclusive lock (observed live: stratum pseudoshares still accepted, ZERO ASYNC-THINK ticks).

Fix (two pieces, same lock-discipline family as #71 drop_tails)

  1. share_tracker.hpp — THINK_P1_VERIFY_BUDGET=100 bounds the per-cycle Phase-1 verify storm. Budget spent -> set m_think_needs_continue -> run_think() reposts a continuation that releases+reacquires the exclusive lock between chunks (the Phase-2 continuation seam at node.cpp:1544/1659-1660 already existed; Phase-1 now feeds it). Same SET of shares verified as p2pool -> GENTX parity preserved; only chunked. Bootstrap mode keeps INT_MAX budget.
  2. node.cpp / node.hpp — compute-thread deadline watchdog. Atomic ns-deadline + generation counter, polled by an IO-thread steady_timer that never touches m_tracker_mutex. On expiry: logs + backtrace, clears deadline, resets m_think_running so a fresh cycle can schedule. Does NOT forcibly unwind the wedged thread (unsafe). Mirrors btc/node.

Verification

  • Budget logic confirmed in committed tree: git grep THINK_P1_VERIFY_BUDGET 974c2e25.
  • Release BuildID fcb6e0fe8137752a0f7e117c7368a930ba0f9d30 (distinct from wedged 842ae99b — no BuildID/SHA conflation).
  • TSAN build + live .157 re-soak under the .82 corpus-replay: confirmation to follow on the thread.

LTC-side only (share_tracker/node); additive, no overlap with btc-heap-opt drop_tails. Consensus/lock-path -> operator merge-tap when green.

Phase-1 head-verification was unbudgeted: a backward reorg that makes
every head tip fail attempt_verify walks the FULL rooted chain x PPLNS
under the exclusive tracker lock -> multi-minute wedge (.157 19:53Z
2026-06-19). The pre-existing io_context watchdog is blind to this (IO
thread only try_to_locks, so io_context stays live while the compute
thread spins under the exclusive lock).

- share_tracker.hpp: THINK_P1_VERIFY_BUDGET=100 bounds the per-cycle
  Phase-1 verify storm; remainder defers to the next run_think()
  continuation (lock released+reacquired between chunks). Same set of
  shares verified as p2pool -> GENTX parity preserved; only chunked.
- node.cpp/.hpp: compute-thread deadline watchdog (atomic ns-deadline
  + generation, IO-thread steady_timer that never touches the tracker
  lock). On expiry: logs + backtrace, resets m_think_running so a fresh
  cycle can schedule. Mirrors btc/node.

Bounds lock hold; does not change verification semantics.
{
// Phase-1 budget exhausted — defer the remaining heads to the
// next run_think() continuation (lock released between chunks).
if (p1_budget_remaining <= 0) { m_think_needs_continue = true; break; }
ltc/node.cpp included <execinfo.h> and called ::backtrace unconditionally
for the Phase-1-budget watchdog stack dump. execinfo.h/backtrace are
glibc-only; MSVC has no such header -> Windows x86_64 build failed with
C1083 (Cannot open include file: execinfo.h) compiling src/impl/ltc/node.cpp
(ltc.vcxproj). Linux/clang/ASan were unaffected (glibc present); btc/node.cpp
has the same include but is not built on Windows, so master stayed green.

Guard both the include and the backtrace block with #ifndef _WIN32, matching
the existing c2pool_refactored.cpp idiom. On MSVC the watchdog still fires,
logs the timeout + pending_adds, and runs the identical recovery (reset
m_think_running); only the native stack dump is skipped. No change to the
Linux runtime path.
@frstrtr
frstrtr merged commit 210cf68 into master Jun 20, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants