Skip to content

fix: disable Non-Returning Functions analyzer under the large-binary fast profile#284

Merged
branover merged 2 commits into
mainfrom
fix/noreturn-analyzer-monolith
Jul 11, 2026
Merged

fix: disable Non-Returning Functions analyzer under the large-binary fast profile#284
branover merged 2 commits into
mainfrom
fix/noreturn-analyzer-monolith

Conversation

@branover

Copy link
Copy Markdown
Owner

What

Add the Non-Returning Functions analyzers (- Discovered and - Known) to the large-binary
fast profile (_slow_analyzer), so they are disabled for images ≥ _FAST_PROFILE_BYTES (100 MB)
alongside the already-dropped Call-Fixup Installer, decompiler passes, and per-processor
constant/scalar propagation.

Why

FindNoReturnFunctionsAnalyzer (the "- Discovered" analyzer), when it flags a routine as
non-returning, runs ClearFlowAndRepairCmd to repair downstream flow. findInstructionFlow walks
the instruction-flow graph following every branch and fall-through, doing several CodeManager DB
lookups per instruction visited. On a monolith with dense or mis-disassembled code (a heavily-called
routine wrongly flagged non-returning is enough), that traversal explodes.

Observed on a ~900 MB stripped image: auto-analysis finished the base passes (functions, thunks,
references) in ~1.5 h, then wedged for >40 h inside
FindNoReturnFunctionsAnalyzer.repairDamagedLocations → ClearFlowAndRepairCmd.findInstructionFlow.
A JVM thread dump confirmed it was still advancing (different leaf frames across dumps, ~1.3 cores)
— not deadlocked — but the reachable flow set is astronomically large and nothing was being
persisted. The - Known variant can hit the same repair path when a widely-called libc routine
(abort/__stack_chk_fail/…) is on its known list, so both are disabled.

Trade-off: without no-return detection, Ghidra won't truncate disassembly after calls to
non-returning functions, so a little spurious code may follow abort()-style calls. This does not
affect the accuracy of the functions HexGraph actually analyzes on demand (xrefs/decompiles), and it
avoids the far worse failure of the analysis never completing on a large monolith.

Verification

  • tests/test_ghidra_fast_profile.py updated: asserts both Non-Returning Functions - Discovered
    and - Known are disabled by _slow_analyzer, that a dotted sub-option is still not disabled
    (the "." rule wins), and the call-graph analyzers stay enabled.
  • just test (offline) green.
  • Behavior spot-checked directly against the edited module.

jonsnow and others added 2 commits July 11, 2026 13:04
…fast profile

FindNoReturnFunctionsAnalyzer's ClearFlowAndRepair walks the instruction-flow
graph (a CodeManager DB lookup per node) to repair downstream flow whenever a
routine is flagged non-returning. On a monolith with dense or mis-disassembled
code that traversal explodes: observed wedged for >40h in
FindNoReturnFunctionsAnalyzer.repairDamagedLocations on a ~900 MB image, still
burning CPU (genuinely advancing through the graph, but the reachable set is
astronomically large) with no forward progress persisted.

Add "Non-Returning Functions" (both "- Discovered" and "- Known") to
_slow_analyzer so the >=100 MB fast profile disables it, alongside the
already-dropped Call-Fixup Installer / decompiler / constant-prop passes. Flow
past no-return calls stays intact; we lose only the automatic no-return marking,
which does not affect HexGraph's on-demand xrefs/decompiles. Test updated to
lock in both variants disabled while the dotted sub-option and call-graph
analyzers stay enabled.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The _slow_analyzer docstring and the test module docstring listed the
fast-profile disabled set but omitted the newly-added Non-Returning
Functions analyzers (doc drift flagged in PR #284 review). No behavior
change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@branover branover left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — disabling the Non-Returning Functions analyzers under the large-binary fast profile is correct, well-motivated, and low-risk.

Independent review. I am not the author. The repo-local pr-reviewer agent type isn't usable from this session's root, so I stood in for it and ran the full rubric (correctness / consistency / tests / /code-review high / security / test gate) manually.

Correctness — re-derived, holds

  • Prefix match is right. Ghidra registers exactly two analyzers under this family: Non-Returning Functions - Discovered (FindNoReturnFunctionsAnalyzer) and Non-Returning Functions - Known (NoReturnFunctionAnalyzer). Both begin with "Non-Returning Functions", so name.startswith(...) disables both and nothing else — no stock analyzer we want to keep begins with that prefix, so no over-match.
  • Ordering is correct. The if "." in name: return False early-return sits before the new check, so a dotted sub-option (Non-Returning Functions - Discovered.Create Analysis Bookmarks) returns False and stays ENABLED — consistent with the existing rule for every other disabled analyzer, and inert anyway once its parent is off. The test pins this.
  • Disabling BOTH variants is the right call, not overreach. The - Known variant is the one most likely to trigger the wedge on a canary-hardened image: it marks known libc noreturns (abort/exit/__assert_fail/__stack_chk_fail) by name, and __stack_chk_fail is called from essentially every stack-protected function — thousands of ClearFlowAndRepairCmd.findInstructionFlow repairs. So disabling only - Discovered would not have fixed it. Good that both are dropped.
  • Trade-off is sound. We lose only the auto no-return marking. Call edges (from CALL instructions) and function/reference discovery are unaffected, so the call graph and on-demand xrefs/decompiles HexGraph actually uses are intact. Worst realistic downside: a little spurious disassembly may follow abort()-style calls, and in rare cases a function reachable only by fall-through after a noreturn call might not be auto-split (such functions almost always have their own xrefs and are still found via Function ID / call analysis). This is cosmetic next to the actual failure mode — analysis never completing (>40h wedge, nothing persisted) yields zero functions. Net strongly positive. Scoped to ≥100 MB (_FAST_PROFILE_BYTES), so small binaries keep full no-return analysis.

Motivation verified as accurate: FindNoReturnFunctionsAnalyzer.repairDamagedLocations → ClearFlowAndRepairCmd.findInstructionFlow is a per-instruction flow-graph traversal with a CodeManager DB lookup per node; on a dense/mis-disassembled monolith the reachable set explodes.

Consistency / docs — one nit, FIXED

  • Doc drift (nit, non-blocking) — fixed in this branch (commit 4f279a5). The _slow_analyzer docstring and the test_ghidra_fast_profile.py module docstring both enumerate the fast-profile disabled set and omitted the new Non-Returning Functions analyzers. I added them so the enumerations stay accurate; no behavior change.
  • No CLAUDE.md / CHANGELOG change needed. Repo CLAUDE.md documents no durable fact about the disabled set and explicitly routes feature history to CHANGELOG.md, which is release-please-generated from conventional commits — the fix: commit will produce the entry automatically. The docstring's "Jython FAST_PROFILE_SCRIPT" is a historical reference only; no such script file exists to keep in sync.

Tests — sufficient and correct

Both variants asserted disabled, the dotted sub-option asserted still-enabled (the . rule wins), call-graph analyzers asserted still-enabled. Covers the meaningful boundaries. Optional future nit: no assertion that a bare Non-Returning Functions (no suffix) is handled, but that's not a real Ghidra name — not worth adding.

/code-review high + security pass

  • /code-review high on the 13-line diff surfaced no correctness/reuse/simplification/efficiency/altitude findings beyond the doc-drift nit above (now fixed). Angles A/B/C: pure addition (no removed invariant), single caller _apply_fast_profile iterates option names under contextlib.suppress — no call-site break.
  • /security-review aborts here ("not a git repository" — this session's cwd isn't the repo), so I did a manual security read. _slow_analyzer is a pure string predicate; name comes from Ghidra's trusted opts.getOptionNames(), not external input; startswith has no injection surface; the diff adds no path/shell/eval/file-I/O/network/deserialization/privilege change. No security issues.

Test gate

  • Targeted tests/test_ghidra_fast_profile.py: 3 passed (after my docstring edit too).
  • Full local just test (offline suite): 1694 passed, 6 skipped, 14 deselected in ~261s. Green.
  • CI (gh pr checks 284): 6/7 green (dependency audit, frontend build, ghidra decompile gate, tests offline py3.11 + py3.12, angr solver gate); the only non-pass is live tests (sandbox image + just test-ci) which was pending, not failing.

Verdict: APPROVE. No blocking findings. One doc-drift nit fixed and pushed. Not merging — leaving that to the orchestrator.

@branover
branover merged commit 567442a into main Jul 11, 2026
7 checks passed
@branover
branover deleted the fix/noreturn-analyzer-monolith branch July 11, 2026 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant