Skip to content

Commit 4d8130b

Browse files
branoverclaude
andauthored
fix: route the xref family to warm Ghidra in any mode, not just headless (#286)
* fix: route the xref family to warm Ghidra in any mode, not just headless The cross-reference tools (re_xrefs / re_function_xrefs / re_data_xrefs / re_call_graph) gated the warm-Ghidra path on `_ghidra_xrefs_active()`, which is true only for headless settings mode. A user running Ghidra in bridge mode (a managed resident bridge, or a researcher bridge) has a warm Ghidra project, but the gate returned false, so these tools fell through to the cold radare2 xrefs probe. That probe finds no warm *r2* project — only Ghidra was analyzed — and returns "No warm radare2 analysis for this target yet. Run re_analyze ...", naming the wrong engine and telling the user to build an analysis they don't use. Fix: - Add `_ghidra_backend_enabled()` (enabled in ANY mode) and gate the xref family on it. Keep the narrow `_ghidra_xrefs_active()` (headless-only) for re_script, which genuinely needs the headless probe's warm project / P-Code surface. - `_ghidra_xrefs` now degrades a registered-but-unreachable managed bridge to the headless warm slot before giving up, instead of dropping the query to r2. - When Ghidra is the active backend but its warm index can't answer, surface a Ghidra-named lead (`_GHIDRA_XREF_LEAD`) — never the r2 "no warm radare2" one. r2 still runs as a genuine last resort, but its cold-miss lead is rewritten so the message always matches the active backend. - `_xrefs` now passes `project_mount` on its r2 path (it previously never reloaded the warm r2 project) and surfaces the cold-miss lead instead of formatting a false-empty sink map. - data_xrefs: a subject that doesn't resolve to an address (most often a string VALUE mistakenly passed, e.g. a URL/path) now points at the resolve-the- string's-address workflow (re_list_strings / re_resolve) rather than a bare "no resolvable references". Tests: broaden the xref-routing wiring to the new gate; add coverage for the bridge-mode gate, the string-value data_xrefs hint, and the bridge-mode cold case returning the Ghidra lead (never radare2). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: honest researcher-bridge xref message + review follow-ups Refines the xref-backend routing from the first commit, addressing the merge review and extending the fix to a sibling that shared the bug: - Researcher jfx bridge (features.ghidra.mode=="bridge") is decompile-only and re_analyze builds no slot in that mode, so routing its xref into a headless cold-miss produced a "run re_analyze" lead that can't help. `_ghidra_bridge_only` now short-circuits re_xrefs/re_function_xrefs/re_data_xrefs/re_call_graph with an honest "switch to headless (the bridge is decompile-only)" message — no futile attempt, no radare2 cold-miss. Mirrors emulation's early mode-aware refusal. - _search_code_scan gated its warm path on the headless-only _ghidra_xrefs_active while _ghidra_search already routes bridge-aware — the exact mis-route this PR fixes for the xref family, on a sibling. Switched it to _ghidra_backend_enabled. It does NOT get the bridge-only refusal: its r2 fallback is a RAW byte scan that needs no warm analysis, so researcher-bridge targets still get real hits. - _ghidra_xrefs now degrades a dead managed bridge to headless ONLY when the bridge call RAISED (unreachable). A LIVE bridge returning an error dict no longer triggers a headless op that would conflict on the project lock the bridge holds — it returns None (caller surfaces the lead / r2) instead. Tests: the dead-managed-bridge→headless degrade (a real GhidraBridgeDecompiler that raises), the re_xrefs path's own cold-miss rewrite, the researcher-bridge refusal message, and search_code falling back to the r2 raw scan in bridge mode. Renamed the mis-named "bridge_mode_cold" test (its primary was headless). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: tighten _ghidra_xrefs degrade docstring + bridge-only message Two LOW wording nits from the #286 re-review: the _ghidra_xrefs docstring said the headless degrade fires when the bridge 'unreachable/errors', but it now degrades only on a raise (an error dict returns None); and the bridge-only message listed 'xrefs or search_code' though search_code never shows it (it falls back to the r2 raw scan in bridge mode). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 07b0bbb commit 4d8130b

3 files changed

Lines changed: 298 additions & 45 deletions

File tree

src/hexgraph/agent/agent_tools.py

Lines changed: 157 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1949,11 +1949,36 @@ def _solve_constraint(ctx: ToolContext, args: dict) -> str:
19491949
return _clip("\n".join(lines))
19501950

19511951

1952+
# The cold-miss / unreachable-backend lead the xref family returns when GHIDRA is the active backend
1953+
# — deliberately Ghidra-named. The r2 xrefs_probe's own lead names "radare2", which is the WRONG engine
1954+
# for a Ghidra user (they built a warm Ghidra project, not an r2 one) and tells them to analyze a
1955+
# backend they don't use — the confusing message this fixes. re_analyze with Ghidra active builds the
1956+
# Ghidra slot; a stale/unhealthy bridge reverts to the warm headless project via re_bridge_stop.
1957+
_GHIDRA_XREF_LEAD = (
1958+
"No warm Ghidra analysis available for this target (Ghidra is the active decompiler backend). Run "
1959+
"re_analyze(target) FIRST — it builds the warm project ONCE (detached; poll until state='analyzed'), "
1960+
"then re-run this and it'll be instant. If a Ghidra bridge is running but unhealthy, re_bridge_stop "
1961+
"and retry (the target reverts to the warm headless project). This tool is warm-only and never runs "
1962+
"a cold analysis itself.")
1963+
1964+
# The honest message for the RESEARCHER jfx bridge (`features.ghidra.mode == "bridge"`), which is
1965+
# decompile-only: it has no warm reference index / P-Code surface, so xrefs / search_code can't be
1966+
# served from it AND re_analyze builds no slot in bridge mode (analysis._active_backend maps bridge →
1967+
# None). So — unlike `_GHIDRA_XREF_LEAD` (a headless-mode cold miss that re_analyze fixes) — the fix is
1968+
# to switch mode, not to re-analyze. (The managed resident bridge runs under headless mode, so it is
1969+
# NOT this case.)
1970+
_GHIDRA_BRIDGE_ONLY_XREF_MSG = (
1971+
"Cross-references need headless Ghidra's warm reference index, but Ghidra is configured in bridge "
1972+
"mode — the researcher bridge is decompile-only (no reference index / P-Code surface for xrefs, and "
1973+
"re_analyze builds no warm project in bridge mode). To get cross-references, set features.ghidra.mode "
1974+
"to headless and run re_analyze, or switch the backend to radare2.")
1975+
1976+
19521977
def _ghidra_xrefs_active() -> bool:
1953-
"""True when headless Ghidra is the active decompiler backend, so cross-reference queries are
1954-
served from its warm persistent project's reference index (analyze-once) instead of the cold r2
1955-
xrefs sweep. Mirrors `get_taint_analyzer`'s selection; a settings hiccup ⇒ False (stay on the
1956-
always-available r2 path)."""
1978+
"""True when HEADLESS Ghidra is the active decompiler backend. Narrow on purpose: re_script needs
1979+
the headless probe's warm project / P-Code surface specifically (radare2 and the bridge have no
1980+
equivalent), so it gates on THIS. For the cross-reference family use `_ghidra_backend_enabled`
1981+
instead — any warm-Ghidra mode (headless OR bridge) serves xrefs. A settings hiccup ⇒ False."""
19571982
try:
19581983
from hexgraph.engine.re.ghidra import ghidra_config
19591984

@@ -1963,30 +1988,81 @@ def _ghidra_xrefs_active() -> bool:
19631988
return False
19641989

19651990

1991+
def _ghidra_backend_enabled() -> bool:
1992+
"""True when Ghidra is the configured decompiler backend in ANY mode (headless OR bridge), so the
1993+
cross-reference family serves from Ghidra's warm reference index rather than the cold r2 xrefs
1994+
sweep. Broader than `_ghidra_xrefs_active` (headless-only): a bridge-mode user — or a live managed
1995+
per-target bridge — is just as much a "Ghidra is the warm backend" case. Routing their xrefs to r2
1996+
wrongly reports "no warm radare2" for an r2 project that was never built (only Ghidra was analyzed);
1997+
gating on `enabled` here is what fixes that mis-route. A settings hiccup ⇒ False (stay on r2)."""
1998+
try:
1999+
from hexgraph.engine.re.ghidra import ghidra_config
2000+
2001+
return bool(ghidra_config().get("enabled"))
2002+
except Exception: # noqa: BLE001 — never let config break xrefs backend selection
2003+
return False
2004+
2005+
2006+
def _ghidra_bridge_only() -> bool:
2007+
"""True when Ghidra is enabled in RESEARCHER-bridge mode (`mode == "bridge"`), a decompile-only
2008+
backend that can't serve xrefs / search_code. Used to give the honest `_GHIDRA_BRIDGE_ONLY_XREF_MSG`
2009+
("switch to headless") instead of routing into a futile headless cold miss whose lead ("run
2010+
re_analyze") doesn't apply in bridge mode. A managed resident bridge runs under headless mode, so
2011+
it is NOT flagged here — and it can't be started in bridge mode anyway (there's no warm slot to
2012+
build one from). A settings hiccup ⇒ False (fall through to the normal path)."""
2013+
try:
2014+
from hexgraph.engine.re.ghidra import ghidra_config
2015+
2016+
g = ghidra_config()
2017+
return bool(g.get("enabled")) and (g.get("mode") or "headless") == "bridge"
2018+
except Exception: # noqa: BLE001 — never let config break xrefs backend selection
2019+
return False
2020+
2021+
19662022
def _ghidra_xrefs(ctx: ToolContext, mode: str, subject: str | None) -> dict | None:
19672023
"""A cross-reference query answered from the warm persistent Ghidra project's reference index
1968-
(`GhidraDecompiler.xrefs`). Returns the probe dict — SAME shape as the r2 xrefs_probe, with a
1969-
`not_found` flag for an unknown symbol/address — on a run that COMPLETED (even an empty /
1970-
not-found one) so the caller trusts it and does NOT fall back to the cold r2 sweep. Returns
1971-
None ONLY when Ghidra could not run (Docker down, not built into the image, a probe fault, or a
1972-
top-level `error`), so the caller falls back to r2. Best-effort: never raises into the caller."""
2024+
(`GhidraDecompiler.xrefs` / the managed bridge). Returns the probe dict — SAME shape as the r2
2025+
xrefs_probe, with a `not_found` flag for an unknown symbol/address — on a run that COMPLETED (even
2026+
an empty / not-found one) so the caller TRUSTS it and does not look further. Returns None ONLY when
2027+
Ghidra genuinely could not run (Docker down, not built into the image, a probe fault, or a top-level
2028+
`error`); a Ghidra-backed caller then surfaces `_GHIDRA_XREF_LEAD`, never the r2 radare2 lead.
2029+
2030+
A live managed per-target bridge OWNS the Ghidra project, so `ghidra_op_backend` routes there first
2031+
(the resident reference index answers with no per-call open and no project-lock conflict). If that
2032+
bridge is registered but UNREACHABLE (its call raises), we RETRY the headless warm project
2033+
explicitly, so a dead bridge degrades to the warm headless slot rather than dropping the query to
2034+
r2. A LIVE bridge that RETURNS an error keeps holding its project lock, so we do NOT run a headless
2035+
op behind it (that returns None). Best-effort: never raises into the caller."""
19732036
from hexgraph.sandbox.runner import docker_available
19742037

19752038
if not docker_available():
19762039
return None
1977-
# A live per-target bridge OWNS the Ghidra project, so route the xref query to it — the resident
1978-
# reference index answers with no per-call open and no project-lock conflict; else the headless
1979-
# warm path. Either way a probe fault / top-level error returns None so the caller degrades to r2.
1980-
from hexgraph.sandbox.decompiler import ghidra_op_backend
2040+
from hexgraph.engine.re.ghidra_bridge import GhidraBridgeDecompiler
2041+
from hexgraph.sandbox.decompiler import GhidraDecompiler, ghidra_op_backend
19812042

2043+
primary = ghidra_op_backend(ctx.target)
2044+
raised = False
19822045
try:
1983-
out = ghidra_op_backend(ctx.target).xrefs(
1984-
ctx.target.path, mode=mode, subject=subject or None, project=ctx.project)
1985-
except Exception: # noqa: BLE001 — a warm-path failure DEGRADES to r2, never aborts the tool
1986-
return None
1987-
if not isinstance(out, dict) or out.get("error"):
1988-
return None
1989-
return out
2046+
out = primary.xrefs(ctx.target.path, mode=mode, subject=subject or None, project=ctx.project)
2047+
except Exception: # noqa: BLE001 — the backend was UNREACHABLE (e.g. a dead managed bridge)
2048+
out, raised = None, True
2049+
if not raised:
2050+
# The backend ANSWERED (a dict — possibly not_found, or an error). Trust it and do NOT run a
2051+
# headless op behind it: a LIVE bridge that returned an error still holds the project lock a
2052+
# headless pass would conflict on. A good dict is the result; anything else ⇒ give up (None).
2053+
return out if isinstance(out, dict) and not out.get("error") else None
2054+
# primary RAISED ⇒ it was unreachable. Degrade a dead managed BRIDGE to the headless warm slot
2055+
# (headless can't conflict with a bridge that isn't answering); a headless primary has nothing to
2056+
# degrade to.
2057+
if isinstance(primary, GhidraBridgeDecompiler):
2058+
try:
2059+
out = GhidraDecompiler().xrefs(
2060+
ctx.target.path, mode=mode, subject=subject or None, project=ctx.project)
2061+
except Exception: # noqa: BLE001
2062+
return None
2063+
if isinstance(out, dict) and not out.get("error"):
2064+
return out
2065+
return None
19902066

19912067

19922068
def _scripting_enabled() -> bool:
@@ -2115,12 +2191,14 @@ def _xrefs(ctx: ToolContext, symbol: str | None) -> str:
21152191
key = f"xrefs:{symbol or '*'}"
21162192
if key in ctx.cache:
21172193
return ctx.cache[key]
2118-
# Prefer the warm persistent Ghidra project's reference index when headless Ghidra is active: an
2194+
if _ghidra_bridge_only(): # researcher jfx bridge: decompile-only, can't serve xrefs — say so
2195+
return _GHIDRA_BRIDGE_ONLY_XREF_MSG
2196+
# Prefer the warm Ghidra reference index whenever Ghidra is the configured backend (any mode): an
21192197
# index lookup over the already-analyzed program, NOT a cold whole-binary r2 `aaa` sweep (which
2120-
# re-analyzes every call and times out on a large target). Fall back to the r2 xrefs probe only
2121-
# when Ghidra can't run.
2122-
out = _ghidra_xrefs(ctx, "callers" if symbol else "sinks", symbol) \
2123-
if _ghidra_xrefs_active() else None
2198+
# re-analyzes every call and times out on a large target). Only a NON-Ghidra target reaches the r2
2199+
# probe; a Ghidra backend that can't answer surfaces the Ghidra lead, never r2's radare2 one.
2200+
ghidra = _ghidra_backend_enabled()
2201+
out = _ghidra_xrefs(ctx, "callers" if symbol else "sinks", symbol) if ghidra else None
21242202
if out is None:
21252203
from hexgraph.sandbox.executor import get_executor
21262204
from hexgraph.sandbox.runner import docker_available
@@ -2131,9 +2209,15 @@ def _xrefs(ctx: ToolContext, symbol: str | None) -> str:
21312209
out = get_executor().run_json_probe(
21322210
"xrefs_probe.py", ctx.target.path,
21332211
extra_args=[symbol] if symbol else None,
2212+
project_mount=_r2_project_mount(ctx),
21342213
)
21352214
except Exception as exc: # noqa: BLE001
21362215
return f"xrefs failed: {exc}"
2216+
# A warm-MISS carries the re_analyze lead — surface it rather than formatting a false-empty
2217+
# sink map. Rewrite it to the Ghidra-named lead when Ghidra is the active backend (the r2
2218+
# probe's own lead names radare2, the wrong engine for a Ghidra user).
2219+
if isinstance(out, dict) and out.get("needs_analysis") and out.get("error"):
2220+
return _GHIDRA_XREF_LEAD if ghidra else out["error"]
21372221
# Record the xref result (a QUERY) — the enrichment extractor tags is_sink on any
21382222
# already-curated dangerous-import symbol; no graph nodes are created here.
21392223
_record_obs(ctx, tool="xrefs", args={"symbol": symbol} if symbol else {},
@@ -2207,13 +2291,26 @@ def _r2_project_mount(ctx: ToolContext) -> str | None:
22072291

22082292

22092293
def _run_xrefs_probe(ctx: ToolContext, subject: str | None, mode: str):
2210-
"""Run a breadth xrefs query in `mode`, returning (out, error_text). Prefers the warm persistent
2211-
Ghidra project's reference index when headless Ghidra is active (analyze-once, fast on a large
2212-
target where the cold r2 sweep times out); falls back to the r2 xrefs_probe (`--mode
2213-
function|data|callgraph`) only when Ghidra can't run. The r2 probe is WARM-ONLY too — it RELOADS
2214-
the persistent r2 project (bind-mounted) and NEVER runs `aaa`; a cold slot returns the re_analyze
2215-
lead. `mode="callers"` passes no --mode flag on the r2 path (legacy compat)."""
2216-
if _ghidra_xrefs_active():
2294+
"""Run a breadth xrefs query in `mode`, returning (out, error_text). Prefers the warm Ghidra
2295+
reference index whenever Ghidra is the configured backend in ANY mode (analyze-once, fast on a
2296+
large target where the cold r2 sweep times out); only a NON-Ghidra target reaches the r2
2297+
xrefs_probe (`--mode function|data|callgraph`). The r2 probe is WARM-ONLY too — it RELOADS the
2298+
persistent r2 project (bind-mounted) and NEVER runs `aaa`; a cold slot returns the re_analyze lead.
2299+
`mode="callers"` passes no --mode flag on the r2 path (legacy compat).
2300+
2301+
When Ghidra IS the backend but its warm index can't answer (cold project, or a bridge registered
2302+
yet unreachable), we surface `_GHIDRA_XREF_LEAD` — NOT the r2 probe's "no warm radare2" lead, which
2303+
names the wrong engine for a Ghidra user. r2 still runs as a genuine last resort (a misconfigured
2304+
Ghidra with a warm r2 project can still answer), but its cold-miss lead is rewritten to the Ghidra
2305+
one so the message always matches the active backend.
2306+
2307+
In researcher-bridge mode (`_ghidra_bridge_only`) the query can't be served at all (decompile-only,
2308+
no slot re_analyze could build), so it returns the honest "switch to headless" message up front —
2309+
no futile headless attempt, no radare2 cold miss."""
2310+
if _ghidra_bridge_only():
2311+
return None, _GHIDRA_BRIDGE_ONLY_XREF_MSG
2312+
ghidra = _ghidra_backend_enabled()
2313+
if ghidra:
22172314
out = _ghidra_xrefs(ctx, mode, subject)
22182315
if out is not None:
22192316
return out, None
@@ -2229,11 +2326,12 @@ def _run_xrefs_probe(ctx: ToolContext, subject: str | None, mode: str):
22292326
project_mount=_r2_project_mount(ctx))
22302327
except Exception as exc: # noqa: BLE001
22312328
return None, f"{mode} xrefs failed: {exc}"
2232-
# A warm-MISS payload carries the re_analyze lead (a cold index mode, e.g. reached with the
2233-
# run_tool gate reporting `unavailable`). Surface it as the error so EVERY caller (function/data
2234-
# xrefs, call graph) points at re_analyze instead of formatting an empty result.
2329+
# A warm-MISS payload carries the re_analyze lead (a cold index mode). Surface it as the error so
2330+
# EVERY caller (function/data xrefs, call graph) points at re_analyze instead of formatting an
2331+
# empty result — but rewrite it to the Ghidra-named lead when Ghidra is the active backend (the r2
2332+
# probe's own lead names radare2, the wrong engine for a Ghidra user).
22352333
if isinstance(out, dict) and out.get("needs_analysis") and out.get("error"):
2236-
return None, out["error"]
2334+
return None, (_GHIDRA_XREF_LEAD if ghidra else out["error"])
22372335
return out, None
22382336

22392337

@@ -2304,6 +2402,22 @@ def _function_xrefs(ctx: ToolContext, function: str) -> str:
23042402
return ctx.cache[key]
23052403

23062404

2405+
def _no_data_xrefs_msg(subject: str) -> str:
2406+
"""The 'no references' message for data_xrefs, tuned to what `subject` was. A hex address that
2407+
resolved to nothing is a genuine empty result; a subject that did NOT resolve to an address (a bare
2408+
name the index doesn't know — most often a string VALUE mistakenly passed here, e.g. a URL/path)
2409+
gets pointed at the resolve-the-string's-address workflow, since data_xrefs keys on an ADDRESS."""
2410+
s = (subject or "").strip()
2411+
if re.fullmatch(r"0x[0-9a-fA-F]+", s):
2412+
return (f"no references to {subject} found — nothing in the analyzed image points at this "
2413+
f"address (or it isn't a mapped address).")
2414+
return (f"{subject!r} did not resolve to an address, so data_xrefs found nothing. This tool keys on "
2415+
f"a hex ADDRESS (or a symbol that resolves to one), not a string VALUE. To find who "
2416+
f"references a string like this, first get its address — re_list_strings to locate it (each "
2417+
f"row carries the address), or re_resolve for a symbol — then re_data_xrefs on that 0x… "
2418+
f"address.")
2419+
2420+
23072421
def _data_xrefs(ctx: ToolContext, address: str) -> str:
23082422
"""Every code/data/string reference TO an address. QUERY."""
23092423
key = f"dxrefs:{address}"
@@ -2313,7 +2427,7 @@ def _data_xrefs(ctx: ToolContext, address: str) -> str:
23132427
if err:
23142428
return err
23152429
if out.get("error") or out.get("not_found"):
2316-
return f"no resolvable references to {address!r}"
2430+
return _no_data_xrefs_msg(address)
23172431
refs = out.get("data_refs") or []
23182432
_record_obs(ctx, tool="data_xrefs", args={"address": address},
23192433
result_kind="data_xrefs", payload=out,
@@ -2590,9 +2704,13 @@ def _search_code_scan(ctx: ToolContext, args: dict, *, bytes_pat, immediate) ->
25902704
return "search_code byte/immediate scan unavailable (Docker/sandbox not running)"
25912705
subj = f"bytes {bytes_pat!r}" if bytes_pat else f"immediate {immediate!r}"
25922706

2593-
# Warm Ghidra first (reuses the warm project / a live bridge); r2 raw scan otherwise.
2707+
# Warm Ghidra first whenever Ghidra is the backend in ANY mode (reuses the warm project / a live
2708+
# managed bridge) — same gate as the xref family, so a bridge-mode user isn't wrongly forced onto
2709+
# the cold path here. Unlike xrefs, the r2 fallback is a RAW byte scan that needs no warm analysis,
2710+
# so researcher-bridge mode (where warm Ghidra search returns an error without spawning) still gets
2711+
# real results from it rather than a "switch to headless" refusal.
25942712
out = _ghidra_search(ctx, bytes_pat=bytes_pat, immediate=immediate) \
2595-
if _ghidra_xrefs_active() else None
2713+
if _ghidra_backend_enabled() else None
25962714
if out is None:
25972715
out = _r2_search(ctx, bytes_pat=bytes_pat, immediate=immediate)
25982716
if isinstance(out, str): # a string error from the r2 fallback

0 commit comments

Comments
 (0)