Skip to content

Semi-Sparse Abstract Interpretation (#1803 rebased on #1804)#1806

Open
bjjwwang wants to merge 27 commits into
SVF-tools:masterfrom
bjjwwang:semi-sparse2
Open

Semi-Sparse Abstract Interpretation (#1803 rebased on #1804)#1806
bjjwwang wants to merge 27 commits into
SVF-tools:masterfrom
bjjwwang:semi-sparse2

Conversation

@bjjwwang

Copy link
Copy Markdown
Contributor

Summary

Rebased version of #1803 on top of #1804, with further simplification.

Adds a new -semi-sparse mode for abstract interpretation that treats top-level SSA variables (ValVars) sparsely and address-taken variables (ObjVars) densely. Since ValVars in LLVM IR are in SSA form (each defined exactly once), carrying them through every ICFG node is redundant — we can always fetch them from their unique definition site on demand. ObjVars are not SSA and still require dense propagation.

This reduces the total abstract state size by 70.6% (97.7% reduction in ValVar entries) across 348/348 passing tests, with no changes to the default (dense) mode behavior.


Changes from #1803

  • Rebased onto Set correct ICFGNode for ValVar subclasses during SVFIR construction #1804: leverages the corrected ICFGNode assignments for ArgValVar, RetValPN, VarArgValPN, GlobalValVar, and GepValVar, eliminating most "orphan" ValVars.
  • Removed orphanVarDefMap entirely: the only remaining def-site mismatch (call-result ValVars whose getICFGNode() returns CallICFGNode but whose value is written at RetICFGNode) is now resolved inline via CallICFGNode::getRetICFGNode() — no pre-built map needed.
  • Removed PreAnalysis.h changes: PreAnalysis no longer has any semi-sparse-specific code.
  • Fixed 3 previously failing tests: replaced set_value external API calls with direct assignments / svf_assert range checks (PR to Test-Suite). All 348 tests now pass.

Key Design

Hybrid merge strategy in mergeStatesFromPredecessors

Edge Type Strategy Notes
IntraCFGEdge joinAddrOnly Only merge _addrToAbsVal (ObjVars). ValVars are pulled on demand from their SSA definition sites.
CallCFGEdge / RetCFGEdge joinWith Full merge to correctly join formal parameters and return values in context-insensitive analysis (e.g., f(2); f(3) → formal param must be [2,3]).

On-demand ValVar lookup (getAbstractValue)

  • Constants: computed directly, no state lookup.
  • Non-constants: fetched from abstractTrace[defNode] where defNode = var->getICFGNode().
  • Call-result ValVars: falls back to CallICFGNode::getRetICFGNode() to find the value written by RetPE.

Sparse state construction (buildSparseState)

  • Before processing each node, collectNeededVarIds identifies which ValVars are needed as operands, then pulls them from def-sites.
  • pullBranchConditionVars ensures branch feasibility checks have the condition variables and load-chain pointers available.

Files Changed

File Changes
Options.h/cpp New -semi-sparse CLI option
AbstractState.h/cpp Add joinAddrOnly method
AbstractInterpretation.h/cpp Core semi-sparse logic: getAbstractValue, buildSparseState, collectNeededVarIds, pullBranchConditionVars, hybrid merge
PreAnalysis.cpp +1 line: #include for CallICFGNode (no semi-sparse-specific logic)

Test Results

  • Semi-sparse mode: 348/348 pass (identical results to dense mode)
  • Dense mode: 348/348 pass (zero regressions)
  • Combined (dense + semi-sparse) via Test-Suite semi-sparse branch: 695/695 pass

State Size Reduction

Measured across 281 comparable test invocations:

Metric Dense Semi-Sparse Reduction
VarEntries 5,507,006 126,480 97.7%
AddrEntries 2,038,704 2,089,102 -2.5%
Total 7,545,710 2,215,582 70.6%

Per-test: min 38.5%, median 79.4%, mean 78.2%. All comparable tests show reduction.


Test Plan

  • ctest -R ae: 348/348 pass in default (dense) mode
  • ctest -R semi_sparse: 347/347 pass in semi-sparse mode
  • 3 previously failing set_value tests fixed in Test-Suite PR

bjjwwang and others added 5 commits March 23, 2026 21:15
ObjVars propagate densely via joinWith; ValVars are pulled on-demand
from their unique def-sites via buildSparseState/getAbstractValue.

Key changes:
- New -semi-sparse CLI option (Options::SemiSparse)
- PreAnalysis::buildOrphanVarDefMap for ValVars with null getICFGNode
- getAbstractValue returns by value, handles constants and orphan vars
- collectNeededVarIds + buildSparseState pull ValVars before execution
- pullBranchConditionVars ensures branch feasibility in sparse mode
- updateStateOnPhi uses getAbstractValue for phi operands

All 348 ae tests pass in both dense and semi-sparse modes.

Made-with: Cursor
…e strategy

Use joinAddrOnly for intra-procedural edges (only merge ObjVars) and
joinWith for inter-procedural edges (preserve formal params/returns).

Key changes:
- Add AbstractState::joinAddrOnly to merge only _addrToAbsVal
- Split mergeStatesFromPredecessors into intra/inter edge handling
- Fix getAbstractValue to use varMap.count() for bottom value detection
- Fix PreAnalysis::buildOrphanVarDefMap to always map RetPE vars

Results: 345/348 tests pass, 97.7% VarEntry reduction, 70.6% total
state size reduction. 3 set_value-related tests remain unsupported.

Made-with: Cursor
After PR SVF-tools#1804 fixed ICFGNode assignments for ValVar subclasses,
the only remaining mismatch is call-result ValVars (RetPE LHS)
whose getICFGNode() returns CallICFGNode but whose value is
written at RetICFGNode. Use CallICFGNode::getRetICFGNode()
directly instead of pre-building a map.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move the @@STATE_SIZE@@ measurement snippet to semi-sparse.md
as documentation for future benchmarking.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@bjjwwang bjjwwang changed the title # Semi-Sparse Abstract Interpretation (#1803 rebased on #1804) Semi-Sparse Abstract Interpretation (#1803 rebased on #1804) Mar 23, 2026
Comment thread svf/lib/Util/Options.cpp Outdated

const Option<bool> Options::SparseAE(
"sparse-ae", "Enable sparse abstract execution via SVFG", false);
const Option<bool> Options::SemiSparse(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to make a strong option for all non-sparse, semi and sparse settings as one option

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to make a strong option for all non-sparse, semi and sparse settings as one option

OK. got it.

@codecov

codecov Bot commented Mar 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 13.20755% with 138 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.52%. Comparing base (402a3fc) to head (bff3244).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
svf/lib/AE/Svfexe/AbstractInterpretation.cpp 13.63% 133 Missing ⚠️
svf/lib/AE/Core/AbstractState.cpp 0.00% 5 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1806      +/-   ##
==========================================
- Coverage   63.84%   63.52%   -0.33%     
==========================================
  Files         247      247              
  Lines       24858    25002     +144     
  Branches     4673     4721      +48     
==========================================
+ Hits        15871    15882      +11     
- Misses       8987     9120     +133     
Files with missing lines Coverage Δ
svf/include/AE/Core/AbstractState.h 89.47% <ø> (ø)
svf/include/AE/Svfexe/AbstractInterpretation.h 100.00% <ø> (ø)
svf/lib/AE/Svfexe/PreAnalysis.cpp 51.02% <ø> (ø)
svf/lib/AE/Core/AbstractState.cpp 61.68% <0.00%> (-1.48%) ⬇️
svf/lib/AE/Svfexe/AbstractInterpretation.cpp 61.41% <13.63%> (-11.27%) ⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

bjjwwang and others added 22 commits March 24, 2026 16:28
… AbstractState

- Reorder getAbstractValue/updateAbstractValue params to (var, node) consistently
- Add AESparsity option to group dense/semi-sparse/sparse modes
- AbsExtAPI now holds AbstractInterpretation& instead of raw abstractTrace reference
- AEDetector::detect takes AbstractInterpretation& instead of AbstractState&
- Add override to AEDetector subclass virtual methods

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Decompose loadValue/storeValue into explicit ValVar lookup + ObjVar load/store
- Replace all as[id] in updateStateOnLoad/Store/Copy/Gep/Select/Phi/Call/Ret/Addr/Binary/Cmp
- Replace all as[id] in AbsExtAPI ext function handlers and SSE_FUNC_PROCESS macro
- Replace as[id] in AEDetector detectExtAPI and handleStubFunctions
- Remaining as[id] only in: API implementations, low-level state utilities (canSafelyAccessMemory, canSafelyDerefPtr, getStrlen)
- Update compatible_dense_semisparse.md with design documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add ICFGNode* node parameter to getStrlen, canSafelyAccessMemory, canSafelyDerefPtr
- Replace as[value_id] with ae.getAbstractValue(var, node) in all three functions
- Update all call sites to pass the node context
- AbsExtAPI.cpp and AEDetector.cpp now have zero as[idx] bracket accesses

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…dling

- Dense mode: read ValVar from current node's state (not def-site)
- Semi-sparse mode: pull ValVar from def-site (unchanged)
- Fix SVFVar* dispatch: check ObjVar before ValVar (ObjVar inherits ValVar)
- Restore direct as[id] for AddrStmt RHS (ObjVar virtual address in _varToVal)
- Fixes 97% of test regressions (345/347 semi-sparse, 344/347 dense pass)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…in Binary/Cmp

Uninitialized ValVars (e.g., uninitialized local variables, function params)
default to bottom when read via as[id]. The old code had explicit guards in
updateStateOnBinary and updateStateOnCmp to set them to top (conservative).
This prevents isBranchFeasible from falsely pruning reachable branches.

Dense: 347/347 pass. Semi-sparse: 345/347 (2 pre-existing).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…irst

In semi-sparse mode, buildSparseState and defensive top initialization
write into the current node's state. getAbstractValue must check the
current state before falling back to def-site lookup, otherwise those
writes are invisible.

Dense: 347/347 pass. Semi-sparse: 347/347 pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…retation

- Remove buildSparseState/collectNeededVarIds/addIfValVar — getAbstractValue
  handles on-demand ValVar lookup from def-sites
- Add getGepElementIndex, getGepByteOffset, getGepObjAddrs to
  AbstractInterpretation, replacing AbstractState methods that used (*this)[id]
- Add predNode parameter to isBranchFeasible/isCmpBranchFeasible/
  isSwitchBranchFeasible, use getAbstractValue to initialize missing ValVars
- Binary/Cmp: treat bottom operands as top locally (no state mutation)
- Update AbsExtAPI handleMemcpy/handleMemset/strRead/getStrlen signatures
  to accept ICFGNode* for getGepObjAddrs calls

Dense: 346/347. Semi-sparse: 341/347.
Remaining failures: uninitialized function parameters (argc etc.)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
getPointeeElement reads pointer value from state via inVarToAddrsTable,
but semi-sparse mode doesn't merge ValVars into the current node's state.
Fix: pull pointer value via ae.getAbstractValue before calling getPointeeElement,
and add null check for safety.

Dense: 346/347. Semi-sparse: 347/347 (100%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dense mode previously used as[id] (map::operator[]) which silently inserts
a default-constructed bottom entry. This bottom has no analysis semantics —
it's just a C++ side-effect — but downstream code (isBranchFeasible narrow)
interprets it as "path infeasible", falsely pruning reachable branches.

Fix: check inVarToValTable/inVarToAddrsTable first. If absent, fall through
to the final top fallback. This correctly models "uninitialized = unknown".

Dense: 347/347 (100%). Semi-sparse: 347/347 (100%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…sFromPredecessors

- updateStateOnPhi: unified phi operand read via getAbstractValue(opVar, opICFGNode)
  instead of separate dense/semi-sparse paths with tmpEs[curId]
- pullBranchConditionVars: remove semiSparse guard — dense mode safely no-ops
  when vars are already in state
- Only two sparsity checks remain: getAbstractValue (lookup strategy) and
  mergeStatesFromPredecessors (joinAddrOnly vs joinWith)

Dense: 347/347 (100%). Semi-sparse: 347/347 (100%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
pullValVarIntoState is no longer needed — isCmpBranchFeasible now pulls
ValVars via getAbstractValue directly.

pullLoadChainPointerIntoState is still needed: isCmpBranchFeasible narrows
ObjVars through load pointers (e.g., narrowing *%a.addr when %0 = load %a.addr
is refined). It needs the pointer's AddressValue in the state to find
which ObjVar to narrow. Inlined the pull logic as a local lambda.

pullBranchConditionVars now only pulls load-chain pointers, not ValVars.

Dense: 347/347 (100%). Semi-sparse: 347/347 (100%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document getAbstractValue/updateAbstractValue (dense vs semi-sparse
strategy, input/output, when top is returned), GEP helpers
(getGepElementIndex, getGepByteOffset, getGepObjAddrs — why they were
lifted from AbstractState), and branch condition helpers
(pullBranchConditionVars, pullLoadChainPointerIntoState — the load-chain
pointer pulling for ObjVar narrowing in isCmpBranchFeasible).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…Only

joinWith now checks Options::AESparsity internally — in semi-sparse mode
it skips ValVar (_varToAbsVal) merge since ValVars are pulled on demand
from def-sites via getAbstractValue.

This removes the last semiSparse check from mergeStatesFromPredecessors.
Now only getAbstractValue(ValVar*) checks AESparsity (lookup strategy).
joinWith is the single merge API for both modes.

Dense: 347/347 (100%). Semi-sparse: 347/347 (100%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…State

These functions pre-pulled load-chain pointers (e.g., %ptr from
`%x = load i32, i32* %ptr`) into predecessor state copies for
isCmpBranchFeasible's ObjVar narrowing.

This is unnecessary: pointer AddressValues originate from AddrStmt and
are propagated via joinWith to all reachable states. ObjVars are always
dense in both modes, so the predecessor state copy already contains all
needed pointer addresses and ObjVar contents.

Dense: 347/347 (100%). Semi-sparse: 347/347 (100%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove 7 functions from AbstractState that internally used (*this)[id]
to read ValVars, which doesn't work in semi-sparse mode:

Deleted (already have AbstractInterpretation equivalents):
  - getGepObjAddrs, getElementIndex, getByteOffset, loadValue, storeValue

Lifted to AbstractInterpretation (now use getAbstractValue):
  - getPointeeElement(SVFVar*, ICFGNode*) — resolves pointer to pointee type
  - getAllocaInstByteSize(AddrStmt*, ICFGNode*) — reads VLA size variables

Also updated:
  - AbsExtAPI::getElementSize: removed AbstractState& param, uses ICFGNode*
  - ae.cpp unit test: inlined load/store logic

AbstractState is now a pure data container: operator[], load/store (ObjVar),
initObjVar, join/meet/widen/narrow, print.

Dense: 347/347 (100%). Semi-sparse: 347/347 (100%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New class AbstractStateManager owns abstractTrace and all state access APIs:
  - getAbstractValue / updateAbstractValue (all overloads)
  - getAbstractState / hasAbstractState
  - getGepElementIndex / getGepByteOffset / getGepObjAddrs
  - getPointeeElement / getAllocaInstByteSize

AbstractInterpretation now owns AbstractStateManager* svfStateMgr and
delegates through inline wrappers. All abstractTrace references in
AbstractInterpretation.cpp replaced with svfStateMgr->getTrace().

Separation of concerns:
  - AbstractStateManager: state storage + value access + GEP/type helpers
  - AbstractInterpretation: analysis engine (WTO, fixpoint, statements)
  - AbstractState: pure data container (operator[], load/store, join/meet)

Dense: 347/347 (100%). Semi-sparse: 347/347 (100%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove all inline wrapper methods from AbstractInterpretation
  (getAbstractValue, updateAbstractValue, getAbstractState, etc.)
- AbsExtAPI now holds AbstractStateManager* instead of AbstractInterpretation&
- AEDetector::detect takes AbstractStateManager& instead of AbstractInterpretation&
- AbstractInterpretation.cpp: all state calls go through svfStateMgr->
- AEDetector.cpp: handleStubFunctions uses getAEInstance().getStateMgr()
  for state access, getAEInstance().getUtils() for checkpoints

Dense: 347/347 (100%). Semi-sparse: 347/347 (100%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Combines ValVar pointer lookup (sparsity-aware via getAbstractValue)
with ObjVar load/store (always dense) into a single API:
  - loadValue(pointer, node): resolve address set + load from ObjVars
  - storeValue(pointer, val, node): resolve address set + store to ObjVars

updateStateOnLoad/Store now delegate to these in one line each.

Dense: 347/347 (100%). Semi-sparse: 347/347 (100%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ar to ValVar

- Remove AbstractStateManager& param from AEDetector::detect() and all
  internal helpers; detectors now fetch the manager via
  AbstractInterpretation::getAEInstance().getStateMgr()
- Remove AbstractState& param from AbsExtAPI::strRead, getStrlen,
  handleMemcpy, handleMemset; state is retrieved internally from the node
- Narrow SVFVar* to ValVar* in getStrlen, getElementSize, strRead,
  handleMemcpy, handleMemset (callers always pass getArgument() results)
- Remove dead dyn_cast<ObjVar>(ValVar*) branches (ObjVar and ValVar are
  siblings under SVFVar; cast always yields null)
- Remove ICFGNode* node param from getElementSize (was unused after
  dead code removal)
- Move getUseSitesOfObjVar, getUseSitesOfValVar, getDefSiteOfValVar,
  getDefSiteOfObjVar from PreAnalysis to AbstractStateManager
- Move SVFG ownership from PreAnalysis to AbstractStateManager;
  AbstractStateManager builds SVFG itself (only when AESparsity==Sparse)
  using the AndersenWaveDiff* passed to its constructor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
if (const AddrStmt* addrStmt = SVFUtil::dyn_cast<AddrStmt>(stmt2))
{
size = as.getAllocaInstByteSize(addrStmt);
size = AbstractInterpretation::getAEInstance().getStateMgr()->getAllocaInstByteSize(addrStmt);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have the statemgr as a field of the AEDetector class?

{
// void SAFE_BUFACCESS(void* data, int size);
AbstractInterpretation::getAEInstance().getUtils()->checkpoints.erase(callNode);
AbstractInterpretation& aeInst = AbstractInterpretation::getAEInstance();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a AI instance in the AEDetector?

// an unknown but valid memory location. This allows the analysis to continue
// while being conservatively sound.
if (!as[value_id].isAddr())
AbstractInterpretation& aeInst = AbstractInterpretation::getAEInstance();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

AbstractInterpretation::getAEInstance() has been invoked too frequently.

if (const AddrStmt* addrStmt = SVFUtil::dyn_cast<AddrStmt>(stmt2))
{
size = as.getAllocaInstByteSize(addrStmt);
size = AbstractInterpretation::getAEInstance().getStateMgr()->getAllocaInstByteSize(addrStmt);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

Comment on lines +199 to 201
std::vector<AbstractState> intraWorkList;
std::vector<AbstractState> interWorkList;
for (auto& edge : node->getInEdges())

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need two worklists here?


bool AbstractInterpretation::isCmpBranchFeasible(const CmpStmt* cmpStmt, s64_t succ,
AbstractState& as)
AbstractState& as, const ICFGNode* predNode)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need an ICFGEdge argument here, no need to have as and preNode?


bool AbstractInterpretation::isSwitchBranchFeasible(const SVFVar* var, s64_t succ,
AbstractState& as)
AbstractState& as, const ICFGNode* predNode)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

}

bool AbstractInterpretation::isBranchFeasible(const IntraCFGEdge* intraEdge,
AbstractState& as)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

if (hasAbstractState(globalNode))
abstractTrace[node] = abstractTrace[globalNode];
if (svfStateMgr->hasAbstractState(globalNode))
svfStateMgr->getTrace()[node] = svfStateMgr->getTrace()[globalNode];

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to get ride of getTrace()[] to make code more readable and aligned with other AE/AS files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants