Skip to content

Invalidate stale rule caches missing bytes_prefix_index#3116

Open
itxsamad1 wants to merge 2 commits into
mandiant:masterfrom
itxsamad1:fix/stale-cache-invalidation
Open

Invalidate stale rule caches missing bytes_prefix_index#3116
itxsamad1 wants to merge 2 commits into
mandiant:masterfrom
itxsamad1:fix/stale-cache-invalidation

Conversation

@itxsamad1

Copy link
Copy Markdown

Summary

Fixes runtime crashes when loading stale rule caches whose pickled _RuleFeatureIndex objects predate the bytes_prefix_index field (#2961).

  • Bump on-disk cache format version from v1 to v2 (invalidates old cache files at load time)
  • Validate cached rulesets after unpickling and treat outdated schemas as cache misses
  • Broaden load error handling to delete corrupt cache files instead of crashing later during matching

Fixes #2961

Test plan

  • pytest tests/test_rule_cache.py (5 passed)
  • Added test_ruleset_cache_rejects_outdated_schema simulating pre-bytes_prefix_index index objects

Bump cache format to v2 and reject cached rulesets whose feature index schema is outdated, instead of crashing at match time with AttributeError.

Fixes mandiant#2961

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add bug fixes, new features, breaking changes and anything else you think is worthwhile mentioning to the master (unreleased) section of CHANGELOG.md. If no CHANGELOG update is needed add the following to the PR description: [x] No CHANGELOG update needed

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the rule cache mechanism to handle outdated internal schemas by incrementing the cache version, expanding exception handling during cache loading, and introducing a validation check to ensure the cached ruleset contains the required attributes. Feedback suggests safely accessing the _feature_indexes_by_scopes attribute using getattr to prevent potential AttributeError crashes, as this check is performed outside the main try-except block.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread capa/rules/cache.py Outdated
Comment on lines +86 to +88
for feature_index in ruleset._feature_indexes_by_scopes.values():
if not hasattr(feature_index, "bytes_prefix_index"):
return False

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If the unpickled ruleset is from an older cache version or is corrupted, it might not have the _feature_indexes_by_scopes attribute. Accessing it directly will raise an AttributeError and crash the application, as this call is outside the try-except block in load_cached_ruleset. Using getattr to safely retrieve the attribute prevents potential crashes.

Suggested change
for feature_index in ruleset._feature_indexes_by_scopes.values():
if not hasattr(feature_index, "bytes_prefix_index"):
return False
feature_indexes = getattr(ruleset, "_feature_indexes_by_scopes", None)
if feature_indexes is None:
return False
for feature_index in feature_indexes.values():
if not hasattr(feature_index, "bytes_prefix_index"):
return False

Avoid AttributeError when unpickled rulesets lack _feature_indexes_by_scopes.
@williballenthin

Copy link
Copy Markdown
Collaborator

@itxsamad1 would you please explain the sequence of events that led to you encountering this issue? i'm surprised that the caching logic doesn't already notice this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stale cache can crash with _RuleFeatureIndex.bytes_prefix_index AttributeError

2 participants