Skip to content

Fix race condition in ExpireAfter when items are removed before expiration fires#1076

Open
dwcullop wants to merge 1 commit intoreactivemarbles:mainfrom
dwcullop:bugfix/expire-after-race
Open

Fix race condition in ExpireAfter when items are removed before expiration fires#1076
dwcullop wants to merge 1 commit intoreactivemarbles:mainfrom
dwcullop:bugfix/expire-after-race

Conversation

@dwcullop
Copy link
Copy Markdown
Member

Description

Fixes a race condition in ExpireAfter.ForSource where the expiration timer callback assumes the item still exists in the source cache. When an item is removed or updated by another thread between when the expiration was scheduled and when it fires, the callback would throw InvalidOperationException from Lookup().Value on a missing key.

Root Cause

The timer callback in OnEditingSource unconditionally called:

_removedItemsBuffer.Add(new(key: proposedExpiration.Key,
    value: updater.Lookup(proposedExpiration.Key).Value));  // throws if missing!
updater.RemoveKey(proposedExpiration.Key);

Fix

  1. Check item existence before removal Lookup() is checked for HasValue before accessing .Value
  2. Check expiration still applies _timeSelector.Invoke(lookup.Value) is not null verifies the item still has a pending expiration
  3. Fix expiration accumulation haveExpirationsChanged |= TrySetExpiration(...) uses |= instead of = to correctly accumulate changes across multiple items in a changeset (previously, only the last item's result was kept)

Impact

This race was reproducible under moderate concurrent load. A stress test calling AddOrUpdate and RemoveKey on a SourceCache with ExpireAfter from multiple threads would intermittently throw. The fix is minimal and surgical: two guard checks and one operator change.

…tion fires

The expiration timer callback now checks that the item still exists and
still has a pending expiration before attempting removal. Prevents
InvalidOperationException when items are concurrently removed or updated
between when the expiration was scheduled and when it fires.

Also uses |= to accumulate expiration changes correctly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a concurrency edge case in ExpireAfter.ForSource where an expiration-management callback could assume an item still exists in the source cache, leading to exceptions under concurrent remove/update activity.

Changes:

  • Guard expiration removals by verifying the key still exists in the source before dereferencing Lookup().Value.
  • Add an additional “still expiring” check before removing an item during expiration management.
  • Fix expiration-change aggregation by using |= when accumulating TrySetExpiration(...) results across a changeset.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +168 to +175
if (lookup.HasValue && _timeSelector.Invoke(lookup.Value) is not null)
{
_removedItemsBuffer.Add(new(
key: proposedExpiration.Key,
value: lookup.Value));

updater.RemoveKey(proposedExpiration.Key);
}
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_timeSelector is user-provided and can throw. Calling _timeSelector.Invoke(lookup.Value) inside the scheduled expiration-management path (OnEditingSource via ManageExpirations) is currently unguarded (no try/catch here or around _source.Edit(_onEditingSource)), so an exception would escape the scheduler callback instead of being propagated via _observer.OnError (as OnSourceNext does). Consider either avoiding timeSelector evaluation here (rely on internal expiration state), or wrapping the invocation/removal block to cancel further scheduling and forward the error to _observer.OnError consistently.

Suggested change
if (lookup.HasValue && _timeSelector.Invoke(lookup.Value) is not null)
{
_removedItemsBuffer.Add(new(
key: proposedExpiration.Key,
value: lookup.Value));
updater.RemoveKey(proposedExpiration.Key);
}
try
{
if (lookup.HasValue && _timeSelector.Invoke(lookup.Value) is not null)
{
_removedItemsBuffer.Add(new(
key: proposedExpiration.Key,
value: lookup.Value));
updater.RemoveKey(proposedExpiration.Key);
}
}
catch (Exception error)
{
_observer.OnError(error);
return;
}

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants