Optimize ImmutableSortedSet<T>.SetEquals to avoid unnecessary allocations

[aw0lid]

Summary

ImmutableSortedSet<T>.SetEquals always creates a new intermediate SortedSet<T> for the other collection, leading to avoidable allocations and GC pressure, especially for large datasets

Optimization Logic

• Type-Specific Fast Paths: Uses pattern matching to detect if other is an ImmutableSortedSet<T> or SortedSet<T>, triggering optimized logic only if their Comparer matches.
• O(1) Early Exit: Performs an immediate ReferenceEquals check and leverages ICollection<T> and IReadOnlyCollection<T> to return false early if other.Count is less than this.Count.
• Sequential Lock-Step Comparison: Replaces the $O(\log n)$ per-element .Contains() check with a dual-enumerator while loop. This leverages the sorted nature of both sets to achieve $O(n)$ linear complexity.
• Zero Allocation Path: For compatible sorted sets, the comparison is performed directly on existing instances, eliminating the memory overhead of temporary collections.
• Refined Fallback: Even when a new SortedSet<T> is required for general IEnumerable types, the final comparison now uses the same efficient $O(n)$ sequential scan instead of repeated lookups.

Click to expand Benchmark Source Code

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;

[MemoryDiagnoser]
[Orderer(BenchmarkDotNet.Order.SummaryOrderPolicy.FastestToSlowest)]
public class ImmutableSortedSetEqualsBenchmark
{
    private ImmutableSortedSet<int> _sourceSet = null!;
    private ImmutableSortedSet<int> _sameReference = null!;
    private ImmutableSortedSet<int> _immutableSortedSetMatch = null!;
    private ImmutableSortedSet<int> _immutableSortedSetDiffComparer = null!;
    private HashSet<int> _hashSetMatch = null!;
    private SortedSet<int> _sortedSetMatch = null!;
    private SortedSet<int> _sortedSetDifferentCount = null!;
    private SortedSet<int> _sortedSetDifferentElements = null!;
    private List<int> _listMatch = null!;
    private int[] _arrayMatch = null!;
    private List<int> _listWithDuplicates = null!;
    private ImmutableSortedSet<int> _emptySet = null!;

    [Params(100, 100000)]
    public int Size;

    [GlobalSetup]
    public void Setup()
    {
        var elements = Enumerable.Range(0, Size).ToList();
        _sourceSet = ImmutableSortedSet.CreateRange(elements);
        _sameReference = _sourceSet;
        _immutableSortedSetMatch = ImmutableSortedSet.CreateRange(elements);
        _hashSetMatch = new HashSet<int>(elements);
        _sortedSetMatch = new SortedSet<int>(elements);
        _sortedSetDifferentCount = new SortedSet<int>(elements.Take(Size / 2));
        _sortedSetDifferentElements = new SortedSet<int>(Enumerable.Range(Size, Size));
        _immutableSortedSetDiffComparer = ImmutableSortedSet.CreateRange(new CustomComparer(), elements);
        _listMatch = elements;
        _arrayMatch = elements.ToArray();
        _listWithDuplicates = elements.Concat(elements.Take(10)).ToList(); 
        _emptySet = ImmutableSortedSet<int>.Empty;
    }



    [Benchmark(Description = "Identity (Same Ref)")]
    public bool Case01_SameReference() => _sourceSet.SetEquals(_sameReference);

    [Benchmark(Description = "Type: ImmutableSortedSet (Match)")]
    public bool Case02_ImmutableSortedMatch() => _sourceSet.SetEquals(_immutableSortedSetMatch);

    [Benchmark(Description = "Type: SortedSet (Match)")]
    public bool Case03_SortedSetMatch() => _sourceSet.SetEquals(_sortedSetMatch);

    [Benchmark(Description = "Diff Count (O1 Check)")]
    public bool Case04_DifferentCount() => _sourceSet.SetEquals(_sortedSetDifferentCount);

    [Benchmark(Description = "Diff Elements (Scan)")]
    public bool Case05_DifferentElements() => _sourceSet.SetEquals(_sortedSetDifferentElements);

    [Benchmark(Description = "Type: HashSet (Match)")]
    public bool Case06_HashSetMatch() => _sourceSet.SetEquals(_hashSetMatch);

    [Benchmark(Description = "Diff Comparer (Fallback)")]
    public bool Case07_DifferentComparer() => _sourceSet.SetEquals(_immutableSortedSetDiffComparer);

    [Benchmark(Description = "Type: List (Fallback)")]
    public bool Case08_ListMatch() => _sourceSet.SetEquals(_listMatch);

    [Benchmark(Description = "Type: Array (Fallback)")]
    public bool Case09_ArrayMatch() => _sourceSet.SetEquals(_arrayMatch);

    [Benchmark(Description = "Type: List (With Duplicates)")]
    public bool Case10_ListDuplicates() => _sourceSet.SetEquals(_listWithDuplicates);

    [Benchmark(Description = "Empty Set")]
    public bool Case11_EmptySet() => _sourceSet.SetEquals(_emptySet);
}

public class CustomComparer : IComparer<int>
{
    public int Compare(int x, int y)
    {
        return y.CompareTo(x);
    }
}

public class Program
{
    public static void Main(string[] args) => BenchmarkRunner.Run<ImmutableSortedSetEqualsBenchmark>();
}

Click to expand Benchmark Results

Benchmark Results (Before Optimization)

Method	Size	Mean	Error	StdDev	Gen0	Gen1	Gen2	Allocated
'Identity (Same Ref)'	100	1.1184 ns	0.2595 ns	0.7104 ns	-	-	-	-
'Empty Set'	100	13.7398 ns	0.3181 ns	0.3786 ns	0.0306	-	-	48 B
'Diff Count (O1 Check)'	100	979.3076 ns	16.9917 ns	15.0627 ns	1.4629	-	-	2296 B
'Diff Elements (Scan)'	100	2,014.6961 ns	25.8909 ns	24.2184 ns	2.8534	-	-	4480 B
'Type: SortedSet (Match)'	100	3,891.6668 ns	54.9797 ns	48.7380 ns	2.8534	-	-	4480 B
'Type: List (Fallback)'	100	4,105.7005 ns	52.5954 ns	46.6244 ns	2.9449	-	-	4624 B
'Type: HashSet (Match)'	100	4,137.2090 ns	51.9803 ns	43.4059 ns	2.9449	-	-	4624 B
'Type: Array (Fallback)'	100	4,155.9947 ns	68.5157 ns	60.7374 ns	2.9449	-	-	4624 B
'Type: List (With Duplicates)'	100	4,468.4364 ns	87.5793 ns	93.7089 ns	2.9678	-	-	4664 B
'Type: ImmutableSortedSet (Match)'	100	7,163.2983 ns	133.9480 ns	227.4540 ns	2.9449	-	-	4624 B
'Diff Comparer (Fallback)'	100	7,252.2769 ns	132.0741 ns	117.0802 ns	2.9449	-	-	4624 B
'Identity (Same Ref)'	100000	0.9957 ns	0.0450 ns	0.0376 ns	-	-	-	-
'Empty Set'	100000	13.3283 ns	0.3092 ns	0.4434 ns	0.0306	-	-	48 B
'Diff Count (O1 Check)'	100000	1,943,135.6939 ns	33,457.9774 ns	29,659.6255 ns	382.8125	277.3438	-	2000618 B
'Diff Elements (Scan)'	100000	4,567,024.2086 ns	57,081.1927 ns	53,393.7831 ns	656.2500	640.6250	-	4000963 B
'Type: Array (Fallback)'	100000	13,327,777.3678 ns	74,136.0679 ns	61,906.9743 ns	671.8750	640.6250	15.6250	4400411 B
'Type: List (Fallback)'	100000	13,534,149.4052 ns	178,140.4990 ns	166,632.7334 ns	687.5000	640.6250	15.6250	4400395 B
'Type: HashSet (Match)'	100000	13,935,060.6116 ns	139,140.5300 ns	123,344.4557 ns	687.5000	640.6250	15.6250	4400509 B
'Type: SortedSet (Match)'	100000	14,638,636.8531 ns	80,417.0718 ns	75,222.1789 ns	671.8750	609.3750	-	4000966 B
'Type: List (With Duplicates)'	100000	15,013,534.3504 ns	201,822.5137 ns	178,910.4016 ns	656.2500	593.7500	-	4400426 B
'Type: ImmutableSortedSet (Match)'	100000	17,013,787.9104 ns	155,637.5145 ns	145,583.4277 ns	687.5000	625.0000	-	4400386 B
'Diff Comparer (Fallback)'	100000	18,090,747.7835 ns	197,699.0992 ns	175,255.1021 ns	718.7500	625.0000	-	4400396 B

---

Benchmark Results (After Optimization)

Method	Size	Mean	Error	StdDev	Gen0	Gen1	Gen2	Allocated
'Identity (Same Ref)'	100	2.543 ns	0.0753 ns	0.0704 ns	-	-	-	-
'Empty Set'	100	3.093 ns	0.0549 ns	0.0487 ns	-	-	-	-
'Diff Count (O1 Check)'	100	3.313 ns	0.0745 ns	0.0582 ns	-	-	-	-
'Diff Elements (Scan)'	100	74.127 ns	0.5865 ns	0.5199 ns	0.0969	-	-	152 B
'Type: SortedSet (Match)'	100	2,060.667 ns	14.6815 ns	12.2597 ns	0.0954	-	-	152 B
'Type: ImmutableSortedSet (Match)'	100	2,261.202 ns	24.3335 ns	21.5710 ns	-	-	-	-
'Type: List (Fallback)'	100	4,198.301 ns	32.2187 ns	26.9041 ns	2.9449	-	-	4624 B
'Type: Array (Fallback)'	100	4,362.325 ns	85.0312 ns	75.3780 ns	2.9449	-	-	4624 B
'Type: HashSet (Match)'	100	4,504.253 ns	52.2721 ns	40.8106 ns	2.9449	-	-	4624 B
'Type: List (With Duplicates)'	100	4,661.714 ns	68.9347 ns	61.1088 ns	2.9678	-	-	4664 B
'Diff Comparer (Fallback)'	100	5,499.184 ns	41.4631 ns	36.7560 ns	2.9449	-	-	4624 B
'Identity (Same Ref)'	100000	1.752 ns	0.0575 ns	0.0538 ns	-	-	-	-
'Empty Set'	100000	3.144 ns	0.0659 ns	0.0584 ns	-	-	-	-
'Diff Count (O1 Check)'	100000	3.682 ns	0.0420 ns	0.0351 ns	-	-	-	-
'Diff Elements (Scan)'	100000	141.683 ns	1.8439 ns	1.7248 ns	0.1988	-	-	312 B
'Type: SortedSet (Match)'	100000	9,030,326.684 ns	48,748.2920 ns	45,599.1826 ns	-	-	-	318 B
'Type: ImmutableSortedSet (Match)'	100000	10,866,865.974 ns	121,176.7182 ns	107,420.0045 ns	-	-	-	-
'Type: List (Fallback)'	100000	13,871,545.590 ns	191,030.7402 ns	178,690.2730 ns	687.5000	625.0000	15.6250	4400390 B
'Type: Array (Fallback)'	100000	13,878,012.071 ns	150,945.4960 ns	141,194.5107 ns	687.5000	640.6250	15.6250	4400419 B
'Type: HashSet (Match)'	100000	14,096,194.868 ns	209,735.4518 ns	185,925.0151 ns	687.5000	625.0000	15.6250	4400444 B
'Type: List (With Duplicates)'	100000	15,361,172.358 ns	152,747.1816 ns	142,879.8085 ns	687.5000	640.6250	15.6250	4400451 B
'Diff Comparer (Fallback)'	100000	16,359,868.051 ns	238,088.5159 ns	211,059.2679 ns	687.5000	593.7500	-	4400396 B

Performance Analysis Summary (100,000 Elements)

🚀 Optimized Direct Paths

• Diff Count (O(1) Check):
  • Execution Time: 527,739x speedup (1,943,135 ns → 3.68 ns).
  • Allocated Memory: 100% reduction (~2 MB → 0 B).
• Diff Elements (Scan):
  • Execution Time: 32,234x speedup (4,567,024 ns → 141.68 ns).
  • Allocated Memory: 99.99% reduction (~4 MB → 312 B).

🛠️ Specialized Type Matching

• Type: SortedSet (Match):
  • Execution Time: 38.3% improvement (14.63 ms → 9.03 ms).
  • Allocated Memory: 99.99% reduction (~4 MB → 318 B).
• Type: ImmutableSortedSet (Match):
  • Execution Time: 36.1% improvement (17.01 ms → 10.86 ms).
  • Allocated Memory: 100% reduction (4.4 MB → 0 B).

⚖️ Fallback & General Scenarios

• Type: HashSet, Array, & List:
  • Execution Time: Steady
  • Allocated Memory: Stable at ~4.4 MB.
• Diff Comparer (Fallback):
  • Execution Time: 9.5% improvement (18.09 ms → 16.35 ms).
  • Allocated Memory: Stable at ~4.4 MB.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-collections
See info in area-owners.md if you want to be subscribed.

[aw0lid]

cc/ @stephentoub

[aw0lid]

I noticed that methods like IsSubsetOf , IsProperSubsetOf can be optimized using the same pattern. Should I include them in this PR for consistency, or open separate PRs?
CC/ @stephentoub @rosebyte