[Efficiency Improver] perf: eliminate yield-iterator allocations in test execution hot path#8093
Conversation
AwesomeAssertions was bumped from 9.3.0 to 9.4.0 (PR #8008) and is the mandated assertion library for this project's test suites. Adding a glossary entry to explain its purpose and relationship to FluentAssertions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add four new terms identified in the 7-day full scan: - DynamicData: MSTest attribute for data-driven tests (RFC 006; highlighted by PR #7925 log-accumulation bug fix) - Lean–C# Correspondence (FV): new FV artifact introduced by PR #7936 FV docs validation workflow - PropertyBag: core MTP extension API for typed test node metadata (PR #7940 TestNodeProperties refactor brought these types into focus) - TestNode: core MTP class representing a discovered/executed test Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GetRetryAttribute() called GetAttributes<RetryBaseAttribute>() which allocates a compiler-generated state machine (IEnumerator<T> via yield return) on every test method construction. Replace with direct iteration of the cached Attribute[] from GetCustomAttributesCached(). Preserves duplicate-attribute detection and the custom TypeInspectionException. TryExecuteDataSourceBasedTestsAsync() called _testMethodInfo.GetAttributes<DataSourceAttribute>() which allocates a yield iterator state machine AND a DataSourceAttribute[] array. Replace with ReflectHelper.Instance.IsAttributeDefined<DataSourceAttribute>(), which iterates the same cached array without any allocation. Since DataSourceAttribute is sealed and does not allow multiple, presence == exactly one instance. Proxy metric: heap allocation count - GetRetryAttribute: 1 allocation eliminated per TestMethodInfo construction - TryExecuteDataSourceBasedTestsAsync: 2 allocations eliminated per test execution (for tests without a DataSourceAttribute -- the common case) - Estimated: ~480 KB GC pressure reduction per 10,000-test run GSF principle: Hardware Efficiency -- fewer allocations = less DRAM and GC CPU per joule; Energy Proportionality -- GC cost scales with allocation rate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR targets runtime efficiency in the MSTest adapter execution hot path by removing allocation-heavy iterator/array creation when checking for certain attributes, and also expands the project glossary with several new terms.
Changes:
- Updated test execution to avoid allocating
DataSourceAttribute[]in the common “no DataSource” path by switching to an allocation-free attribute presence check. - Updated retry-attribute discovery to iterate the cached
Attribute[]directly instead of using ayield-based iterator. - Expanded
docs/glossary.mdwith new glossary entries (e.g., AwesomeAssertions, DynamicData, PropertyBag, TestNode, FV correspondence terms).
Show a summary per file
| File | Description |
|---|---|
| src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodRunner.cs | Removes per-test allocations by using IsAttributeDefined<DataSourceAttribute> in the DataSource execution decision. |
| src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.cs | Removes iterator allocations by scanning cached attributes directly to find a single RetryBaseAttribute. |
| docs/glossary.md | Adds/updates glossary entries for testing libraries and platform concepts. |
Copilot's findings
- Files reviewed: 3/3 changed files
- Comments generated: 2
| // IsAttributeDefined avoids the array allocation of GetAttributes<DataSourceAttribute>(). | ||
| // DataSourceAttribute is sealed and does not allow multiple, so presence == exactly one. | ||
| if (!ReflectHelper.Instance.IsAttributeDefined<DataSourceAttribute>(_testMethodInfo.MethodInfo)) | ||
| { |
There was a problem hiding this comment.
Addressed in 12d5a07. TryExecuteDataSourceBasedTestsAsync now scans cached attributes directly and executes the DataSource path only when exactly one DataSourceAttribute is present (0 => false, >1 => false), preserving prior semantics without allocation.
Evangelink
left a comment
There was a problem hiding this comment.
Summary
Workflow: Test Expert Reviewer 🧪
Date: 2026-05-11
Repository: microsoft/testfx
Key Findings
This PR contains no test file changes. The two production-code changes are allocation-free refactorings in the test execution hot path:
-
TestMethodInfo.GetRetryAttribute()— replaces a yield-iterator enumeration with a directforeachover the cachedAttribute[]. Semantically equivalent, but the "throw when multipleRetryBaseAttributeare present" guard is not covered by any existing test (confirmed via grep ofTestMethodInfoTests.csandThrowMultipleAttributesException). -
TestMethodRunner.TryExecuteDataSourceBasedTestsAsync()— replacesGetAttributes<DataSourceAttribute>()[Length == 1]withIsAttributeDefined<DataSourceAttribute>(). Safe becauseDataSourceAttributehasAllowMultiple = falseby default; the subtle semantic shift from "exactly one" to "any" is therefore unreachable in practice.
Recommendations
- Add a unit test in
TestMethodInfoTests.csverifying that applying twoRetryBaseAttributesubclass instances to a method causes the expected exception. This would pin the refactored iteration logic against regression. (See inline comment.)
Generated by Test Expert Reviewer
🧪 Test quality reviewed by Test Expert Reviewer 🧪
Evangelink
left a comment
There was a problem hiding this comment.
Summary
Workflow: PR Nitpick Reviewer 🔍
Date: 2026-05-11
Repository: microsoft/testfx
Key Findings
The two code changes are well-motivated and the logic is correct. Two minor comment/naming issues are worth a quick pass:
-
TestMethodRunner.csL182 – comment precision: The phrase "sealed and does not allow multiple" conflatessealed(prevents subclassing) withAllowMultiple = false(prevents multiple attribute instances). Both properties matter but for different reasons, and the current wording could mislead a reader about which one controls what. -
TestMethodInfo.csL293 – variable name leaks implementation detail:cachedAttributesmirrors the helper method name's caching detail. A neutral name (methodAttributes) would be more resilient to future renames of the underlying API.
Positive Highlights
- The guard-clause inversion in
TryExecuteDataSourceBasedTestsAsyncis a clear improvement in readability. - Both explanatory comments accurately describe why the allocation is avoided — helpful for future maintainers.
- The glossary additions are thorough and well-written.
Recommendations
- Clarify the comment on L182 to separate the roles of
sealedandAttributeUsage.AllowMultiple = false. - Rename
cachedAttributes→methodAttributes(orattributes) to avoid coupling the local name to the helper's implementation detail.
🔍 Meticulously inspected by PR Nitpick Reviewer
🔍 Meticulously inspected by PR Nitpick Reviewer 🔍
Evangelink
left a comment
There was a problem hiding this comment.
Summary
Workflow: Expert Code Reviewer 🧠
Date: 2026-05-11
Repository: microsoft/testfx
Key Findings
No correctness, threading, security, or API-compatibility issues found. The two code changes are semantically equivalent to what they replace — the optimizations are sound.
TestMethodInfo.GetRetryAttribute — replaces a yield-iterator chain (GetAttributes<RetryBaseAttribute> → GetEnumerator) with a direct foreach over the already-cached Attribute[]. The filtering (is RetryBaseAttribute), the dual-match detection (ThrowMultipleAttributesException, [DoesNotReturn]), and the null-return for no match are all preserved. Because ThrowMultipleAttributesException never returns, the loop exits on the second match exactly as the old enumerator code did.
TestMethodRunner.TryExecuteDataSourceBasedTestsAsync — replaces GetAttributes<DataSourceAttribute>() (yield iterator + spread-to-array) and a { Length: 1 } check with IsAttributeDefined<DataSourceAttribute>. The comment's claim "presence == exactly one" is verified: DataSourceAttribute is sealed and [AttributeUsage(Inherited = false)] with AllowMultiple = false, so it is physically impossible to have more than one instance in the cached array.
Both paths ultimately route through ReflectHelper.Instance.GetCustomAttributesCached(MethodInfo), so there is no change in which attributes are seen or how inheritance is handled.
Recommendations
None required — the changes are correct and the allocation savings are genuine hot-path wins (these methods run for every test execution).
Generated by Expert Code Reviewer 🧠 🧠
🧠 Reviewed by Expert Code Reviewer 🧠
|
@copilot address review comments |
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| // IsAttributeDefined avoids the array allocation of GetAttributes<DataSourceAttribute>(). | ||
| // DataSourceAttribute is sealed (so subclasses cannot redefine AttributeUsage) | ||
| // and has AllowMultiple = false, so presence == exactly one. | ||
| if (!ReflectHelper.Instance.IsAttributeDefined<DataSourceAttribute>(_testMethodInfo.MethodInfo)) | ||
| { | ||
| await ExecuteTestFromDataSourceAttributeAsync(results).ConfigureAwait(false); | ||
| return true; | ||
| return false; | ||
| } | ||
|
|
||
| return false; | ||
| await ExecuteTestFromDataSourceAttributeAsync(results).ConfigureAwait(false); | ||
| return true; |
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
✨ Enhancement
What does this improve?
This PR reduces allocations in MSTest adapter test-execution hot paths while preserving behavior:
RetryBaseAttributediscovery now iterates cached method attributes directly instead of using ayielditerator path.DataSourceAttributehandling now uses a direct cached-attribute scan that remains allocation-free and preserves prior cardinality semantics (execute only when exactly oneDataSourceAttributeis present).Why is this valuable?
These paths run during test execution, so avoiding iterator/array allocations reduces overhead in common runtime scenarios without changing intended outcomes.
Implementation approach:
ReflectHelper.Instance.GetCustomAttributesCached(...).RetryBaseAttributeerror path (ThrowMultipleAttributesException) to lock in behavior.Testing