Skip to content

[Efficiency Improver] perf: eliminate yield-iterator allocations in test execution hot path#8093

Open
Evangelink wants to merge 7 commits into
mainfrom
efficiency/avoid-yield-iterator-get-retry-attribute-be0151402d39b1b6
Open

[Efficiency Improver] perf: eliminate yield-iterator allocations in test execution hot path#8093
Evangelink wants to merge 7 commits into
mainfrom
efficiency/avoid-yield-iterator-get-retry-attribute-be0151402d39b1b6

Conversation

@Evangelink
Copy link
Copy Markdown
Member

@Evangelink Evangelink commented May 11, 2026

✨ Enhancement

What does this improve?
This PR reduces allocations in MSTest adapter test-execution hot paths while preserving behavior:

  • RetryBaseAttribute discovery now iterates cached method attributes directly instead of using a yield iterator path.
  • DataSourceAttribute handling now uses a direct cached-attribute scan that remains allocation-free and preserves prior cardinality semantics (execute only when exactly one DataSourceAttribute is present).

Why is this valuable?
These paths run during test execution, so avoiding iterator/array allocations reduces overhead in common runtime scenarios without changing intended outcomes.

Implementation approach:

  • Replaced allocation-heavy attribute enumeration patterns with direct iteration over ReflectHelper.Instance.GetCustomAttributesCached(...).
  • Kept/clarified guard behavior for multiple retry attributes.
  • Added unit-test coverage for the multiple-RetryBaseAttribute error path (ThrowMultipleAttributesException) to lock in behavior.
  • Applied minor naming/comment clarity improvements related to the refactor.

Testing

  • Added targeted unit coverage for the retry multi-attribute exception path.
  • Targeted adapter unit test runs were attempted, but restore/test execution is currently blocked in this environment by transient package feed download failures.

github-actions Bot and others added 3 commits May 11, 2026 10:40
AwesomeAssertions was bumped from 9.3.0 to 9.4.0 (PR #8008) and is the
mandated assertion library for this project's test suites. Adding a
glossary entry to explain its purpose and relationship to FluentAssertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add four new terms identified in the 7-day full scan:

- DynamicData: MSTest attribute for data-driven tests (RFC 006;
  highlighted by PR #7925 log-accumulation bug fix)
- Lean–C# Correspondence (FV): new FV artifact introduced by PR #7936
  FV docs validation workflow
- PropertyBag: core MTP extension API for typed test node metadata
  (PR #7940 TestNodeProperties refactor brought these types into focus)
- TestNode: core MTP class representing a discovered/executed test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GetRetryAttribute() called GetAttributes<RetryBaseAttribute>() which allocates
a compiler-generated state machine (IEnumerator<T> via yield return) on every
test method construction. Replace with direct iteration of the cached Attribute[]
from GetCustomAttributesCached(). Preserves duplicate-attribute detection and
the custom TypeInspectionException.

TryExecuteDataSourceBasedTestsAsync() called _testMethodInfo.GetAttributes<DataSourceAttribute>()
which allocates a yield iterator state machine AND a DataSourceAttribute[] array.
Replace with ReflectHelper.Instance.IsAttributeDefined<DataSourceAttribute>(),
which iterates the same cached array without any allocation. Since DataSourceAttribute
is sealed and does not allow multiple, presence == exactly one instance.

Proxy metric: heap allocation count
- GetRetryAttribute: 1 allocation eliminated per TestMethodInfo construction
- TryExecuteDataSourceBasedTestsAsync: 2 allocations eliminated per test execution
  (for tests without a DataSourceAttribute -- the common case)
- Estimated: ~480 KB GC pressure reduction per 10,000-test run

GSF principle: Hardware Efficiency -- fewer allocations = less DRAM and GC CPU
per joule; Energy Proportionality -- GC cost scales with allocation rate.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 11, 2026 09:04
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR targets runtime efficiency in the MSTest adapter execution hot path by removing allocation-heavy iterator/array creation when checking for certain attributes, and also expands the project glossary with several new terms.

Changes:

  • Updated test execution to avoid allocating DataSourceAttribute[] in the common “no DataSource” path by switching to an allocation-free attribute presence check.
  • Updated retry-attribute discovery to iterate the cached Attribute[] directly instead of using a yield-based iterator.
  • Expanded docs/glossary.md with new glossary entries (e.g., AwesomeAssertions, DynamicData, PropertyBag, TestNode, FV correspondence terms).
Show a summary per file
File Description
src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodRunner.cs Removes per-test allocations by using IsAttributeDefined<DataSourceAttribute> in the DataSource execution decision.
src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.cs Removes iterator allocations by scanning cached attributes directly to find a single RetryBaseAttribute.
docs/glossary.md Adds/updates glossary entries for testing libraries and platform concepts.

Copilot's findings

  • Files reviewed: 3/3 changed files
  • Comments generated: 2

Comment on lines 181 to 184
// IsAttributeDefined avoids the array allocation of GetAttributes<DataSourceAttribute>().
// DataSourceAttribute is sealed and does not allow multiple, so presence == exactly one.
if (!ReflectHelper.Instance.IsAttributeDefined<DataSourceAttribute>(_testMethodInfo.MethodInfo))
{
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot address review comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 12d5a07. TryExecuteDataSourceBasedTestsAsync now scans cached attributes directly and executes the DataSource path only when exactly one DataSourceAttribute is present (0 => false, >1 => false), preserving prior semantics without allocation.

Comment thread docs/glossary.md Outdated
Copy link
Copy Markdown
Member Author

@Evangelink Evangelink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Workflow: Test Expert Reviewer 🧪
Date: 2026-05-11
Repository: microsoft/testfx

Key Findings

This PR contains no test file changes. The two production-code changes are allocation-free refactorings in the test execution hot path:

  1. TestMethodInfo.GetRetryAttribute() — replaces a yield-iterator enumeration with a direct foreach over the cached Attribute[]. Semantically equivalent, but the "throw when multiple RetryBaseAttribute are present" guard is not covered by any existing test (confirmed via grep of TestMethodInfoTests.cs and ThrowMultipleAttributesException).

  2. TestMethodRunner.TryExecuteDataSourceBasedTestsAsync() — replaces GetAttributes<DataSourceAttribute>()[Length == 1] with IsAttributeDefined<DataSourceAttribute>(). Safe because DataSourceAttribute has AllowMultiple = false by default; the subtle semantic shift from "exactly one" to "any" is therefore unreachable in practice.

Recommendations

  • Add a unit test in TestMethodInfoTests.cs verifying that applying two RetryBaseAttribute subclass instances to a method causes the expected exception. This would pin the refactored iteration logic against regression. (See inline comment.)

Generated by Test Expert Reviewer

🧪 Test quality reviewed by Test Expert Reviewer 🧪

Copy link
Copy Markdown
Member Author

@Evangelink Evangelink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Workflow: PR Nitpick Reviewer 🔍
Date: 2026-05-11
Repository: microsoft/testfx

Key Findings

The two code changes are well-motivated and the logic is correct. Two minor comment/naming issues are worth a quick pass:

  1. TestMethodRunner.cs L182 – comment precision: The phrase "sealed and does not allow multiple" conflates sealed (prevents subclassing) with AllowMultiple = false (prevents multiple attribute instances). Both properties matter but for different reasons, and the current wording could mislead a reader about which one controls what.

  2. TestMethodInfo.cs L293 – variable name leaks implementation detail: cachedAttributes mirrors the helper method name's caching detail. A neutral name (methodAttributes) would be more resilient to future renames of the underlying API.

Positive Highlights

  • The guard-clause inversion in TryExecuteDataSourceBasedTestsAsync is a clear improvement in readability.
  • Both explanatory comments accurately describe why the allocation is avoided — helpful for future maintainers.
  • The glossary additions are thorough and well-written.

Recommendations

  • Clarify the comment on L182 to separate the roles of sealed and AttributeUsage.AllowMultiple = false.
  • Rename cachedAttributesmethodAttributes (or attributes) to avoid coupling the local name to the helper's implementation detail.

🔍 Meticulously inspected by PR Nitpick Reviewer

🔍 Meticulously inspected by PR Nitpick Reviewer 🔍

Comment thread src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodRunner.cs Outdated
Comment thread src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.cs Outdated
Copy link
Copy Markdown
Member Author

@Evangelink Evangelink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Workflow: Expert Code Reviewer 🧠
Date: 2026-05-11
Repository: microsoft/testfx

Key Findings

No correctness, threading, security, or API-compatibility issues found. The two code changes are semantically equivalent to what they replace — the optimizations are sound.

TestMethodInfo.GetRetryAttribute — replaces a yield-iterator chain (GetAttributes<RetryBaseAttribute>GetEnumerator) with a direct foreach over the already-cached Attribute[]. The filtering (is RetryBaseAttribute), the dual-match detection (ThrowMultipleAttributesException, [DoesNotReturn]), and the null-return for no match are all preserved. Because ThrowMultipleAttributesException never returns, the loop exits on the second match exactly as the old enumerator code did.

TestMethodRunner.TryExecuteDataSourceBasedTestsAsync — replaces GetAttributes<DataSourceAttribute>() (yield iterator + spread-to-array) and a { Length: 1 } check with IsAttributeDefined<DataSourceAttribute>. The comment's claim "presence == exactly one" is verified: DataSourceAttribute is sealed and [AttributeUsage(Inherited = false)] with AllowMultiple = false, so it is physically impossible to have more than one instance in the cached array.

Both paths ultimately route through ReflectHelper.Instance.GetCustomAttributesCached(MethodInfo), so there is no change in which attributes are seen or how inheritance is handled.

Recommendations

None required — the changes are correct and the allocation savings are genuine hot-path wins (these methods run for every test execution).


Generated by Expert Code Reviewer 🧠 🧠

🧠 Reviewed by Expert Code Reviewer 🧠

@Evangelink
Copy link
Copy Markdown
Member Author

@copilot address review comments

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI commented May 11, 2026

@copilot address review comments

Done in ee2c9f5. I addressed the actionable review items (coverage test for multiple RetryBaseAttribute, DataSource comment wording, and local variable naming).

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 11, 2026 11:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 4/4 changed files
  • Comments generated: 1

Comment on lines +181 to +190
// IsAttributeDefined avoids the array allocation of GetAttributes<DataSourceAttribute>().
// DataSourceAttribute is sealed (so subclasses cannot redefine AttributeUsage)
// and has AllowMultiple = false, so presence == exactly one.
if (!ReflectHelper.Instance.IsAttributeDefined<DataSourceAttribute>(_testMethodInfo.MethodInfo))
{
await ExecuteTestFromDataSourceAttributeAsync(results).ConfigureAwait(false);
return true;
return false;
}

return false;
await ExecuteTestFromDataSourceAttributeAsync(results).ConfigureAwait(false);
return true;
Copilot AI and others added 2 commits May 11, 2026 11:38
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 11, 2026 11:39
@Evangelink Evangelink review requested due to automatic review settings May 11, 2026 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants