Skip to content

Commit 7ecc5b3

Browse files
committed
Add tvp test split plan.
1 parent 27df831 commit 7ecc5b3

2 files changed

Lines changed: 27 additions & 1 deletion

File tree

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
## Plan: Split TvpTest.cs Into Independent Test Classes
2+
3+
The current `TestMain()` runs 500+ test cases sequentially via baseline comparison, making it slow, hard to debug, and retry-unfriendly. The tests are already logically grouped into helper classes (`StreamInputParam`, `SqlVariantParam`, `DateTimeVariantTest`, `OutputParameter`) plus inline methods, with no shared state between groups—making them ideal candidates for splitting.
4+
5+
### Steps (Each Step = One Git Commit)
6+
7+
1.**Add per-group baseline files and update `TestMain()` to validate using concatenated split files** — Extract sections from [SqlParameterTest_DebugMode.bsl](src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/SqlParameterTest_DebugMode.bsl), [SqlParameterTest_ReleaseMode.bsl](src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/SqlParameterTest_ReleaseMode.bsl), and Azure variants into new files (`StreamInputParameter_DebugMode.bsl`, `StreamInputParameter_ReleaseMode.bsl`, `TvpColumnBoundaries_DebugMode.bsl`, `TvpColumnBoundaries_ReleaseMode.bsl`, etc.). Modify `FindDiffFromBaseline()` to concatenate the split baseline files in order (selecting Debug or Release variants based on build configuration) and compare against test output instead of using the original combined file. **If the split is incorrect, `TestMain()` fails—proving equivalence when CI passes.**
8+
9+
2.**Add new test classes that run alongside `TestMain()`** — Create [StreamInputParameterTests.cs](src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/), [TvpColumnBoundariesTests.cs](src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/), [TvpQueryHintsTests.cs](src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/), [SqlVariantParameterTests.cs](src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/), [DateTimeVariantTests.cs](src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/), [OutputParameterTests.cs](src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/). Each class has its own baseline comparison method pointing to its split baseline files (Debug and Release variants), using `#if DEBUG` to select the appropriate baseline. Original `TestMain()` remains unchanged. **CI runs both old and new tests—all must pass.**
10+
11+
3.**Delete `TestMain()` and original combined baseline files** — Remove `TestMain()`, `RunTestCoreAndCompareWithBaseline()`, `RunTest()`, `FindDiffFromBaseline()` from [TvpTest.cs](src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/TvpTest.cs). Delete [SqlParameterTest_*.bsl](src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/) files. Keep shared helper methods/classes in TvpTest.cs or move to a shared utilities file. Per-group Debug and Release baseline files remain. **Reviewer can verify deletion is safe since steps 1-2 already validated equivalence and new tests passed in CI.**
12+
13+
4.**[Repeatable per group] Migrate one test class from baseline comparison to xUnit assertions** — For each group (start with smallest, e.g., `OutputParameterTests`): replace `Console.WriteLine` comparisons with `Assert.Equal()`, `Assert.True()`, etc., then delete that group's Debug and Release baseline files. Use `[Theory]` with `[MemberData]` pointing to static methods that yield all test case permutations with unique identifiers (e.g., `"StreamInput_Sync_DataLen100000_ParamLen-1_OldFalse"`)—xUnit displays each identifier in test output, allowing reviewers to diff test names against baseline to verify completeness. For DEBUG-only test cases, use `#if DEBUG` or `[Trait("Configuration", "Debug")]` to conditionally include them. **Each migration is a separate commit; reviewer sees direct assertion logic instead of string comparison.**
14+
15+
### Baseline Section Boundaries
16+
17+
| Test Group | Start Marker | End Marker | Debug | Release | Debug Azure | Release Azure |
18+
|------------|--------------|------------|-------|---------|-------------|---------------|
19+
| **StreamInputParameter** | `Starting test 'TvpTest'` (line 1) | Line before `+++++++ Iteration 0 ++++++++` | 1–478 | 1–17 | 1–478 | 1–17 |
20+
| **TvpColumnBoundaries** | `+++++++ Iteration 0 ++++++++` | Line before `------- Sort order + uniqueness #1` | 479–943 | 18–482 | 479–943 | 18–482 |
21+
| **TvpQueryHints** | `------- Sort order + uniqueness #1: simple -------` | Last `-------------` line before empty line | 944–974 | 483–513 | 944–974 | 483–513 |
22+
| **SqlVariantParameter** | Empty line + `Starting test 'SqlVariantParam'` | `End test 'SqlVariantParam'` (inclusive) | 975–1073 | 514–612 | 975–1073 | 514–612 |
23+
| **DateTimeVariant** | Line after `End test 'SqlVariantParam'` | Line before `Starting 'OutputParameter' tests` | 1074–2231 | 613–1770 | 1074–2231 | 613–1770 |
24+
| **OutputParameter** | `Starting 'OutputParameter' tests` | `Done` (end of file) | 2232–2235 | 1771–1774 | 2232–2235 | 1771–1774 |
25+
26+
**Note:** Azure baseline files have identical line numbers to their non-Azure counterparts—differences are in content (different error messages/behaviors on Azure SQL), not structure.

.github/prompts/minimize-test-overlap.prompt.md renamed to .github/prompts/refine-test-overlap.prompt.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
name: minimize-test-overlap
2+
name: test-minimize-overlap
33
description: Run coverage overlap analysis and suggest test suite optimizations
44
argument-hint: Test filter (e.g. FullyQualifiedName~MyTests) or describe the tests you want to analyze
55
---

0 commit comments

Comments
 (0)