diff --git a/src/StaticWebAssetsSdk/AGENTS.md b/src/StaticWebAssetsSdk/AGENTS.md index 8a6bb0f3b50d..50117fdcd6d8 100644 --- a/src/StaticWebAssetsSdk/AGENTS.md +++ b/src/StaticWebAssetsSdk/AGENTS.md @@ -66,6 +66,112 @@ These invariants must hold at all times. Violating any of them is a bug. - `AssetKind` → `All` (or `Build` when `CopyToPublishDirectory` is `Never`). - `AssetMode` → `All`; `AssetRole` → `Primary`. +## Triage Heuristic: "Sequence contains more than one element" (and other aggregation crashes) + +> **The crash site is never the fix.** A manifest/endpoint task that throws on a collision is the +> *symptom*, not the cause. Work out **why two assets reached the same target-path + `AssetKind` slot**, then +> fix the layer that *should have stopped the second asset from existing*. The most durable fix **prevents the +> surplus asset from being emitted at all**, at its producer — so exactly one asset ever lands on the route — +> rather than letting two land and de-duplicating them downstream. It is **never** the throwing task; but it +> is **also not** automatically "the SDK resolution." Localize where the second asset is *born*, then decide +> whether it should be born conditionally (best) or de-duplicated later (a defensible-but-weaker compensation). + +**Symptom.** A Static Web Assets task throws +`InvalidOperationException: Sequence contains more than one element`, or otherwise finds more +than one asset where it expects at most one. The canonical site is +`ChooseNearestAssetKind(group, kind).SingleOrDefault()` in `GenerateStaticWebAssetEndpointsManifest` +(and the identical pattern in `GenerateStaticWebAssetsDevelopmentManifest`). + +**What it actually means.** It is a violation of the **target-path uniqueness invariant** documented +above: *at a given target path, at most one asset per `AssetKind` slot (`Build`/`Publish`/`All`)*. +`ChooseNearestAssetKind` is documented to **assume the manifest is already correct** — it deliberately +does not validate or de-duplicate, and yields multiple assets on malformed input precisely so the error +surfaces. So when this throws, **upstream input is malformed**: some producer emitted two assets that +resolve to the same target path + kind. + +**The traps (what NOT to do).** Two reflexes both fail here: + +1. **Symptom-site softening.** Do not "fix" it at the throw site by replacing `SingleOrDefault()` with + `First()`, by collapsing/de-duplicating the colliding assets in the consuming task, or by relaxing the + uniqueness invariant. That masks the real defect and contradicts the task's own contract. +2. **The *unconditional* package remodel.** Do not retype the package's fallback (e.g. model + `blazor.modules.json` as a `Framework` asset) while still emitting it **unconditionally**. That still + leaves **two** assets on the route — it just relabels one — so the collision survives. + +Both traps were hit on dotnet/sdk#54779 (a symptom-site task "fix" *and* an unconditional `Framework`-asset +remodel) and **both were rejected in review**. **Crucially, the fix that ultimately landed *was* a package +change** (dotnet/aspnetcore#67375, merged) — but a **conditional** one: it makes the package emit its +fallback only when the consumer has no asset of its own, so the second asset never exists. "Change the +package" is not the trap; **changing it *unconditionally*** is. The discriminator is always: *does exactly +one asset end up on the route, or do two?* + +**Playbook (do this instead).** + +1. **Enumerate every colliding asset** at the offending target path with full provenance — + `Identity`, `SourceId`, `SourceType`, `AssetKind`, `AssetMode`. Get this from the failing build's + binlog (search the target path, e.g. `_framework/blazor.modules.json`) or by temporarily logging the + group contents. You usually have the provenance already; the gap is interpreting it. +2. **Identify the producer** of each colliding asset — which referenced project, NuGet package, or target + generated it (`SourceType=Package`/`Project`/`Computed`/`Framework` tells you where to look). +3. **Localize the *resolution gap*, not just the producer.** Ask two questions, in order: + 1. **Is the input genuinely malformed?** Is there a duplicate that should *never* exist under any correct + resolution — e.g. two unrelated sources each emitting a primary asset at one target path with no + grouping relationship between them? If so, the producer of the spurious asset is at fault. + 2. **Or is the input legitimately grouped, but the surplus variant was emitted anyway?** Assets that + *intentionally* share a target path do so via **asset groups** (advanced extensibility): a package ships + a fallback that must be **dropped** once the consumer supplies its own asset for the same path. + Historically this was modeled with a **deferred** group resolved at **build**, while the build manifest + deliberately **retained all variants** so transitive consumers could re-resolve against their own graph. + **Key signal:** if the project **builds** cleanly and only **publish** throws, the build-time resolution + wasn't reflected at publish — so the variant that *should* have been dropped survives. Two layers can own + this: the **package** that emits the fallback (it can decline to emit it when the consumer already has an + asset), and the **SDK** that reloads the build manifest at publish (it can re-apply the build-time + resolution). The durable fix is the former. +4. **Fix the layer that can stop the second asset from existing.** + - **A grouped/fallback collision that only manifests at publish** → **preferred: make the package's + fallback conditional** so exactly one asset is ever produced. The package's build targets run *inside the + consumer's build* and can see whether the consumer already contributes an asset for the path (e.g. + `@(_ExistingBuildJSModules) == ''`); gate the fallback on that, and the surplus variant is never emitted — + no deferred group, no downstream de-duplication, no SDK change. This is what landed for #54779 + (dotnet/aspnetcore#67375). **Acceptable alternative (weaker): carry the build resolution into publish in + the SDK** — persist the *resolved* (no-longer-`Deferred`) groups into the build manifest and re-apply that + **unscoped** decision when the manifest is reloaded at publish (dotnet/sdk#54941). It is a real, working + fix, but it *de-duplicates two assets after the fact* rather than preventing the second one, and it was + **superseded** by the package fix. Whichever layer you choose, do **NOT** retype the package's asset as a + `Framework` asset while still emitting it **unconditionally** (it leaves two assets and a redundant + endpoint), and do **NOT** add a consumer-scoped `Remove`/`SourceId != $(PackageId)` guard in the publish + targets — at publish the group filter is consumer-scoped (`Source="$(PackageId)"`) and *structurally + cannot* filter a group owned by a referenced project or package, so a scoped patch is both wrong and + incomplete. + - **A genuine duplicate from a referenced project** (no legitimate grouping) → fix that project. + - **A genuine duplicate emitted by the SDK's own targets** → fix the emitting target. + In every case, **preserve** the target-path uniqueness invariant and never soften the throwing task. + +**Why "blame the producer" is half-right.** Provenance tells you *which* asset collided and *where it came +from*; it does **not** by itself tell you where the fix belongs. The fix belongs where the surplus asset can +be **prevented**. For #54779 that turned out to be the producing package after all — but not because "the +producer is always at fault," and not via an unconditional remodel. The package's targets execute in the +consumer's build, so the package *can* see whether the consumer already owns the asset and decline to emit its +fallback. The earlier belief that "only the SDK can resolve this, because the package can't see the consumer's +graph" was the false premise: at build time, inside the consumer, the package's own targets see exactly the +signal they need (`_ExistingBuildJSModules`). Preventing the asset at its source beats carrying a resolution +forward to delete it later. + +**Worked example.** dotnet/sdk#54779 — the `Microsoft.AspNetCore.Components.WebView` package shipped a +fallback `_framework/blazor.modules.json` *unconditionally* (originally via a **deferred** static-web-asset +group). The app *built* fine but failed to *publish* with `Sequence contains more than one element`, because +the fallback survived alongside the app's own modules manifest — two assets on one route. **The fix that +landed is in the package** (dotnet/aspnetcore#67375, merged; backport dotnet/aspnetcore#67401): it removes the +deferred-group machinery and instead materializes the empty (`[]`) fallback as the consumer's own asset +**only when the consumer has no JS modules of its own**, so exactly one asset ever lands on the route and the +collision cannot form — with **no SDK change**. An SDK-side carry-forward (dotnet/sdk#54941) was built and +*empirically verified to work*, but it compensates downstream and was **not merged** (superseded). The earlier +attempts to soften the throwing task and to remodel the package's asset as a `Framework` asset *unconditionally* +were both rejected. + +See [Architecture.md](Architecture.md) §"Deferred Groups and `SkipDeferred` Filtering" for the mechanism. A +regression eval for this exact reasoning failure lives in [`evals/`](evals/README.md). + ## Development Workflow For the full development workflow (inner loop, patching, testing, validation), use the **Static Web Assets** agent defined in `.github/agents/static-web-assets-agent.agent.md`. It has the step-by-step process, commands, and test strategy. diff --git a/src/StaticWebAssetsSdk/Architecture.md b/src/StaticWebAssetsSdk/Architecture.md index 7fb7b3258283..c467ab32cfec 100644 --- a/src/StaticWebAssetsSdk/Architecture.md +++ b/src/StaticWebAssetsSdk/Architecture.md @@ -294,6 +294,49 @@ When a project targets multiple frameworks (`TargetFrameworks`), each inner buil Groups are keyed by `(SourceId, Name)` and stored in a `Dictionary<(string SourceId, string Name), StaticWebAssetGroup>`. They travel alongside assets in manifests and allow a consuming project to read provider-specific settings without coupling to the provider's implementation. An asset's `AssetGroups` property (semicolon-separated `Name=Value` pairs) can reference these groups. +#### Deferred Groups and `SkipDeferred` Filtering + +A group can be marked **`Deferred`**. Deferred groups model assets whose final resolution depends on the +*consumer* — most often a **fallback** asset that a package ships but that must be **excluded** once the +consuming project (or one of its references) supplies a more specific asset for the same target path. + +`FilterStaticWebAssetGroups` (`Tasks/FilterStaticWebAssetGroups.cs`) applies group declarations and runs +in two modes, controlled by its `SkipDeferred` parameter: + +- **`SkipDeferred="true"`** (pre-filter pass): groups marked `Deferred` are **skipped** — their exclusions + are *not* applied yet. The **publish** path runs in this mode + (`Microsoft.NET.Sdk.StaticWebAssets.Publish.targets`). +- **`SkipDeferred="false"`** (final pass): all groups must be concrete; an error is raised if any group is + still `Deferred`. + +> **Failure mode (target-path collision) — and where it is actually fixed.** The build manifest +> (`staticwebassets.build.json`) intentionally **retains every variant** of a grouped asset so that +> transitive consumers can re-resolve the group against their own asset graph. At **build**, the unscoped +> `FilterStaticWebAssetGroups` pass resolves the deferred group and drops the excluded fallback. At +> **publish**, the SDK reloads that build manifest but historically **never re-resolved the deferred group**: +> there is no publish-side resolution pass, and the publish `FilterStaticWebAssetGroups` is *consumer-scoped* +> (`Source="$(PackageId)"`), so it structurally **cannot** filter a group owned by a referenced project or +> package. The fallback therefore survives and reaches `GenerateStaticWebAssetEndpointsManifest` *alongside* +> the consumer's own asset — two assets at the same target path + `AssetKind` slot — and +> `ChooseNearestAssetKind(...).SingleOrDefault()` throws `Sequence contains more than one element`. **Key +> signal: the project builds fine; only publish fails.** +> **Where it was actually fixed (and the precision point).** The durable fix does **not** carry the resolution +> forward — it stops the second asset from being emitted at all. The producing package's build targets run +> *inside the consumer's build*, so they can gate the fallback on whether the consumer already contributes an +> asset (e.g. `@(_ExistingBuildJSModules) == ''`) and materialize it **only when there is none**. Then exactly +> one asset ever lands on the route and there is nothing to collide — no deferred group, no SDK change. This is +> what landed for dotnet/sdk#54779: dotnet/aspnetcore#67375 (merged; backport dotnet/aspnetcore#67401) removes +> the deferred-group machinery from the `Microsoft.AspNetCore.Components.WebView` package and makes the +> `blazor.modules.json` fallback **conditional**. A *different* SDK-side fix that re-applies the resolved +> groups at publish (dotnet/sdk#54941) was built and **empirically verified to work**, but it de-duplicates two +> assets after the fact rather than preventing the second one, and was **not merged** (superseded). Two further +> approaches were tried and **rejected in review**: softening the throwing task, and retyping the package's +> asset as a **`Framework`** asset while still emitting it **unconditionally** (that just relabels one of two +> assets — the collision survives). The distinction that matters: a *conditional* package fallback (one asset, +> correct) versus an *unconditional* package remodel (two assets, still broken). See the triage heuristic in +> [AGENTS.md](AGENTS.md). + + ### Discovery Patterns `StaticWebAssetsDiscoveryPattern` (`Tasks/Data/StaticWebAssetsDiscoveryPattern.cs`) describes a glob pattern for discovering files dynamically at runtime (for the dev server). diff --git a/src/StaticWebAssetsSdk/Tasks/Data/StaticWebAsset.cs b/src/StaticWebAssetsSdk/Tasks/Data/StaticWebAsset.cs index ab1f9f9bd151..24bd417d30e8 100644 --- a/src/StaticWebAssetsSdk/Tasks/Data/StaticWebAsset.cs +++ b/src/StaticWebAssetsSdk/Tasks/Data/StaticWebAsset.cs @@ -620,6 +620,16 @@ public static StaticWebAsset FromTaskItem(ITaskItem item, TaskEnvironment env, b // than one type of asset we will just return all of them. // One exception to this is the `All` kind of assets, where we will just return the first two we find. The reason for it is // to avoid having to allocate a buffer to collect all the `All` assets. + // CONTRACT: callers must treat "more than one asset returned" as malformed input for this pipeline — a + // violation of the target-path uniqueness invariant (at most one asset per AssetKind slot per target + // path). Surface the error; do NOT silently collapse the duplicates here. The cause is upstream: either + // a genuine producer duplicate, or (commonly) a package fallback that was emitted even though the consumer + // already supplied its own asset for the same path. The durable fix removes the surplus asset at its + // source — make the package's fallback conditional so exactly one asset is ever produced + // (dotnet/aspnetcore#67375, the fix that landed for dotnet/sdk#54779). An SDK-side carry-forward that + // re-applies the build-time group resolution at publish (dotnet/sdk#54941) also works but compensates + // downstream and was superseded. Either way, do NOT soften this call. See + // StaticWebAssetsSdk/AGENTS.md "Triage Heuristic". internal static IEnumerable ChooseNearestAssetKind(IEnumerable group, string assetKind) { StaticWebAsset allKindAssetCandidate = null; diff --git a/src/StaticWebAssetsSdk/Tasks/GenerateStaticWebAssetEndpointsManifest.cs b/src/StaticWebAssetsSdk/Tasks/GenerateStaticWebAssetEndpointsManifest.cs index f996a75b7168..0cd6c1f3a938 100644 --- a/src/StaticWebAssetsSdk/Tasks/GenerateStaticWebAssetEndpointsManifest.cs +++ b/src/StaticWebAssetsSdk/Tasks/GenerateStaticWebAssetEndpointsManifest.cs @@ -226,6 +226,16 @@ private IEnumerable ComputeManifestAssets(IEnumerable You are investigating a build failure in the .NET SDK. A .NET MAUI Blazor Hybrid app (a project using +> `Microsoft.NET.Sdk.Razor` that references the `Microsoft.AspNetCore.Components.WebView` package **and** a +> Razor Class Library that contributes JS module assets) fails to **publish** on SDK +> `11.0.100-preview.6` with the error below. The same app published cleanly on `preview.5`. **The project +> `build`s without error on `preview.6`; the failure occurs only during `publish`.** +> +> ``` +> error : InvalidOperationException: Sequence contains more than one element +> at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source) +> at Microsoft.AspNetCore.StaticWebAssets.Tasks.GenerateStaticWebAssetEndpointsManifest +> .ComputeManifestAssets(IEnumerable`1 assets, String kind)+MoveNext() +> at Microsoft.AspNetCore.StaticWebAssets.Tasks.GenerateStaticWebAssetEndpointsManifest.Execute() +> ``` +> +> From the failing build's binlog, three `StaticWebAsset`s share the same target path +> `_framework/blazor.modules.json` going into the publish endpoints manifest: +> +> | SourceId | SourceType | AssetKind | AssetMode | Identity (abbreviated) | +> |----------|-----------|-----------|-----------|------------------------| +> | `Microsoft.AspNetCore.Components.WebView` | `Package` | `All` | `All` | `…/microsoft.aspnetcore.components.webview/…/staticwebassets/blazor.modules.json` | +> | `MyApp` | `Computed` | `Build` | `All` | `…/obj/.../jsmodules/jsmodules.build.manifest.json` (promoted to `blazor.modules.json`) | +> | `MyApp` | `Computed` | `Publish` | `All` | `…/obj/.../jsmodules/jsmodules.publish.manifest.json` | +> +> The `Publish`-kind publish manifest (row 3) is **new in `preview.6`**; `preview.5` produced only the +> `Build`-kind build manifest. At publish, the `Build`-kind asset is filtered out, so the endpoints manifest +> group for that target path contains the package's `All` asset and the app's `Publish` asset. +> +> Determine the **root cause** and recommend **where the fix belongs**. State the specific code or +> project/package you would change, and why. + +## Rubric + +Score the agent's **final conclusion/recommendation** on a three-tier scale. The crux of this eval is +distinguishing *removing the cause* (one asset ever lands on the route) from *compensating for the symptom* +(two assets land and something de-duplicates them). + +### The root cause (what the agent must ultimately localize) + +The `Microsoft.AspNetCore.Components.WebView` package **unconditionally** contributes a fallback +`blazor.modules.json` onto the shared target path `_framework/blazor.modules.json` (historically via a +deferred asset group plus an `AssetKind=All` promotion of the app's own manifest). That guarantees **two +primary assets on one route** whenever the app *also* has its own modules manifest — a collision that +*something* downstream must then de-duplicate. Pre-`preview.6` the de-duplication happened to survive into +publish; `preview.6` added a third collider (a real `Publish`-kind app manifest) and the de-duplication was +never re-applied at publish, so the fallback survived and `ChooseNearestAssetKind(...).SingleOrDefault()` +threw. **The durable fix is to stop creating the second asset unless it is actually needed** — i.e. make the +package's fallback **conditional** on the consumer having no JS modules of its own. Then exactly one asset is +ever on the route, in every phase, and no group / carry-forward / de-dup machinery is required anywhere. + +### PASS (full credit) — removes the cause + +Requires ALL of: + +1. **Correct cause.** Identifies that the **package** is the source of the second asset on the route: it + ships/materializes a fallback `blazor.modules.json` *unconditionally*, so two primary assets occupy one + target path whenever the app supplies its own — and that surplus asset is the thing that has to be + de-duplicated (and isn't, at publish). +2. **Correct remedy — eliminate, don't de-duplicate.** Recommends making the package's fallback + **conditional**: materialize the empty (`[]`) fallback as the consumer's own static web asset **only when + the app contributes no JS modules of its own** (decided during input resolution, before the manifest + conflict check). Net effect named or clearly implied: **exactly one asset ever lands on the route**, so + there is nothing to disambiguate and **no SDK change is needed**. (This is dotnet/aspnetcore#67375.) +3. **Uses the build-vs-publish asymmetry correctly** — recognizes the project builds but only publish throws, + and explains it as the surplus fallback surviving into publish, *without* concluding that the remedy is to + teach publish to drop it. (Bonus for noting that any downstream de-dup — at the throw site, via a + consumer-scoped `Remove`, or via build→publish carry-forward — is compensating for a duplicate the package + should never have emitted.) +4. **Rejects all four non-durable destinations** (see PARTIAL/FAIL): does not soften the throwing task, does + not retype the fallback as a `Framework` asset (that still leaves two assets + a redundant endpoint), does + not add a consumer-scoped publish-targets guard, and does not stop at the SDK build→publish carry-forward + as *the* fix. + +### PARTIAL (acceptable but superseded) — fixes a real defect, but compensates downstream + +Award PARTIAL when the agent lands on the **SDK build→publish carry-forward** fix (dotnet/sdk#54941): persist +the resolved (no-longer-`Deferred`) groups into the build manifest and re-apply the **unscoped** group filter +when the manifest is reloaded at publish, dropping the excluded variant before the endpoints manifest is +computed, preserving the target-path uniqueness invariant. + +This is **not a failure**: it correctly localizes a genuine SDK resolution gap, is at a defensible layer, was +**empirically verified to fix the bug** (see Baseline), and explicitly rejects the symptom-site, package-retype, +and consumer-scoped traps. It falls short of full credit only because it **de-duplicates the two assets at +publish instead of preventing the second asset** — a downstream compensation the owner ultimately superseded +with the simpler package fix. An agent that proposes #54941 *and* notes that the cleaner fix is to make the +package fallback conditional should be scored **PASS**. + +### FAIL — wrong layer or wrong mechanism + +Any of: + +- **Symptom-site softening.** Recommends changing `GenerateStaticWebAssetEndpointsManifest` / + `ChooseNearestAssetKind` / `GenerateStaticWebAssetsDevelopmentManifest` to disambiguate, pick one, de-dup, + or swap `SingleOrDefault` → `First()` **as the fix**. (A clearer *diagnostic* at the throw site is fine only + if explicitly framed as not being the fix.) +- **Wrong package fix (unconditional retype).** Recommends remodeling the package's `blazor.modules.json` as a + **`Framework`** asset (or any retyping that keeps the fallback present unconditionally). This still leaves + **two assets on the route** and a redundant endpoint — it relocates the disambiguation rather than removing + the duplicate — and it is the approach that was **tried and rejected in review**. *Note the sharp line: a + package fix is correct only if it makes the fallback **conditional** (one asset). An **unconditional** + package change is FAIL.* +- **Consumer-scoped SDK patch.** A `Remove` / `SourceId != $(PackageId)` guard in the publish targets — + structurally incomplete, because the publish group filter cannot scope to a referenced project's or + package's group. +- **No localization.** Concludes the regression is "just" some other SDK PR/commit to revert, or proposes only + a consumer-project workaround (remove the ProjectReference, disable the feature), without identifying the + package's unconditional-fallback as the source of the surplus asset. + +## Baseline + +> **v3 recalibration (2026-06-24) — the correct answer moved a THIRD time, and it landed in the package.** +> Ground truth for this bug has now shifted three times: **(v1)** model `blazor.modules.json` as a `Framework` +> asset (package) — *rejected in review*; **(v2)** carry deferred-group resolution into publish +> (dotnet/sdk#54941, SDK) — *built and empirically verified, but not merged → superseded*; **(v3)** make the +> package fallback **conditional** so exactly one asset ever lands on the route (dotnet/aspnetcore#67375, +> package) — **MERGED**. The Rubric above is **v3**, calibrated to that landed fix. +> +> **Why this is not a contradiction of v2's "don't fix the package" rule.** v1 and v3 are *both* package +> changes but are opposites in mechanism: v1 keeps the fallback **unconditional** (just retyped) so two assets +> still collide and must be disambiguated — which is exactly why it was rejected; v3 makes the fallback +> **conditional** so the second asset never exists. The v2 rubric's hard-FAIL clause ("any package change = +> FAIL") was over-broad: it correctly caught the v1 *unconditional* retype but would have wrongly failed the +> v3 *conditional* fix that actually shipped. v3 narrows the FAIL clause to *unconditional* package changes +> only. +> +> **Methodology lesson (now fully realized).** Never freeze an eval's "correct answer" from an unsettled +> investigation. The obvious producer fix (v1) was a later-rejected intermediate; the elegant SDK fix (v2) was +> a verified-but-superseded intermediate; only the merged fix (v3) is authoritative. We deliberately never +> pushed the guidance branch, never opened an SDK PR, and never recalibrated to v2 as final — which is the +> only reason this rewrite is a clean re-aim rather than a shipped mistake. + +### Empirical verification of the landed fix (2026-06-24) + +A reproduction harness (Razor-SDK app → real public WebView `preview.6` package + an RCL with a JS module) was +run on a genuinely pre-fix SDK, swapping **only the package** between runs: + +| Scenario | Public package (pre-fix) | Patched package (dotnet/aspnetcore#67375) | +|----------|--------------------------|-------------------------------------------| +| App + RCL-with-JS-module, `publish` | **crash** (`Sequence contains more than one element`, `GenerateStaticWebAssetEndpointsManifest.cs:229`) | **PASS** — 1 route, content = app's `_content/rcl/…lib.module.js` | +| App with no JS modules, `build`+`publish` | n/a | **PASS** — 1 route, content `[]` | +| Incremental no-module → add-module (no clean) | n/a | **PASS** — stale `[]` Discovered fallback fully evicted; no double-add | +| App **and** RCL both *directly* reference the package, neither has a JS module | n/a | **FAIL (genuine uncovered edge)** — two `[]` fallbacks collide → `Conflicting assets…` *build* error (loud/early, not the silent publish crash); the new target has no class-library guard. Not covered by the merged tests. | + +**Conclusion:** the package-only conditional fallback resolves the targeted regression with **no SDK change**, +confirming v3. One non-blocking edge (dual *direct* package reference with no modules) was reported upstream as +a follow-up. dotnet/sdk#54941 remains a valid *alternative* fix (PARTIAL) but was superseded. + +#### Real-MAUI confirmation (2026-06-24) — `dotnet new maui-blazor` on the genuine preview.6 toolchain + +The synthetic harness above was independently confirmed on a **real MAUI Blazor Hybrid app** built and +published against the **official `11.0.100-preview.6.26323.106` SDK** (genuinely pre-fix, no dotnet/sdk#54941), +with the base `Microsoft.AspNetCore.Components.WebView` pinned to the matching `preview.6` value and referenced +exactly as `…WebView.Maui` references it (`ExcludeAssets="Build;Analyzers"`), plus an RCL contributing +`rcl.lib.module.js`: + +- **Baseline (real MAUI, pre-fix package):** same crash — + `InvalidOperationException: Sequence contains more than one element` at + `GenerateStaticWebAssetEndpointsManifest.ComputeManifestAssets` (`Publish.targets(55,5)`); the binlog shows + **two** assets on `_framework/blazor.modules.json` (the package's `[]` fallback + the SDK-generated RCL + manifest). **MAUI preview.6 is genuinely affected — not immune.** +- **Fix (swap *only* the base package to the #67375-shaped build, same SDK, same app):** + `GenerateStaticWebAssetEndpointsManifest` **succeeds**; exactly **one** endpoint on the route, content = + the real `_content/rcl/…lib.module.js` module list (not `[]`). Publish then proceeds past SWA (later hitting + an unrelated AOT/IL1032 native-build issue, orthogonal to this pipeline). +- **Mechanism correction worth keeping:** `…WebView.Maui`'s `ExcludeAssets="Build"` does **not** prevent the + conditional-fallback target from running. The package's targets reach the consumer via + `buildTransitive → buildMultiTargeting → build/…targets` (an explicit path-based ``); `Exclude="Build"` + only suppresses the *automatic* `build/` import, not the path-based one. So `_AddBlazorWebViewModulesFallback` + **does** run inside real MAUI apps — skipped when the RCL contributes a module (RCL wins), materializing the + `[]` fallback when none exists. This refutes the pre-merge worry that the package fix might never fire under + MAUI's `Exclude="Build"`, and the symmetric worry that a no-modules app would lose its manifest: it gets a + valid `[]` manifest either way. **The package's targets see the local signal (`_ExistingBuildJSModules`) + they need, inside the consumer's build — which is exactly why a *conditional* package fix is possible.** +- **Flow note:** as of WebView `…26324.102` the fix had not yet shipped in any preview.6 package; MAUI + preview.6 needs the backport (dotnet/aspnetcore#67401) to ship to pick it up. Scenario D (dual direct + reference, neither owner owns a module) was not re-run in MAUI shape and remains the one open edge. + +### v1 runs (superseded rubric — retained for the methodology trail) + +> Scored against the original v1 rubric, which scored the `Framework`-asset package remodel as PASS. Under v3 +> every row below is **FAIL** (the v1 treatment landed on the *unconditional* package retype that was rejected +> in review). + +| Date | Model / setup | v1 score | v3 re-score | Evidence | +|------|---------------|----------|-------------|----------| +| 2026-06-22 | Copilot agent, preview.6 era, **before** any guidance | FAIL | **FAIL** | Drafted an SDK issue whose "Expected behavior" was that the publish manifest task should *disambiguate* Build/Publish/All assets "not throwing", and opened two SDK-task PRs (softening `SingleOrDefault`; adding a diagnostic). Symptom-site anchoring. | +| 2026-06-23 | Sonnet-4.6 · **control** (clean `main`) · repo-confined · no GitHub/web/git-history | FAIL | **FAIL** | Found the publish-side `_ExistingPublishJSModules` filter lacks the `SourceId != $(PackageId)` guard its build-side sibling has, and proposed adding it. Consumer-scoped SDK patch; no producer/deferred-group framing. | +| 2026-06-23 | Sonnet-4.6 · **treatment** (guidance, spoilers redacted) · repo-confined · no GitHub/web/git-history | FAIL (near-miss) | **FAIL** | Reconstructed the deferred-group mechanism and named the `Framework`-asset remedy, but chose an SDK-side compensating `Remove` in `JSModules.targets` as the concrete change. | +| 2026-06-23 | Sonnet-4.6 · **control**, un-confined · no GitHub/web/git-history | FAIL | **FAIL** | Located the fix in SDK `StaticWebAsset.ChooseNearestAssetKind` (suppress the `All` candidate once a specific kind is seen). Symptom-site softening. | +| 2026-06-23 | Sonnet-4.6 · **treatment** (guidance, spoilers redacted), un-confined · no GitHub/web/git-history | **PASS (v1)** | **FAIL** | Concluded the fix belongs in the WebView package by changing `blazor.modules.json` from an `All`-kind deferred-group asset to a `SourceType="Framework"` asset. Right *layer*, but the **unconditional** retype that was rejected in review — it leaves a redundant endpoint and still puts two assets on the route. This is precisely the v3 "wrong package fix" FAIL clause. | + +### v2 runs (superseded rubric — SDK carry-forward scored as PASS) + +> The v2 rubric scored the SDK build→publish carry-forward as PASS and **any** package change as FAIL. Under +> v3 the SDK carry-forward is **PARTIAL** (acceptable but superseded), and the FAIL-on-package clause is +> narrowed to *unconditional* package changes only. + +| Date | Model / setup | v2 score | v3 re-score | Evidence | +|------|---------------|----------|-------------|----------| +| 2026-06-23 | Sonnet-4.6 · **control** (clean `main`, no guidance), un-confined · no GitHub/web/git-history | FAIL | **FAIL** | Rejected both the task-softening and the package remodel, but landed on a **consumer-scoped `Remove`** in `JSModules.targets` `ResolveJSModuleManifestPublishStaticWebAssets`. Structurally incomplete (consumer-scoped). | +| 2026-06-23 | Sonnet-4.6 · **treatment** (recalibrated v2 guidance, spoilers redacted), un-confined · no GitHub/web/git-history | PASS | **PARTIAL** | Concluded the SDK resolves deferred groups at build but never carries that into publish, and the publish filter is consumer-scoped so it structurally can't filter a package-owned group; fix in SDK = persist resolved groups + re-apply the unscoped filter at publish. Independently matches dotnet/sdk#54941 — a real, verified fix at a defensible layer, **but a downstream de-dup that was superseded** by the package conditional fallback. Did not reach "stop creating the second asset." | + +### v3 runs (current rubric — authoritative) + +| Date | Model / setup | Result | Evidence | +|------|---------------|--------|----------| +| _pending_ | — | — | No v3 treatment run yet: the companion guidance (AGENTS.md / Architecture.md / code comments) still teaches the v2 SDK-carry-forward thesis and must be recalibrated to the landed package conditional-fallback before a fair treatment A/B can be run. A v3 **control** (no guidance) is expected to FAIL on the symptom-site or consumer-scoped traps as before; the open question v3 measures is whether recalibrated guidance moves a fresh agent past the *defensible-but-superseded* SDK carry-forward (PARTIAL) to "make the package fallback conditional" (PASS). | + +> Update this table when re-running. A guidance change "works" when a fresh agent session, given only the +> **Prompt** above, produces a conclusion that scores **PASS** against the v3 **Rubric** (full credit = +> conditional package fallback; the SDK carry-forward is honest PARTIAL credit, not a failure). diff --git a/src/StaticWebAssetsSdk/evals/README.md b/src/StaticWebAssetsSdk/evals/README.md new file mode 100644 index 000000000000..8082746361a5 --- /dev/null +++ b/src/StaticWebAssetsSdk/evals/README.md @@ -0,0 +1,66 @@ +# Static Web Assets — Agent Reasoning Evals + +These are **reasoning evals**, not unit tests. Each one captures a real investigation where an +AI agent (or a person) plausibly reaches the *wrong* conclusion, and pins down what the *right* +conclusion is. They exist so that changes to agent guidance (`AGENTS.md`, `Architecture.md`, +diagnostics, code comments) can be **measured**: does the new guidance actually move an agent from +the wrong conclusion to the right one? + +## Why this exists + +The Static Web Assets pipeline crashes tend to surface **far from their cause**. A duplicate asset +produced by a NuGet package throws inside an SDK MSBuild task, so the obvious-but-wrong instinct is +to "fix the task that threw." Getting this right requires reasoning from *invariants* and *provenance* +to the **producer**, not patching the symptom site. That reasoning step is exactly what a doc/diagnostic +change is supposed to install — and the only way to know if it worked is to re-run a fixed scenario and +check the conclusion. + +## Format + +Each eval is a Markdown file, `.eval.md`, with three required sections: + +- **`## Prompt`** — what the agent is given. The crash, minimal repro context, and the artifacts a real + investigator would have (e.g. a binlog excerpt). It must **not** contain the answer or leading hints. + **Ground every datum in the real artifact.** Asset facts (target path, `AssetKind`, `AssetMode`, + `SourceType`, provenance) must be transcribed verbatim from the actual binlog and cited (build/job). + A single wrong `AssetKind` can change which collider looks "foreign" and quietly invert the eval — a + fabricated value here is worse than no value. +- **`## Rubric`** — objective PASS/FAIL criteria. Written as checkable assertions about the agent's final + conclusion, so a human or a grader model can score a transcript without judgment calls. +- **`## Baseline`** — the last recorded result (date, model, PASS/FAIL, and the evidence). This is what a + guidance change is trying to improve. + +## How to run (until a harness exists) + +1. Start a fresh agent session in a clean checkout of this repo (so it sees current `AGENTS.md` etc.). +2. Paste the eval's **Prompt** verbatim. Let the agent investigate to a final recommendation. +3. Score the transcript against the **Rubric**. Update **Baseline** with date, model, result, evidence. + +**Do not floor out cross-repo attribution.** If the eval's correct answer is a fix in a *different* repo +(a referenced package, the consuming app), the run setup must let the agent recommend that. An agent +confined to read/write only this repo will put the fix where it can edit it — masking the very +producer-attribution behavior under test. State explicitly in the run setup: *"you may recommend a fix in +any repo — this SDK, the consuming app, or a referenced package's source — name the specific repo and +file."* Keep other constraints (no internet, no git history) as needed for a fair reasoning test. (This is +a sound general rule; note, though, that for `54779-endpoints-manifest-collision` the correct fix turned out +to be in *this* repo — the SDK — so confinement was not the decisive flaw there. Mis-calibration was; see the +next rule.) + +**Do not calibrate against an unsettled investigation.** An eval's "correct answer" must be the *final* +resolution, not an in-flight hypothesis. `54779-endpoints-manifest-collision` was first calibrated to the +producer-side `Framework`-asset fix that dotnet/aspnetcore#67375 *originally* proposed; that approach was +**rejected in review** and the real fix landed in the SDK (dotnet/sdk#54941, with the package authoring +unchanged), so the PASS criterion had to be flipped. If the source PR/issue is still open or under active +review, mark the baseline **provisional** and re-confirm the rubric once it merges — the obvious "fix the +producer" answer can be a later-rejected intermediate. + +A transcript-grading harness (feed Prompt → capture conclusion → grade against Rubric with a model) can +be added later; the file format is designed to be machine-readable for that purpose. The point of +committing these now is that the **target outcome and the rubric are version-controlled and reviewable**, +independent of any harness. + +## Catalog + +| Eval | Scenario | Right conclusion | +|------|----------|------------------| +| [`54779-endpoints-manifest-collision`](54779-endpoints-manifest-collision.eval.md) | `GenerateStaticWebAssetEndpointsManifest` throws `Sequence contains more than one element` for a MAUI Blazor Hybrid app (builds fine; only publish fails). | SDK build→publish carry-forward: a package's deferred-group fallback `blazor.modules.json` is resolved at build but never re-applied at publish, so it survives and collides. Fix the **SDK** (persist resolved groups; re-apply the unscoped group filter when the manifest is reloaded at publish); leave the package unchanged. NOT an SDK-task disambiguation, and NOT a package remodel to a `Framework` asset. |