Skip to content

Commit 992de6b

Browse files
Separate YAML emission conformance surfaces
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent b68b5b2 commit 992de6b

3 files changed

Lines changed: 217 additions & 75 deletions

File tree

README.md

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -175,20 +175,24 @@ The archive contains 402 inputs:
175175
| Syntax and composition | 400 | 2 | 0 | 0 |
176176
| Representation events | 308 | 0 | 0 | 94 |
177177
| JSON projection | 277 | 2 | 0 | 123 |
178-
| `out.yaml` projection | 241 | 0 | 0 | 161 |
179-
| Emit and round trip | 306 | 0 | 0 | 96 |
178+
| `out.yaml` projection | 241 | 1 | 0 | 160 |
179+
| Official `emit.yaml` fixtures | 55 | 0 | 0 | 347 |
180+
| Module self-round-trip | 306 | 2 | 0 | 94 |
180181

181182
All 94 fixtures marked invalid are rejected. The valid `2JQS` and `X38W`
182183
inputs are syntactically recognized and produce matching representation
183184
events, then are rejected during load validation because representation
184-
mapping keys must be unique. They are not unsupported grammar.
185+
mapping keys must be unique. They are not unsupported grammar. Both are
186+
reported as policy differences for module self-round-trip; `X38W`, the one
187+
case with an `out.yaml` fixture, is also reported that way on that surface.
185188

186189
The two JSON projection differences are `565N`, where `!!binary` intentionally
187190
becomes `byte[]` instead of a Base64 string, and `J7PZ`, where legacy `!!omap`
188191
intentionally becomes `System.Collections.Specialized.OrderedDictionary`
189192
instead of remaining a sequence of one-entry mappings. No event mismatch is
190-
classified as policy. The deterministic runner reports no unexplained
191-
failures.
193+
classified as policy. All 55 official `emit.yaml` fixtures are read and
194+
validated independently of the 402-input module self-round-trip. The
195+
deterministic runner reports no unexplained failures.
192196

193197
These results are a pinned compatibility measurement, not a claim that a finite
194198
corpus proves complete YAML 1.2.2 compliance.

tests/Conformance.Tests.ps1

Lines changed: 48 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,8 @@ BeforeAll {
3333
-CompareJson `
3434
-CompareEvents `
3535
-CompareOutYaml `
36-
-CompareEmitRoundTrip
36+
-CompareEmitYaml `
37+
-CompareSelfRoundTrip
3738
)
3839
$emptySuitePath = Join-Path $TestDrive 'empty-suite'
3940
$null = New-Item -Path $emptySuitePath -ItemType Directory
@@ -235,19 +236,57 @@ ship-to:
235236
It 'accounts for out.yaml representation comparisons' {
236237
@($suiteResults | Where-Object OutYamlResult -EQ 'Pass').Count | Should -Be 241
237238
@($suiteResults | Where-Object OutYamlResult -EQ 'PolicyDifference').Count |
238-
Should -Be 0
239+
Should -Be 1
239240
@($suiteResults | Where-Object OutYamlResult -EQ 'Fail').Count | Should -Be 0
240241
@($suiteResults | Where-Object OutYamlResult -EQ 'NotApplicable').Count |
241-
Should -Be 161
242+
Should -Be 160
243+
$outPolicy = $suiteResults | Where-Object OutYamlResult -EQ 'PolicyDifference'
244+
$outPolicy.Case | Should -Be 'X38W'
245+
$outPolicy.OutYamlReason | Should -Be 'RepresentationMappingKeyUniqueness'
246+
}
247+
248+
It 'validates the 55 official emit.yaml fixtures separately' {
249+
@($suiteResults | Where-Object HasEmitYaml).Count | Should -Be 55
250+
@($suiteResults | Where-Object EmitYamlResult -EQ 'Pass').Count | Should -Be 55
251+
@($suiteResults | Where-Object EmitYamlResult -EQ 'PolicyDifference').Count |
252+
Should -Be 0
253+
@($suiteResults | Where-Object EmitYamlResult -EQ 'Fail').Count | Should -Be 0
254+
@($suiteResults | Where-Object EmitYamlResult -EQ 'NotApplicable').Count |
255+
Should -Be 347
256+
}
257+
258+
It 'reads official emit.yaml content rather than counting its presence' {
259+
$mutatedSuitePath = Join-Path $TestDrive 'mutated-emit-fixture'
260+
$null = New-Item -Path $mutatedSuitePath -ItemType Directory -Force
261+
Copy-Item -LiteralPath (Join-Path $suiteDataPath '2LFX') `
262+
-Destination $mutatedSuitePath -Recurse
263+
'--- altered' | Set-Content `
264+
-LiteralPath (Join-Path $mutatedSuitePath '2LFX\emit.yaml') `
265+
-Encoding utf8NoBOM
266+
267+
$mutatedResult = & $runnerPath -Path $mutatedSuitePath -CompareEmitYaml
268+
269+
$mutatedResult.EmitYamlResult | Should -Be 'Fail'
270+
$mutatedResult.EmitYamlReason | Should -Be 'EmitYamlRepresentationMismatch'
242271
}
243272

244-
It 'accounts for emitter and round-trip comparisons' {
245-
@($suiteResults | Where-Object EmitResult -EQ 'Pass').Count | Should -Be 306
246-
@($suiteResults | Where-Object EmitResult -EQ 'PolicyDifference').Count |
273+
It 'accounts honestly for general module self-round-trips' {
274+
@($suiteResults | Where-Object SelfRoundTripResult -EQ 'Pass').Count |
275+
Should -Be 306
276+
@($suiteResults | Where-Object SelfRoundTripResult -EQ 'PolicyDifference').Count |
277+
Should -Be 2
278+
@($suiteResults | Where-Object SelfRoundTripResult -EQ 'Fail').Count |
247279
Should -Be 0
248-
@($suiteResults | Where-Object EmitResult -EQ 'Fail').Count | Should -Be 0
249-
@($suiteResults | Where-Object EmitResult -EQ 'NotApplicable').Count |
250-
Should -Be 96
280+
@($suiteResults | Where-Object SelfRoundTripResult -EQ 'NotApplicable').Count |
281+
Should -Be 94
282+
$policyResults = @(
283+
$suiteResults |
284+
Where-Object SelfRoundTripResult -EQ 'PolicyDifference' |
285+
Sort-Object Case
286+
)
287+
@($policyResults.Case) | Should -Be @('2JQS', 'X38W')
288+
@($policyResults.SelfRoundTripReason | Select-Object -Unique) |
289+
Should -Be @('RepresentationMappingKeyUniqueness')
251290
}
252291

253292
It 'keeps the previously failing multi-document JSON cases green' {

0 commit comments

Comments
 (0)