Skip to content

Conformance harness part A#72

Draft
fangmobile wants to merge 24 commits into
BBC6BAE9:mainfrom
fangmobile:comformance-harness-a
Draft

Conformance harness part A#72
fangmobile wants to merge 24 commits into
BBC6BAE9:mainfrom
fangmobile:comformance-harness-a

Conversation

@fangmobile

@fangmobile fangmobile commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Implement conformance harness - Part A in #59
a2ui-json related test cases will fail until #70 is merged.

a second batch of tests fail due to other non conformance
Draft for now.

all changes to 4 files in Sources/A2UISwiftUICore are "addition", to meet the conformance suite requirements.

Errors.swift

Adds two error types mirroring the Python reference validator: A2uiIntegrityError (code INTEGRITY_ERROR) for structural violations of the component graph, and A2uiRecursionError (code RECURSION_ERROR) for recursion/depth-limit violations. They exist so the new A2uiPayloadValidator can throw the same error taxonomy the conformance suite asserts on.
Related cases: validator.yaml — integrity cases such as test_validate_duplicate_ids_v09, test_validate_missing_root_v09, test_validate_dangling_references_v09; recursion cases such as test_validate_self_reference_v09 and test_validate_function_call_recursion*.

Transport/A2UIResponseParser.swift (new)

Non-streaming counterpart to the stream parser, mirroring upstream Python parser.py. Provides parseFull (splits a complete agent response into text / a2ui parts, throwing a strict A2uiParseError on malformed JSON since the whole payload is available up front), fixPayload (repairs common LLM JSON damage — trailing commas, auto-wrapping a bare object into an array), and hasA2uiParts (detects whether a response contains any A2UI block).
Related cases: the entire parser.yaml suite — 9 parse_full cases (e.g. test_parse_response_with_text_and_tags, test_parse_response_multiple_blocks, test_parse_response_invalid_json), 7 fix_payload cases (e.g. test_fix_payload_trailing_comma_list, test_fix_payload_auto_wrap), 3 has_parts cases.

Transport/A2UIStreamParser.swift

Previously the parser buffered an entire block and only decoded it when the close tag arrived. The upstream Python streaming.py state machine instead emits incrementally ("sniff & cut"): messages are yielded as each top-level array element completes, partial components are yielded early when safe, and the conformance suite asserts on that per-chunk event granularity. This change ports that behavior: incremental brace/string scanning, auto-closing unterminated strings only on catalog-cuttable keys, holding back partial components missing catalog-required fields, seen-component reachability yields gated on the Row loading placeholder, per-key data-model deltas, and cycle/self-reference detection. A new A2UIStreamParserConfig carries the catalog-derived cuttable keys and required-field map.
Related cases: all 76 process_chunk cases in streaming_parser.yaml — e.g. test_incremental_yielding, test_partial_json_and_incremental_yielding, test_waiting_for_root_component, test_circular_reference_detection, test_multiple_a2ui_blocks.

Validation/A2uiPayloadValidator.swift (new)

Semantic (graph/topology) pre-flight validator that sits above the per-message Codable envelope decode, mirroring Python validating/{validator,integrity_checker,topology_analyzer}.py. Enforces: duplicate component ids, root reachability, dangling references, orphaned components, self-references and cycles, functionCall nesting depth, global logical and data-model depth limits, JSON-Pointer path syntax, and catalog-declared property schemas. It is stateful across validate calls so incremental updates (payloads without createSurface) can reference ids delivered in earlier steps while still enforcing the always-on rules.

Related cases:

all 45 validate cases in validator.yaml — e.g. test_validator_0_9, test_custom_catalog_0_9, test_validate_duplicate_ids_v09, test_validate_orphaned_component_v09, and the test_incremental_update_* series.

Test suite green with agent related tests skipped. (not for renderers)

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new test target, A2UIConformanceTests, which includes a custom YAML parser, a test harness, and several conformance test suites to validate the A2UI specification. The review feedback primarily focuses on enhancing the custom YAML parser to correctly decode escape sequences in quoted strings and to robustly support Windows-style line endings (\r\n) across sequence and mapping checks. Additionally, it is recommended to replace a hardcoded sleep in the test harness with a deterministic wait loop to prevent test flakiness on slower CI environments.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +182 to +199
if ch == "[" {
result.append(try parseFlowSequence())
consumeRestOfLine()
} else if ch == "{" {
result.append(try parseFlowMapping())
consumeRestOfLine()
} else if ch == "|" || ch == ">" {
result.append(try parseBlockScalar(keyIndent: seqIndent + 2))
} else if lookaheadIsMappingKey(from: pos) {
// Inline mapping: first key starts here, continuation lines
// are at seqIndent + 2.
result.append(try parseInlineBlockMapping(keyIndent: seqIndent + 2))
} else {
let raw = collectScalar()
result.append(parseScalar(raw))
consumeRestOfLine()
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Inline block sequence items that are quoted strings (e.g., - "value") are currently collected as raw scalars, which means escape sequences (like \\n, \\") are never decoded. We should explicitly parse them using parseDoubleQuotedString() or parseSingleQuotedString().

                let ch = source[pos]
                if ch == "[" {
                    result.append(try parseFlowSequence())
                    consumeRestOfLine()
                } else if ch == "{" {
                    result.append(try parseFlowMapping())
                    consumeRestOfLine()
                } else if ch == "|" || ch == ">" {
                    result.append(try parseBlockScalar(keyIndent: seqIndent + 2))
                } else if lookaheadIsMappingKey(from: pos) {
                    // Inline mapping: first key starts here, continuation lines
                    // are at seqIndent + 2.
                    result.append(try parseInlineBlockMapping(keyIndent: seqIndent + 2))
                } else if ch == "\"" {
                    result.append(try parseDoubleQuotedString())
                    consumeRestOfLine()
                } else if ch == "'" {
                    result.append(parseSingleQuotedString())
                    consumeRestOfLine()
                } else {
                    let raw = collectScalar()
                    result.append(parseScalar(raw))
                    consumeRestOfLine()
                }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — block-sequence items now dispatch to parseDoubleQuotedString()/parseSingleQuotedString() for "/' (YAMLDecoder.swift, commit 6583846).

Comment on lines +261 to +274
let ch = source[pos]
if ch == "[" {
value = try parseFlowSequence()
consumeRestOfLine()
} else if ch == "{" {
value = try parseFlowMapping()
consumeRestOfLine()
} else if ch == "|" || ch == ">" {
value = try parseBlockScalar(keyIndent: mapIndent + 2)
} else {
let raw = collectScalar()
value = parseScalar(raw)
consumeRestOfLine()
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Block mapping values that are quoted strings (e.g., key: "value") are currently collected as raw scalars, which means escape sequences (like \\n, \\") are never decoded. We should explicitly parse them using parseDoubleQuotedString() or parseSingleQuotedString().

            let ch = source[pos]
            if ch == "[" {
                value = try parseFlowSequence()
                consumeRestOfLine()
            } else if ch == "{" {
                value = try parseFlowMapping()
                consumeRestOfLine()
            } else if ch == "|" || ch == ">" {
                value = try parseBlockScalar(keyIndent: mapIndent + 2)
            } else if ch == "\"" {
                value = try parseDoubleQuotedString()
                consumeRestOfLine()
            } else if ch == "'" {
                value = parseSingleQuotedString()
                consumeRestOfLine()
            } else {
                let raw = collectScalar()
                value = parseScalar(raw)
                consumeRestOfLine()
            }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — block-mapping quoted values now dispatch to parseDoubleQuotedString()/parseSingleQuotedString() (YAMLDecoder.swift, commit 6583846).

Comment on lines +94 to +101
while i < source.count && source[i] != "\n" {
let ch = source[i]
if ch == ":" {
let next = (i + 1 < source.count) ? source[i + 1] : "\0"
if next == " " || next == "\n" || next == "\t" { return true }
}
i += 1
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In environments with \r\n (Windows-style) line endings, the lookahead check for mapping keys will fail to recognize keys followed by an empty value or a newline because \r is not handled as a line terminator or whitespace. We should update the loop condition and the whitespace check to include \r.

Suggested change
while i < source.count && source[i] != "\n" {
let ch = source[i]
if ch == ":" {
let next = (i + 1 < source.count) ? source[i + 1] : "\0"
if next == " " || next == "\n" || next == "\t" { return true }
}
i += 1
}
while i < source.count && source[i] != "\n" && source[i] != "\r" {
let ch = source[i]
if ch == ":" {
let next = (i + 1 < source.count) ? source[i + 1] : "\0"
if next == " " || next == "\n" || next == "\r" || next == "\t" { return true }
}
i += 1
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed differently: rather than patching every individual \r check, YAMLScanner.init now normalizes \r\n/\r\n once up front (YAML 1.2 §5.4 line-break normalization), so every downstream scan only needs to check \n. Covers this and the other four \r-handling comments below in one place (YAMLDecoder.swift, commit 6583846).

Comment on lines +158 to +159
let next = (look + 1 < source.count) ? source[look + 1] : "\0"
guard next == " " || next == "\n" || next == "\t" else { break }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When parsing block sequences, a Windows-style newline (\r\n) after a hyphen (-) will not be recognized as a valid sequence entry because \r is not included in the allowed whitespace characters for the next character check.

Suggested change
let next = (look + 1 < source.count) ? source[look + 1] : "\0"
guard next == " " || next == "\n" || next == "\t" else { break }
let next = (look + 1 < source.count) ? source[look + 1] : "\0"
guard next == " " || next == "\n" || next == "\r" || next == "\t" else { break }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — see the reply on the mapping-key-lookahead comment above: line-ending normalization in YAMLScanner.init handles this case too (commit 6583846).


skipLineWhitespace()

if pos >= source.count || source[pos] == "\n" {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Ensure \r is treated as a newline when checking for empty/newline sequence entries to prevent Windows-style line endings from being parsed as inline content.

Suggested change
if pos >= source.count || source[pos] == "\n" {
if pos >= source.count || source[pos] == "\n" || source[pos] == "\r" {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed via the same line-ending normalization in YAMLScanner.init (commit 6583846).

Comment on lines +220 to +222
if source[look] == "-" {
let nx = (look + 1 < source.count) ? source[look + 1] : "\0"
if nx == " " || nx == "\n" { break }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When checking if a block mapping contains a sequence item, we should also check for \r to support Windows-style line endings.

Suggested change
if source[look] == "-" {
let nx = (look + 1 < source.count) ? source[look + 1] : "\0"
if nx == " " || nx == "\n" { break }
if source[look] == "-" {
let nx = (look + 1 < source.count) ? source[look + 1] : "\0"
if nx == " " || nx == "\n" || nx == "\r" { break }
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed via the same line-ending normalization in YAMLScanner.init (commit 6583846).

skipLineWhitespace()

let value: Any
if pos >= source.count || source[pos] == "\n" {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In parseKeyValuePair, treat \r as a newline to correctly parse empty values followed by Windows-style line endings.

Suggested change
if pos >= source.count || source[pos] == "\n" {
if pos >= source.count || source[pos] == "\n" || source[pos] == "\r" {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed via the same line-ending normalization in YAMLScanner.init (commit 6583846).

Comment on lines +571 to +574
let c = source[pos]
if c == ":" {
let nx = (pos + 1 < source.count) ? source[pos + 1] : "\0"
if nx == " " || nx == "\n" || nx == "\t" { break }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When parsing keys, treat \r as a valid line terminator following a colon to support Windows-style line endings.

            if c == ":" {
                let nx = (pos + 1 < source.count) ? source[pos + 1] : "\0"
                if nx == " " || nx == "\n" || nx == "\r" || nx == "\t" { break }
            }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed via the same line-ending normalization in YAMLScanner.init (commit 6583846).

Comment on lines +83 to +87
var events: [ParsedEvent] = []
func append(_ e: ParsedEvent) { events.append(e) }
func drain() -> [ParsedEvent] { let r = events; events = []; return r }
}
let buffer = EventBuffer()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Add an eventsCount property to the EventBuffer actor to allow deterministic waiting in tests without exposing the mutable events array directly.

    actor EventBuffer {
        var events: [ParsedEvent] = []
        var eventsCount: Int { events.count }
        func append(_ e: ParsedEvent) { events.append(e) }
        func drain() -> [ParsedEvent] { let r = events; events = []; return r }
    }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed with a different shape: kept events private on the actor and added drainStable(expectedCount:), which polls buffer.count (an existing internal accessor) up to the expected count before falling back to the stability-window poll — same deterministic-wait goal without exposing a new public counter (ConformanceHarness.swift, commit 6583846).

Comment on lines +101 to +103
// Sleep briefly to give the cooperative scheduler time to drain events reliably.
try await Task.sleep(nanoseconds: 10_000_000)
let events = await buffer.drain()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using a hardcoded Task.sleep of 10ms to wait for cooperative scheduler tasks is a known source of test flakiness on slower CI runners. Since we know the expected number of events when step.expect is specified, we can implement a deterministic wait loop with a timeout to make the tests both faster and more robust.

Suggested change
// Sleep briefly to give the cooperative scheduler time to drain events reliably.
try await Task.sleep(nanoseconds: 10_000_000)
let events = await buffer.drain()
await parser.add(input)
// Wait for expected events if any are specified, otherwise sleep briefly
let expectedCount = (step.expect as? [Any])?.count ?? 0
if expectedCount > 0 {
let start = DispatchTime.now()
while await buffer.eventsCount < expectedCount {
if DispatchTime.now().uptimeNanoseconds - start.uptimeNanoseconds > 1_000_000_000 {
break
}
try? await Task.sleep(nanoseconds: 1_000_000)
}
} else {
try? await Task.sleep(nanoseconds: 10_000_000)
}
let events = await buffer.drain()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed along the same lines as suggested — drainStable now takes an expectedCount (derived from step.expect's flattened event count) and polls for it before falling back to the fixed sleep, bounded at ~1s (ConformanceHarness.swift, commit 6583846).

@fangmobile
fangmobile marked this pull request as draft July 6, 2026 18:36
fangmobile and others added 10 commits July 6, 2026 11:47
…am conformance files

Vendor 5 upstream YAML suite files (streaming_parser, parser, validator, catalog, schema_manager) and 5 JSON test data files (simplified_s2c_v08/v09, simplified_catalog_v08/v09, simplified_common_types_v09) from https://github.com/a2ui-project/a2ui for conformance testing. Add corresponding SPM test target with Resources bundle.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pure-Swift character-level parser handling the YAML subset used by the
5 conformance suite files — block/flow sequences and mappings, block
scalars, quoted strings, and scalar coercion. All 9 YAMLDecoder tests
pass including exact case counts (validator=45, streaming_parser=76,
catalog=24).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements ConformanceHarness.swift with alignErrorMatch, assertErrorMatches,
skipAgentOnlyAction, runProcessChunk, runParseFull, and runValidate dispatchers.
Adds ConformanceHarnessTests.swift covering alignErrorMatch and skipAgentOnlyAction.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…g error test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement StreamingParserConformanceTests XCTestCase that loads and runs the 76-case streaming_parser.yaml conformance suite. The test iterates through all cases, skips agent-only actions, and dispatches process_chunk operations via the conformance harness.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…catalog, and schema_manager

Add 4 new XCTestCase implementations for the remaining conformance suites:
- ParserConformanceTests: handles parse_full and fix_payload actions
- ValidatorConformanceTests: handles validate action
- CatalogConformanceTests: skips all catalog actions (agent-side only)
- SchemaManagerConformanceTests: skips all schema_manager actions (agent-side only)

All classes follow the same pattern as StreamingParserConformanceTests, using
the shared ConformanceHarness dispatcher functions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix dead XCTAssertFalse guard in CatalogConformanceTests and SchemaManagerConformanceTests
- Fix race condition in runProcessChunk (Task.sleep instead of yield loop; await consumeTask.value)
- Fix runValidate to skip v0.8 catalog cases instead of decoding with wrong schema
- Fix silent swallow of load failures: XCTAssertFalse → guard+XCTSkip in all 5 test classes
- Convert ConformanceCaseTests and YAMLDecoderTests from Swift Testing to XCTestCase

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Migrates ConformanceHarnessTests from Swift Testing (@Suite/@Test/@expect)
to XCTestCase (func testXxx/XCTAssert*). All 9 tests pass. Replaces:
- import Testing with import XCTest
- @suite(…) struct with final class: XCTestCase
- @test func with func testXxx
- #expect comparisons with XCTAssertEqual
- #expect(bool) with XCTAssertTrue
- Manual throw detection with XCTAssertThrowsError/do-catch

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@fangmobile
fangmobile force-pushed the comformance-harness-a branch from ca83b1f to 1c6b341 Compare July 6, 2026 18:48
fangmobile and others added 13 commits July 6, 2026 14:50
Adds A2uiPayloadValidator, a pre-flight graph/topology validator for v0.9
payloads that mirrors the Python reference validator
(a2ui_core/core/validating): duplicate-id, missing-root, dangling-reference,
orphaned-component, self-reference, circular-reference, functionCall depth,
global logical/data-model depth, JSON-Pointer path syntax, and catalog-declared
property type checks.

Wires the ValidatorConformanceTests harness (runValidate/validatePayload) to
run the semantic validator after the envelope Codable decode, using one
validator instance per case so incremental-update steps validate against ids
accumulated from earlier steps.

Adds A2uiIntegrityError and A2uiRecursionError to Errors.swift.

Fixes all 16 v0.9 assertions in ValidatorConformanceTests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Implement the non-streaming counterpart to A2UIStreamParser as a
self-contained type in A2UISwiftCore (zero edits to the streaming parser
so the concurrent incremental-emission work merges cleanly).

- parseFull(_:): splits a complete agent response into ordered
  A2uiResponsePart values (leading/trailing prose + decoded <a2ui-json>
  payloads). Strict: throws A2uiParseError when no block is present
  ("not found in response"), a block is empty ("A2UI JSON part is empty"),
  or the JSON is malformed ("Failed to parse ..."), matching the upstream
  Python parser's messages.
- fixPayload(_:): repairs common LLM JSON mistakes — smart/curly quote
  normalization, trailing-comma removal (string-literal aware, nested
  objects/arrays), and auto-wrapping a single object into an array.
- hasA2uiParts(_:): true only when a complete open+close tag pair exists.
- New A2uiParseError (code PARSE_ERROR) conforming to A2uiError.
- Markdown code fences inside tags are stripped before decoding.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rewrite the JSON-mode of A2UIStreamParser to scan the <a2ui-json> block
character-by-character (brace/string state machine) and emit .message events
as each top-level array element, partial component, or partial data-model
update becomes complete — instead of only after </a2ui-json>. Ports the
upstream Python streaming.py behavior: fixJson string/brace healing gated on
cuttable keys, empty-dict discarding, empty-key pruning, catalog required-field
enforcement, createSurface-ordering buffering for updateComponents, and
cycle/self-reference detection. Adds A2UIStreamParserConfig (cuttable keys +
required-fields map) with a no-arg default that preserves the structural
fallback and complete-block behavior used by A2UISwiftCoreTests.

Harness: runProcessChunk flattens each YAML expect part (text + N-message
a2ui array) into the ordered event sequence and drains via count-stable
polling; builds the parser config from the case catalog + custom_cuttable_keys.

Takes StreamingParserConformanceTests from 78 failures to 11.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…L decoder

The conformance YAML decoder parsed a quoted scalar value after `key:`
(e.g. `input: "Hello\n<a2ui-json>[{\"id\": \"test\"}]</a2ui-json>"`) via
the raw collectScalar path, which kept backslash escapes literal. That
corrupted embedded JSON (`\"` stayed as backslash-quote, `\n` as
backslash-n), so parser.yaml's double-quoted parse_full inputs decoded to
invalid JSON.

Route `"`- and `'`-prefixed block-mapping values through the existing
escape-aware parseDoubleQuotedString / parseSingleQuotedString readers.
YAMLDecoderTests and all other conformance suites are unaffected (validator
failure count unchanged at 15 before/after).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…arser

Rewrite the parse_full harness dispatch to drive the new
A2UIResponseParser instead of wrapping A2UIStreamParser (which silently
yielded nothing for the error cases and could not validate fix_payload
outcomes).

- runParseFull now branches on action: parse_full asserts thrown
  A2uiParseError against expect_error, else compares each decoded
  A2uiResponsePart's text AND a2ui payload to the YAML expect entries via
  a new order-independent jsonEqual helper.
- fix_payload decodes the repaired payload and asserts structural JSON
  equality with the expected (always-array) value.
- Add runHasParts for the has_parts boolean cases, and route parse_full /
  fix_payload / has_parts through the ParserConformanceTests switch before
  the agent-only skip so a mid-loop XCTSkip no longer aborts the later
  fix_payload cases.
- Drop the now-unused parseFullResponse/encodeMessageToAny helpers.

Parser suite: 21 assertion failures -> 0 (all 20 applicable cases now run
and pass; full suite drops from 116 to 85 pre-existing failures owned by
the concurrent streaming/validator work).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ponseParser

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Covers each validation rule independently of the YAML conformance harness,
including positive-boundary cases the conformance suite only exercises on the
throwing side: a shallow functionCall is valid, a component chain at the logical
depth limit is valid, and a well-formed JSON Pointer is valid. These lock in the
reference-matching behavior and guard against false positives on valid payloads.

The deeply-nested functionCall test documents that the v0.9 wire form
(functionCall wrapper around each {call,args}) increments funcDepth twice per
logical call — matching the Python reference, which throws on the same fixture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Wrap in-block decode failures in A2uiValidationError("Validation failed: ...")
  so error-category cases match the harness pattern.
- Data-model sniff: add the upstream rsplit-on-comma fallback (via fixAndDecode)
  so a dangling "key with no value heals to the last valid entry; re-yield only
  when a top-level value changed (value-based delta), and keep explicitly-opened
  empty dicts (items:{}) instead of pruning them.
- Component sniff keeps fixJson-only (no comma-strip) so a partial non-cuttable
  value (e.g. a path) holds the whole component back.
- Read custom_cuttable_keys from the catalog block (its actual YAML location).

StreamingParserConformanceTests: 11 -> 4 failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cache components as they close (nested or top-level), then yield a partial
updateComponents once per chunk (and on complete top-level updateComponents)
whenever the set of components reachable from `root` changes — mirroring
upstream yield_reachable off _seen_components rather than a perfectly-balanced
outer object (recovers the malformed trailing braces some conformance cases
feed). Reachability rules:
- a child ref target that is seen is followed;
- an unseen target through a ChildList/ComponentId-typed field is renderable
  only when the catalog can synthesise the loading placeholder (defines Row);
- an unseen target through any other field holds the parent back.
Complete top-level updateComponents are decode-validated so a structurally
invalid one still surfaces .error; keyNotFound/valueNotFound are phrased to
match the harness error-alignment. Harness config now derives child-ref
fields and placeholder availability from the catalog schema.

StreamingParserConformanceTests: 4 -> 1 failure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The final reachability rule: a child-ref-named field (child/children/…) whose
target is unseen renders as a loading placeholder, and the parent is yieldable
only when the catalog can synthesise that placeholder (defines the v0.9 `Row`
type). This distinguishes cases with identical schema shape but opposite
expectations (empty-children-dict discards + yields with a placeholder because
Row exists; partial-children-lists holds because Row is absent). Drops the
now-redundant ChildList/ComponentId ref-field detection.

StreamingParserConformanceTests: 1 -> 0 failures (all 38 running v0.9 cases pass).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@fangmobile
fangmobile marked this pull request as ready for review July 6, 2026 23:38
@fangmobile

Copy link
Copy Markdown
Contributor Author

@BBC6BAE9 Please take a look?

@fangmobile

Copy link
Copy Markdown
Contributor Author

@gemini-code-assist[bot] review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive validation and parsing capabilities for v0.9 A2UI payloads, including a new A2uiPayloadValidator for semantic graph and topology checks, an A2UIResponseParser for complete responses, and incremental scanning support in A2UIStreamParser. It also adds a robust suite of conformance tests. The review feedback highlights critical issues to address: a bug on Darwin platforms where NSNumber bridging to Bool breaks numeric type validation, a state leak in A2UIStreamParser when processing deleteSurface messages, and a performance bottleneck caused by $O(N^2)$ string slicing in the stream parser's substring helper.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +454 to +461
private func isNumber(_ value: Any) -> Bool {
if value is Bool { return false }
if let number = value as? NSNumber {
// Exclude boolean NSNumbers.
return CFGetTypeID(number) != CFBooleanGetTypeID()
}
return value is Int || value is Double || value is Float
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

On Darwin platforms (iOS/macOS), NSNumber bridges to Bool, which means value is Bool evaluates to true for any numeric NSNumber (e.g., NSNumber(value: 1) or NSNumber(value: 3.14)). This causes isNumber to incorrectly return false for all numbers, breaking validation for numeric properties. To reliably distinguish between booleans and numbers, we should check the CFGetTypeID of the bridged NSNumber first. We can introduce an isBool helper to handle this correctly.

    private func isNumber(_ value: Any) -> Bool {
        if let number = value as? NSNumber {
            // Exclude boolean NSNumbers.
            return CFGetTypeID(number) != CFBooleanGetTypeID()
        }
        return value is Int || value is Double || value is Float
    }

    private func isBool(_ value: Any) -> Bool {
        if let number = value as? NSNumber {
            return CFGetTypeID(number) == CFBooleanGetTypeID()
        }
        return value is Bool
    }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed. Verified empirically: on Darwin, JSONSerialization-decoded integers 0/1 bridge to is Bool == true (ObjC BOOL/char aliasing) — e.g. {"v":1} decodes to a value where is Bool is true but is Int/is NSNumber are also true. Added isBool(_:) using CFGetTypeID as you suggested, and reworked isNumber to rely solely on CFGetTypeID (dropping the value is Bool early-return, which was the actual source of the misclassification) so integer 0/1 are never excluded from isNumber. Added a regression test (test_booleanProperty_rejectsIntegerZeroOrOne) that fails without this fix.

Comment on lines +429 to +450
private func validateType(_ value: Any, expected: String) throws {
let ok: Bool
switch expected {
case "string":
ok = value is String
case "number", "integer":
ok = isNumber(value)
case "boolean":
ok = value is Bool && !isNumber(value)
case "object":
ok = value is [String: Any]
case "array":
ok = value is [Any]
case "null":
ok = value is NSNull
default:
ok = true
}
if !ok {
throw A2uiValidationError("\(describe(value)) is not of type '\(expected)'")
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Update validateType to use the new isBool helper to ensure boolean type validation is robust and does not incorrectly match numeric NSNumber values.

    private func validateType(_ value: Any, expected: String) throws {
        let ok: Bool
        switch expected {
        case "string":
            ok = value is String
        case "number", "integer":
            ok = isNumber(value)
        case "boolean":
            ok = isBool(value)
        case "object":
            ok = value is [String: Any]
        case "array":
            ok = value is [Any]
        case "null":
            ok = value is NSNull
        default:
            ok = true
        }
        if !ok {
            throw A2uiValidationError("\(describe(value)) is not of type '\(expected)'")
        }
    }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — validateType's "boolean" case now calls isBool(value).

Comment on lines +464 to +480
private func describe(_ value: Any) -> String {
if let string = value as? String { return "'\(string)'" }
if value is Bool { return "\(value)" }
if let number = value as? NSNumber, CFGetTypeID(number) != CFBooleanGetTypeID() {
// Print integers without a trailing ".0".
if number.doubleValue == number.doubleValue.rounded() {
return "\(number.intValue)"
}
return "\(number.doubleValue)"
}
if let int = value as? Int { return "\(int)" }
if let double = value as? Double {
if double == double.rounded() { return "\(Int(double))" }
return "\(double)"
}
return "\(value)"
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Update describe to use the new isBool helper to prevent numeric NSNumber values (like 1 or 0) from being incorrectly described as "true" or "false" due to Swift's Bool bridging.

Suggested change
private func describe(_ value: Any) -> String {
if let string = value as? String { return "'\(string)'" }
if value is Bool { return "\(value)" }
if let number = value as? NSNumber, CFGetTypeID(number) != CFBooleanGetTypeID() {
// Print integers without a trailing ".0".
if number.doubleValue == number.doubleValue.rounded() {
return "\(number.intValue)"
}
return "\(number.doubleValue)"
}
if let int = value as? Int { return "\(int)" }
if let double = value as? Double {
if double == double.rounded() { return "\(Int(double))" }
return "\(double)"
}
return "\(value)"
}
private func describe(_ value: Any) -> String {
if let string = value as? String { return "'\(string)'" }
if isBool(value) { return "\(value)" }
if let number = value as? NSNumber, CFGetTypeID(number) != CFBooleanGetTypeID() {
// Print integers without a trailing ".0".
if number.doubleValue == number.doubleValue.rounded() {
return "\(number.intValue)"
}
return "\(number.doubleValue)"
}
if let int = value as? Int { return "\(int)" }
if let double = value as? Double {
if double == double.rounded() { return "\(Int(double))" }
return "\(double)"
}
return "\(value)"
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — describe now calls isBool(value) instead of value is Bool.

Comment on lines +482 to +489
private func jsonEquals(_ lhs: Any, _ rhs: Any) -> Bool {
if let l = lhs as? String, let r = rhs as? String { return l == r }
if isNumber(lhs) && isNumber(rhs) {
return (lhs as? NSNumber)?.doubleValue == (rhs as? NSNumber)?.doubleValue
}
if let l = lhs as? Bool, let r = rhs as? Bool { return l == r }
return false
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Update jsonEquals to use the new isBool helper to prevent numeric NSNumber values from being incorrectly compared and matched as equal to boolean values.

Suggested change
private func jsonEquals(_ lhs: Any, _ rhs: Any) -> Bool {
if let l = lhs as? String, let r = rhs as? String { return l == r }
if isNumber(lhs) && isNumber(rhs) {
return (lhs as? NSNumber)?.doubleValue == (rhs as? NSNumber)?.doubleValue
}
if let l = lhs as? Bool, let r = rhs as? Bool { return l == r }
return false
}
private func jsonEquals(_ lhs: Any, _ rhs: Any) -> Bool {
if let l = lhs as? String, let r = rhs as? String { return l == r }
if isNumber(lhs) && isNumber(rhs) {
return (lhs as? NSNumber)?.doubleValue == (rhs as? NSNumber)?.doubleValue
}
if isBool(lhs) && isBool(rhs) {
return (lhs as? Bool) == (rhs as? Bool)
}
return false
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — jsonEquals now uses isBool(lhs) && isBool(rhs) instead of the as? Bool double-cast, consistent with the other three sites.

Comment on lines +411 to +418
if obj["deleteSurface"] is [String: Any] {
// A deleteSurface before the surface exists is dropped (upstream `_delete_surface`
// simply clears state); otherwise emit it.
let sid = (obj["deleteSurface"] as? [String: Any])?["surfaceId"] as? String
if let sid, !startedSurfaces.contains(sid) { return }
emitDecodedOrError(obj)
return
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When a deleteSurface message is processed, the parser should clear all cached state associated with that surface (such as startedSurfaces, pendingComponentSurfaces, seenComponents, lastReachableSignature, and yieldedDataModelValues). Failing to do so causes a memory leak and can lead to stale state or incorrect cycle/reachability validation if the same surface ID is reused later in the stream.

Suggested change
if obj["deleteSurface"] is [String: Any] {
// A deleteSurface before the surface exists is dropped (upstream `_delete_surface`
// simply clears state); otherwise emit it.
let sid = (obj["deleteSurface"] as? [String: Any])?["surfaceId"] as? String
if let sid, !startedSurfaces.contains(sid) { return }
emitDecodedOrError(obj)
return
}
if obj["deleteSurface"] is [String: Any] {
let sid = (obj["deleteSurface"] as? [String: Any])?["surfaceId"] as? String
if let sid {
if !startedSurfaces.contains(sid) { return }
startedSurfaces.remove(sid)
pendingComponentSurfaces.remove(sid)
seenComponents.removeValue(forKey: sid)
lastReachableSignature.removeValue(forKey: sid)
yieldedDataModelValues.removeValue(forKey: sid)
}
emitDecodedOrError(obj)
return
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed as suggested — deleteSurface now removes the surface id from startedSurfaces, pendingComponentSurfaces, seenComponents, lastReachableSignature, and yieldedDataModelValues before emitting. Added a regression test (deleteSurfaceClearsReachabilityState) that reuses a surfaceId after delete with an identical component shape — without the fix, the second reachability yield was silently dropped as a dup of the stale pre-delete signature.

Comment on lines +982 to +985
private func substring(_ s: String, from offset: Int) -> String {
guard offset >= 0, offset <= s.count else { return "" }
return String(s[s.index(s.startIndex, offsetBy: offset)...])
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In Swift, String is not a random-access collection, so s.count and s.index(s.startIndex, offsetBy: offset) are $O(N)$ operations. Since substring is called frequently during streaming, this leads to quadratic time complexity $O(N^2)$ on large inputs, which can cause significant CPU overhead and UI thread lag. Consider refactoring jsonBuffer to be of type [Character] (where count is $O(1)$ and slicing is highly efficient) or storing String.Index directly in braceStack instead of integer offsets.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, though via a narrower change than the suggested String→ full rewrite: converted jsonBuffer to [Character] (O(1) count/indexing, O(k) slicing) and updated its call sites (substring, dropConsumedObject, the two contains(...) key probes). Left buffer (the outer text-mode buffer) as String since its index(offsetBy:) calls are bounded by small tag-prefix lengths, not the full buffer, so they don't have the same quadratic exposure.

@BBC6BAE9

BBC6BAE9 commented Jul 19, 2026

Copy link
Copy Markdown
Owner

@BBC6BAE9 Please take a look?

@fangmobile Sorry for the long wait, and thanks for all your hard work. I think it would be best to put this on hold for now and resume the work once WebCore 1.0 is officially released.

@fangmobile
fangmobile marked this pull request as draft July 20, 2026 17:29
@fangmobile

Copy link
Copy Markdown
Contributor Author

@BBC6BAE9 Please take a look?

@fangmobile Sorry for the long wait, and thanks for all your hard work. I think it would be best to put this on hold for now and resume the work once WebCore 1.0 is officially released.

Sure. I put it back to draft for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants