feat(formats): implement Express and Elemental `InferenceFormat`s by gspencergoog · Pull Request #1982 · a2ui-project/a2ui

gspencergoog · 2026-07-10T01:59:31Z

Summary

This PR unifies the structured inference SDK API and evaluation harness in Python to support multiple inference formats (standard JSON transport, experimental Express DSL, and experimental Elemental HTML). It encapsulates format-specific details using polymorphic implementations of the InferenceFormat interface.

Changes

Polymorphic format abstractions: Defined the InferenceFormat base class contract and created polymorphic implementations TransportFormat, ExpressFormat, and ElementalFormat.
Decoupled strategy implementations: Relocated format-specific prompt generators, parsers, and decompilers under the unified formats hierarchy.
Elemental compiler enhancements:
- Restricted ElementalCompiler root validation to <body>.
- Wrapped HTML layout parsing in a virtual <root> tag dynamically during compilation to support top-level sibling tags (like <link> outside <body>).
- Added property type coercion (boolean, float, integer) and fallback expression parsing for string array/object attributes (such as options).
Inference sentinel constants: Extracted <a2ui> and </a2ui> tags into A2UI_INFERENCE_OPEN_TAG and A2UI_INFERENCE_CLOSE_TAG constants.
Evaluation solver & scorer improvements:
- Consolidated separate format solvers into a single polymorphic format.py solver.
- Made experiments settings conditional on version == "1.0" in both solver and scorer to avoid enabling experimental version 1.0 features for older version tests.
Documentation: Added docstrings describing properties of the ClientUiCapabilities TypedDict.

Impact & Risks

This refactor changes internal SDK package layouts and imports, but preserves existing public API boundaries. It is non-breaking for external integrations.

Testing

Verify the changes by running all python unit tests and the evaluation suite runs:

uv run pytest
uv run main.py --strategies express,elemental

- Make experiments parameter conditional on version == "1.0" in eval format strategy resolver. - Support resolving surface ID from both a2ui and body tags in completion solver regex. - Auto-close unclosed a2ui tags during streaming compilation in ElementalParser. - Ensure prompt generator delegates example decompilation using its pruned catalog decompiler if available. - Call _ensure_catalog before instantiating prompt generator in ExpressFormat property. - Fix prefix check for action properties in ElementalDecompiler to look for "on-".

…s and parser classes

…tants

…in compiler

…alid compiler root tag

… nested HTML compiler test tag wrapping

…l & clean up tests - Wrap Elemental HTML input in a virtual `<root>` tag to parse top-level sibling elements (like `<link>` outside `<body>`) without requiring LLM format changes. - Add attribute type coercion (boolean, number, integer) and fallback JSON-like array/object parsing to handle LLM format variations. - Make the `experiments` configuration in evals conditional on `version == "1.0"` in both solver and scorer to avoid loading v1.0 schemas for older formats. - Merge the redundant `test_refactoring.py` assertions into `test_formats.py` and unify catalog provider test structures. - Delete the empty `eval/tests/conftest.py` file.

…tion

…trategy

… validation checks

nan-yu · 2026-07-17T17:16:30Z

        self._catalog = catalog
        from urllib.parse import urljoin
-        from jsonschema import Draft202012Validator
+        from jsonschema import Draft202012Validator, FormatChecker


Not for this PR, but A2uiValidatorWrapperV10 uses jsonschema validation, while v0.9 uses pydantic model validation. I'll update the a2ui-core to consolidate the validation across multiple versions separately.

…yout attributes

…ments in functions

…own cleaning - Refactor unwrap_response in TransportParser to use BlockLexer for robust boundary detection. - Unify markdown code block wrapper cleaning in BlockLexer into a single _clean_markdown method that handles both conversational text and inner blocks. - Remove duplicate _sanitize_json_string and unused imports. - Add unit tests for inner markdown block cleaning and nested close tags in JSON string literals.

# Conflicts: # eval/a2ui_eval/strategies/express.py # eval/a2ui_eval/strategies/subagent_tool.py # eval/tests/test_strategies.py # pyproject.toml

…tegies' into unified-inference-formats-strategies

…trip codecs - Consolidated compilation and decompilation under the Parser class, turning them into symmetric codecs. - Modified InferenceFormat to remove the separate decompiler property. - Retained format decompiler classes as private internal helpers (_TransportDecompiler, _ExpressDecompiler, _ElementalDecompiler) to preserve SRP. - Updated all unit, integration, and roundtrip tests, as well as helper scripts, to use the parser decompile APIs.

…entUiCapabilities - Replaced custom ClientUiCapabilities TypedDict in a2ui_agent with the standardized V09Capabilities model from a2ui_core. - Updated base prompt interfaces, template manager, and format-specific prompt generators to support Union[dict, V09Capabilities]. - Implemented dict-to-model validation coercion in TransportFormat._select_catalog with fallback for missing optional fields in conformance tests. - Deleted src/a2ui/schema/capabilities.py.

github-project-automation Bot added this to A2UI Jul 10, 2026

github-project-automation Bot moved this to Todo in A2UI Jul 10, 2026

gspencergoog force-pushed the unified-inference-formats-strategies branch 5 times, most recently from 34a0c18 to 0101648 Compare July 10, 2026 02:38

a2ui-project deleted a comment from gemini-code-assist Bot Jul 10, 2026

This was referenced Jul 10, 2026

refactor(formats): introduce InferenceFormat base API #1981

Merged

refactor(eval): unify evaluation strategy solvers using generic InferenceFormat solver #1983

Merged

This comment was marked as resolved.

Sign in to view

gspencergoog requested a review from jacobsimionato July 10, 2026 03:18

gspencergoog force-pushed the unified-inference-formats-strategies branch 3 times, most recently from 6238957 to 82bd501 Compare July 10, 2026 16:56

gspencergoog force-pushed the unified-inference-formats-foundation branch from 89a547a to 1134c08 Compare July 10, 2026 16:59

gspencergoog force-pushed the unified-inference-formats-strategies branch 10 times, most recently from 5266cef to 77a52dd Compare July 13, 2026 21:12

gspencergoog mentioned this pull request Jul 13, 2026

feat(agent-sdk): Add streaming support to Python agent SDK formats #1996

Open

gspencergoog force-pushed the unified-inference-formats-foundation branch from dd30576 to 914f571 Compare July 13, 2026 21:44

gspencergoog force-pushed the unified-inference-formats-strategies branch 2 times, most recently from c0e7429 to 33fc7c2 Compare July 13, 2026 21:49

Merge branch 'main' into unified-inference-formats-strategies

9b0b6cc

This comment was marked as resolved.

Sign in to view

gspencergoog added 8 commits July 16, 2026 14:52

docs(python-sdk): document public interfaces and properties of format…

81e3f58

…s and parser classes

refactor(python-sdk): pull out inference sentinel tags as schema cons…

e5049c3

…tants

Merge branch 'main' into unified-inference-formats-strategies

f58b0bb

refactor(elemental): enforce body root container tag instead of a2ui …

b41996e

…in compiler

test(elemental): add unit tests for multiple action prefixing and inv…

57ece8a

…alid compiler root tag

docs(python-sdk): document ClientUiCapabilities attributes and update…

6aaeeb0

… nested HTML compiler test tag wrapping

This comment was marked as resolved.

Sign in to view

gspencergoog added 5 commits July 16, 2026 17:11

fix(review): address PR 1982 code assist review feedback

a850f5a

fix(agent-adk): resolve format and registry validation bugs in evalua…

c6df921

…tion

fix(agent-adk): resolve format and registry validation bugs in evalua…

c2af6f3

…tion

feat(eval): split evaluation datasets and optimize elemental prompt s…

5fd6239

…trategy

feat(experimental/elemental): improve type mapping, layout rules, and…

17d7ab8

… validation checks

nan-yu reviewed Jul 17, 2026

View reviewed changes

gspencergoog added 5 commits July 17, 2026 10:26

feat(elemental): clarify list template uniqueness and warn against la…

88747e0

…yout attributes

feat(elemental): add warning against mixing positional and named argu…

7193ea3

…ments in functions

Merge branch 'main' into unified-inference-formats-strategies

7f3c5d7

# Conflicts: # eval/a2ui_eval/strategies/express.py # eval/a2ui_eval/strategies/subagent_tool.py # eval/tests/test_strategies.py # pyproject.toml

Merge remote-tracking branch 'upstream/unified-inference-formats-stra…

4868407

…tegies' into unified-inference-formats-strategies

gspencergoog force-pushed the unified-inference-formats-strategies branch from 9c08077 to 4868407 Compare July 17, 2026 19:19

gspencergoog added 2 commits July 17, 2026 12:44

nan-yu approved these changes Jul 17, 2026

View reviewed changes

Merge branch 'main' into unified-inference-formats-strategies

6072e02

gspencergoog merged commit 2472202 into main Jul 17, 2026
26 checks passed

github-project-automation Bot moved this from Todo to Done in A2UI Jul 17, 2026

gspencergoog deleted the unified-inference-formats-strategies branch July 17, 2026 21:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(formats): implement Express and Elemental `InferenceFormat`s #1982

feat(formats): implement Express and Elemental `InferenceFormat`s #1982
gspencergoog merged 38 commits into
mainfrom
unified-inference-formats-strategies

gspencergoog commented Jul 10, 2026 •

edited

Loading

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

nan-yu Jul 17, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

gspencergoog commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Impact & Risks

Testing

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

nan-yu Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gspencergoog commented Jul 10, 2026 •

edited

Loading