Skip to content

fix: prompt builder misleading LLM to return free text amongst A2UI messages #841

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gspencergoog merged 1 commit into from
Apr 8, 2026
Merged

fix: prompt builder misleading LLM to return free text amongst A2UI messages #841

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/genui/lib/src/facade/prompt_builder.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,7 @@ To update an existing UI:
_fenced('''
When constructing UI, you must output a VALID A2UI JSON object representing one of the A2UI message types ($_operationsFormatted).
- You can treat the A2UI schema as a specification for the JSON you typically output.
- You may include a brief conversational explanation before or after the JSON block if it helps the user, but the JSON block must be valid and complete.
- The JSON block must be valid and complete.
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While removing the explicit permission for conversational text is a good step, LLMs often still include it unless explicitly prohibited. Additionally, since the instructions in the 'CONTROLLING THE UI' section describe multiple steps requiring multiple JSON messages (e.g., for creating a new UI), using plural phrasing or 'Each' is more accurate and avoids confusing the model into thinking it should only output a single block.

Suggested change
- The JSON block must be valid and complete.
- Each JSON block must be valid and complete. Do not include any conversational text or explanations outside of the JSON blocks.

- Ensure your JSON is fenced with ```json and ```.
''', sectionName: 'OUTPUT FORMAT'),
);
Expand Down
2 changes: 1 addition & 1 deletion ...ages/genui/test/facade/prompt_builder_test.golden/all_operations_with_dataModel_false.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ To update an existing UI:
-----OUTPUT_FORMAT_START-----
When constructing UI, you must output a VALID A2UI JSON object representing one of the A2UI message types (`createSurface`, `updateComponents`, `deleteSurface`).
- You can treat the A2UI schema as a specification for the JSON you typically output.
- You may include a brief conversational explanation before or after the JSON block if it helps the user, but the JSON block must be valid and complete.
- The JSON block must be valid and complete.
- Ensure your JSON is fenced with ```json and ```.
-----OUTPUT_FORMAT_END-----

Expand Down
2 changes: 1 addition & 1 deletion packages/genui/test/facade/prompt_builder_test.golden/all_operations_with_dataModel_true.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ To update an existing UI:
-----OUTPUT_FORMAT_START-----
When constructing UI, you must output a VALID A2UI JSON object representing one of the A2UI message types (`createSurface`, `updateComponents`, `deleteSurface`, `updateDataModel`).
- You can treat the A2UI schema as a specification for the JSON you typically output.
- You may include a brief conversational explanation before or after the JSON block if it helps the user, but the JSON block must be valid and complete.
- The JSON block must be valid and complete.
- Ensure your JSON is fenced with ```json and ```.
-----OUTPUT_FORMAT_END-----

Expand Down
2 changes: 1 addition & 1 deletion ...s/genui/test/facade/prompt_builder_test.golden/create_and_update_with_dataModel_false.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ To update an existing UI:
-----OUTPUT_FORMAT_START-----
When constructing UI, you must output a VALID A2UI JSON object representing one of the A2UI message types (`createSurface`, `updateComponents`).
- You can treat the A2UI schema as a specification for the JSON you typically output.
- You may include a brief conversational explanation before or after the JSON block if it helps the user, but the JSON block must be valid and complete.
- The JSON block must be valid and complete.
- Ensure your JSON is fenced with ```json and ```.
-----OUTPUT_FORMAT_END-----

Expand Down
2 changes: 1 addition & 1 deletion ...es/genui/test/facade/prompt_builder_test.golden/create_and_update_with_dataModel_true.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ To update an existing UI:
-----OUTPUT_FORMAT_START-----
When constructing UI, you must output a VALID A2UI JSON object representing one of the A2UI message types (`createSurface`, `updateComponents`, `updateDataModel`).
- You can treat the A2UI schema as a specification for the JSON you typically output.
- You may include a brief conversational explanation before or after the JSON block if it helps the user, but the JSON block must be valid and complete.
- The JSON block must be valid and complete.
- Ensure your JSON is fenced with ```json and ```.
-----OUTPUT_FORMAT_END-----

Expand Down
2 changes: 1 addition & 1 deletion packages/genui/test/facade/prompt_builder_test.golden/create_only_with_dataModel_false.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ IMPORTANT: DO NOT update or modify surfaces created in previous turns. If the UI
-----OUTPUT_FORMAT_START-----
When constructing UI, you must output a VALID A2UI JSON object representing one of the A2UI message types (`createSurface`, `updateComponents`).
- You can treat the A2UI schema as a specification for the JSON you typically output.
- You may include a brief conversational explanation before or after the JSON block if it helps the user, but the JSON block must be valid and complete.
- The JSON block must be valid and complete.
- Ensure your JSON is fenced with ```json and ```.
-----OUTPUT_FORMAT_END-----

Expand Down
2 changes: 1 addition & 1 deletion packages/genui/test/facade/prompt_builder_test.golden/create_only_with_dataModel_true.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ IMPORTANT: DO NOT update or modify surfaces created in previous turns. If the UI
-----OUTPUT_FORMAT_START-----
When constructing UI, you must output a VALID A2UI JSON object representing one of the A2UI message types (`createSurface`, `updateComponents`, `updateDataModel`).
- You can treat the A2UI schema as a specification for the JSON you typically output.
- You may include a brief conversational explanation before or after the JSON block if it helps the user, but the JSON block must be valid and complete.
- The JSON block must be valid and complete.
- Ensure your JSON is fenced with ```json and ```.
-----OUTPUT_FORMAT_END-----

Expand Down
2 changes: 1 addition & 1 deletion packages/genui/test/facade/prompt_builder_test.golden/update_only_with_dataModel_false.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ To update an existing UI:
-----OUTPUT_FORMAT_START-----
When constructing UI, you must output a VALID A2UI JSON object representing one of the A2UI message types (`updateComponents`).
- You can treat the A2UI schema as a specification for the JSON you typically output.
- You may include a brief conversational explanation before or after the JSON block if it helps the user, but the JSON block must be valid and complete.
- The JSON block must be valid and complete.
- Ensure your JSON is fenced with ```json and ```.
-----OUTPUT_FORMAT_END-----

Expand Down
2 changes: 1 addition & 1 deletion packages/genui/test/facade/prompt_builder_test.golden/update_only_with_dataModel_true.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ To update an existing UI:
-----OUTPUT_FORMAT_START-----
When constructing UI, you must output a VALID A2UI JSON object representing one of the A2UI message types (`updateComponents`, `updateDataModel`).
- You can treat the A2UI schema as a specification for the JSON you typically output.
- You may include a brief conversational explanation before or after the JSON block if it helps the user, but the JSON block must be valid and complete.
- The JSON block must be valid and complete.
- Ensure your JSON is fenced with ```json and ```.
-----OUTPUT_FORMAT_END-----

Expand Down
Loading