Skip to content

Retain UI render world data.#24893

Open
pcwalton wants to merge 9 commits into
bevyengine:mainfrom
pcwalton:retained-ui
Open

Retain UI render world data.#24893
pcwalton wants to merge 9 commits into
bevyengine:mainfrom
pcwalton:retained-ui

Conversation

@pcwalton

@pcwalton pcwalton commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

At present, Bevy re-extracts all render world data for the UI from scratch every frame. This is wasteful, as most UI widgets don't change from frame to frame.

This PR changes Bevy UI to retain phase items and render-world entities from frame to frame in the render world. It uses ECS change detection to only re-extract UI widgets that have changed in some way.

UI widgets differ from 2D and 3D meshes in that a single UI widget in the main world may extract to multiple phase items in the render world. For UI nodes, multiple systems process the same node repeatedly in order to generate the various rendered pieces (the base node, drop shadows, backgrounds, etc.) To handle this, I split out the change detection and invalidation logic into a separate system, extract_uinode_changes, that runs before any of the main extraction systems. When that system detects a change to any of the components that are relevant to rendering, all render-world entities corresponding to the changed main-world entity are despawned and created anew. This isn't as efficient as it could be, but it's fine for now and is a sizable improvement to the current situation.

Note that one unfortunate side effect of these changes is that we now mark UI nodes as changed if their visuals change in any way, as opposed to only when their layout changes. The layout code uses the Changed flag to determine whether reflow needs to be performed, so reflows will now be needlessly performed if e.g. the color of a node changes, even though color only affects visuals and not layout and so the reflow is unnecessary. I believe that this use of the changed flag by layout was always an abuse, because Changed is supposed to be set whenever a component changes, not just when some part of a component relevant to some arbitrary system (in this case, layout) changes. In other words, the Changed flag should be for every system to use; layout shouldn't have exclusive use of it. If we care about making these types of visual-only changes fast again, layout should be fixed to maintain its own dirty flags in a follow-up.

Several potential improvements have been delegated to follow-ups. I opted to perform these as follow-ups to this PR, as this patch is already quite large.

  1. The phase items in the SortedRenderPhase should be retained as well. This will result in a large speedup, as extract_ui_camera_view and queue_uinodes time will drop to zero if no changes were made from the previous frame, and sort_phase_system will likely improve as well as the data will be in mostly-sorted order. (See attached profile.)

  2. The UiBatch component is redundant, as the phase items and ExtractedUiNodes already contain all the relevant data. I opted to leave it for now and clean it up later.

  3. A large chunk of the GPU data that prepare_uinodes and similar (prepare_shadows, etc.) emit is redundant, as it's duplicated for every vertex. This should be moved to instance-rate shading (and, in fact, as every UI phase item represents a quad, the entire UI subsystem can be moved to only emit per-instance data).

  4. prepare_uinodes and similar duplicate in an ad-hoc way what batch_and_prepare_sorted_render_phase does. These systems should be removed in favor of the generic functionality.

  5. Much of the work that prepare_uinodes and similar do should be moved to the extraction phase so that it can be retained from frame to frame.

  6. UI should support GPU preprocessing to eliminate the work that remains.

As mentioned before, this work should be a series of follow-ups.

On many_buttons, with this patch I observed the median frame time to decrease from 48.29 ms to 45.16 ms, a 1.07× speedup. Note that the true value of this patch as far as performance is concerned isn't what the patch itself does but rather what it unlocks in the future.

many_buttons before and after:
Screenshot 2026-07-06 084300

Overall picture after this patch for many_buttons:
Screenshot 2026-07-06 085331
Note the sizable performance increases that this patch should unlock by eliminating extract_ui_camera_view and queue_uinodes, as well as prepare_uinodes in the future.

At present, Bevy re-extracts all render world data for the UI from
scratch every frame. This is wasteful, as most UI widgets don't change
from frame to frame.

This PR changes Bevy UI to retain phase items and render-world entities
from frame to frame in the render world. It uses ECS change detection to
only re-extract UI widgets that have changed in some way.

UI widgets differ from 2D and 3D meshes in that a single UI widget in
the main world may extract to multiple phase items in the render world.
For UI nodes, multiple systems process the same node repeatedly in order
to generate the various rendered pieces (the base node, drop shadows,
backgrounds, etc.) To handle this, I split out the change detection and
invalidation logic into a separate system, `extract_uinode_changes`,
that runs before any of the main extraction systems. When that system
detects a change to any of the components that are relevant to
rendering, all render-world entities corresponding to the changed
main-world entity are despawned and created anew. This isn't as
efficient as it could be, but it's fine for now and is a sizable
improvement to the current situation.

Note that one unfortunate side effect of these changes is that we now
mark UI nodes as changed if their visuals change in any way, as opposed
to only when their layout changes. The layout code uses the `Changed`
flag to determine whether reflow needs to be performed, so reflows will
now be needlessly performed if e.g. the color of a node changes, even
though color only affects visuals and not layout and so the reflow is
unnecessary. I believe that this use of the changed flag by layout was
always an abuse, because `Changed` is supposed to be set whenever a
component changes, not just when some part of a component relevant to
some arbitrary system (in this case, layout) changes. In other words,
the `Changed` flag should be for every system to use; layout shouldn't
have exclusive use of it. If we care about making these types of
visual-only changes fast again, layout should be fixed to maintain its
own dirty flags in a follow-up.

Several potential improvements have been delegated to follow-ups. I
opted to perform these as follow-ups to this PR, as this patch is
already quite large.

1. The phase items in the `SortedRenderPhase` should be retained as
   well. This will result in a large speedup, as
   `extract_ui_camera_view` and `queue_uinodes` time will drop to zero
   if no changes were made from the previous frame, and
   `sort_phase_system` will likely improve as well as the data will be
   in mostly-sorted order. (See attached profile.)

2. The `UiBatch` component is redundant, as the phase items and
   `ExtractedUiNodes` already contain all the relevant data. I opted to
   leave it for now and clean it up later.

3. A large chunk of the GPU data that `prepare_uinodes` and similar
   (`prepare_shadows`, etc.) emit is redundant, as it's duplicated for
   every vertex. This should be moved to instance-rate shading (and, in
   fact, as every UI phase item represents a quad, the entire UI
   subsystem can be moved to *only* emit per-instance data).

4. `prepare_uinodes` and similar duplicate in an ad-hoc way what
   `batch_and_prepare_sorted_render_phase` does. These systems should be
   removed in favor of the generic functionality.

5. Much of the work that `prepare_uinodes` and similar do should be
   moved to the extraction phase so that it can be retained from frame
   to frame.

6. UI should support GPU preprocessing to eliminate the work that
   remains.

As mentioned before, this work should be a series of follow-ups.

On `many_buttons`, with this patch I observed the median frame time to
decrease from 48.29 ms to 45.16 ms, a 1.07× speedup. Note that the true
value of this patch as far as performance is concerned isn't what the
patch itself does but rather what it unlocks in the future.
@pcwalton pcwalton requested review from Zeophlite and ickshonpe July 6, 2026 15:55
@pcwalton pcwalton added the A-Rendering Drawing game state to the screen label Jul 6, 2026
@github-project-automation github-project-automation Bot moved this to Needs SME Triage in Rendering Jul 6, 2026
@pcwalton pcwalton added C-Performance A change motivated by improving speed, memory usage or compile times S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Jul 6, 2026
@alice-i-cecile alice-i-cecile added the M-Release-Note Work that should be called out in the blog due to impact label Jul 6, 2026
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

It looks like your PR has been selected for a highlight in the next release blog post, but you didn't provide a release note.

Please review the instructions for writing release notes, then expand or revise the content in the release notes directory to showcase your changes.

@ickshonpe ickshonpe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minimizing any of the examples results in a panic:

cargo run --example text --features="debug"
thread 'Compute Task Pool (2)' (29568) panicked at crates\bevy_anti_alias\src\taa\mod.rs:337:14:
Camera entity wasn't synced.: InvalidEntityError { entity: 320v0, current_generation: EntityGeneration(1) }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

thread 'Compute Task Pool (2)' (29568) panicked at crates\bevy_ecs\src\error\handler.rs:141:1:
Encountered an error in system `bevy_anti_alias::taa::extract_taa_settings`: System panicked

Encountered a panic in system `bevy_anti_alias::taa::extract_taa_settings`!

thread '<unnamed>' (34620) panicked at crates\bevy_ui_render\src\lib.rs:1175:14:
Camera entity wasn't synced.: InvalidEntityError { entity: 320v0, current_generation: EntityGeneration(1) }

thread '<unnamed>' (34620) panicked at crates\bevy_ecs\src\error\handler.rs:141:1:
Encountered an error in system `bevy_ui_render::extract_ui_camera_view`: System panicked

Encountered a panic in system `bevy_ui_render::extract_ui_camera_view`!
2026-07-06T19:53:27.392979Z  WARN bevy_ecs::world::command_queue: CommandQueue has un-applied commands being dropped. Did you forget to call SystemState::apply?
2026-07-06T19:53:27.393097Z  WARN bevy_ecs::world::command_queue: CommandQueue has un-applied commands being dropped. Did you forget to call SystemState::apply?
2026-07-06T19:53:27.393198Z  WARN bevy_ecs::world::command_queue: CommandQueue has un-applied commands being dropped. Did you forget to call SystemState::apply?

Comment thread crates/bevy_ui_render/src/lib.rs Outdated
cached_ui_view_data.insert(
main_entity,
CachedUiViewData {
extracted_view_entity: entity_commands.id(),

@ickshonpe ickshonpe Jul 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the crash is because it stores the wrong view entity:

Suggested change
extracted_view_entity: entity_commands.id(),
extracted_view_entity: ui_camera_view,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@pcwalton pcwalton requested a review from ickshonpe July 6, 2026 20:34
Comment thread crates/bevy_ui_render/src/lib.rs Outdated
let retained_view_entity =
RetainedViewEntity::new(main_entity.into(), None, UI_CAMERA_SUBVIEW);
// Creates the UI view.
let ui_camera_view = commands

@ickshonpe ickshonpe Jul 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a leak here I think. A new view entity is spawned on changes to the camera target. But old view entities are only despawned if the camera is removed or not renderable, so everytime you resize the window it leaves behind an orphaned view entity.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be fixed. I tested resizing the window and it doesn’t leak now.

}
}

// If this node is a `TextSpan`, then we need to invalidate the ancestor

@ickshonpe ickshonpe Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Text also needs to be updated on changes to any TextSpan entity's text style components that are applied per section: TextColor, TextBackgroundColor, Underline, Strikethrough, StrikethroughColor, and UnderlineColor.

You can see in the text_background_colors example that the colors are no longer animated with this PR.

@pcwalton pcwalton Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I added a separate query for that and included it in the big chained iteration. text_background_colors now animates properly.

@pcwalton pcwalton requested a review from ickshonpe July 7, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-Rendering Drawing game state to the screen C-Performance A change motivated by improving speed, memory usage or compile times M-Release-Note Work that should be called out in the blog due to impact S-Needs-Review Needs reviewer attention (from anyone!) to move forward

Projects

Status: Needs SME Triage

Development

Successfully merging this pull request may close these issues.

3 participants