Skip to content

Optimize large-manifest browsing and archive startup#689

Open
NotOfficer wants to merge 7 commits into
devfrom
codex/optimize-asset-tree
Open

Optimize large-manifest browsing and archive startup#689
NotOfficer wants to merge 7 commits into
devfrom
codex/optimize-asset-tree

Conversation

@NotOfficer

@NotOfficer NotOfficer commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Large-manifest browsing

  • build package folders off the UI dispatcher with full-path dictionary lookups, a consecutive-folder fast path, lazy asset/search views, and batched collection publication
  • virtualize the folder tree and realize only the indexed target path for reliable, responsive deep Go To navigation
  • fast-path breadcrumb navigation to already-realized ancestors and suspend asset previews while Search Go To is moving between folders
  • limit package/type/preview resolver concurrency to four jobs across the application
  • auto-refresh package searches after 500 ms of idle input, caching parsed terms or a compiled regex once per refresh

Archive startup

  • batch archive-list additions and mount-state updates on the dispatcher instead of synchronously invoking it for every archive event
  • address archive rows by reader identity, coalesce transient provider status updates, and virtualize/recycle the archive list
  • overlap mappings and independent native-tool initialization with archive registration and mounting
  • update CUE4Parse to the optimization commit from CUE4Parse#379, which optimizes IoStore traversal, package indexing, TOC reads, and repeated file metadata

Performance

Across the paired FModel/CUE4Parse Fortnite startup changes:

  • total CPU: 25,273 -> 19,519 units (22.8% lower)
  • IoStore mount: 8,687 -> 6,100 (29.8% lower)
  • FileProviderDictionary.AddFiles: 2,681 -> 763 (71.5% lower)

The large-manifest browsing work was profiled and manually exercised with a Fortnite Live manifest containing 2,215,162 packages.

Validation

  • dotnet build FModel\FModel.csproj --no-restore -p:OutputPath=C:\Users\Marlon\AppData\Local\Temp\FModelCodexBuild\ (0 errors)
  • CUE4Parse relevant tests: 6 passed
  • Fortnite pakchunk32-WindowsClient.utoc package-index output matches the expected 247,486 normal packages
  • manually verified deep Go To navigation and breadcrumb navigation through Fortnite Game Features paths

@NotOfficer NotOfficer changed the title Optimize asset tree construction for large manifests Optimize asset browsing for large manifests Jul 16, 2026
@NotOfficer NotOfficer changed the title Optimize asset browsing for large manifests Optimize large-manifest browsing and archive startup Jul 16, 2026
@NotOfficer
NotOfficer requested a review from Copilot July 16, 2026 19:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR focuses on improving responsiveness and throughput when working with very large manifests by moving heavy work off the UI thread, adding UI virtualization for folder/asset browsing, batching dispatcher updates, and limiting resolver concurrency; it also reduces archive-startup overhead by coalescing status updates and batching archive-list mutations.

Changes:

  • Added debounced (idle) auto-refresh for package search text input and improved Go To navigation flow.
  • Virtualized and optimized folder tree realization / selection, including programmatic container realization support.
  • Batched archive list/status updates and introduced lightweight/lazy collections for assets and search results.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
FModel/Views/SearchView.xaml.cs Adds auto-search debounce and switches Go To navigation to async + preview-suspension.
FModel/Views/SearchView.xaml Wires search TextChanged/Closed events and updates watermark text.
FModel/Views/Resources/Resources.xaml Enables virtualization/recycling and updates bindings to use AssetsList.Count.
FModel/Views/Resources/Controls/TiledExplorer/Resources.xaml.cs Updates folder auto-expand logic to use new AssetsList.Count.
FModel/Views/Resources/Controls/TiledExplorer/FolderButton3.xaml Updates count binding to AssetsList.Count.
FModel/Views/Resources/Controls/TiledExplorer/FolderButton2.xaml Updates count binding to AssetsList.Count.
FModel/Views/Resources/Controls/NavigableVirtualizingStackPanel.cs Exposes BringIndexIntoView for virtualized navigation.
FModel/Views/Resources/Controls/Breadcrumb.xaml.cs Switches breadcrumb navigation to async Go To.
FModel/ViewModels/SearchViewModel.cs Caches parsed filters/regex per refresh and lazily materializes the results view.
FModel/ViewModels/GameFileViewModel.cs Caps resolver concurrency and suspends previews during navigation.
FModel/ViewModels/GameDirectoryViewModel.cs Batches archive-list additions/updates and adds flush/scheduling helpers.
FModel/ViewModels/CUE4ParseViewModel.cs Flushes queued archive UI changes and uses new Clear() APIs.
FModel/ViewModels/Commands/LoadCommand.cs Uses new AssetsFolder.Clear() and SearchVm.Clear() methods.
FModel/ViewModels/Commands/GoToCommand.cs Replaces synchronous tree walking with async realized-path selection.
FModel/ViewModels/AssetsListViewModel.cs Lazily creates asset view-models and supports a cached filter predicate.
FModel/ViewModels/AssetsFolderViewModel.cs Reworks tree construction with path dictionary lookup and fast paths.
FModel/ViewModels/ApplicationViewModel.cs Coalesces provider status updates and adds preview-suspension flag.
FModel/MainWindow.xaml.cs Adds async folder selection into virtualized tree + preview refresh helper.
FModel/MainWindow.xaml Updates UI bindings to use AssetsList.Count.
FModel/Framework/RangeObservableCollection.cs Optimizes AddRange notifications and adds AddWithoutNotification.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread FModel/Views/SearchView.xaml.cs Outdated
Comment thread FModel/ViewModels/GameDirectoryViewModel.cs
NotOfficer and others added 2 commits July 16, 2026 22:17
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants