feat(gallery): Speed up load times and clean gallery entries - #9211
Conversation
| ) | ||
|
|
||
| // Model-level estimate result cache — keyed by model ID, avoids re-running | ||
| // the full estimation pipeline (HTTP HEAD, GGUF reads, HF API) on every |
There was a problem hiding this comment.
mmm readers are already cached in this file, I'm missing what's the issue trying to solve here? the first loading of the page is slow because it runs estimations, but if you go back to page already loaded that's not loaded from scratch- unless I'm missing something?
There was a problem hiding this comment.
I found it to be very slow and this speeds it up mainly because it does the VRAM estimation asynchronously, but yeah I got into a mess here, there is a lot of duplication of VRAM estimation and we are doing quite a bit of unnecessary work. So I've reworked all of it into a single function and replaced the TTL cache with a generational cache and stuff like this.
86fda78 to
e98d30f
Compare
e98d30f to
a525398
Compare
|
any chance to revive this PR before cutting a new rel ? 🫶 |
probably not unfortunately, the code changes are pretty much done, but I have to fixup at least all of the popular models in the gallery which I have an agent to do that, but it still requires some manual review. Probably we also don't want to rush that into a release |
8f9a9cc to
2e5ab62
Compare
4c17120 to
90db416
Compare
|
OK, seems ready to go. |
|
@richiejp there are merge conflicts, can't merge as is |
|
@copilot resolve the merge conflicts in this pull request |
90db416 to
81fca12
Compare
Signed-off-by: Richard Palethorpe <io@richiejp.com> Assisted-by: Claude:claude-opus-4-7[1m] [Claude Code]
…del entries Signed-off-by: Richard Palethorpe <io@richiejp.com>
81fca12 to
2bcb5f6
Compare
3 participants
Description
Improve the gallery load times and clean up the gallery entry meta-data, in particular the known_usecases.
clip-04-06_11-49-23.mp4
Notes for Reviewers
Signed commits