Optimize HTMX profile fragments: works-deposits and mysql-data view speed

[MartinPaulEve]

Context

Distributed Locust runs against /htmx/works-deposits/<user>/ and /htmx/mysql-data/<user>/ reveal both endpoints as hot — they fan in on every profile page view (each profile renders the shell plus seven HTMX fragments) and each one does enough uncached work that they dominate the profile-page latency budget.

Tracing both views shows several layered wastes that compound: HTTP requests we don't need to make, citeproc renders we don't need to run, MySQL round-trips we don't need to repeat, and a redundant API object instantiation per request.

Proposed Solution

Five wins, in priority order. Ship 1-3 first, measure, decide on 4-5.

1. works_deposits view — chart-gating + fragment cache

• htmx.py:110 computes api.works_chart_json unconditionally even when api.profile.show_works is False. Gate it: when works are hidden, set chart="{}" directly. Pure logic fix.
• Wrap the whole works_deposits() view body in a Redis cache.get_or_set("htmx_works_deposits:{user}:{style}", …, 60). The output is deterministic from (username, style, profile.show_works, profile.works_order). A 60s TTL lets two consecutive page hits of the same profile cost one Redis GET.

2. mysql_data view — drop the redundant viewer API instance

• htmx.py:366 builds a second full API(request, request.user.username, …) purely to call get_profile_info() on it and pass logged_in_profile to the template. That duplicates Profile.objects.get, the mastodon handle resolver, and the sanitize_html calls per request when an authenticated user views someone else's profile.
• Audit partials/mysql_data.html for which fields of logged_in_profile are actually used; replace the API instantiation with a minimal Profile.objects.values(...).get(username=request.user.username) or just thread request.user directly.

3. Cache the two remaining uncached MySQL methods

In newprofile/api.py, get_groups() (uncached cross-DB query with annotations + select_related) and follower_count() (uncached WpBpFollow.objects.filter(...).count()) fire on every mysql_data request. Add cache.get/set wrappers with the same 600s TTL pattern as get_memberships()/get_user_blogs()/get_activity(). Key includes the username; invalidation via a post_save signal on the underlying WP models (or just let the TTL handle it — these aren't security-sensitive).

4. Citeproc + vega-chart caching

works.py: get_works() already caches the raw Works API JSON. But get_formatted_works(style=...) and get_vega_chart_json() run citeproc / data-reshape on every request — CPU-heavy and deterministic from the cached raw JSON. Cache the formatted HTML at xprofile-works-formatted-{user}-{style} and the chart JSON at xprofile-works-chart-{user} with the same key version + TTL as the raw cache. Invalidate together when works_order changes.

5. mysql_data fragment-level cache

Same pattern as (1) but for mysql_data(). Keying is trickier — output varies by (username, viewer_user_id or "anon") and a handful of profile flags. Hold until (2) and (3) are in and the per-call cost is dominated by Redis GETs rather than DB queries.

Rationale

The profile page is the busiest read path in the system and at 8 sub-requests per page view, anything cheap that the views do unnecessarily multiplies. Wins (1)-(3) are local, low-risk, and don't change semantics. Wins (4)-(5) need invalidation thinking; defer until the simpler wins have been measured.

Acceptance Criteria

• htmx works-deposits p95 latency drops; fragment cache hits visible in Redis (htmx_works_deposits:* keys present).
• Chart JSON is \"{}\" when show_works=False; verified by a unit test on the view.
• htmx mysql-data no longer issues Profile.objects.get twice per authenticated viewer.
• get_groups() and follower_count() show cache hits in Redis under repeat load.
• All existing tests in knowledge_commons_profiles/newprofile/tests/ still pass.