Skip to content

recent_activity hydrate_context has N+1 entity lookup pattern #710

@phernandez

Description

@phernandez

Problem

recent_activity is called frequently (112 times in the last 12h) and its shape_response phase has the same N+1 entity lookup pattern as search hydration (issue #707).

Inside shape_response, memory.hydrate_context.lookup_entities fetches entities one-by-one, paying Neon network latency per lookup. This averages 0.3s but spikes to 1.8s for larger result sets.

Evidence (last 12h, 112 calls)

Phase Avg P50 P95 Calls
build_context 0.45s 0.43s 0.62s 112
shape_response 0.30s 0.31s 0.39s 112

Worst case traces drilling into shape_response:

Trace 1 (shape_response: 1.84s):

shape_response                        1.837s
└── memory.hydrate_context            1.834s
    ├── lookup_entities               1.817s  ← N+1 queries
    └── shape_results                 0.013s

Trace 2 (shape_response: 0.46s):

shape_response                        0.462s
└── memory.hydrate_context            0.460s
    ├── lookup_entities               0.430s  ← N+1 queries
    └── shape_results                 0.026s

Trace 3 (shape_response: 0.43s):

shape_response                        0.430s
└── memory.hydrate_context            0.429s
    ├── lookup_entities               0.415s  ← N+1 queries
    └── shape_results                 0.011s

In all cases, lookup_entities dominates the phase.

Proposed Fix

Same approach as #707: collect all entity IDs/permalinks needed for hydration upfront, then batch-fetch in a single find_by_ids query instead of one query per entity.

Impact

recent_activity is one of the most-called endpoints (112 calls/12h). Batching the entity lookup would save ~0.3s avg per call and eliminate the 1.8s worst-case spikes.

Related: #707 (same N+1 pattern in search hydration)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions