(feat) Runner should rebuild session context from records when a session cannot be resumed warm

[mmabrouk]

Problem

Today the front end resends the entire message history to the runner on every turn (web/packages/agenta-playground/src/state/execution/agentRequest.ts, the history array). This is a pre-sessions workaround, and it has real costs: the payload grows with every turn, and any attachment bytes in the history are re-uploaded again and again.

The product direction is the opposite split of responsibility:

• The front end sends only the last message on each turn.
• The runner owns conversation context. When a session is warm, the harness already holds the context. When a session cannot be resumed warm (runner restart, sandbox eviction, crash), the runner should read the session's records and rebuild the conversation itself before running the new turn.

The substrate for this already exists. The runner persists every event to the API's records log independent of any browser (services/runner/src/sessions/persist.ts, POST /sessions/records/ingest), and records can be queried back (POST /sessions/records/query, RecordsRouter in api/oss/src/apis/fastapi/sessions/router.py). What is missing is the runner-side rebuild: on a cold start, query the records for the session and reconstruct the harness conversation from them, instead of relying on the front end to resend everything.

Todo

• Runner: on cold session start, query /sessions/records/query for the session and rebuild the conversation context from records before running the new turn.
• Front end: once the runner rebuild lands, stop resending full history and send only the last message.
• Make the cold-replay rendering faithful (today a past image is replayed as the literal string "[image]", services/runner/src/engines/sandbox_agent/transcript.ts). The agent multi-modality design (PR [docs] Plan agent multi-modality #5439) makes records carry small attachment references instead of bytes, which is what a faithful rebuild needs.

Related

• (feat) Warm sessions should hold client tool calls open instead of replaying the turn #5384 covers the warm case (hold client tool calls open instead of replaying the turn). This issue is the cold case: the session cannot be continued and must be rebuilt from records.
• PR [docs] Plan agent multi-modality #5439 (agent multi-modality design) assumes this direction: records store a reference to a durable original, so a rebuilt conversation can restore real content instead of placeholder text.

[linear-code]

AGE-3982

[Aman-goel-04]

I want to work on this. Could this be assigned to me?

[Tarunvoff]

Hi! I'd be interested in contributing to this issue.

From my understanding, the goal is to move conversation reconstruction entirely into the runner by rebuilding the session from /sessions/records/query on cold starts, allowing the frontend to send only the latest message. I also noticed the replay fidelity improvements for multimodal records mentioned in the related design.

I've worked with FastAPI, AI application backends, session/state management, and LLM pipelines, and I'd like to take this on if it's still available. Is anyone already working on it, or would it be okay for me to start?

[Yashwant-Rangrej]

Hi @mmabrouk,

I would love to contribute to this issue if it hasn't been assigned yet!

As an AI/ML engineer with solid experience in both Python backend development and frontend web development, I’d be very comfortable handling the full scope of this feature—both rebuilding the session context on the runner side and optimizing the frontend to send only the latest message payload.

I see a few others have also expressed interest, but I'm ready to dive in and take this on if it's still available. Let me know!