Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ A simple vibe-coded agent trace prefix [analyzer](https://v0-llm-agent-dashboard
| [DSPy](https://github.com/stanfordnlp/dspy) | 🔄 | The framework for programming—not prompting—foundation models |
| [ThinkGPT](https://github.com/alaeddine-13/thinkgpt) | 🔄 | Agent techniques to augment your LLM and push it beyond its limits |
| [PyCodeAGI](https://github.com/chakkaradeep/pyCodeAGI) | ✅ | Synthetic trace from GPT-4 chat pipeline (5 sessions × 5 steps). Strong substring cache case (84% hit rate); modest prefix hit (18.5%) from shared boilerplate. |
| Coordination Agent | ✅ | Synthetic trace for public artifact review, policy checks, receipt writing, and owner-visible handoffs. Demonstrates stable coordination instructions plus accumulating receipts across sessions. |
| [SuperAGI](https://github.com/TransformerOptimus/SuperAGI) | ⏳ | SuperAGI - A dev-first open source autonomous AI agent framework |
| [MemGPT(letta)](https://github.com/letta-ai/letta) | ✅ | Letta is the platform for building stateful agents: open AI with advanced memory that can learn and self-improve over time. |

Expand All @@ -34,4 +35,4 @@ References:
* https://www.arxiv.org/pdf/2510.04618 (Agentic Context Engineering)
* https://arxiv.org/pdf/2511.02230 (CONTINUUM)
* https://arxiv.org/pdf/2410.02506 (AgentPrune)
* https://arxiv.org/pdf/2510.12872 (KVCOMM)
* https://arxiv.org/pdf/2510.12872 (KVCOMM)
66 changes: 66 additions & 0 deletions coordination_agent/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Coordination Agent

## Overview

This is a small synthetic trace for a coordination agent that repeatedly reads
public artifacts, checks policy boundaries, writes receipts, and hands off the
next action.

The trace is designed to model a common multi-agent operations pattern:

1. Orient on a public task.
2. Build a source map from public artifacts.
3. Check coordination and privacy policy.
4. Write a concise action receipt.
5. Hand off the next owner-visible follow-up.

## Synthetic Disclosure

This trace is synthetic and representative. It does not contain private
transcripts, private messages, raw logs, credentials, personal data, hidden
reasoning, or environment state. Public URLs in the trace are illustrative
examples, and the responses are hand-authored to demonstrate the repeated
context pattern.

The generator is dependency-free and deterministic. It embeds all scenario text
in `generate_trace.py` and writes `trace.jsonl`; it does not read from the local
filesystem beyond writing its output file.

## Why This Is Useful for Cache Analysis

Coordination agents often reuse a stable instruction block while accumulating
task specific receipts over several calls. That creates two reuse patterns:

- Cross-session prefix reuse from the stable system and workflow framing.
- Within-session substring reuse as each step carries forward prior receipts.

This makes the trace a compact example for comparing strict prefix reuse with
broader substring-style reuse on agent operations that are not code-editing
tasks.

## Trace Details

- Sessions: 3
- LLM calls per session: 5
- Total trace entries: 15
- Timestamp format: relative seconds from trace start
- Session IDs: MD5 hex digests of the synthetic scenario names
- Format: one JSON object per line with `timestamp`, `input`, `output`, and
`session_id`

## Reproduce the Trace

```bash
python coordination_agent/generate_trace.py
```

## Run Cache Analysis

From the repository root:

```bash
python prefix_analysis.py -i coordination_agent/trace.jsonl -o coordination_agent_result.png --tokenizer gpt2
```

The analysis command is optional. This contribution includes the trace and
generator only, not a performance claim.
152 changes: 152 additions & 0 deletions coordination_agent/generate_trace.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
#!/usr/bin/env python3
"""Generate a synthetic coordination-agent trace.

The trace is deliberately hand-authored and deterministic. It models the shape
of a public coordination workflow without reading private logs, transcripts, or
environment state.
"""

from __future__ import annotations

import hashlib
import json
from pathlib import Path


STATIC_SYSTEM = """[System]
You are a coordination agent for a public open-source project.
You only use public artifacts, explicit receipts, and reproducible checks.
You must separate observation, inference, action, and follow-up.
Never include private transcripts, credentials, personal data, or hidden notes.
"""


SCENARIOS = [
{
"name": "public_issue_followup",
"task": "Review a public GitHub issue after a collaborator replies.",
"public_artifacts": [
"GitHub issue URL: https://github.com/example-org/example-project/issues/64",
"Project status page: https://example.org/project/status",
"Previous public comment: clarify owner-of-record before implementation",
],
"checks": [
"Read the latest public issue comments.",
"Confirm the reply requests protocol clarification rather than code.",
"Check that no private support channel is referenced.",
],
"action": "Post one concise public clarification and update the watchlist.",
"handoff": "Wait for maintainer confirmation before opening a PR.",
},
{
"name": "resolution_nudge_measurement",
"task": "Measure whether a public market-resolution nudge attracted attention.",
"public_artifacts": [
"Market URL: https://example.com/markets/public-resolution-example",
"Pre-comment snapshot: traders=43, volume=4610.58",
"Public comment label: resolution-status nudge",
],
"checks": [
"Fetch the public market page after the six-hour window.",
"Compare trader and volume deltas against the pre-comment snapshot.",
"Do not post a second top-level comment without a reply or exception.",
],
"action": "Record a post-window metric receipt with delta traders and volume.",
"handoff": "Schedule the 24-hour measurement window.",
},
{
"name": "blog_receipt_review",
"task": "Review a public blog draft for provenance and uncertainty receipts.",
"public_artifacts": [
"Draft title: Small Samples Need Assumption Receipts",
"Public source card: small-N forecasting note",
"Site policy: posts need owner, inference, uncertainty, and falsifier fields",
],
"checks": [
"Scan front matter for author and owner fields.",
"Flag claims that cite anecdotal examples without source caveats.",
"Keep lint advisory-only so publication is not blocked by old posts.",
],
"action": "Publish the post only after adding source caveats and an owner receipt.",
"handoff": "Add the lint finding to the daily culture queue.",
},
]


def session_id(name: str) -> str:
return hashlib.md5(name.encode("utf-8")).hexdigest()


def build_rows() -> list[dict[str, object]]:
rows: list[dict[str, object]] = []
timestamp = 0

for scenario in SCENARIOS:
sid = session_id(scenario["name"])
memory = ""

steps = [
(
"Orient",
"Identify the task, public artifacts, and private-data boundary.",
(
f"Task: {scenario['task']}\n"
"Boundary: use public artifacts only; no private logs or secrets."
),
),
(
"Source Map",
"Build a source map from the public artifacts.",
"Source map:\n- " + "\n- ".join(scenario["public_artifacts"]),
),
(
"Policy Check",
"Apply the coordination rules before taking action.",
"Checks:\n- " + "\n- ".join(scenario["checks"]),
),
(
"Receipt",
"Write the minimal receipt for the safe action.",
f"Action receipt: {scenario['action']}",
),
(
"Handoff",
"Name the next owner-visible follow-up.",
f"Next action: {scenario['handoff']}",
),
]

for step_name, instruction, output in steps:
input_text = (
f"{STATIC_SYSTEM}\n"
f"[Workflow]\nName: {scenario['name']}\nTask: {scenario['task']}\n"
f"{memory}\n"
f"[User]\nStep: {step_name}\n{instruction}"
)
rows.append(
{
"timestamp": timestamp,
"input": input_text,
"output": output,
"session_id": sid,
}
)
memory += f"\n[{step_name} Output]\n{output}\n"
timestamp += 120

timestamp += 600

return rows


def main() -> None:
out_path = Path(__file__).with_name("trace.jsonl")
rows = build_rows()
with out_path.open("w", encoding="utf-8") as f:
for row in rows:
f.write(json.dumps(row, ensure_ascii=True) + "\n")
print(f"Wrote {len(rows)} rows to {out_path}")


if __name__ == "__main__":
main()
Loading