Add generic agent activity tracking and Alembic migrations - #6
Merged
Conversation
Records every agent run and tool call (status, duration, token usage, model, error) tied to its newsletter in a new agent_activity table. Introduces a newsletter lifecycle (create placeholder up front, finalize after generation) so newsletter_id exists before agents run. Moves agent runtime helpers (chat_client, guardrails, make_agent, tracking) into agents/runtime/ and wires ActivityTracker + ToolTracker into every agent via a shared make_agent factory. Adds Alembic migrations that run automatically in Docker on startup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds generic agent activity tracking so every agent run and tool call in the newsroom pipeline is recorded in a queryable
agent_activitytable, tied to its newsletter. Tracking is wired in automatically via a sharedmake_agentfactory, so no per-agent code is needed now or in the future.Related issues
Closes #5
Changes
agent_activitytable: kind, name, model, status, duration, token counts (input/output/total), error, metadata JSONBcreate_newsletter(pending placeholder up front) andfinalize_newsletter(fills content + flips status), sonewsletter_idexists before agents runActivityTrackerandToolTrackermiddleware auto-injected into every agent viamake_agent; token counts read fromAgentResponse.usage_details; tool args and result length recorded in metadataagents/runtime/subpackage consolidates agent runtime helpers:chat_client.py,guardrails.py,make_agent.py,tracking.pyagents/sections.pymoved fromutils/;utils/package removedalembic/) with a hand-written initial migration covering all three tables;docker-entrypoint.shrunsalembic upgrade headon startupHow to test
conda run -n agentic-mediapulse pytest— 107 tests passconda run -n agentic-mediapulse ruff check . && ruff format --check .— cleanDATABASE_URL:python src/app.py test --email=you@example.com, then queryselect name, kind, status, duration_ms, total_tokens from agent_activity order by created_at;