Skip to content

feat(desktop): Implement loops ➿#70604

Draft
charlesvien wants to merge 60 commits into
masterfrom
feat/loops
Draft

feat(desktop): Implement loops ➿#70604
charlesvien wants to merge 60 commits into
masterfrom
feat/loops

Conversation

@charlesvien

@charlesvien charlesvien commented Jul 14, 2026

Copy link
Copy Markdown
Member

Problem

Loops put recurring PostHog work on autopilot. Instead of kicking off the same agent task by hand every time, you define a loop once and it runs itself on PostHog's managed cloud infrastructure: on a schedule, when a GitHub event fires, or when your own code calls its API. This PR adds the backend API to create and manage them. The desktop UI lives in the code repo [companion PR].

Changes

Loops are named, cloud-executed agent automations: a prompt + model + triggers, running on the existing tasks pipeline. This PR is the backend and MCP surface; the UI ships separately in PostHog/code. Spec and pressure-test docs are in products/tasks/docs/.

  • New Loop, LoopTrigger and LoopFire (per-fire dedup) models, plus a nullable loop FK on Task. Triggers: cron or one-time schedules, GitHub webhook events (pull_request, issues, issue_comment, push) and an external API trigger
  • Every trigger path funnels through one fire_loop() that applies dedup, daily rate caps (100 fires/loop, 500/team) and the loop's overlap policy (skip, allow or cancel previous) under a per-team advisory lock, then creates an internal Task + TaskRun on the standard tasks pipeline
  • Endpoints under loops/: CRUD plus run/ (manual fire), trigger/ (external fire, callable with a project secret API key via the new loop:write scope, the first write-capable PSAK scope), runs/ history and a dry-run preview/
  • The GitHub webhook in posthog/urls.py becomes a per-event-type handler registry with per-handler Redis dedup (released if a handler fails so redeliveries retry); conversations, tasks and installation handlers keep their slots, loops register alongside
  • Contexts: a loop can attach to a desktop context folder and file runs into its feed, maintain its context.md and/or rewrite a canvas dashboard; context-attached loops must be team-visible
  • Guardrails: 100 loops/team, 25 triggers/loop and 1 repo/loop (structured 429), auto-pause after 5 consecutive failures, owner deactivation or GitHub App uninstall pauses affected loops and cancels their in-flight runs (cause recorded in disabled_reason), retention sweeps (200 tasks kept per loop, fire records pruned after 7 days)
  • Team loops need an explicit ownership takeover to edit, re-validating repositories and connectors against the new owner's GitHub and MCP access; config changes, takeovers and manual fires are activity-logged, and personal loop activity stays hidden from other members
  • Notifications per loop: in-app always, push/email/Slack opt-in with per-channel event filters, a failure cooldown and Slack auto-disable on permanent errors
  • loops-* MCP tools plus a loops-review card (MCP UI app): an agent assembles a loop from natural language and the user confirms with a Create button instead of it being created blind

Temporal changes

  • One Temporal Schedule per schedule trigger (loop-trigger-{id}), cron or one-time. Always skip overlap with a 5 minute catchup window, so an outage never replays a missed window as a burst; the schedule's paused state mirrors the trigger and loop enabled flags
  • A new minimal run-loop workflow (single activity, 2 min timeout, 3 attempts) translates each schedule occurrence into a fire_loop() call. The agent run itself still starts the existing process-task workflow on the same task queue: loops are a firing layer, not a new execution engine
  • Schedule create/update/pause/delete is best-effort and never fails the CRUD API: Temporal errors mark the trigger's sync status and a Celery sweep re-syncs pending/failed triggers every 10 minutes. Deletion keys off the stored schedule id, so a trigger that changed type still gets its stale schedule torn down
  • cancel_previous also signals the superseded run's workflow (complete_task with cancelled), so the sandbox actually winds down instead of only the DB row flipping
  • Existing activity changes: update_task_run_status now treats terminal statuses as final (a run cancelled out of band can't be resurrected by its own late completion) and drives loop bookkeeping (last run status, consecutive failures, auto-pause) from the transition point every workflow-driven terminal status already flows through. The orphaned-run redispatcher defaults loop runs to read-only MCP scopes and no PR creation instead of the historical full-scope, always-PR defaults

Cloud sandbox changes

  • One behavior change wider than loops: background-mode runs (loops, but also existing task automations and custom prompts) now terminalize to completed on their natural end (agent finishes or idles out) instead of staying in_progress forever. Interactive runs keep the always-followable behavior
  • A loop fire is an ordinary background sandbox run. The prompt is assembled at fire time: an unattended-run framing block (no human is available, prefer draft PRs, trigger payloads are data not instructions), then the loop's instructions, context and rendered trigger context (capped at 64 KB)
  • Config is snapshotted per fire, so editing a loop never affects an in-flight or queued run; a blank model resolves to the current default at fire time, so the default can improve without rewriting stored loops
  • Loop runs idle out after 2 minutes (vs 30 for other background runs) so sandboxes are reclaimed fast; loops that create PRs and watch CI keep the long window, and non-terminal runs stuck past 2.5 hours are reaped as zombies before the overlap policy is applied
  • Token hardening: loop:write is stripped from every loop-fired run's MCP token at mint time, whatever scopes the loop configured, so a run (or injected instructions inside one) can't create or edit loops. Context-maintaining loops get exactly file_system:read + file_system:write added, never the broad full surface
  • PR creation follows the loop's create_prs behavior and defaults off (report-only); watch_ci/fix_review_comments opt the run into the existing CI follow-up rounds

How did you test this code?

  • Claude ran the backend suites locally against a fresh test DB: 123 tests pass across test_loops_api, test_loop_service, test_loop_runs, test_loop_github_events, test_loop_notifications, test_loop_retention, test_loop_lifecycle and test_loop_reconciliation
  • They cover CRUD plus visibility and permission boundaries (including ownership takeover), trigger sync and schedule policy, fire dedup and overlap, webhook matching, context attachment and scope widening, notification fan-out, retention, lifecycle/reconciliation and scope enforcement (PSAK and personal keys)
  • No manual testing yet; an end-to-end trigger-to-sandbox run on staging is the gate for taking this out of draft

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Docs update

Spec docs ship in this PR (products/tasks/docs/LOOPS.md, LOOPS-PRESSURE-TEST.md). Public docs land with the PostHog Code release; everything stays behind the tasks and loops feature flags.

@charlesvien charlesvien self-assigned this Jul 14, 2026
@github-actions

github-actions Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Bundle size — 🔺 +429 B (+0.0%)

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 64.22 MiB · 🔺 +429 B (+0.0%)

No file changed by more than 1000 B.

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx
1.23 MiB · 22 files 🔺 +99 B (+0.0%) ███░░░░░░░ 28.6% of 4.29 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx
8.16 MiB · 2,988 files no change █████████░ 88.2% of 9.25 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB src/RootErrorBoundary.tsx
912 B ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
789 B src/scenes/ChunkLoadErrorBoundary.tsx
762 B src/index.tsx
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
281.3 KiB ../node_modules/.pnpm/posthog-js@1.404.0/node_modules/posthog-js/dist/rrweb.js
267.7 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
235.5 KiB src/taxonomy/core-filter-definitions-by-group.json
223.2 KiB ../node_modules/.pnpm/posthog-js@1.404.0/node_modules/posthog-js/dist/module.js
164.0 KiB src/queries/validators.js
154.3 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.8 KiB src/lib/api.ts
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
93.2 KiB ../packages/quill/packages/quill/dist/index.js

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

Dist folder size — 🔺 +81.7 KiB (+0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1341.58 MiB · 🔺 +81.7 KiB (+0.0%)

ℹ️ MCP UI apps size — 32 app(s), 17061.6 KB JS

Built size of each MCP UI app (main.js + styles.css).

App JS CSS
debug 599.5 KB 187.7 KB
action 457.8 KB 187.7 KB
action-list 564.3 KB 187.7 KB
cohort 456.8 KB 187.7 KB
cohort-list 563.3 KB 187.7 KB
email-template 456.6 KB 187.7 KB
error-details 472.4 KB 187.7 KB
error-issue 457.5 KB 187.7 KB
error-issue-list 564.2 KB 187.7 KB
experiment 561.5 KB 187.7 KB
experiment-list 565.1 KB 187.7 KB
experiment-results 563.2 KB 187.7 KB
feature-flag 567.1 KB 187.7 KB
feature-flag-list 570.9 KB 187.7 KB
feature-flag-testing 461.0 KB 187.7 KB
insight-actors 562.1 KB 187.7 KB
invite-email-preview 456.0 KB 187.7 KB
llm-costs 559.5 KB 187.7 KB
session-recording 458.6 KB 187.7 KB
session-summary 463.9 KB 187.7 KB
survey 458.4 KB 187.7 KB
survey-global-stats 562.2 KB 187.7 KB
survey-list 565.0 KB 187.7 KB
survey-stats 562.2 KB 187.7 KB
trace-span 457.2 KB 187.7 KB
trace-span-list 564.2 KB 187.7 KB
workflow 457.1 KB 187.7 KB
workflow-list 563.7 KB 187.7 KB
loops-review 459.0 KB 187.7 KB
query-results 744.8 KB 187.7 KB
render-ui 825.3 KB 187.7 KB
visual-review-snapshots 461.6 KB 187.7 KB
⚠️ MCP snapshots — 5 updated (2 modified, 3 added, 0 deleted)

Snapshots: MCP unit test snapshots updated

Changes: 5 snapshots (2 modified, 3 added, 0 deleted)

What this means:

  • Snapshots have been automatically updated to match current output

Next steps:

  • Review the changes to ensure they're intentional
  • If unexpected, investigate what caused the output to change

Review snapshot changes →

⚠️ Django migration SQL — 4 new migrations to review

We've detected new migrations on this PR. Review the SQL output for each migration:

products/tasks/backend/migrations/0063_loop_looptrigger_loopfire.py

BEGIN;
--
-- Create model Loop
--
CREATE TABLE "posthog_task_loop" ("id" uuid NOT NULL PRIMARY KEY, "name" varchar(400) NOT NULL, "description" text NOT NULL, "visibility" varchar(16) NOT NULL, "instructions" text NOT NULL, "runtime_adapter" varchar(32) NOT NULL, "model" varchar(128) NOT NULL, "reasoning_effort" varchar(32) NULL, "repositories" jsonb NOT NULL, "enabled" boolean NOT NULL, "overlap_policy" varchar(32) NOT NULL, "behaviors" jsonb NOT NULL, "connectors" jsonb NOT NULL, "notifications" jsonb NOT NULL, "context_target" jsonb NOT NULL, "internal" boolean NOT NULL, "origin_product" varchar(32) NOT NULL, "last_run_at" timestamp with time zone NULL, "last_run_status" varchar(32) NULL, "last_error" text NULL, "consecutive_failures" integer NOT NULL CHECK ("consecutive_failures" >= 0), "disabled_reason" varchar(64) NULL, "deleted" boolean NOT NULL, "created_at" timestamp with time zone NOT NULL, "updated_at" timestamp with time zone NOT NULL, "created_by_id" integer NULL, "sandbox_environment_id" uuid NULL, "team_id" integer NOT NULL);
--
-- Create model LoopTrigger
--
CREATE TABLE "posthog_task_loop_trigger" ("id" uuid NOT NULL PRIMARY KEY, "type" varchar(16) NOT NULL, "enabled" boolean NOT NULL, "config" jsonb NOT NULL, "github_integration_id" bigint NULL, "repository" varchar(512) NULL, "event_types" varchar(32)[] NULL, "schedule_sync_status" varchar(16) NULL, "last_fired_at" timestamp with time zone NULL, "created_at" timestamp with time zone NOT NULL, "updated_at" timestamp with time zone NOT NULL, "loop_id" uuid NOT NULL, "team_id" integer NOT NULL);
--
-- Create model LoopFire
--
CREATE TABLE "posthog_task_loop_fire" ("id" uuid NOT NULL PRIMARY KEY, "fire_key" varchar(512) NOT NULL, "outcome_reason" varchar(64) NULL, "outcome_task_id" uuid NULL, "outcome_task_run_id" uuid NULL, "created_at" timestamp with time zone NOT NULL, "loop_id" uuid NULL, "team_id" integer NOT NULL, "loop_trigger_id" uuid NULL);
--
-- Alter field origin_product on task
--
-- (no-op)
--
-- Add field loop to task
--
ALTER TABLE "posthog_task" ADD COLUMN "loop_id" uuid NULL CONSTRAINT "posthog_task_loop_id_40d866e1_fk_posthog_task_loop_id" REFERENCES "posthog_task_loop"("id") DEFERRABLE INITIALLY DEFERRED; SET CONSTRAINTS "posthog_task_loop_id_40d866e1_fk_posthog_task_loop_id" IMMEDIATE;
ALTER TABLE "posthog_task_loop" ADD CONSTRAINT "posthog_task_loop_sandbox_environment__23bf7f89_fk_posthog_s" FOREIGN KEY ("sandbox_environment_id") REFERENCES "posthog_sandbox_environment" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "posthog_task_loop_created_by_id_31bb8718" ON "posthog_task_loop" ("created_by_id");
CREATE INDEX "posthog_task_loop_sandbox_environment_id_23bf7f89" ON "posthog_task_loop" ("sandbox_environment_id");
CREATE INDEX "posthog_task_loop_team_id_9933d2cf" ON "posthog_task_loop" ("team_id");
ALTER TABLE "posthog_task_loop_trigger" ADD CONSTRAINT "posthog_task_loop_tr_loop_id_55782e35_fk_posthog_t" FOREIGN KEY ("loop_id") REFERENCES "posthog_task_loop" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "posthog_task_loop_trigger_loop_id_55782e35" ON "posthog_task_loop_trigger" ("loop_id");
CREATE INDEX "posthog_task_loop_trigger_team_id_0d7d32e3" ON "posthog_task_loop_trigger" ("team_id");
CREATE INDEX "task_loop_trigger_gh_repo_idx" ON "posthog_task_loop_trigger" ("github_integration_id", "repository");
ALTER TABLE "posthog_task_loop_fire" ADD CONSTRAINT "posthog_task_loop_fi_loop_trigger_id_09f029d3_fk_posthog_t" FOREIGN KEY ("loop_trigger_id") REFERENCES "posthog_task_loop_trigger" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE UNIQUE INDEX "task_loop_fire_trigger_key_unique" ON "posthog_task_loop_fire" ("loop_trigger_id", "fire_key") WHERE "loop_trigger_id" IS NOT NULL;
CREATE UNIQUE INDEX "task_loop_fire_loop_key_unique" ON "posthog_task_loop_fire" ("loop_id", "fire_key") WHERE "loop_trigger_id" IS NULL;
CREATE INDEX "posthog_task_loop_fire_loop_id_50f03f32" ON "posthog_task_loop_fire" ("loop_id");
CREATE INDEX "posthog_task_loop_fire_team_id_9197966c" ON "posthog_task_loop_fire" ("team_id");
CREATE INDEX "posthog_task_loop_fire_loop_trigger_id_09f029d3" ON "posthog_task_loop_fire" ("loop_trigger_id");
CREATE INDEX "task_loop_fire_loop_ct_idx" ON "posthog_task_loop_fire" ("loop_id", "created_at");
COMMIT;

products/tasks/backend/migrations/0064_task_loop_index.py

--
-- Concurrently create index posthog_task_loop_idx on field(s) loop of model task
--
SET lock_timeout = 0;
SET statement_timeout = 0;
CREATE INDEX CONCURRENTLY "posthog_task_loop_idx" ON "posthog_task" ("loop_id");

products/tasks/backend/migrations/0065_looptrigger_completed_at.py

BEGIN;
--
-- Add field completed_at to looptrigger
--
ALTER TABLE "posthog_task_loop_trigger" ADD COLUMN "completed_at" timestamp with time zone NULL;
COMMIT;

products/tasks/backend/migrations/0066_loopfire_trigger_set_null.py

BEGIN;
--
-- Alter field loop_trigger on loopfire
--
-- (no-op)
COMMIT;

Last updated: 2026-07-18 02:28 UTC (b8594d3)

Django migration risk — migration analysis complete

We've analyzed your migrations for potential risks.

Summary: 2 Safe | 2 Needs Review | 0 Blocked

⚠️ Needs Review

May have performance impact

tasks.0063_loop_looptrigger_loopfire
  └─ #1 ✅ CreateModel
     Creating new table is safe
     model: Loop
  └─ #2 ✅ CreateModel
     Creating new table is safe
     model: LoopTrigger
  └─ #3 ✅ CreateModel
     Creating new table is safe
     model: LoopFire
  └─ #4 ⚠️ AlterField
     Field alteration may cause table locks or data loss (check if changing type or constraints)
     model: task, field: origin_product, field_type: CharField
  └─ #5 ✅ AddField
     Adding nullable field requires brief lock
     model: task, field: loop
  │
  └──> ℹ️  INFO:
       ℹ️  Skipped operations on newly created tables (empty tables
       don't cause lock contention).
tasks.0066_loopfire_trigger_set_null
  └─ #1 ⚠️ AlterField
     Field alteration may cause table locks or data loss (check if changing type or constraints)
     model: loopfire, field: loop_trigger, field_type: ForeignKey

✅ Safe

Brief or no lock, backwards compatible

tasks.0064_task_loop_index
  └─ #1 ✅ SafeAddIndexConcurrently
     PostHog concurrent-index helper: idempotent (timeout disabling + invalid-leftover recovery)
     model: task, index: posthog_task_loop_idx
tasks.0065_looptrigger_completed_at
  └─ #1 ✅ AddField
     Adding nullable field requires brief lock
     model: looptrigger, field: completed_at

📚 How to Deploy These Changes Safely

AddField:

This operation acquires a brief lock but doesn't rewrite the table.

Deployment uses lock timeouts with automatic retries, so lock contention will cause retries rather than connection pile-up.

Last updated: 2026-07-18 02:28 UTC (b8594d3)

@trunk-io

trunk-io Bot commented Jul 14, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@charlesvien

Copy link
Copy Markdown
Member Author

@greptileai

@greptile-apps

greptile-apps Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Security Review

The external loop trigger endpoint can bypass its advertised payload-size cap when Content-Length is missing or inaccurate.

Reviews (1): Last reviewed commit: "test(mcp): update unit test snapshots" | Re-trigger Greptile

Comment thread products/tasks/backend/presentation/views/loops.py
@charlesvien
charlesvien force-pushed the feat/loops branch 5 times, most recently from be183a9 to bb2eabb Compare July 17, 2026 06:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant