Skip to content

feat(tasks): add isolated cloud computer use#72186

Open
richardsolomou wants to merge 6 commits into
masterfrom
posthog-code/feat/cloud-computer-use
Open

feat(tasks): add isolated cloud computer use#72186
richardsolomou wants to merge 6 commits into
masterfrom
posthog-code/feat/cloud-computer-use

Conversation

@richardsolomou

@richardsolomou richardsolomou commented Jul 19, 2026

Copy link
Copy Markdown
Member

Problem

Cloud task agents need a computer they can control without reaching back into the user's local machine. Each opted-in run should get an isolated desktop without weakening browser or network isolation.

Changes

  • Add a computer_use run option and carry it through task state into agent-server startup.
  • Run opted-in tasks cold on the real-kernel VM runtime so WebKit keeps its process sandbox enabled.
  • Start the Xvfb/Openbox desktop inside agentsh when the task has restricted egress.
  • Include the browser, terminal, screenshot, mouse, keyboard, font, and desktop integration packages in the sandbox image.

Companion client and agent changes: PostHog/code#3569

How did you test this code?

  • Ran the desktop package stack in an Ubuntu 24.04 cloud task, exercised coordinate clicks and keyboard input through xdotool, verified terminal output, visually inspected a screenshot, and confirmed idempotent startup.
  • Ran focused tests for VM selection, sandboxed desktop startup, and agent-server computer-use configuration: 3 passed. These guard against disabling WebKit isolation or launching restricted desktop processes outside agentsh.
  • Ran Ruff and hogli ci:preflight --fix: no failures.
  • Ran repo-wide mypy: changed files pass; the command reports one unrelated existing unused-ignore error in posthog/personhog_client/converters.py.
  • The DB-backed warm-task regression test cannot run locally because this task has no Postgres service. CI identified the early warm-reuse path and a run-only field leaking into Task.objects.create(); computer use now skips warm reuse and is removed before model creation.
  • Docker, Depot, and Modal credentials are unavailable here, so I could not build or launch the final VM image. CI and the external security review validate the pushed configuration.

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Docs update

The companion PostHog Code PR updates the computer-use documentation and setting copy.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

PostHog Code implemented the cloud-owned desktop architecture. I used /improving-drf-endpoints, /writing-tests, /depot-container-builds, /debugging-ci-failures, and /security-audit. CI covered the DB-backed cold-create path, while security review shaped the use of the existing VM and agentsh boundaries.


Created with PostHog Code

Generated-By: PostHog Code
Task-Id: c6d78416-85c8-4304-bd69-e309562468c4
@richardsolomou richardsolomou self-assigned this Jul 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Hey @richardsolomou! 👋

It looks like your git author email on this PR isn't your @posthog.com address (richard@solomou.dev). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

  • Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
  • Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

@github-actions

github-actions Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

🤖 CI report

Bundle size — no change

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 64.22 MiB · no change

No file changed by more than 1000 B.

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx
1.23 MiB · 22 files no change ███░░░░░░░ 28.6% of 4.29 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx
8.16 MiB · 2,988 files no change █████████░ 88.2% of 9.25 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB src/RootErrorBoundary.tsx
912 B ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
789 B src/scenes/ChunkLoadErrorBoundary.tsx
762 B src/index.tsx
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
281.3 KiB ../node_modules/.pnpm/posthog-js@1.404.1/node_modules/posthog-js/dist/rrweb.js
267.7 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
235.5 KiB src/taxonomy/core-filter-definitions-by-group.json
223.4 KiB ../node_modules/.pnpm/posthog-js@1.404.1/node_modules/posthog-js/dist/module.js
164.0 KiB src/queries/validators.js
154.3 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.8 KiB src/lib/api.ts
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
93.2 KiB ../packages/quill/packages/quill/dist/index.js

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

Dist folder size — 🔺 +1.1 KiB (+0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1341.67 MiB · 🔺 +1.1 KiB (+0.0%)

⚠️ Backend coverage — 68.0% of changed backend lines covered — 25 uncovered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ██████████████░░░░░░ 68.0% (55 / 80)

File Patch Uncovered changed lines
products/tasks/backend/temporal/process_task/activities/tests/test_get_task_processing_context.py 0.0% 733–734, 738, 750
products/tasks/backend/temporal/process_task/activities/tests/test_start_agent_server.py 0.0% 184–188, 192–193, 196, 200, 204, 207, 212, 221
products/tasks/backend/temporal/process_task/activities/get_task_processing_context.py 33.3% 117, 366–367, 372
products/tasks/backend/logic/services/docker_sandbox.py 40.0% 941–943
products/tasks/backend/logic/services/modal_sandbox.py 80.0% 1123

🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 29693067589 -n patch-coverage), or the coverage-data block at the end of this comment.

Per-product line coverage (touched products)
Product Coverage Lines
demo ███████████░░░░░░░░░ 56.2% 1,497 / 2,663
tasks █████████████░░░░░░░ 67.0% 25,987 / 38,765
signals ████████████████░░░░ 79.4% 19,493 / 24,553
data_modeling ████████████████░░░░ 80.0% 4,834 / 6,045
cdp ████████████████░░░░ 80.7% 3,118 / 3,864
notebooks █████████████████░░░ 85.1% 7,096 / 8,338
cohorts █████████████████░░░ 86.2% 4,065 / 4,717
agent_platform █████████████████░░░ 86.4% 3,807 / 4,405
actions █████████████████░░░ 86.6% 717 / 828
product_tours █████████████████░░░ 87.5% 1,266 / 1,447
exports ██████████████████░░ 88.4% 6,933 / 7,842
conversations ██████████████████░░ 89.0% 16,183 / 18,186
mcp_analytics ██████████████████░░ 89.2% 2,514 / 2,819
dashboards ██████████████████░░ 89.4% 5,847 / 6,540
engineering_analytics ██████████████████░░ 89.5% 5,507 / 6,154
alerts ██████████████████░░ 89.9% 4,045 / 4,499
early_access_features ██████████████████░░ 90.1% 1,031 / 1,144
error_tracking ██████████████████░░ 90.1% 9,885 / 10,966
streamlit_apps ██████████████████░░ 90.4% 2,501 / 2,767
slack_app ██████████████████░░ 90.6% 9,018 / 9,955
marketing_analytics ██████████████████░░ 90.8% 11,514 / 12,684
stamphog ██████████████████░░ 91.0% 3,993 / 4,387
product_analytics ██████████████████░░ 91.3% 5,757 / 6,304
data_warehouse ██████████████████░░ 92.4% 18,771 / 20,309
workflows ███████████████████░ 92.8% 5,482 / 5,909
ai_observability ███████████████████░ 92.8% 14,916 / 16,077
web_analytics ███████████████████░ 92.9% 13,853 / 14,913
surveys ███████████████████░ 93.0% 5,724 / 6,157
posthog_ai ███████████████████░ 93.2% 1,325 / 1,421
approvals ███████████████████░ 93.3% 3,395 / 3,640
reminders ███████████████████░ 93.4% 468 / 501
endpoints ███████████████████░ 94.1% 8,606 / 9,143
revenue_analytics ███████████████████░ 94.5% 3,598 / 3,809
skills ███████████████████░ 94.5% 2,881 / 3,049
review_hog ███████████████████░ 94.6% 6,806 / 7,191
logs ███████████████████░ 95.3% 9,844 / 10,325
experiments ███████████████████░ 95.7% 24,417 / 25,527
replay_vision ███████████████████░ 95.8% 13,776 / 14,383
annotations ███████████████████░ 96.2% 732 / 761
feature_flags ███████████████████░ 96.3% 16,233 / 16,859
user_interviews ███████████████████░ 96.4% 2,242 / 2,325
warehouse_sources ███████████████████░ 96.4% 287,423 / 298,013
customer_analytics ███████████████████░ 97.2% 7,482 / 7,700
data_catalog ███████████████████░ 97.4% 2,303 / 2,365
pulse ████████████████████ 98.4% 2,017 / 2,049

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

Generated-By: PostHog Code
Task-Id: c6d78416-85c8-4304-bd69-e309562468c4
Generated-By: PostHog Code
Task-Id: c6d78416-85c8-4304-bd69-e309562468c4
@github-actions

github-actions Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

🦔 Hogbox preview · ✅ ready

▶ Open the preview

🔑 Login test@posthog.com / 12345678 (demo data)
🧩 Running this PR's backend and frontend, on the PostHog :master base
🔗 Link stable across rebuilds — a re-push swaps the box underneath, the URL stays
🔒 Access tailnet only (PostHog VPN)
🛠️ Admin inspect & debug state in hogland
💤 Idle sleeps after ~30 min idle (snapshot to S3, zero node cost) and wakes on your next visit in ~30s, behind a brief "waking up" screen

commit 44a6a0e · box box-a5085cdbeaf2 · ready in 749s (push → usable) · build log · rebuilds on every push, torn down on close

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team July 19, 2026 14:38
Comment thread products/tasks/backend/logic/services/sandbox.py Outdated
Comment thread products/tasks/backend/sandbox/images/start-virtual-desktop.sh
@veria-ai

veria-ai Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

Generated-By: PostHog Code
Task-Id: c6d78416-85c8-4304-bd69-e309562468c4
Generated-By: PostHog Code
Task-Id: c6d78416-85c8-4304-bd69-e309562468c4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant