Skip to content

feat(agent): report cloud rtk token savings#3347

Merged
tatoalo merged 3 commits into
mainfrom
posthog-code/report-cloud-rtk-savings
Jul 12, 2026
Merged

feat(agent): report cloud rtk token savings#3347
tatoalo merged 3 commits into
mainfrom
posthog-code/report-cloud-rtk-savings

Conversation

@tatoalo

@tatoalo tatoalo commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Problem

Cloud runs can use RTK to compress verbose command output, but current production telemetry only compares total run tokens between rollout cohorts. That comparison is affected by workload and run length, so it cannot measure the command-output savings directly.

Changes

  • Read RTK's cumulative gain summary at terminal cleanup without exposing process secrets to the subprocess.
  • Emit a once-only _posthog/rtk_savings event before event ingest closes, including runtime and model context.
  • Keep the reporting path best-effort so missing, malformed, or failing RTK telemetry never affects a run.
  • Isolate terminal-failure tests from RTK resolution so they continue to assert event ordering deterministically.

Generated-By: PostHog Code
Task-Id: 0cd57c96-660f-4e98-8153-e736f0ef5649
@tatoalo tatoalo self-assigned this Jul 10, 2026
@trunk-io

trunk-io Bot commented Jul 10, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown

React Doctor found no issues in the changed files. 🎉

Reviewed by React Doctor for commit 9279717.

@tatoalo tatoalo added the Stamphog This will request an autostamp by stamphog on small changes label Jul 10, 2026
@tatoalo tatoalo marked this pull request as ready for review July 10, 2026 17:19
github-actions[bot]
github-actions Bot previously approved these changes Jul 10, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-contained additive feature: emits a new RTK savings notification once per run with proper idempotency guard, secure env scrubbing on the subprocess call, timeout, and silent error handling. Good test coverage for all edge cases, no existing code paths changed.

@greptile-apps

greptile-apps Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "fix(agent): stabilize rtk savings shutdo..." | Re-trigger Greptile

Comment thread packages/agent/src/server/rtk-savings.ts
@github-actions github-actions Bot dismissed their stale review July 12, 2026 09:13

New commits pushed (delta classified non_trivial_delta) — stamphog approval dismissed; re-review running automatically.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additive feature that emits RTK token savings once per run at session cleanup. The env-scrubbing concern raised in review was fixed (RTK_DB_PATH preserved in allowlist), idempotency is guarded by rtkSavingsAttempted flag, errors are silently swallowed, subprocess has a timeout, and test coverage is thorough.

@tatoalo tatoalo enabled auto-merge (squash) July 12, 2026 09:16
@tatoalo tatoalo merged commit c54878c into main Jul 12, 2026
26 checks passed
@tatoalo tatoalo deleted the posthog-code/report-cloud-rtk-savings branch July 12, 2026 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Stamphog This will request an autostamp by stamphog on small changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant