Skip to content

feat(tasks): add runtime_adapter label to run token metrics#71447

Merged
tatoalo merged 1 commit into
masterfrom
posthog-code/tasks-run-tokens-runtime-adapter-label
Jul 16, 2026
Merged

feat(tasks): add runtime_adapter label to run token metrics#71447
tatoalo merged 1 commit into
masterfrom
posthog-code/tasks-run-tokens-runtime-adapter-label

Conversation

@tatoalo

@tatoalo tatoalo commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Problem

The terminal-run token metrics (tasks_run_total_tokens, tasks_run_tokens_total) carry an rtk_enabled cohort label but no runtime adapter, so the rtk cohort panels on the cloud-background-agents-runs dashboard can't separate Claude runs (where rtk is applied deterministically via a PreToolUse rewrite hook) from Codex runs (where rtk adoption is instruction-level, added in PostHog/code#3491). Without the split there's no way to see whether the Codex integration actually moves tokens per run.

Why: rtk command-output compression is being extended to cloud Codex runs, and the per-adapter cohort is the graph that validates (or falsifies) that change.

Changes

  • record_run_token_usage now takes the run's runtime_adapter (read from TaskRun.state, where run creation already persists it) and labels both token metrics with it, bounded to claude/codex/other/unknown to cap cardinality.
  • Corrected the sandbox Dockerfile.sandbox-base comment that claimed "the agent" auto-detects rtk on PATH — that is true only for Claude; Codex has no command-rewrite channel and is instructed instead.

Companion dashboard change (splitting the rtk cohort panels by this label) goes to the grafana-dashboards repo.

The tasks_run_total_tokens / tasks_run_tokens_total metrics carry an rtk_enabled cohort label but no adapter, so the rtk cohort comparison can't separate Claude runs (deterministic PreToolUse rewriting) from Codex runs (instruction-level adoption). Label terminal-run token metrics with the run's runtime adapter, bounded to claude/codex/other/unknown. Also corrects the sandbox Dockerfile comment that claimed rtk is auto-detected by "the agent" — that is true only for Claude.

Generated-By: PostHog Code
Task-Id: b005dede-2677-4364-94a2-c2a7e90e376a
@trunk-io

trunk-io Bot commented Jul 16, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@PostHog PostHog deleted a comment from github-actions Bot Jul 16, 2026
@tatoalo
tatoalo marked this pull request as ready for review July 16, 2026 10:00
@assign-reviewers-posthog
assign-reviewers-posthog Bot requested a review from a team July 16, 2026 10:01
@greptile-apps

greptile-apps Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "feat(tasks): add runtime_adapter label t..." | Re-trigger Greptile

@tatoalo tatoalo added the stamphog Request AI approval (no full review) label Jul 16, 2026
@stamphog

stamphog Bot commented Jul 16, 2026

Copy link
Copy Markdown

Note

🤖 stamphog reviewed 41b1c55bb3055ff30360f262564c5aafab354c46 — verdict: REFUSED

Gate verdict is DENIED (deny-list hit on toolchain/infra plus T2-never tier), which forces routing to a human regardless of how contained the diff looks; the diff itself (metrics label + Dockerfile comment fix) appears low-risk and Greptile left a 👍 with no concerns.

  • 👍 on the PR from greptile-apps[bot], hex-security-app[bot].
  • Gates DENIED: deny-list matched deps_toolchain and infra_cicd (Dockerfile.sandbox-base change), and the PR was classified as T2-never (never auto-approved regardless of size).
Gate mechanics and policy version
Gate Result
prerequisites all clear
deny-list matches: deps_toolchain, infra_cicd
size 22L, 3F substantive, 24L/4F incl. docs/generated/snapshots — within ceiling
tier classified as T2-never: T2-never (24L, 4F, single-area, feat)
stamphog 2.0.0b3 .stamphog/policy.yml @ 9267ac7 · reviewed head 41b1c55

@stamphog stamphog Bot removed the stamphog Request AI approval (no full review) label Jul 16, 2026
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Backend coverage — 28.0% of changed backend lines covered — 5 uncovered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ██████░░░░░░░░░░░░░░ 28.0% (2 / 7)

File Patch Uncovered changed lines
products/tasks/backend/temporal/process_task/activities/tests/test_update_task_run_status.py 0.0% 120
products/tasks/backend/temporal/process_task/activities/update_task_run_status.py 0.0% 107
products/tasks/backend/temporal/metrics.py 40.0% 79–81

🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 29490050673 -n patch-coverage), or the coverage-data block at the end of this comment.

Per-product line coverage (touched products)
Product Coverage Lines
tasks █████████████░░░░░░░ 67.4% 25,787 / 38,257

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

@tatoalo
tatoalo merged commit b7f65e4 into master Jul 16, 2026
370 of 373 checks passed
@tatoalo
tatoalo deleted the posthog-code/tasks-run-tokens-runtime-adapter-label branch July 16, 2026 15:14
@deployment-status-posthog

deployment-status-posthog Bot commented Jul 16, 2026

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-07-16 15:52 UTC Run
prod-us ✅ Deployed 2026-07-16 16:08 UTC Run
prod-eu ✅ Deployed 2026-07-16 16:07 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants