Skip to content

feat(turns): extract turns from agent-tokens artifact in workflow_run_id mode#20

Merged
adamhenson merged 4 commits into
mainfrom
feat/turns-from-artifact
Jun 21, 2026
Merged

feat(turns): extract turns from agent-tokens artifact in workflow_run_id mode#20
adamhenson merged 4 commits into
mainfrom
feat/turns-from-artifact

Conversation

@adamhenson

Copy link
Copy Markdown
Contributor

Summary

  • Adds optional turns field to AgentTokensArtifact type
  • parseAgentTokensZip now passes turns through when present in the artifact JSON
  • fetchAgentTokens returns artifactTurns: number | null alongside tokens
  • WorkflowRunData now carries artifactTurns
  • run.ts resolution priority: explicit turns input → artifactTurns (artifact) → extractTurnsFromOutput(agent_output)

Why turns were never populated for CI runs

The action already had extractTurnsFromOutput wired, but the workflow_run_id path fetches tokens from the agent-tokens artifact rather than processing agent_output directly. The artifact format didn't include turns, so they were always null.

How to wire turns in an agent workflow

In the workflow that produces the artifact, add turns to agent-tokens.json:

- name: Save agent tokens
  run: |
    echo '{
      "input_tokens": ${{ steps.agent.outputs.input_tokens }},
      "output_tokens": ${{ steps.agent.outputs.output_tokens }},
      "cache_read_tokens": ${{ steps.agent.outputs.cache_read_tokens }},
      "cache_write_tokens": ${{ steps.agent.outputs.cache_write_tokens }},
      "turns": ${{ steps.agent.outputs.num_turns }}
    }' > agent-tokens.json
  - uses: actions/upload-artifact@v4
    with:
      name: agent-tokens
      path: agent-tokens.json

The field is optional — existing workflows that don't include it will continue to work with artifactTurns: null.

Test plan

  • 2 new tests: extracts turns from artifact when present, sets artifactTurns to null when absent
  • All 111 existing tests pass

@github-actions

Copy link
Copy Markdown

Findings:

  • src/workflow-run.ts:183-206 The conclusion-job gate fails closed on any listJobsForWorkflowRun error and skips the run entirely. That avoids duplicates, but it also turns a transient GitHub API hiccup or token-scope issue into permanent data loss for that agent run. Since this gate is only for deduping gh-aw, consider retrying once and/or proceeding when the API call fails instead of dropping the ingest.
  • src/comment.ts:305-319 findExistingComment() swallows all list/pagination errors and returns null, and upsertComment() then creates a new comment. If listComments is rate-limited or intermittently fails, the action will duplicate the AgentMeter comment instead of updating the existing one. A lookup failure should either retry or abort comment posting, not fall through to create.

Overall the implementation is solid, but those two paths can fail in production in ways that either lose a run or spam duplicate comments.

@github-actions

Copy link
Copy Markdown

Findings:

  • src/run.ts:102-105 and src/workflow-run.ts:114 can produce an empty workflowName in workflow_run mode. If getWorkflowRun() fails or returns no name, the code overwrites resolvedWorkflowName with '', so the ingest payload and PR comment end up with a blank workflow column. Since resolveWorkflowRun() is explicitly allowed to return partial data on failure, this should preserve a non-empty fallback or abort instead of writing an empty name.

  • src/workflow-run.ts:337-347 overmatches issue branches. The regex \bagent\/issue-(\d+)\b will match any branch containing that substring, such as feature/agent/issue-12-fix, not just the intended exact agent/issue-N convention. That can misattribute runs to the wrong issue. Anchor the pattern to the full branch name.

  • src/run.ts:141-146 does not safely format manual trigger_number overrides in workflow_run mode. If automatic trigger resolution fails and only trigger_number is provided, buildTriggerRef() falls back to #N unless trigger_event is also set, so PRs can be mislabeled as issues. Either infer PR-vs-issue from the resolved trigger type or require trigger_event when trigger_number is used.

@github-actions

Copy link
Copy Markdown
  1. README.md:288-302 - The new non-gh-aw example now always prints "turns":%s, but the note says turns is optional. If TURNS is unset, this writes invalid JSON into /tmp/agent-tokens.json and breaks the artifact. Make it conditional like the gh-aw snippet above, or explicitly set a default numeric value only when you truly have one.

The code changes in src/workflow-run.ts and src/run.ts look consistent otherwise; I didn’t spot a runtime regression in the turn-resolution flow.

@adamhenson
adamhenson merged commit 8a8068c into main Jun 21, 2026
8 checks passed
@adamhenson
adamhenson deleted the feat/turns-from-artifact branch June 21, 2026 17:20
adamhenson added a commit that referenced this pull request Jun 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant