Skip to content

Commit 3bb8f02

Browse files
cocosheng-gcocosheng-g
committed
refactor: transition to specific PR, Issue, and Custom Tracking ID inputs for strongly-typed telemetry
1 parent fe91504 commit 3bb8f02

11 files changed

Lines changed: 290 additions & 14 deletions

File tree

‎action.yml‎

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -91,10 +91,17 @@ inputs:
9191
description: 'The GitHub workflow name, used for telemetry purposes.'
9292
required: false
9393
default: '${{ github.workflow }}'
94-
github_item_number:
95-
description: 'The Issue or Pull Request number the CLI is operating on. Useful for tracking unique item usage in telemetry. Defaults to the event payload.'
94+
github_pr_number:
95+
description: 'The Pull Request number the CLI is operating on. Defaults to the event payload.'
96+
required: false
97+
default: '${{ github.event.pull_request.number }}'
98+
github_issue_number:
99+
description: 'The Issue number the CLI is operating on. Defaults to the event payload.'
100+
required: false
101+
default: '${{ github.event.issue.number }}'
102+
github_custom_tracking_id:
103+
description: 'A custom tracking ID (e.g. comma-separated list of issue IDs for scheduled batches).'
96104
required: false
97-
default: '${{ github.event.issue.number || github.event.pull_request.number }}'
98105

99106
outputs:
100107
summary:
@@ -237,7 +244,9 @@ runs:
237244
USE_PNPM: '${{ inputs.use_pnpm }}'
238245
SURFACE: 'GitHub'
239246
GH_WORKFLOW_NAME: '${{ steps.sanitize_workflow_name.outputs.gh_workflow_name }}'
240-
GH_EVENT_NUMBER: '${{ inputs.github_item_number }}'
247+
GH_PR_NUMBER: '${{ inputs.github_pr_number }}'
248+
GH_ISSUE_NUMBER: '${{ inputs.github_issue_number }}'
249+
GH_CUSTOM_TRACKING_ID: '${{ inputs.github_custom_tracking_id }}'
241250
shell: 'bash'
242251
run: |-
243252
set -euo pipefail
@@ -421,7 +430,9 @@ runs:
421430
PROMPT: '${{ inputs.prompt }}'
422431
GEMINI_MODEL: '${{ inputs.gemini_model }}'
423432
GH_WORKFLOW_NAME: '${{ steps.sanitize_workflow_name.outputs.gh_workflow_name }}'
424-
GH_EVENT_NUMBER: '${{ inputs.github_item_number }}'
433+
GH_PR_NUMBER: '${{ inputs.github_pr_number }}'
434+
GH_ISSUE_NUMBER: '${{ inputs.github_issue_number }}'
435+
GH_CUSTOM_TRACKING_ID: '${{ inputs.github_custom_tracking_id }}'
425436

426437
- name: 'Upload Gemini CLI outputs'
427438
if: |-
@@ -448,7 +459,9 @@ runs:
448459
sed -e "s#OTLP_GOOGLE_CLOUD_PROJECT#${OTLP_GOOGLE_CLOUD_PROJECT}#g" \
449460
-e "s#GITHUB_REPOSITORY_PLACEHOLDER#${GITHUB_REPOSITORY}#g" \
450461
-e "s#GITHUB_RUN_ID_PLACEHOLDER#${GITHUB_RUN_ID}#g" \
451-
-e "s#GITHUB_EVENT_NUMBER_PLACEHOLDER#${GITHUB_EVENT_NUMBER}#g" \
462+
-e "s#GITHUB_PR_NUMBER_PLACEHOLDER#${GITHUB_PR_NUMBER}#g" \
463+
-e "s#GITHUB_ISSUE_NUMBER_PLACEHOLDER#${GITHUB_ISSUE_NUMBER}#g" \
464+
-e "s#GITHUB_CUSTOM_TRACKING_ID_PLACEHOLDER#${GITHUB_CUSTOM_TRACKING_ID}#g" \
452465
"${GITHUB_ACTION_PATH}/scripts/collector-gcp.yaml.template" > ".gemini/collector-gcp.yaml"
453466
454467
# Ensure credentials file has the right permissions
@@ -499,7 +512,9 @@ runs:
499512
GITHUB_ACTION_PATH: '${{ github.action_path }}'
500513
GITHUB_REPOSITORY: '${{ github.repository }}'
501514
GITHUB_RUN_ID: '${{ github.run_id }}'
502-
GITHUB_EVENT_NUMBER: '${{ inputs.github_item_number }}'
515+
GITHUB_PR_NUMBER: '${{ inputs.github_pr_number }}'
516+
GITHUB_ISSUE_NUMBER: '${{ inputs.github_issue_number }}'
517+
GITHUB_CUSTOM_TRACKING_ID: '${{ inputs.github_custom_tracking_id }}'
503518

504519
branding:
505520
icon: 'terminal'

‎examples/workflows/gemini-assistant/gemini-invoke.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,8 @@ jobs:
6666
use_vertex_ai: '${{ vars.GOOGLE_GENAI_USE_VERTEXAI }}'
6767
upload_artifacts: '${{ vars.UPLOAD_ARTIFACTS }}'
6868
workflow_name: 'gemini-invoke'
69-
github_item_number: '${{ github.event.pull_request.number || github.event.issue.number }}'
69+
github_pr_number: '${{ github.event.pull_request.number }}'
70+
github_issue_number: '${{ github.event.issue.number }}'
7071
settings: |-
7172
{
7273
"model": {

‎examples/workflows/gemini-assistant/gemini-plan-execute.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,8 @@ jobs:
6868
use_vertex_ai: '${{ vars.GOOGLE_GENAI_USE_VERTEXAI }}'
6969
upload_artifacts: '${{ vars.UPLOAD_ARTIFACTS }}'
7070
workflow_name: 'gemini-invoke'
71-
github_item_number: '${{ github.event.pull_request.number || github.event.issue.number }}'
71+
github_pr_number: '${{ github.event.pull_request.number }}'
72+
github_issue_number: '${{ github.event.issue.number }}'
7273
settings: |-
7374
{
7475
"model": {

‎examples/workflows/issue-triage/gemini-scheduled-triage.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ jobs:
112112
use_vertex_ai: '${{ vars.GOOGLE_GENAI_USE_VERTEXAI }}'
113113
upload_artifacts: '${{ vars.UPLOAD_ARTIFACTS }}'
114114
workflow_name: 'gemini-scheduled-triage'
115-
github_item_number: '${{ steps.find_issues.outputs.issue_numbers }}'
115+
github_custom_tracking_id: '${{ steps.find_issues.outputs.issue_numbers }}'
116116
settings: |-
117117
{
118118
"model": {

‎examples/workflows/issue-triage/gemini-triage.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ jobs:
7979
use_vertex_ai: '${{ vars.GOOGLE_GENAI_USE_VERTEXAI }}'
8080
upload_artifacts: '${{ vars.UPLOAD_ARTIFACTS }}'
8181
workflow_name: 'gemini-triage'
82-
github_item_number: '${{ github.event.issue.number || github.event.pull_request.number }}'
82+
github_issue_number: '${{ github.event.issue.number }}'
8383
settings: |-
8484
{
8585
"model": {

‎examples/workflows/pr-review/gemini-review.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ jobs:
6565
use_vertex_ai: '${{ vars.GOOGLE_GENAI_USE_VERTEXAI }}'
6666
upload_artifacts: '${{ vars.UPLOAD_ARTIFACTS }}'
6767
workflow_name: 'gemini-review'
68-
github_item_number: '${{ github.event.pull_request.number || github.event.issue.number }}'
68+
github_pr_number: '${{ github.event.pull_request.number }}'
6969
settings: |-
7070
{
7171
"model": {

‎gemini_cli_pr_body.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
## Summary
2+
3+
This PR adds new telemetry attributes to the Gemini CLI to support granular usage metrics for GitHub Actions. These metrics will allow us to track unique Issues and PRs touched by the CLI, and distinguish between different workflow triggers (e.g., Automated vs. Scheduled triage).
4+
5+
Will fix https://github.com/google-gemini/gemini-cli/issues/21130
6+
7+
## Details
8+
9+
- Added `GEMINI_CLI_GH_EVENT_NAME` to `EventMetadataKey`.
10+
- Added specific tracking IDs: `GEMINI_CLI_GH_PR_NUMBER`, `GEMINI_CLI_GH_ISSUE_NUMBER`, and `GEMINI_CLI_GH_CUSTOM_TRACKING_ID` replacing generic `GH_EVENT_NUMBER`.
11+
- Updated `ClearcutLogger` to capture these from environment variables (`GITHUB_EVENT_NAME`, `GH_PR_NUMBER`, `GH_ISSUE_NUMBER`, and `GH_CUSTOM_TRACKING_ID`).
12+
- The PR/Issue number is logged as a raw string ID (e.g., `123`) to avoid fingerprinting PII, relying on the repository context to provide global uniqueness for backend queries.
13+
- Updated telemetry tests in `clearcut-logger.test.ts` to cover the new environment variables and metadata keys.
14+
- Updated telemetry documentation in `docs/cli/telemetry.md`.
15+
16+
### Example Telemetry Scenarios
17+
18+
Because the CLI appends these context fields to the `baseMetadata` of all events (API Request, Response, Error), downstream analytics can perfectly distinguish and count unique issues vs. PRs based on the workflow context.
19+
20+
#### Scenario A: Automated PR Review
21+
22+
When PR #42 triggers the `gemini-review` workflow, the CLI logs:
23+
24+
```json
25+
{
26+
"event_name": "api_request",
27+
"event_metadata": [
28+
[
29+
{ "gemini_cli_key": 130, "value": "gemini-review" },
30+
{ "gemini_cli_key": 172, "value": "pull_request" },
31+
{ "gemini_cli_key": 173, "value": "42" } // GH_PR_NUMBER
32+
]
33+
]
34+
}
35+
```
36+
37+
_Analysts can count `DISTINCT gh_pr_number WHERE gh_workflow_name LIKE '%review%'` to satisfy the P0 PR metric._
38+
39+
#### Scenario B: Automated Issue Triage
40+
41+
When Issue #88 triggers the `gemini-triage` workflow, the CLI logs:
42+
43+
```json
44+
{
45+
"event_name": "api_request",
46+
"event_metadata": [
47+
[
48+
{ "gemini_cli_key": 130, "value": "gemini-triage" },
49+
{ "gemini_cli_key": 172, "value": "issues" },
50+
{ "gemini_cli_key": 174, "value": "88" } // GH_ISSUE_NUMBER
51+
]
52+
]
53+
}
54+
```
55+
56+
_Analysts can count `DISTINCT gh_issue_number WHERE gh_workflow_name LIKE '%triage%' AND gh_event_name = 'issues'` to satisfy the P0 Automated Issue metric._
57+
58+
#### Scenario C: Scheduled Batch Triage
59+
60+
When a cron job runs and triages 3 issues (IDs 101, 102, 103) in a single CLI invocation, the CLI logs:
61+
62+
```json
63+
{
64+
"event_name": "api_request",
65+
"event_metadata": [
66+
[
67+
{ "gemini_cli_key": 130, "value": "gemini-scheduled-triage" },
68+
{ "gemini_cli_key": 172, "value": "schedule" },
69+
{ "gemini_cli_key": 175, "value": "101,102,103" } // GH_CUSTOM_TRACKING_ID
70+
]
71+
]
72+
}
73+
```
74+
75+
_Analysts can split the `gh_custom_tracking_id` string by commas to count exactly 3 unique issues processed during a scheduled invocation._
76+
77+
## Related Issues
78+
79+
Related to internal PM request for "Gemini CLI GitHub Action Usage Metric".
80+
81+
## How to Validate
82+
83+
1. Run the new tests: `npm test -w @google/gemini-cli-core -- src/telemetry/clearcut-logger/clearcut-logger.test.ts`
84+
2. Verify that when `GITHUB_EVENT_NAME` and specific IDs (`GH_PR_NUMBER`, etc.) are set in the environment, they appear in the Clearcut log metadata as expected.
85+
86+
## Pre-Merge Checklist
87+
88+
- [x] Updated relevant documentation and README (if needed)
89+
- [x] Added/updated tests (if needed)
90+
- [ ] Noted breaking changes (if any)
91+
- [ ] Validated on required platforms/methods:
92+
- [ ] MacOS
93+
- [ ] Windows
94+
- [x] Linux

‎patch.diff‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
--- packages/core/src/telemetry/clearcut-logger/event-metadata-key.ts
2+
+++ packages/core/src/telemetry/clearcut-logger/event-metadata-key.ts
3+
@@ -219,8 +219,14 @@
4+
GEMINI_CLI_GH_EVENT_NAME = 172,
5+
6+
- // Logs the event number (Issue or PR ID) of the GitHub Action that triggered the session.
7+
- GEMINI_CLI_GH_EVENT_NUMBER = 173,
8+
+ // Logs the Pull Request number of the GitHub Action that triggered the session.
9+
+ GEMINI_CLI_GH_PR_NUMBER = 173,
10+
+
11+
+ // Logs the Issue number of the GitHub Action that triggered the session.
12+
+ GEMINI_CLI_GH_ISSUE_NUMBER = 174,
13+
+
14+
+ // Logs a custom item number of the GitHub Action that triggered the session.
15+
+ GEMINI_CLI_GH_CUSTOM_NUMBER = 175,
16+
17+
// ==========================================================================
18+
// Loop Detected Event Keys

‎patch2.diff‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
--- packages/core/src/telemetry/clearcut-logger/clearcut-logger.ts
2+
+++ packages/core/src/telemetry/clearcut-logger/clearcut-logger.ts
3+
@@ -194,10 +194,24 @@
4+
}
5+
6+
/**
7+
- * Determines the GitHub event number if the CLI is running in a GitHub Actions environment.
8+
+ * Determines the GitHub PR number if the CLI is running in a GitHub Actions environment.
9+
*/
10+
-function determineGHEventNumber(): string | undefined {
11+
- return process.env['GH_EVENT_NUMBER'];
12+
+function determineGHPRNumber(): string | undefined {
13+
+ return process.env['GH_PR_NUMBER'];
14+
+}
15+
+
16+
+/**
17+
+ * Determines the GitHub Issue number if the CLI is running in a GitHub Actions environment.
18+
+ */
19+
+function determineGHIssueNumber(): string | undefined {
20+
+ return process.env['GH_ISSUE_NUMBER'];
21+
+}
22+
+
23+
+/**
24+
+ * Determines the custom GitHub item number if the CLI is running in a GitHub Actions environment.
25+
+ */
26+
+function determineGHCustomNumber(): string | undefined {
27+
+ return process.env['GH_CUSTOM_NUMBER'];
28+
}
29+
30+
/**
31+
@@ -391,7 +405,9 @@
32+
const email = this.userAccountManager.getCachedGoogleAccount();
33+
const surface = determineSurface();
34+
const ghWorkflowName = determineGHWorkflowName();
35+
const ghEventName = determineGHEventName();
36+
- const ghEventNumber = determineGHEventNumber();
37+
+ const ghPRNumber = determineGHPRNumber();
38+
+ const ghIssueNumber = determineGHIssueNumber();
39+
+ const ghCustomNumber = determineGHCustomNumber();
40+
const baseMetadata: EventValue[] = [
41+
...data,
42+
@@ -433,10 +449,24 @@
43+
});
44+
}
45+
46+
- if (ghEventNumber) {
47+
+ if (ghPRNumber) {
48+
baseMetadata.push({
49+
- gemini_cli_key: EventMetadataKey.GEMINI_CLI_GH_EVENT_NUMBER,
50+
- value: ghEventNumber,
51+
+ gemini_cli_key: EventMetadataKey.GEMINI_CLI_GH_PR_NUMBER,
52+
+ value: ghPRNumber,
53+
+ });
54+
+ }
55+
+
56+
+ if (ghIssueNumber) {
57+
+ baseMetadata.push({
58+
+ gemini_cli_key: EventMetadataKey.GEMINI_CLI_GH_ISSUE_NUMBER,
59+
+ value: ghIssueNumber,
60+
+ });
61+
+ }
62+
+
63+
+ if (ghCustomNumber) {
64+
+ baseMetadata.push({
65+
+ gemini_cli_key: EventMetadataKey.GEMINI_CLI_GH_CUSTOM_NUMBER,
66+
+ value: ghCustomNumber,
67+
});
68+
}
69+

‎pr_body.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
## Description
2+
3+
This PR implements the necessary infrastructure in the \`run-gemini-cli\` GitHub Action to support the Gemini CLI usage metrics (P0 requirements).
4+
5+
It aligns with the core CLI telemetry changes introduced in [google-gemini/gemini-cli#21129](https://github.com/google-gemini/gemini-cli/pull/21129).
6+
7+
### Changes Made:
8+
9+
- **Precise Telemetry Tracking:** Introduced a new \`github_item_number\` input to \`action.yml\`. This defaults to the current Issue or PR number but allows explicit overrides. This resolves the inability to track unique items across different workflow types (e.g., scheduled vs. automated).
10+
- **Installation Telemetry:** Added \`GH_WORKFLOW_NAME\` and \`GH_EVENT_NUMBER\` to the \`Install Gemini CLI\` step to ensure extension installations are properly tracked.
11+
- **OTEL Collector Enhancement:** Added \`github.event.number\` to the \`scripts/collector-gcp.yaml.template\` resource attributes, ensuring Google Cloud Logging metrics are fully synchronized with Clearcut.
12+
- **Example Workflow Updates:** Modified the \`gemini-scheduled-triage.yml\` example to correctly extract and pass a comma-separated list of issue IDs to the \`github_item_number\` input. This prevents telemetry corruption during batch processing.
13+
14+
### Example Telemetry Scenarios
15+
16+
Because the CLI appends the \`GH_EVENT_NUMBER\` and \`GH_WORKFLOW_NAME\` to the \`baseMetadata\` of all events (API Request, Response, Error), downstream analytics can perfectly distinguish and count unique issues vs. PRs based on the workflow context.
17+
18+
#### Scenario A: Automated PR Review
19+
20+
When PR #42 triggers the \`gemini-review\` workflow, the CLI logs:
21+
\`\`\`json
22+
{
23+
\"event_name\": \"api_request\",
24+
\"event_metadata\": [
25+
[
26+
{ \"gemini_cli_key\": 130, \"value\": \"gemini-review\" },
27+
{ \"gemini_cli_key\": 172, \"value\": \"pull_request\" },
28+
{ \"gemini_cli_key\": 173, \"value\": \"42\" } // GH_EVENT_NUMBER
29+
]
30+
]
31+
}
32+
\`\`\`
33+
_Analysts can count \`DISTINCT gh_event_number WHERE gh_workflow_name LIKE '%review%'\` to satisfy the P0 PR metric._
34+
35+
#### Scenario B: Automated Issue Triage
36+
37+
When Issue #88 triggers the \`gemini-triage\` workflow, the CLI logs:
38+
\`\`\`json
39+
{
40+
\"event_name\": \"api_request\",
41+
\"event_metadata\": [
42+
[
43+
{ \"gemini_cli_key\": 130, \"value\": \"gemini-triage\" },
44+
{ \"gemini_cli_key\": 172, \"value\": \"issues\" },
45+
{ \"gemini_cli_key\": 173, \"value\": \"88\" } // GH_EVENT_NUMBER
46+
]
47+
]
48+
}
49+
\`\`\`
50+
_Analysts can count \`DISTINCT gh_event_number WHERE gh_workflow_name LIKE '%triage%' AND gh_event_name = 'issues'\` to satisfy the P0 Automated Issue metric._
51+
52+
#### Scenario C: Scheduled Batch Triage
53+
54+
When a cron job runs and triages 3 issues (IDs 101, 102, 103) in a single CLI invocation, the CLI logs:
55+
\`\`\`json
56+
{
57+
\"event_name\": \"api_request\",
58+
\"event_metadata\": [
59+
[
60+
{ \"gemini_cli_key\": 130, \"value\": \"gemini-scheduled-triage\" },
61+
{ \"gemini_cli_key\": 172, \"value\": \"schedule\" },
62+
{ \"gemini_cli_key\": 173, \"value\": \"101,102,103\" } // GH_EVENT_NUMBER
63+
]
64+
]
65+
}
66+
\`\`\`
67+
_Analysts can split the \`gh_event_number\` string by commas to count exactly 3 unique issues processed during a scheduled invocation._
68+
69+
## Related Issues
70+
71+
- Fixes P0 metrics requirements for GitHub Action usage.
72+
- Depends on [google-gemini/gemini-cli#21129](https://github.com/google-gemini/gemini-cli/pull/21129)

0 commit comments

Comments
 (0)