Skip to content

Commit 100dc11

Browse files
authored
feat(ci): add days-since-last-pin daily workflow (DataDog#22951)
* feat(ci): add days-since-last-pin daily workflow - Add .github/workflows/days-since-last-pin.yml that runs daily at 9:42 UTC - Computes days since INTEGRATIONS_CORE_VERSION was last updated in datadog-agent/release.json - Posts gauge metric integrations_core.days_since_last_pin{team:agent-integrations} to Datadog API v2 Rationale: AI-6462 — need a CI dashboard counter that turns red when the agent repo hasn't been pinned in >4 days * refactor(ci): extract inline script and fix error handling in days-since-last-pin - Extract 90-line embedded Python heredoc to .github/workflows/scripts/days_since_last_pin.py - Add actions/checkout step so the workflow can access the script file - Replace silent break-on-error with raise to fail the job on fetch errors - Split single step into compute + submit for step-level failure attribution - Add comment explaining the per_page=30 assumption Rationale: PR review found inline heredoc diverged from repo convention, and a mid-walk fetch failure would silently submit a falsely-healthy metric This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * fix(ci): bound commit walk by 30-day window instead of fixed page count - Replace per_page=30 with a since=30-days-ago query parameter and full pagination - release.json is updated for many dep changes (JMXFETCH, OMNIBUS_RUBY, etc.), so a fixed page count could exhaust without finding the pin-change commit - When no commits found in the window, report days=30 (pin is at least that old) Rationale: the sparse-commit assumption was wrong; time-bounded window is the correct approach This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * refactor(ci): replace GITHUB_TOKEN with dd-octo-sts for cross-repo access - Add dd-octo-sts-action step to exchange OIDC token for a scoped token on DataDog/datadog-agent (contents:read only) - Pass dd-octo-sts token as GITHUB_TOKEN to the compute step instead of the built-in secrets.GITHUB_TOKEN - Add id-token:write permission to the job for OIDC federation Trust policy PR: DataDog/datadog-agent#48035 Rationale: scoped short-lived token is more secure than using the default GITHUB_TOKEN for cross-repo access This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * refactor(ci): add type hints and fix spurious page increment in days-since-last-pin - Add type annotations to fetch_json and get_integrations_core_version - Guard page += 1 behind `if not pin_changed` to avoid incrementing after pin change is detected Rationale: page was incremented even when pin_changed became True at the end of the loop body, which is unnecessary; also adds explicit types for clarity This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md)
1 parent f0b0b59 commit 100dc11

2 files changed

Lines changed: 127 additions & 0 deletions

File tree

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
name: Days Since Last Pin
2+
3+
on:
4+
schedule:
5+
- cron: "42 9 * * *"
6+
workflow_dispatch:
7+
8+
jobs:
9+
compute-days-since-last-pin:
10+
runs-on: ubuntu-22.04
11+
permissions:
12+
contents: read
13+
id-token: write
14+
steps:
15+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
16+
17+
- uses: DataDog/dd-octo-sts-action@acaa02eee7e3bb0839e4272dacb37b8f3b58ba80 # v1.0.3
18+
id: octo-sts
19+
with:
20+
scope: DataDog/datadog-agent
21+
policy: integrations-core.github.read-release-json.schedule
22+
23+
- name: Compute days since last pin
24+
id: compute
25+
env:
26+
GITHUB_TOKEN: ${{ steps.octo-sts.outputs.token }}
27+
run: python3 .github/workflows/scripts/days_since_last_pin.py
28+
29+
- name: Submit metric to Datadog
30+
env:
31+
DD_API_KEY: ${{ secrets.DD_API_KEY }}
32+
DAYS: ${{ steps.compute.outputs.days }}
33+
run: |
34+
python3 - <<'EOF'
35+
import json, os, time, urllib.request
36+
days = int(os.environ["DAYS"])
37+
payload = json.dumps({"series": [{"metric": "integrations_core.days_since_last_pin", "type": 3, "points": [{"timestamp": int(time.time()), "value": days}], "tags": ["team:agent-integrations"]}]}).encode()
38+
req = urllib.request.Request("https://api.datadoghq.com/api/v2/series", data=payload, headers={"DD-API-KEY": os.environ["DD_API_KEY"], "Content-Type": "application/json"}, method="POST")
39+
with urllib.request.urlopen(req) as resp:
40+
print(f"Datadog API response: {json.loads(resp.read().decode())}")
41+
EOF
Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
import json
2+
import os
3+
import urllib.request
4+
from datetime import datetime, timedelta, timezone
5+
from typing import Any
6+
7+
AGENT_REPO = "DataDog/datadog-agent"
8+
RELEASE_JSON_URL = f"https://raw.githubusercontent.com/{AGENT_REPO}/main/release.json"
9+
COMMITS_API_URL = f"https://api.github.com/repos/{AGENT_REPO}/commits"
10+
GITHUB_TOKEN = os.environ["GITHUB_TOKEN"]
11+
12+
13+
def fetch_json(url: str, headers: dict[str, str] | None = None) -> Any:
14+
req = urllib.request.Request(url, headers=headers or {})
15+
with urllib.request.urlopen(req) as resp:
16+
return json.loads(resp.read().decode())
17+
18+
19+
def get_integrations_core_version(sha: str) -> str:
20+
raw_url = f"https://raw.githubusercontent.com/{AGENT_REPO}/{sha}/release.json"
21+
req = urllib.request.Request(raw_url, headers={"Authorization": f"token {GITHUB_TOKEN}"})
22+
with urllib.request.urlopen(req) as resp:
23+
release = json.loads(resp.read().decode())
24+
return release["dependencies"]["INTEGRATIONS_CORE_VERSION"]
25+
26+
27+
# Step 1: get current pin
28+
req = urllib.request.Request(RELEASE_JSON_URL, headers={"Authorization": f"token {GITHUB_TOKEN}"})
29+
with urllib.request.urlopen(req) as resp:
30+
release_data = json.loads(resp.read().decode())
31+
current_pin = release_data["dependencies"]["INTEGRATIONS_CORE_VERSION"]
32+
print(f"Current pin: {current_pin}")
33+
34+
# Steps 2 & 3: fetch commits to release.json from the last month, walk newest→oldest.
35+
# release.json is updated for many reasons beyond the integrations-core pin, so we bound by
36+
# time window rather than a fixed page count. If the pin has been unchanged for the full month,
37+
# last_pin_commit will hold the oldest commit in the window (a conservative undercount, but
38+
# still well above the 4-day alert threshold).
39+
since = (datetime.now(timezone.utc) - timedelta(days=30)).strftime("%Y-%m-%dT%H:%M:%SZ")
40+
last_pin_commit: dict | None = None
41+
pin_changed = False
42+
page = 1
43+
44+
while not pin_changed:
45+
page_commits = fetch_json(
46+
f"{COMMITS_API_URL}?path=release.json&per_page=100&since={since}&page={page}",
47+
headers={"Authorization": f"token {GITHUB_TOKEN}", "Accept": "application/vnd.github+json"},
48+
)
49+
if not page_commits:
50+
break
51+
print(f"Page {page}: fetched {len(page_commits)} commits touching release.json")
52+
if last_pin_commit is None:
53+
last_pin_commit = page_commits[0] # most recent commit, used as fallback
54+
for commit in page_commits:
55+
sha = commit["sha"]
56+
try:
57+
pin_at_sha = get_integrations_core_version(sha)
58+
except Exception as e:
59+
print(f"Error: could not fetch release.json at {sha}: {e}")
60+
raise # fail the job; don't submit a potentially wrong metric
61+
if pin_at_sha == current_pin:
62+
last_pin_commit = commit
63+
else:
64+
pin_changed = True
65+
break # last_pin_commit is the oldest commit still on the current pin
66+
if not pin_changed:
67+
page += 1
68+
69+
# Step 4: compute days
70+
now_utc = datetime.now(timezone.utc)
71+
if last_pin_commit is None:
72+
# No commits to release.json in the last 30 days — pin is at least 30 days old
73+
days = 30
74+
print("No commits to release.json found in the last 30 days; reporting days=30")
75+
else:
76+
committed_at_str = last_pin_commit["commit"]["committer"]["date"]
77+
committed_at = datetime.fromisoformat(committed_at_str.replace("Z", "+00:00"))
78+
days = (now_utc - committed_at).days
79+
print(f"Last pin commit: {last_pin_commit['sha']} at {committed_at_str}")
80+
print(f"Days since last pin: {days}")
81+
82+
# Write days to GITHUB_OUTPUT for the submit step
83+
github_output = os.environ.get("GITHUB_OUTPUT")
84+
if github_output:
85+
with open(github_output, "a") as f:
86+
f.write(f"days={days}\n")

0 commit comments

Comments
 (0)