Skip to content

Commit 1d2054c

Browse files
jbachorikclaude
jbachorik
and
claude
committed
Add upstream contribution candidate analysis scripts
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
1 parent 715b5db commit 1d2054c

3 files changed

Lines changed: 563 additions & 0 deletions

File tree

‎.claude/commands/contribute-upstream.md‎

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
---
2+
description: Analyze divergences from upstream async-profiler, propose grouped PRs, and open draft PRs.
3+
---
4+
5+
# Contribute Upstream Workflow
6+
7+
You are orchestrating the contribution of java-profiler divergences back to the upstream async-profiler project. These are all changes in our repo relative to upstream — not just uncommitted local modifications. Follow these steps precisely.
8+
9+
## Configuration
10+
11+
- **Fork repo**: `git@github.com:DataDog/async-profiler.git`
12+
- **Upstream repo**: `async-profiler/async-profiler` (for PR target)
13+
- **Upstream branch**: `master`
14+
- **Analysis script**: `utils/check_contribution_candidates.sh`
15+
- **Report dir**: `build/contribution-reports/`
16+
17+
## Step 1: Run Analysis
18+
19+
Execute the analysis script to generate reports:
20+
21+
```bash
22+
./utils/check_contribution_candidates.sh
23+
```
24+
25+
If it fails, diagnose and report the error to the user. Do not proceed.
26+
27+
## Step 2: Parse Results
28+
29+
Find the most recent JSON report in `build/contribution-reports/` (highest timestamp). Read it to get the list of files with contributable hunks.
30+
31+
Also read the corresponding markdown report to understand the actual diff hunks for each file.
32+
33+
If there are zero candidates, tell the user and stop.
34+
35+
## Step 3: Group into PR Proposals
36+
37+
Analyze the contributable hunks across all candidate files and group them into logical PR proposals. Guidelines:
38+
39+
- **Related changes go together**: e.g., a bug fix touching `stackWalker.cpp` and `vmStructs.cpp` for the same issue = one PR
40+
- **Independent changes are separate**: unrelated fixes in different files = separate PRs
41+
- **Each PR should be self-contained**: it should make sense on its own, compile on its own, and have a clear rationale
42+
- **Keep PRs small**: prefer multiple small PRs over one large one
43+
44+
For each proposed PR, prepare:
45+
- A descriptive title (e.g., "Fix null pointer check in stackWalker", "Add bounds validation in VMStruct")
46+
- The list of files and hunks it covers
47+
- A brief rationale explaining why this change benefits upstream
48+
49+
## Step 4: Present Proposals to User
50+
51+
Show the user a numbered list of proposed PRs with:
52+
- Title
53+
- Files involved
54+
- Brief description of the change
55+
- Number of hunks
56+
57+
Use `AskUserQuestion` with `multiSelect: true` to let the user pick which PRs to create. Offer all proposals as options.
58+
59+
If the user selects none, stop.
60+
61+
## Step 5: Create Selected PRs
62+
63+
For each selected PR, perform the following:
64+
65+
### 5a. Clone the Fork (once)
66+
67+
Clone `git@github.com:DataDog/async-profiler.git` to a temp directory. Reuse this clone for all PRs.
68+
69+
```bash
70+
FORK_DIR=$(mktemp -d "${TMPDIR:-/tmp}/async-profiler-fork.XXXXXX")
71+
git clone git@github.com:DataDog/async-profiler.git "$FORK_DIR"
72+
cd "$FORK_DIR"
73+
git remote add upstream https://github.com/async-profiler/async-profiler.git
74+
git fetch upstream
75+
git checkout -b temp upstream/master
76+
git branch -D master 2>/dev/null || true
77+
git checkout -b master
78+
git branch -D temp
79+
```
80+
81+
### 5b. Create Feature Branch
82+
83+
For each PR, create a branch from upstream master:
84+
85+
```bash
86+
cd "$FORK_DIR"
87+
git checkout master
88+
BRANCH_NAME="contribute/<slug>-$(date +%Y%m%d)"
89+
git checkout -b "$BRANCH_NAME"
90+
```
91+
92+
Where `<slug>` is a short kebab-case description derived from the PR title.
93+
94+
### 5c. Port Changes
95+
96+
Apply only the relevant contributable hunks for this PR to the upstream files:
97+
98+
1. For each file in the PR, find the corresponding upstream file in `src/` of the fork
99+
2. Apply the contributable hunks using careful manual editing (use the Edit tool)
100+
3. **Critical**: Ensure NO Datadog-specific references leak through (DD_, ddprof, Datadog, datadog, DDPROF, context.h, counters.h, tagger, QueueItem)
101+
4. If a hunk cannot be cleanly applied because the upstream file diverged, skip it and note it for the user
102+
103+
### 5d. Verify Build
104+
105+
Attempt a basic build check:
106+
107+
```bash
108+
cd "$FORK_DIR"
109+
make
110+
```
111+
112+
If the build fails, analyze the error. If it's a simple fix (e.g., missing include), fix it. If it's complex, note it for the user and proceed anyway (the PR is draft).
113+
114+
### 5e. Commit and Push
115+
116+
```bash
117+
cd "$FORK_DIR"
118+
git add -A
119+
git commit -m "<concise description of the change>"
120+
git push origin "$BRANCH_NAME"
121+
```
122+
123+
### 5f. Open Draft PR
124+
125+
Use `gh` to create a draft PR against upstream. **Important**: Before creating the first PR, fetch the target project's PR template from the upstream repo (`gh api repos/async-profiler/async-profiler/contents/.github/PULL_REQUEST_TEMPLATE.md` and decode the base64 content). The PR body **must** follow that template exactly — use all its sections, checkboxes, and footer verbatim. Fill in each section with the relevant content for this change.
126+
127+
```bash
128+
gh pr create \
129+
--repo async-profiler/async-profiler \
130+
--base master \
131+
--head "DataDog:$BRANCH_NAME" \
132+
--draft \
133+
--title "<PR title>" \
134+
--body "<body following the upstream PR template>"
135+
```
136+
137+
## Step 6: Report
138+
139+
After all selected PRs are created, show the user:
140+
- A summary table of created PRs with their URLs
141+
- Any hunks that could not be applied
142+
- Any build issues encountered
143+
- The temp directory path in case manual follow-up is needed
144+
145+
## Error Handling
146+
147+
- If `gh` is not authenticated, tell the user to run `gh auth login` and stop
148+
- If the fork clone fails, check SSH key setup and report
149+
- If a branch already exists on the fork, append a counter suffix (e.g., `-2`)
150+
- Always clean up on fatal errors (remove temp directory)

‎utils/check_contribution_candidates.sh‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
#!/bin/bash
2+
# Check divergences from upstream async-profiler for potential contributions
3+
4+
set -e
5+
6+
show_help() {
7+
cat <<EOF
8+
Usage: $(basename "$0") [OPTIONS] [BASELINE]
9+
10+
Identify divergences from upstream async-profiler that could be contributed back.
11+
12+
ARGUMENTS:
13+
BASELINE Baseline tag/commit in upstream to compare against (default: v4.2.1)
14+
15+
OPTIONS:
16+
-h, --help Show this help message and exit
17+
18+
DESCRIPTION:
19+
This script clones the upstream async-profiler repository and, for each tracked
20+
file, diffs the upstream version (at the baseline tag) against our repo version.
21+
Hunks are classified as Datadog-specific or potential contribution candidates.
22+
23+
Tracked files are those in ddprof-lib/src/main/cpp that:
24+
- Contain async-profiler copyright headers
25+
- Have corresponding files in upstream async-profiler
26+
27+
OUTPUT:
28+
Reports are saved to: build/contribution-reports/
29+
- contribution_candidates_YYYYMMDD_HHMMSS.md (human-readable)
30+
- contribution_candidates_YYYYMMDD_HHMMSS.json (machine-readable)
31+
- tracked_files.txt (list of tracked files)
32+
33+
EXAMPLES:
34+
# Compare against upstream v4.2.1 (default baseline)
35+
$(basename "$0")
36+
37+
# Compare against a different baseline
38+
$(basename "$0") v3.0
39+
40+
EXIT STATUS:
41+
0 Success (with or without candidates detected)
42+
1 Error (clone failed, invalid arguments, etc.)
43+
44+
EOF
45+
}
46+
47+
# Parse arguments
48+
if [ "$1" = "-h" ] || [ "$1" = "--help" ]; then
49+
show_help
50+
exit 0
51+
fi
52+
53+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
54+
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
55+
56+
# Configuration
57+
UPSTREAM_REPO_URL="https://github.com/async-profiler/async-profiler.git"
58+
TMPDIR="${TMPDIR:-/tmp}"
59+
UPSTREAM_CLONE_DIR="$(mktemp -d "$TMPDIR/async-profiler-contrib.XXXXXX")"
60+
LOCAL_CPP_DIR="$PROJECT_ROOT/ddprof-lib/src/main/cpp"
61+
REPORT_DIR="$PROJECT_ROOT/build/contribution-reports"
62+
63+
BASELINE="${1:-v4.2.1}"
64+
65+
echo "=== Contribution Candidate Checker ==="
66+
echo ""
67+
echo "Configuration:"
68+
echo " Local cpp dir: $LOCAL_CPP_DIR"
69+
echo " Baseline: $BASELINE"
70+
echo " Report dir: $REPORT_DIR"
71+
echo ""
72+
73+
# Create report directory
74+
mkdir -p "$REPORT_DIR"
75+
76+
# Clone upstream repository
77+
echo "[1/4] Cloning upstream async-profiler..."
78+
git clone -q --depth 200 "$UPSTREAM_REPO_URL" "$UPSTREAM_CLONE_DIR"
79+
cd "$UPSTREAM_CLONE_DIR"
80+
git fetch -q --tags
81+
echo " Cloned to $UPSTREAM_CLONE_DIR"
82+
echo ""
83+
84+
# Generate tracked files list
85+
echo "[2/4] Discovering tracked files..."
86+
TRACKED_FILES="$REPORT_DIR/tracked_files.txt"
87+
"$SCRIPT_DIR/generate_tracked_files.sh" \
88+
"$LOCAL_CPP_DIR" \
89+
"$UPSTREAM_CLONE_DIR/src" \
90+
> "$TRACKED_FILES"
91+
92+
FILE_COUNT=$(wc -l < "$TRACKED_FILES" | tr -d ' ')
93+
echo " Tracking $FILE_COUNT files"
94+
echo ""
95+
96+
# Analyze contribution candidates
97+
echo "[3/4] Analyzing divergences from upstream..."
98+
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
99+
MARKDOWN_REPORT="$REPORT_DIR/contribution_candidates_${TIMESTAMP}.md"
100+
JSON_REPORT="$REPORT_DIR/contribution_candidates_${TIMESTAMP}.json"
101+
102+
"$SCRIPT_DIR/find_contribution_candidates.sh" \
103+
"$UPSTREAM_CLONE_DIR" \
104+
"$BASELINE" \
105+
"$LOCAL_CPP_DIR" \
106+
"$TRACKED_FILES" \
107+
"$MARKDOWN_REPORT" \
108+
"$JSON_REPORT"
109+
110+
# Check results
111+
if [ ! -f "$JSON_REPORT" ]; then
112+
echo " No contribution candidates detected."
113+
echo ""
114+
echo "All divergences are Datadog-specific."
115+
else
116+
CANDIDATES=$(jq -r '.files_with_candidates' "$JSON_REPORT")
117+
TOTAL_CHANGED=$(jq -r '.files_with_changes' "$JSON_REPORT")
118+
119+
echo ""
120+
echo " Candidates found!"
121+
echo " Files with divergences: $TOTAL_CHANGED"
122+
echo " Files with contributable hunks: $CANDIDATES"
123+
echo ""
124+
echo "Reports generated:"
125+
echo " Markdown: $MARKDOWN_REPORT"
126+
echo " JSON: $JSON_REPORT"
127+
echo ""
128+
echo "View the markdown report:"
129+
echo " cat $MARKDOWN_REPORT"
130+
fi
131+
132+
# Cleanup
133+
echo ""
134+
echo "[4/4] Cleaning up..."
135+
rm -rf "$UPSTREAM_CLONE_DIR"
136+
echo " Removed temporary clone"
137+
138+
echo ""
139+
echo "Done!"
140+
echo ""
141+
echo "Tracked files list saved to:"
142+
echo " $TRACKED_FILES"

0 commit comments

Comments
 (0)