Skip to content

Commit b1df040

Browse files
Copilotfunctionstackxcquil11
authored
Move pip install to dedicated GitHub Actions steps (#206)
* Refactor pip install to use GitHub Actions steps with Python caching Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com> * Remove cache: 'pip', fix dependencies, and restore -q flag Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com> * add pyyaml install to label validation workflow --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com> Co-authored-by: Cameron Quilici <cjquilici@gmail.com>
1 parent 706b279 commit b1df040

7 files changed

Lines changed: 129 additions & 30 deletions

‎.github/workflows/collect-results.yml‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,14 @@ jobs:
1919
token: ${{ secrets.REPO_PAT }}
2020
fetch-depth: 0
2121

22+
- name: Set up Python
23+
uses: actions/setup-python@v6
24+
with:
25+
python-version: '3.12'
26+
27+
- name: Install dependencies
28+
run: pip install -q matplotlib
29+
2230
- name: Download JSON artifacts
2331
uses: actions/download-artifact@v6
2432
with:
@@ -38,9 +46,7 @@ jobs:
3846
path: agg_${{ inputs.exp-name || 'all' }}.json
3947

4048
- name: Plot performance
41-
run: |
42-
pip install -q matplotlib
43-
python3 utils/plot_perf.py results/ ${{ inputs.exp-name || 'all' }}
49+
run: python3 utils/plot_perf.py results/ ${{ inputs.exp-name || 'all' }}
4450
- name: Upload performance graphs
4551
uses: actions/upload-artifact@v5
4652
with:

‎.github/workflows/e2e-tests.yml‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,16 @@ jobs:
1818
- name: Checkout code
1919
uses: actions/checkout@v5
2020

21+
- name: Set up Python
22+
uses: actions/setup-python@v6
23+
with:
24+
python-version: '3.12'
25+
26+
- name: Install dependencies
27+
run: pip install pydantic pyyaml
28+
2129
- id: get-jobs
2230
run: |
23-
pip install pydantic
2431
CONFIG_JSON=$(python3 ${GITHUB_WORKSPACE}/utils/matrix-logic/generate_sweep_configs.py ${{ inputs.generate-cli-command }})
2532
echo "search-space-config=$CONFIG_JSON" >> $GITHUB_OUTPUT
2633
@@ -70,15 +77,20 @@ jobs:
7077
token: ${{ secrets.REPO_PAT }}
7178
fetch-depth: 0
7279

80+
- name: Set up Python
81+
uses: actions/setup-python@v6
82+
with:
83+
python-version: '3.12'
84+
85+
- name: Install dependencies
86+
run: pip install PyGithub
87+
7388
- name: Download results artifacts
7489
uses: actions/download-artifact@v6
7590
with:
7691
path: ${{ env.RESULTS_DIR }}
7792
pattern: results_*
7893

79-
- name: Install python dependencies
80-
run: pip install PyGithub
81-
8294
- name: Calculate success rate
8395
run: python3 utils/calc_success_rate.py $STATS_FILENAME
8496

‎.github/workflows/full-sweep-1k1k-scheduler.yml‎

Lines changed: 24 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,16 @@ jobs:
1414
- name: Checkout code
1515
uses: actions/checkout@v5
1616

17+
- name: Set up Python
18+
uses: actions/setup-python@v6
19+
with:
20+
python-version: '3.12'
21+
22+
- name: Install dependencies
23+
run: pip install pydantic pyyaml
24+
1725
- id: get-dsr1-configs
1826
run: |
19-
pip install pydantic
2027
CONFIG_JSON=$(python3 ${GITHUB_WORKSPACE}/utils/matrix-logic/generate_sweep_configs.py full-sweep --config-files ${GITHUB_WORKSPACE}/.github/configs/nvidia-master.yaml ${GITHUB_WORKSPACE}/.github/configs/amd-master.yaml --seq-lens 1k1k --model-prefix dsr1)
2128
echo "search-space-config=$CONFIG_JSON" >> $GITHUB_OUTPUT
2229
@@ -28,9 +35,16 @@ jobs:
2835
- name: Checkout code
2936
uses: actions/checkout@v5
3037

38+
- name: Set up Python
39+
uses: actions/setup-python@v6
40+
with:
41+
python-version: '3.12'
42+
43+
- name: Install dependencies
44+
run: pip install pydantic pyyaml
45+
3146
- id: get-gptoss-configs
3247
run: |
33-
pip install pydantic
3448
CONFIG_JSON=$(python3 ${GITHUB_WORKSPACE}/utils/matrix-logic/generate_sweep_configs.py full-sweep --config-files ${GITHUB_WORKSPACE}/.github/configs/nvidia-master.yaml ${GITHUB_WORKSPACE}/.github/configs/amd-master.yaml --seq-lens 1k1k --model-prefix gptoss)
3549
echo "search-space-config=$CONFIG_JSON" >> $GITHUB_OUTPUT
3650
@@ -159,15 +173,20 @@ jobs:
159173
token: ${{ secrets.REPO_PAT }}
160174
fetch-depth: 0
161175

176+
- name: Set up Python
177+
uses: actions/setup-python@v6
178+
with:
179+
python-version: '3.12'
180+
181+
- name: Install dependencies
182+
run: pip install PyGithub
183+
162184
- name: Download results artifacts
163185
uses: actions/download-artifact@v6
164186
with:
165187
path: ${{ env.RESULTS_DIR }}
166188
pattern: results_*
167189

168-
- name: Install python dependencies
169-
run: pip install PyGithub
170-
171190
- name: Calculate success rate
172191
run: python3 utils/calc_success_rate.py $STATS_FILENAME
173192

‎.github/workflows/full-sweep-1k8k-scheduler.yml‎

Lines changed: 24 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,16 @@ jobs:
1414
- name: Checkout code
1515
uses: actions/checkout@v5
1616

17+
- name: Set up Python
18+
uses: actions/setup-python@v6
19+
with:
20+
python-version: '3.12'
21+
22+
- name: Install dependencies
23+
run: pip install pydantic pyyaml
24+
1725
- id: get-dsr1-configs
1826
run: |
19-
pip install pydantic
2027
CONFIG_JSON=$(python3 ${GITHUB_WORKSPACE}/utils/matrix-logic/generate_sweep_configs.py full-sweep --config-files ${GITHUB_WORKSPACE}/.github/configs/nvidia-master.yaml ${GITHUB_WORKSPACE}/.github/configs/amd-master.yaml --seq-lens 1k8k --model-prefix dsr1)
2128
echo "search-space-config=$CONFIG_JSON" >> $GITHUB_OUTPUT
2229
@@ -28,9 +35,16 @@ jobs:
2835
- name: Checkout code
2936
uses: actions/checkout@v5
3037

38+
- name: Set up Python
39+
uses: actions/setup-python@v6
40+
with:
41+
python-version: '3.12'
42+
43+
- name: Install dependencies
44+
run: pip install pydantic pyyaml
45+
3146
- id: get-gptoss-configs
3247
run: |
33-
pip install pydantic
3448
CONFIG_JSON=$(python3 ${GITHUB_WORKSPACE}/utils/matrix-logic/generate_sweep_configs.py full-sweep --config-files ${GITHUB_WORKSPACE}/.github/configs/nvidia-master.yaml ${GITHUB_WORKSPACE}/.github/configs/amd-master.yaml --seq-lens 1k8k --model-prefix gptoss)
3549
echo "search-space-config=$CONFIG_JSON" >> $GITHUB_OUTPUT
3650
@@ -114,15 +128,20 @@ jobs:
114128
token: ${{ secrets.REPO_PAT }}
115129
fetch-depth: 0
116130

131+
- name: Set up Python
132+
uses: actions/setup-python@v6
133+
with:
134+
python-version: '3.12'
135+
136+
- name: Install dependencies
137+
run: pip install PyGithub
138+
117139
- name: Download results artifacts
118140
uses: actions/download-artifact@v6
119141
with:
120142
path: ${{ env.RESULTS_DIR }}
121143
pattern: results_*
122144

123-
- name: Install python dependencies
124-
run: pip install PyGithub
125-
126145
- name: Calculate success rate
127146
run: python3 utils/calc_success_rate.py $STATS_FILENAME
128147

‎.github/workflows/full-sweep-8k1k-scheduler.yml‎

Lines changed: 24 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,16 @@ jobs:
1414
- name: Checkout code
1515
uses: actions/checkout@v5
1616

17+
- name: Set up Python
18+
uses: actions/setup-python@v6
19+
with:
20+
python-version: '3.12'
21+
22+
- name: Install dependencies
23+
run: pip install pydantic pyyaml
24+
1725
- id: get-dsr1-configs
1826
run: |
19-
pip install pydantic
2027
CONFIG_JSON=$(python3 ${GITHUB_WORKSPACE}/utils/matrix-logic/generate_sweep_configs.py full-sweep --config-files ${GITHUB_WORKSPACE}/.github/configs/nvidia-master.yaml ${GITHUB_WORKSPACE}/.github/configs/amd-master.yaml --seq-lens 8k1k --model-prefix dsr1)
2128
echo "search-space-config=$CONFIG_JSON" >> $GITHUB_OUTPUT
2229
@@ -28,9 +35,16 @@ jobs:
2835
- name: Checkout code
2936
uses: actions/checkout@v5
3037

38+
- name: Set up Python
39+
uses: actions/setup-python@v6
40+
with:
41+
python-version: '3.12'
42+
43+
- name: Install dependencies
44+
run: pip install pydantic pyyaml
45+
3146
- id: get-gptoss-configs
3247
run: |
33-
pip install pydantic
3448
CONFIG_JSON=$(python3 ${GITHUB_WORKSPACE}/utils/matrix-logic/generate_sweep_configs.py full-sweep --config-files ${GITHUB_WORKSPACE}/.github/configs/nvidia-master.yaml ${GITHUB_WORKSPACE}/.github/configs/amd-master.yaml --seq-lens 8k1k --model-prefix gptoss)
3549
echo "search-space-config=$CONFIG_JSON" >> $GITHUB_OUTPUT
3650
@@ -159,15 +173,20 @@ jobs:
159173
token: ${{ secrets.REPO_PAT }}
160174
fetch-depth: 0
161175

176+
- name: Set up Python
177+
uses: actions/setup-python@v6
178+
with:
179+
python-version: '3.12'
180+
181+
- name: Install dependencies
182+
run: pip install PyGithub
183+
162184
- name: Download results artifacts
163185
uses: actions/download-artifact@v6
164186
with:
165187
path: ${{ env.RESULTS_DIR }}
166188
pattern: results_*
167189

168-
- name: Install python dependencies
169-
run: pip install PyGithub
170-
171190
- name: Calculate success rate
172191
run: python3 utils/calc_success_rate.py $STATS_FILENAME
173192

‎.github/workflows/full-sweep-test.yml‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -48,11 +48,18 @@ jobs:
4848
- name: Checkout code
4949
uses: actions/checkout@v5
5050

51+
- name: Set up Python
52+
uses: actions/setup-python@v6
53+
with:
54+
python-version: '3.12'
55+
56+
- name: Install dependencies
57+
run: pip install pydantic pyyaml
58+
5159
# This looks complicated, but it is just calling generate_sweep_configs.py conditioned on
5260
# discrete inputs (i.e., run_1k1k, run_h100, etc.) to split the test sweep into discrete jobs
5361
- id: generate-configs
5462
run: |
55-
pip install pydantic
5663
5764
set -x
5865
# Build runner type filters based on inputs
@@ -427,15 +434,20 @@ jobs:
427434
token: ${{ secrets.REPO_PAT }}
428435
fetch-depth: 0
429436

437+
- name: Set up Python
438+
uses: actions/setup-python@v6
439+
with:
440+
python-version: '3.12'
441+
442+
- name: Install dependencies
443+
run: pip install PyGithub
444+
430445
- name: Download results artifacts
431446
uses: actions/download-artifact@v6
432447
with:
433448
path: ${{ env.RESULTS_DIR }}
434449
pattern: results_*
435450

436-
- name: Install python dependencies
437-
run: pip install PyGithub
438-
439451
- name: Calculate success rate
440452
run: python3 utils/calc_success_rate.py $STATS_FILENAME
441453

‎.github/workflows/label-validation.yml‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,14 @@ jobs:
1818
- name: Checkout code
1919
uses: actions/checkout@v5
2020

21+
- name: Set up Python
22+
uses: actions/setup-python@v6
23+
with:
24+
python-version: '3.12'
25+
26+
- name: Install dependencies
27+
run: pip install pydantic pyyaml
28+
2129
- id: get-jobs
2230
shell: python
2331
run: |
@@ -49,7 +57,6 @@ jobs:
4957
# Generate configs for standard labels
5058
all_configs = []
5159
if matching:
52-
subprocess.run(['pip', 'install', 'pydantic'], check=True)
5360
5461
for label in matching:
5562
result = subprocess.run([
@@ -125,15 +132,20 @@ jobs:
125132
token: ${{ secrets.REPO_PAT }}
126133
fetch-depth: 0
127134

135+
- name: Set up Python
136+
uses: actions/setup-python@v6
137+
with:
138+
python-version: '3.12'
139+
140+
- name: Install dependencies
141+
run: pip install PyGithub
142+
128143
- name: Download results artifacts
129144
uses: actions/download-artifact@v6
130145
with:
131146
path: ${{ env.RESULTS_DIR }}
132147
pattern: results_*
133148

134-
- name: Install python dependencies
135-
run: pip install PyGithub
136-
137149
- name: Calculate success rate
138150
run: python3 utils/calc_success_rate.py $STATS_FILENAME
139151

0 commit comments

Comments
 (0)