Skip to content

Commit 3084a36

Browse files
Copilotpancetta
Copilot
and
pancetta
committed
Add dependency management timeline visualization
- Document weekly workflow with clear timeline - Show how automated system prevents CI failures - Illustrate different update scenarios - Explain two-layer protection system Co-authored-by: pancetta <7158893+pancetta@users.noreply.github.com>
1 parent ade7cca commit 3084a36

1 file changed

Lines changed: 159 additions & 0 deletions

File tree

‎DEPENDENCY_TIMELINE.md‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
# Weekly Dependency Management Timeline
2+
3+
This document illustrates how the automated dependency management system works throughout the week.
4+
5+
## Weekly Timeline
6+
7+
```
8+
Sunday 2:00 AM UTC
9+
│
10+
├─ Lock File Update Workflow Runs
11+
│ └─ Regenerates all lock files with latest compatible versions
12+
│ └─ Creates PR if changes detected
13+
│
14+
└─ Sunday (rest of day)
15+
└─ Review lock file update PR (if created)
16+
└─ CI tests the new versions
17+
└─ Merge if tests pass
18+
19+
Monday 5:01 AM UTC
20+
│
21+
├─ Weekly CI Run
22+
│ └─ Tests with current dependencies
23+
│ └─ If fails: Automated failure PR created
24+
│ └─ If lock file PR was merged: Tests with updated versions
25+
│
26+
└─ Monday (rest of day)
27+
└─ Review any failure PRs
28+
└─ Apply fixes if needed
29+
30+
Tuesday - Saturday
31+
│
32+
└─ Normal Development
33+
└─ PRs tested with current dependencies
34+
└─ Lock files stable unless manually updated
35+
```
36+
37+
## Dependency Update Scenarios
38+
39+
### Scenario 1: Compatible Update (Happy Path)
40+
41+
```
42+
Sunday:
43+
1. NumPy 1.26.3 → 1.26.4 (patch update)
44+
2. Lock file workflow creates PR
45+
3. CI tests pass ✅
46+
4. PR merged automatically or by maintainer
47+
48+
Monday:
49+
5. Weekly CI runs with NumPy 1.26.4
50+
6. Tests pass ✅
51+
7. No action needed
52+
```
53+
54+
### Scenario 2: Breaking Update (Caught Early)
55+
56+
```
57+
Sunday:
58+
1. SciPy 1.11.4 → 1.14.0 (minor update with breaking change)
59+
2. Lock file workflow creates PR
60+
3. CI tests FAIL ❌
61+
4. PR not merged - issue identified early
62+
63+
Monday:
64+
5. Weekly CI runs with SciPy 1.11.4 (old version)
65+
6. Tests pass ✅
66+
7. Lock file PR remains open for investigation
67+
68+
Later:
69+
8. Maintainer reviews lock file PR
70+
9. Identifies SciPy 1.14.0 breaking change
71+
10. Updates upper bound: scipy>=0.17.1,<1.14
72+
11. Regenerates lock files
73+
12. Opens issue to track code updates for SciPy 1.14
74+
```
75+
76+
### Scenario 3: Major Version Update (Blocked)
77+
78+
```
79+
Sunday:
80+
1. NumPy 3.0.0 released (major version)
81+
2. Lock file workflow runs
82+
3. NumPy 3.0.0 blocked by upper bound (<3.0) ✅
83+
4. No lock file changes
84+
5. No PR created
85+
86+
Monday:
87+
6. Weekly CI runs normally
88+
7. Tests pass ✅
89+
8. System stable
90+
91+
Later:
92+
9. Maintainer decides to support NumPy 3.0
93+
10. Updates code for NumPy 3.0 compatibility
94+
11. Changes upper bound to <4.0
95+
12. Lock files update to NumPy 3.0.x on next Sunday
96+
```
97+
98+
## Two-Layer Protection
99+
100+
### Layer 1: Upper Bounds (Source Files)
101+
```yaml
102+
# pyproject.toml & environment.yml
103+
dependencies:
104+
- numpy>=1.15.4,<3.0 # Blocks major version 3.x
105+
- scipy>=0.17.1,<2.0 # Blocks major version 2.x
106+
```
107+
108+
**Purpose**: Prevent unexpected major version updates
109+
**Scope**: All installations (dev, CI, production)
110+
111+
### Layer 2: Lock Files (Generated Weekly)
112+
```yaml
113+
# etc/lockfiles/environment-base-lock.yml
114+
dependencies:
115+
- numpy==1.26.4 # Exact version
116+
- scipy==1.11.4 # Exact version
117+
```
118+
119+
**Purpose**: Exact reproducibility when needed
120+
**Scope**: Optional (can be used in CI for 100% reproducibility)
121+
122+
## Benefits Summary
123+
124+
### For Weekly CI Runs
125+
✅ Lock files update Sunday (before Monday run)
126+
✅ Breaking changes caught in lock file PR (not Monday CI)
127+
✅ Monday CI tests known-good versions
128+
✅ No unexpected failures from random dependency updates
129+
130+
### For Developers
131+
✅ Source files remain simple and readable
132+
✅ Can use lock files for reproducibility
133+
✅ Can use source files for flexibility
134+
✅ No manual constraint updates needed
135+
136+
### For Maintainers
137+
✅ Automated PRs show exactly what's changing
138+
✅ Clear decision point (merge lock file PR or not)
139+
✅ Early warning system for breaking changes
140+
✅ Minimal manual intervention required
141+
142+
## Key Configuration Files
143+
144+
- **Source Files**: `pyproject.toml`, `etc/environment-*.yml`, `pySDC/projects/*/environment.yml`
145+
- Loose constraints with upper bounds
146+
- Human-readable and maintainable
147+
148+
- **Lock Files**: `etc/lockfiles/*-lock.yml`, `pySDC/projects/*/lockfiles/environment-lock.yml`
149+
- Exact versions
150+
- Auto-generated weekly
151+
- Committed to repo
152+
153+
- **Workflows**:
154+
- `.github/workflows/update_lockfiles.yml` - Sunday 2 AM UTC
155+
- `.github/workflows/ci_pipeline.yml` - Monday 5:01 AM UTC
156+
157+
- **Scripts**:
158+
- `etc/scripts/update_lockfile.sh` - Update single lock file
159+
- `etc/scripts/update_all_lockfiles.sh` - Update all lock files

0 commit comments

Comments
 (0)