Skip to content

Commit e196aa7

Browse files
mmckymmcky
authored
chore: add quantecon/ fork maintenance tooling (#14)
* chore: add quantecon/ fork maintenance tooling - Add quantecon/build.sh — rebuilds the `quantecon` integration branch by merging all active feature branches on top of `main`, then patches the version string with a `-qe` suffix so `myst --version` identifies the QuantEcon build. - Add quantecon/features.txt — registry of active carry-patches. - Add quantecon/README.md — documents the branching model, sync workflow, and conflict resolution strategy. - Add warning banner to README.md identifying this as the QuantEcon fork. * chore: disable release and docs workflows on QuantEcon fork * chore: update sync docs to use git merge (not --ff-only) for fork main * docs: rewrite quantecon/README with clearer workflow explanation * chore: revert README banner and workflow changes from PR Only the quantecon/ folder belongs on main. Workflow disabling is handled via the GitHub UI; README banner is unnecessary given the fork name. * fix: address copilot review comments on build.sh - Use awk for features.txt parsing (POSIX-safe, replaces non-portable \s grep) - Add git fetch origin before ff-only merge so refs are current - Create local tracking branch for origin-only feature branches before merging - Fail hard (not warn) if main cannot be fast-forwarded - Patch packages/mystmd/package.json version instead of gitignored version.ts; copy:version propagates the version at build time - Update README to reflect correct version patching mechanism * docs: branch features from upstream/main to keep PR diffs clean - Update diagram and branching model table - New 'Develop a new feature' section uses 'git checkout -b ... upstream/main' - Add migration section for existing branches that include quantecon/ commits (git rebase --onto upstream/main main) - Update keep-current and conflict-resolution sections to use upstream/main - Explain why this works (git merges changes, not full trees)
1 parent ebe742a commit e196aa7

3 files changed

Lines changed: 386 additions & 0 deletions

File tree

‎quantecon/README.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# QuantEcon fork of `mystmd` — maintenance guide
2+
3+
This fork lets QuantEcon develop and use new `mystmd` features before they have been reviewed and merged by the upstream `jupyter-book/mystmd` team. The goal is to make it as easy as possible to open upstream PRs from feature branches, while also being able to run a combined build that includes all in-progress features.
4+
5+
## How it works — the key idea
6+
7+
```
8+
jupyter-book/mystmd:main ─────┐
9+
│ (sync periodically) │ (feature branches start here)
10+
▼ │
11+
QuantEcon/mystmd:main │ ← mirror + quantecon/ tooling
12+
│ │
13+
│ feature/foo ◄─────┤ ← branched from upstream/main; PR target = upstream main
14+
│ feature/bar ◄─────┘ ← branched from upstream/main; PR target = upstream main
15+
▼ (built by build.sh, which merges feature branches into quantecon)
16+
QuantEcon/mystmd:quantecon ← combined build: main + all feature branches
17+
```
18+
19+
**Feature branches serve two purposes simultaneously:**
20+
21+
1. They are the source of upstream PRs — targeting `jupyter-book/mystmd:main`, showing only the diff for that one feature. They are branched from `upstream/main` so they don't carry the `quantecon/` folder.
22+
2. They are consumed by `build.sh`, which merges them onto the throwaway `quantecon` branch (which inherits the `quantecon/` folder from `main`).
23+
24+
This means **you never need to choose** between "make it easy to upstream" and "make it available to use now". The feature branch does both.
25+
26+
## Branching model
27+
28+
| Branch | Purpose |
29+
|---|---|
30+
| `main` | Mirrors `jupyter-book/mystmd:main` exactly. Synced via the GitHub "Sync fork" button or `git merge upstream/main`. **No direct commits.** |
31+
| `feature/<name>` | One branch per logical patch. **Branched from `upstream/main`** (not `main`), kept rebased on `upstream/main`. This is the branch you open the upstream PR from. |
32+
| `quantecon` | Throwaway combined build. Rebuilt by `build.sh` as `main` + all active feature branches. **Never commit here directly — it is always discarded and regenerated.** |
33+
34+
## One-time setup
35+
36+
```bash
37+
git remote add upstream https://github.com/jupyter-book/mystmd.git
38+
```
39+
40+
Verify your remotes look like this:
41+
42+
```
43+
origin https://github.com/QuantEcon/mystmd.git (fetch/push)
44+
upstream https://github.com/jupyter-book/mystmd.git (fetch/push)
45+
```
46+
47+
## Regular workflow
48+
49+
### Sync `main` with upstream
50+
51+
Either use the **"Sync fork"** button on the GitHub web UI (simplest), or locally:
52+
53+
```bash
54+
git fetch upstream
55+
git checkout main
56+
git merge upstream/main
57+
git push origin main
58+
```
59+
60+
After syncing, rebuild the `quantecon` branch (see below) so it includes the latest upstream changes.
61+
62+
### Develop a new feature
63+
64+
> **Important:** branch from `upstream/main`, **not** from `main`. This keeps the `quantecon/` folder out of your feature branch, so the upstream PR diff shows only your actual changes.
65+
66+
```bash
67+
git fetch upstream
68+
git checkout -b feature/<name> upstream/main
69+
# make your changes and commit them
70+
git push origin feature/<name>
71+
```
72+
73+
Then:
74+
1. Add the branch name to `quantecon/features.txt` (on `main`, then commit and push)
75+
2. Open a PR on GitHub: **base: `jupyter-book/mystmd:main`**, **compare: `QuantEcon/mystmd:feature/<name>`**
76+
3. Rebuild the `quantecon` branch so the feature is available immediately
77+
78+
> **Why this works:** `build.sh` runs from `main` (which has the `quantecon/` folder), checks out the `quantecon` branch, and merges your feature branch into it. Because git merges *changes* (not full trees), the feature branch doesn't need to contain `quantecon/` — the folder comes from `main`, and your feature's changes are layered on top.
79+
80+
### Migrating an existing feature branch to use upstream/main as base
81+
82+
If you already have a feature branch that was created from `main` (and therefore includes the `quantecon/` folder commits), rebase it onto `upstream/main`:
83+
84+
```bash
85+
git fetch upstream
86+
git checkout feature/<name>
87+
git rebase --onto upstream/main main
88+
git push --force-with-lease origin feature/<name>
89+
```
90+
91+
This replays only your feature's commits on top of `upstream/main`, dropping the `quantecon/` folder commits from the branch's history. The upstream PR diff will then show only your actual changes.
92+
93+
### Rebuild the `quantecon` branch
94+
95+
```bash
96+
./quantecon/build.sh # local only (safe, no push)
97+
./quantecon/build.sh --push # build and push to origin/quantecon
98+
```
99+
100+
The script:
101+
1. Syncs local `main` from `origin/main`
102+
2. Resets `quantecon` to `main` (discarding any previous build)
103+
3. Merges each branch listed in `features.txt` in order (merge commits, not squash)
104+
4. Patches `packages/mystmd/package.json` version to append `-qe` (e.g. `1.9.0-qe`). The `copy:version` build step propagates this to `version.ts` at build time, so `myst --version` identifies the QuantEcon build.
105+
5. Optionally pushes to `origin/quantecon`
106+
107+
Run this after every upstream sync, after updating any feature branch, or after adding/removing a feature branch from `features.txt`.
108+
109+
### Update a feature branch (e.g. to address upstream reviewer feedback)
110+
111+
```bash
112+
git checkout feature/<name>
113+
# make changes, amend or add commits
114+
git push --force-with-lease origin feature/<name>
115+
# the upstream PR updates automatically
116+
./quantecon/build.sh --push # rebuild quantecon with the updated branch
117+
```
118+
119+
### Keep a feature branch current with upstream
120+
121+
If `upstream/main` has moved since you branched:
122+
123+
```bash
124+
git fetch upstream
125+
git checkout feature/<name>
126+
git rebase upstream/main
127+
git push --force-with-lease origin feature/<name>
128+
./quantecon/build.sh --push
129+
```
130+
131+
## Resolving merge conflicts
132+
133+
Conflicts are **always fixed on the feature branch**, never on `quantecon` (which is throwaway and rebuilt from scratch each time).
134+
135+
`build.sh` will abort cleanly and tell you which branch caused the conflict.
136+
137+
**Feature conflicts with upstream** (upstream changed code the feature touches):
138+
```bash
139+
git fetch upstream
140+
git checkout feature/<name>
141+
git rebase upstream/main # resolve conflicts here, then: git add . && git rebase --continue
142+
git push --force-with-lease origin feature/<name>
143+
./quantecon/build.sh --push
144+
```
145+
146+
**Feature A conflicts with feature B** (two patches touch the same lines):
147+
```bash
148+
# Rebase the later branch on top of the earlier one to establish ordering
149+
git checkout feature/<name-b>
150+
git rebase feature/<name-a>
151+
git push --force-with-lease origin feature/<name-b>
152+
# Make sure features.txt lists feature/<name-a> before feature/<name-b>
153+
./quantecon/build.sh --push
154+
```
155+
156+
## When upstream merges a feature
157+
158+
Once the upstream PR is merged into `jupyter-book/mystmd:main`:
159+
160+
1. Sync `main` with upstream — it now contains the feature
161+
2. Remove the branch from `features.txt`
162+
3. Delete the feature branch
163+
4. Rebuild `quantecon` — the feature is now in `main` itself, so nothing is lost
164+
165+
```bash
166+
git fetch upstream
167+
git checkout main && git merge upstream/main && git push origin main
168+
# edit features.txt to remove the branch
169+
git branch -d feature/<name> && git push origin --delete feature/<name>
170+
./quantecon/build.sh --push
171+
```
172+
173+
## Active patches
174+
175+
See [features.txt](features.txt) for the current list of carry-patches and links to their upstream PRs.

‎quantecon/build.sh‎

Lines changed: 199 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
#!/usr/bin/env bash
2+
# quantecon/build.sh
3+
#
4+
# Rebuilds the `quantecon` integration branch by merging all active feature
5+
# branches (listed in quantecon/features.txt) on top of current `main`.
6+
#
7+
# Usage:
8+
# ./quantecon/build.sh [--push]
9+
#
10+
# Options:
11+
# --push Push the resulting branch to origin after a successful build.
12+
#
13+
# Conflict resolution:
14+
# This script aborts cleanly on any merge conflict. Conflicts must be
15+
# resolved on the feature branch itself (never on `quantecon`):
16+
#
17+
# Feature vs. main:
18+
# git checkout <branch> && git rebase main
19+
# # resolve conflicts, git add, git rebase --continue
20+
# git push --force-with-lease origin <branch>
21+
# ./quantecon/build.sh [--push]
22+
#
23+
# Feature A vs. feature B:
24+
# git checkout <branch-B> && git rebase <branch-A>
25+
# # resolve conflicts, update features.txt so branch-A appears before branch-B
26+
# git push --force-with-lease origin <branch-B>
27+
# ./quantecon/build.sh [--push]
28+
29+
set -euo pipefail
30+
31+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
32+
FEATURES_FILE="$SCRIPT_DIR/features.txt"
33+
INTEGRATION_BRANCH="quantecon"
34+
BASE_BRANCH="main"
35+
PUSH=false
36+
37+
# ---------------------------------------------------------------------------
38+
# Parse args
39+
# ---------------------------------------------------------------------------
40+
for arg in "$@"; do
41+
case "$arg" in
42+
--push) PUSH=true ;;
43+
*) echo "Unknown argument: $arg" >&2; exit 1 ;;
44+
esac
45+
done
46+
47+
# ---------------------------------------------------------------------------
48+
# Preflight checks
49+
# ---------------------------------------------------------------------------
50+
if ! git rev-parse --git-dir > /dev/null 2>&1; then
51+
echo "ERROR: not inside a git repository." >&2
52+
exit 1
53+
fi
54+
55+
if [[ -n "$(git status --porcelain)" ]]; then
56+
echo "ERROR: working tree is dirty. Commit or stash changes before running." >&2
57+
exit 1
58+
fi
59+
60+
# Read feature branches (strip comments and blank lines, extract first token)
61+
mapfile -t FEATURES < <(awk '/^[[:space:]]*#/{next} /^[[:space:]]*$/{next} {print $1}' "$FEATURES_FILE")
62+
63+
if [[ ${#FEATURES[@]} -eq 0 ]]; then
64+
echo "No feature branches listed in $FEATURES_FILE. Nothing to do."
65+
exit 0
66+
fi
67+
68+
echo "==> Feature branches to merge:"
69+
for f in "${FEATURES[@]}"; do
70+
echo " $f"
71+
done
72+
73+
# ---------------------------------------------------------------------------
74+
# Ensure local main is up to date
75+
# ---------------------------------------------------------------------------
76+
echo ""
77+
echo "==> Fetching origin..."
78+
git fetch origin
79+
80+
echo ""
81+
echo "==> Switching to $BASE_BRANCH and fast-forwarding from origin..."
82+
git checkout "$BASE_BRANCH"
83+
# main should be a pure upstream mirror — ff-only guards against accidental commits
84+
if ! git merge --ff-only origin/"$BASE_BRANCH"; then
85+
echo "ERROR: could not fast-forward $BASE_BRANCH from origin/$BASE_BRANCH." \
86+
"Sync main with upstream first (git merge upstream/main && git push origin main)." >&2
87+
exit 1
88+
fi
89+
90+
BASE_SHA=$(git rev-parse "$BASE_BRANCH")
91+
echo " Base commit: $BASE_SHA"
92+
93+
# ---------------------------------------------------------------------------
94+
# Verify all feature branches exist (locally or on origin)
95+
# ---------------------------------------------------------------------------
96+
echo ""
97+
echo "==> Verifying feature branches exist..."
98+
for branch in "${FEATURES[@]}"; do
99+
if git rev-parse --verify "$branch" > /dev/null 2>&1; then
100+
echo " $branch (local)"
101+
elif git rev-parse --verify "origin/$branch" > /dev/null 2>&1; then
102+
# Create a local tracking branch so 'git merge <branch>' works
103+
git branch --track "$branch" "origin/$branch" 2>/dev/null || git branch -f "$branch" "origin/$branch"
104+
echo " $branch (fetched from origin)"
105+
else
106+
echo "ERROR: branch '$branch' not found locally or on origin." >&2
107+
exit 1
108+
fi
109+
done
110+
111+
# ---------------------------------------------------------------------------
112+
# Rebuild the integration branch
113+
# ---------------------------------------------------------------------------
114+
echo ""
115+
echo "==> Resetting '$INTEGRATION_BRANCH' to '$BASE_BRANCH'..."
116+
git checkout -B "$INTEGRATION_BRANCH" "$BASE_BRANCH"
117+
118+
MERGED=()
119+
for branch in "${FEATURES[@]}"; do
120+
echo ""
121+
echo "==> Merging $branch..."
122+
if git merge --no-ff "$branch" -m "chore: merge $branch into $INTEGRATION_BRANCH"; then
123+
MERGED+=("$branch")
124+
else
125+
echo "" >&2
126+
echo "ERROR: merge conflict when merging '$branch'." >&2
127+
echo "" >&2
128+
echo "Conflicting files:" >&2
129+
git diff --name-only --diff-filter=U >&2
130+
echo "" >&2
131+
echo "Aborting. The '$INTEGRATION_BRANCH' branch has been reset; no changes were pushed." >&2
132+
git merge --abort
133+
git checkout "$BASE_BRANCH"
134+
git branch -D "$INTEGRATION_BRANCH"
135+
echo "" >&2
136+
echo "How to fix:" >&2
137+
if [[ ${#MERGED[@]} -gt 0 ]]; then
138+
echo " '$branch' conflicts with one of: ${MERGED[*]}" >&2
139+
echo " Rebase '$branch' on top of the conflicting branch:" >&2
140+
echo " git checkout $branch && git rebase <earlier-branch>" >&2
141+
echo " Then update features.txt so the earlier branch appears first." >&2
142+
else
143+
echo " '$branch' conflicts with $BASE_BRANCH." >&2
144+
echo " Rebase it: git checkout $branch && git rebase $BASE_BRANCH" >&2
145+
fi
146+
echo " Resolve conflicts, push the feature branch, then re-run this script." >&2
147+
exit 1
148+
fi
149+
done
150+
151+
# ---------------------------------------------------------------------------
152+
# Patch version string with -qe suffix
153+
# ---------------------------------------------------------------------------
154+
# version.ts is generated/gitignored — patch packages/mystmd/package.json instead.
155+
# The copy:version build step will propagate the version to version.ts at build time.
156+
PACKAGE_JSON="packages/mystmd/package.json"
157+
echo ""
158+
echo "==> Patching version in $PACKAGE_JSON with '-qe' suffix..."
159+
160+
CURRENT_VERSION=$(node -p "require('./$PACKAGE_JSON').version")
161+
QE_VERSION="${CURRENT_VERSION}-qe"
162+
163+
# Use node to patch the JSON safely (avoids sed quoting issues with JSON)
164+
node -e "
165+
const fs = require('fs');
166+
const pkg = JSON.parse(fs.readFileSync('$PACKAGE_JSON', 'utf8'));
167+
pkg.version = '$QE_VERSION';
168+
fs.writeFileSync('$PACKAGE_JSON', JSON.stringify(pkg, null, 2) + '\n');
169+
"
170+
171+
git add "$PACKAGE_JSON"
172+
git commit -m "chore: set version to ${QE_VERSION} for QuantEcon build"
173+
echo " Version set to ${QE_VERSION}"
174+
175+
# ---------------------------------------------------------------------------
176+
# Summary
177+
# ---------------------------------------------------------------------------
178+
echo ""
179+
echo "==> Build successful."
180+
echo " Branch '$INTEGRATION_BRANCH' contains $BASE_BRANCH + ${#MERGED[@]} feature branch(es):"
181+
for f in "${MERGED[@]}"; do
182+
echo " $f"
183+
done
184+
echo " Version: ${QE_VERSION}"
185+
186+
# ---------------------------------------------------------------------------
187+
# Optionally push
188+
# ---------------------------------------------------------------------------
189+
if $PUSH; then
190+
echo ""
191+
echo "==> Pushing '$INTEGRATION_BRANCH' to origin (force-with-lease)..."
192+
git push --force-with-lease origin "$INTEGRATION_BRANCH"
193+
echo " Done."
194+
else
195+
echo ""
196+
echo "Tip: run with --push to push to origin/$INTEGRATION_BRANCH."
197+
fi
198+
199+
git checkout "$BASE_BRANCH"

‎quantecon/features.txt‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Active QuantEcon feature branches
2+
# One branch name per line. Lines starting with # are ignored.
3+
# Order matters: branches are merged in sequence.
4+
# If two branches conflict with each other, rebase the later one on top
5+
# of the earlier one and update both entries accordingly.
6+
#
7+
# Format:
8+
# <branch-name> [# optional description / upstream PR link]
9+
#
10+
# Example:
11+
# feature/xref-improvements # https://github.com/jupyter-book/mystmd/pull/1
12+

0 commit comments

Comments
 (0)