Skip to content

Commit 4ef822e

Browse files
committed
Merge branch 'main' into spatial_start
2 parents 6a2ba44 + 5912bb7 commit 4ef822e

66 files changed

Lines changed: 2856 additions & 2373 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/ISSUE_TEMPLATE/release.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
---
2+
name: Release
3+
about: Checklist for releasing a new version of ggsql
4+
title: "Release ggsql X.Y.Z"
5+
labels: release
6+
assignees: ''
7+
---
8+
9+
- [ ] Create new branch `release-version-X-Y-Z`.
10+
- [ ] Check examples on ggsql.org for visual regressions
11+
- [ ] Bump version numbers, all to match:
12+
- ggsql
13+
- Main package `version` in `[workspace.package]` section in `Cargo.toml`.
14+
- All packages `version` in `[workspace.dependencies]` section in `Cargo.toml`.
15+
- tree-sitter-ggsql
16+
- `version` in `tree-sitter-ggsql/package.json`.
17+
- `version` in `metadata` in `tree-sitter-ggsql/tree-sitter.json`.
18+
- `version` in `[project]` section in `tree-sitter-ggsql/pyproject.toml`.
19+
- `__version__` in `tree-sitter-ggsql/bindings/python/__init__.py`.
20+
- ggsql-jupyter
21+
- `version` in `[project]` section in `ggsql-jupyter/pyproject.toml`.
22+
- ggsql-vscode
23+
- `version` in `ggsql-vscode/package.json`.
24+
- [ ] Update lock files:
25+
- Run `cargo build`, ensure `Cargo.lock` updates.
26+
- Run `(cd ggsql-wasm && ./build-wasm.sh && cd demo && npm install)`, ensure that `ggsql-wasm/demo/package-lock.json` updates.
27+
- Run `(cd ggsql-vscode && npm install)`, ensure that `ggsql-vscode/package-lock.json` updates.
28+
- [ ] Ensure news bullets are up to date, add a new header for this release.
29+
- `CHANGELOG.md`
30+
- `ggsql-vscode/CHANGELOG.md`
31+
- [ ] Push the branch, make a PR.
32+
- [ ] Watch CI. Once all green, and we are happy, squash-merge the PR.
33+
- [ ] Create a new GitHub release, create a new tag in the form `vX.Y.Z`
34+
- Select the commit we just squash merged in for the new tag.
35+
- Click the button to generate a changelog automatically so our friends all get tagged in the release.
36+
- Check the box to set as the latest release.
37+
- After releasing, watch CI for issues. If anything is red, we must fix, bump patch version, and try again.
38+
- For minor issues, we might want to manually fix up and publish. Currently George's npm and cargo accounts must be used to do this.
39+
- After all green, open Issues in downstream repos asking to update their dependency on ggsql to the latest version,
40+
- [ ] [ggsql-r](https://github.com/posit-dev/ggsql-r)
41+
- [ ] [ggsql-python](https://github.com/posit-dev/ggsql-python)
42+
- [ ] [ggsql-duckdb](https://github.com/posit-dev/ggsql-duckdb)
43+
- [ ] Update the ggsql skill at [posit-dev/skills](https://github.com/posit-dev/skills)

.github/workflows/actions/fix-macos-wheel-dylibs/action.yml

Lines changed: 0 additions & 115 deletions
This file was deleted.
Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
name: "Fix wheel library load paths"
2+
description: >
3+
Relocate bundled shared libraries from the py3-none wheel's top-level
4+
package directory into the wheel's .data/data/lib/ dir, so they install to
5+
<prefix>/lib/ (Python-version-independent) rather than
6+
<prefix>/lib/pythonX.Y/site-packages/. Also rewrites the binary's load
7+
commands (macOS) or RPATH (Linux) to match.
8+
inputs:
9+
dist-dir:
10+
description: "Directory containing the wheel(s) to rewrite in-place."
11+
required: true
12+
binary-name:
13+
description: "Name of the binary inside the wheel's data/scripts dir."
14+
required: false
15+
default: "ggsql-jupyter"
16+
17+
runs:
18+
using: "composite"
19+
steps:
20+
- name: Fix wheel libraries (macOS)
21+
if: runner.os == 'macOS'
22+
shell: bash
23+
env:
24+
DIST_DIR: ${{ inputs.dist-dir }}
25+
BINARY_NAME: ${{ inputs.binary-name }}
26+
run: |
27+
set -euo pipefail
28+
python3 -m pip install --quiet wheel
29+
cd "$DIST_DIR"
30+
31+
shopt -s nullglob
32+
wheels=(*.whl)
33+
if [ ${#wheels[@]} -eq 0 ]; then
34+
echo "::error title=No wheels found::$DIST_DIR contains no .whl files"
35+
exit 1
36+
fi
37+
38+
LIBS_SRC="ggsql_jupyter.dylibs"
39+
DEST_RELATIVE="lib/ggsql-jupyter-dylibs"
40+
OLD_PREFIX="@loader_path/../$LIBS_SRC/"
41+
NEW_PREFIX="@loader_path/../$DEST_RELATIVE/"
42+
43+
for whl in "${wheels[@]}"; do
44+
echo "::group::$whl"
45+
46+
workdir=$(mktemp -d)
47+
python3 -m wheel unpack "$whl" -d "$workdir"
48+
unpacked=$(find "$workdir" -maxdepth 1 -mindepth 1 -type d | head -n 1)
49+
50+
src_dir="$unpacked/$LIBS_SRC"
51+
data_root=$(find "$unpacked" -maxdepth 1 -type d -name '*.data' | head -n 1)
52+
if [ -z "$data_root" ]; then
53+
echo "::error title=Wheel layout::no *.data/ directory in $whl"
54+
exit 1
55+
fi
56+
dest_dir="$data_root/data/$DEST_RELATIVE"
57+
binary=$(find "$data_root/scripts" -type f -name "$BINARY_NAME" | head -n 1)
58+
59+
if [ -z "$binary" ]; then
60+
echo "::error title=Binary not found::$BINARY_NAME not found in $whl"
61+
exit 1
62+
fi
63+
64+
if [ ! -d "$src_dir" ]; then
65+
echo "::warning title=No library bundle::$LIBS_SRC/ not present in $whl (nothing to relocate)"
66+
else
67+
echo "Moving $LIBS_SRC/ -> $(basename "$data_root")/data/$DEST_RELATIVE/"
68+
mkdir -p "$(dirname "$dest_dir")"
69+
mv "$src_dir" "$dest_dir"
70+
fi
71+
72+
echo "Rewriting load commands on $(basename "$binary")"
73+
oldpaths=$(otool -L "$binary" | awk -v p="$OLD_PREFIX" 'index($1, p) == 1 {print $1}')
74+
if [ -z "$oldpaths" ]; then
75+
echo "::warning title=Nothing to rewrite::No $OLD_PREFIX load commands in $binary"
76+
else
77+
for old in $oldpaths; do
78+
new="$NEW_PREFIX${old##*/}"
79+
echo " $old -> $new"
80+
install_name_tool -change "$old" "$new" "$binary"
81+
done
82+
fi
83+
84+
echo "Re-signing modified Mach-O files"
85+
codesign --force --sign - "$binary"
86+
if [ -d "$dest_dir" ]; then
87+
while IFS= read -r -d '' f; do
88+
codesign --force --sign - "$f"
89+
done < <(find "$dest_dir" -type f \( -name '*.dylib' -o -name '*.so' \) -print0)
90+
fi
91+
codesign --verify --verbose "$binary"
92+
93+
targets=("$binary")
94+
if [ -d "$dest_dir" ]; then
95+
while IFS= read -r -d '' f; do targets+=("$f"); done < <(find "$dest_dir" -type f \( -name '*.dylib' -o -name '*.so' \) -print0)
96+
fi
97+
stale=0
98+
for f in "${targets[@]}"; do
99+
if otool -L "$f" | awk -v p="$OLD_PREFIX" 'index($1, p) == 1 {found=1} END {exit !found}'; then
100+
echo "::error title=Stale load path::$f still references $OLD_PREFIX"
101+
otool -L "$f"
102+
stale=1
103+
fi
104+
done
105+
if [ "$stale" -ne 0 ]; then exit 1; fi
106+
107+
rm -f "$whl"
108+
python3 -m wheel pack "$unpacked" --dest-dir .
109+
rm -rf "$workdir"
110+
111+
echo "::endgroup::"
112+
done
113+
114+
- name: Fix wheel libraries (Linux)
115+
if: runner.os == 'Linux'
116+
shell: bash
117+
env:
118+
DIST_DIR: ${{ inputs.dist-dir }}
119+
BINARY_NAME: ${{ inputs.binary-name }}
120+
run: |
121+
set -euo pipefail
122+
python3 -m pip install --quiet wheel
123+
sudo apt-get update -qq && sudo apt-get install -y -qq patchelf
124+
# Wheels built inside a manylinux Docker container are owned by root
125+
sudo chown -R "$(id -u):$(id -g)" "$DIST_DIR"
126+
cd "$DIST_DIR"
127+
128+
shopt -s nullglob
129+
wheels=(*.whl)
130+
if [ ${#wheels[@]} -eq 0 ]; then
131+
echo "::error title=No wheels found::$DIST_DIR contains no .whl files"
132+
exit 1
133+
fi
134+
135+
LIBS_SRC="ggsql_jupyter.libs"
136+
DEST_RELATIVE="lib/ggsql-jupyter-dylibs"
137+
138+
for whl in "${wheels[@]}"; do
139+
echo "::group::$whl"
140+
141+
workdir=$(mktemp -d)
142+
python3 -m wheel unpack "$whl" -d "$workdir"
143+
unpacked=$(find "$workdir" -maxdepth 1 -mindepth 1 -type d | head -n 1)
144+
145+
src_dir="$unpacked/$LIBS_SRC"
146+
data_root=$(find "$unpacked" -maxdepth 1 -type d -name '*.data' | head -n 1)
147+
if [ -z "$data_root" ]; then
148+
echo "::error title=Wheel layout::no *.data/ directory in $whl"
149+
exit 1
150+
fi
151+
dest_dir="$data_root/data/$DEST_RELATIVE"
152+
binary=$(find "$data_root/scripts" -type f -name "$BINARY_NAME" | head -n 1)
153+
154+
if [ -z "$binary" ]; then
155+
echo "::error title=Binary not found::$BINARY_NAME not found in $whl"
156+
exit 1
157+
fi
158+
159+
if [ ! -d "$src_dir" ]; then
160+
echo "::warning title=No library bundle::$LIBS_SRC/ not present in $whl (nothing to relocate)"
161+
else
162+
echo "Moving $LIBS_SRC/ -> $(basename "$data_root")/data/$DEST_RELATIVE/"
163+
mkdir -p "$(dirname "$dest_dir")"
164+
mv "$src_dir" "$dest_dir"
165+
fi
166+
167+
new_rpath="\$ORIGIN/../$DEST_RELATIVE"
168+
echo "Setting RPATH on $(basename "$binary") to $new_rpath"
169+
patchelf --set-rpath "$new_rpath" "$binary"
170+
171+
if [ -d "$dest_dir" ]; then
172+
while IFS= read -r -d '' sofile; do
173+
echo "Setting RPATH on $(basename "$sofile") to \$ORIGIN"
174+
patchelf --set-rpath '$ORIGIN' "$sofile"
175+
done < <(find "$dest_dir" -type f -name '*.so*' -print0)
176+
fi
177+
178+
echo "Verifying RPATH on $(basename "$binary")"
179+
rpath_out=$(patchelf --print-rpath "$binary")
180+
echo " RPATH = $rpath_out"
181+
if [[ "$rpath_out" != *"$DEST_RELATIVE"* ]]; then
182+
echo "::error title=RPATH verification failed::Expected RPATH containing $DEST_RELATIVE, got: $rpath_out"
183+
exit 1
184+
fi
185+
186+
rm -f "$whl"
187+
python3 -m wheel pack "$unpacked" --dest-dir .
188+
rm -rf "$workdir"
189+
190+
echo "::endgroup::"
191+
done

.github/workflows/publish.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,9 @@ jobs:
8686
working-directory: ggsql-wasm/demo
8787
run: cp -r dist ../../doc/wasm
8888

89+
- name: Copy CHANGELOG.md to root
90+
run: cp CHANGELOG.md doc/
91+
8992
- name: Render and Publish
9093
if: github.event_name == 'push' && github.ref == 'refs/heads/main' || github.event_name == 'workflow_dispatch'
9194
uses: quarto-dev/quarto-actions/publish@v2

.github/workflows/release-jupyter.yml

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ jobs:
5555
uses: PyO3/maturin-action@v1
5656
with:
5757
target: ${{ matrix.target }}
58-
args: --release --out dist
58+
args: --release --out dist --auditwheel=repair
5959
working-directory: ggsql-jupyter
6060
manylinux: 2_28
6161
docker-options: -e GGSQL_SKIP_GENERATE=1
@@ -66,6 +66,11 @@ jobs:
6666
apt-get update && apt-get install -y unixodbc-dev
6767
fi
6868
69+
- name: Fix wheel library load paths
70+
uses: ./.github/workflows/actions/fix-wheel-libs
71+
with:
72+
dist-dir: ggsql-jupyter/dist
73+
6974
- uses: actions/upload-artifact@v4
7075
with:
7176
name: jupyter-wheels-linux-${{ matrix.target }}
@@ -101,8 +106,8 @@ jobs:
101106
args: --release --out dist --auditwheel=repair
102107
working-directory: ggsql-jupyter
103108

104-
- name: Fix wheel dylib load paths
105-
uses: ./.github/workflows/actions/fix-macos-wheel-dylibs
109+
- name: Fix wheel library load paths
110+
uses: ./.github/workflows/actions/fix-wheel-libs
106111
with:
107112
dist-dir: ggsql-jupyter/dist
108113

0 commit comments

Comments
 (0)