Skip to content

Commit aba3e8d

Browse files
sushanthpysushanthpy
committed
Fixed build issues.
1 parent 2a3a048 commit aba3e8d

5 files changed

Lines changed: 407 additions & 25 deletions

File tree

‎.github/workflows/release.yml‎

Lines changed: 225 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,48 @@
22
# ToonDB Python SDK Release Pipeline
33
# =============================================================================
44
#
5-
# Downloads pre-built binaries from main ToonDB repo releases and packages
6-
# them into platform-specific Python wheels for PyPI.
5+
# WORKFLOW FLOW:
6+
#
7+
# ┌─────────────────────────────────────────────────────────────────┐
8+
# │ 1. Download binaries from toondb/toondb + Build (PARALLEL) │
9+
# ├─────────────────────────────────────────────────────────────────┤
10+
# │ ├─ build-wheel (Linux x86_64) → downloads binary + builds │
11+
# │ ├─ build-wheel (macOS ARM64) → downloads binary + builds │
12+
# │ ├─ build-wheel (Windows x64) → downloads binary + builds │
13+
# │ └─ build-sdist → builds source distribution │
14+
# └─────────────────────────────────────────────────────────────────┘
15+
# ↓
16+
# ┌─────────────────────────────────────────────────────────────────┐
17+
# │ 2. Publish packages (PARALLEL) │
18+
# ├─────────────────────────────────────────────────────────────────┤
19+
# │ ├─ create-release → Creates GitHub release + attaches wheels │
20+
# │ └─ publish → Uploads wheels to PyPI │
21+
# └─────────────────────────────────────────────────────────────────┘
22+
# ↓
23+
# ┌─────────────────────────────────────────────────────────────────┐
24+
# │ 3. summary → Shows comprehensive status │
25+
# └─────────────────────────────────────────────────────────────────┘
26+
#
27+
# This workflow packages pre-built ToonDB binaries into Python wheels.
28+
#
29+
# IMPORTANT: This SDK pulls pre-compiled binaries from the main ToonDB
30+
# repository (toondb/toondb) and wraps them in Python wheels. Each wheel
31+
# contains platform-specific binaries:
32+
# - toondb-bulk (CLI tool for bulk operations)
33+
# - toondb-server (standalone server)
34+
# - toondb-grpc-server (gRPC server)
35+
# - libtoondb_storage.* (FFI library)
36+
# - libtoondb_index.* (FFI library)
37+
#
38+
# The version input MUST match an existing release in toondb/toondb repo.
739
#
840
# Platforms supported:
941
# - Linux x86_64 (manylinux_2_17)
1042
# - macOS ARM64 (Apple Silicon)
1143
# - Windows x64
1244
#
45+
# Python versions supported: 3.9, 3.10, 3.11, 3.12, 3.13
46+
#
1347
# =============================================================================
1448
# REQUIRED SETUP:
1549
# =============================================================================
@@ -18,6 +52,9 @@
1852
# Uses OIDC Trusted Publisher (configure at PyPI project settings)
1953
# https://pypi.org/manage/project/toondb-client/settings/publishing/
2054
#
55+
# GitHub Releases - Requires write permission
56+
# Uses GITHUB_TOKEN with contents: write permission
57+
#
2158
# =============================================================================
2259

2360
name: Release
@@ -26,9 +63,13 @@ on:
2663
workflow_dispatch:
2764
inputs:
2865
version:
29-
description: 'Release version (e.g., 0.3.1) - must match a toondb/toondb release'
66+
description: 'Release version (e.g., 0.3.1) - MUST match an existing toondb/toondb release tag'
3067
required: true
3168
type: string
69+
toondb_version:
70+
description: 'ToonDB binary version (defaults to same as version if not specified)'
71+
required: false
72+
type: string
3273
dry_run:
3374
description: 'Dry run (validate without publishing)'
3475
required: false
@@ -82,7 +123,12 @@ jobs:
82123
- name: Download binaries from main ToonDB release
83124
shell: bash
84125
run: |
85-
VERSION="${{ inputs.version }}"
126+
# Use toondb_version if provided, otherwise use version
127+
VERSION="${{ inputs.toondb_version }}"
128+
if [ -z "$VERSION" ]; then
129+
VERSION="${{ inputs.version }}"
130+
fi
131+
86132
TAG="v${VERSION}"
87133
TARGET="${{ matrix.target }}"
88134
@@ -93,7 +139,10 @@ jobs:
93139
fi
94140
95141
DOWNLOAD_URL="https://github.com/${{ env.TOONDB_REPO }}/releases/download/${TAG}/${ASSET_NAME}"
96-
echo "Downloading from: $DOWNLOAD_URL"
142+
echo "Downloading ToonDB binaries from main repository:"
143+
echo " Repository: ${{ env.TOONDB_REPO }}"
144+
echo " Version: ${VERSION}"
145+
echo " URL: $DOWNLOAD_URL"
97146
98147
curl -L -f -o release-archive.${{ matrix.archive_ext }} "$DOWNLOAD_URL"
99148
@@ -227,7 +276,110 @@ jobs:
227276
retention-days: 5
228277

229278
# ===========================================================================
230-
# Publish to PyPI
279+
# Create GitHub Release
280+
# ===========================================================================
281+
create-release:
282+
name: Create GitHub Release
283+
runs-on: ubuntu-latest
284+
needs: [build-wheel, build-sdist]
285+
permissions:
286+
contents: write
287+
steps:
288+
- name: Checkout repository
289+
uses: actions/checkout@v4
290+
291+
- name: Download all artifacts
292+
uses: actions/download-artifact@v4
293+
with:
294+
path: dist/
295+
merge-multiple: true
296+
297+
- name: List release packages
298+
run: |
299+
echo "=== Packages for release ==="
300+
ls -la dist/
301+
echo ""
302+
echo "Total packages: $(ls dist/ | wc -l)"
303+
304+
- name: Generate release notes
305+
id: release_notes
306+
run: |
307+
cat > release_notes.md << 'EOF'
308+
## ToonDB Python SDK v${{ inputs.version }}
309+
310+
Python SDK wrapping pre-built binaries from [toondb/toondb v${{ inputs.toondb_version || inputs.version }}](https://github.com/toondb/toondb/releases/tag/v${{ inputs.toondb_version || inputs.version }})
311+
312+
### Installation
313+
314+
```bash
315+
pip install toondb-client==${{ inputs.version }}
316+
```
317+
318+
### What's Included
319+
320+
This release contains platform-specific wheels with pre-compiled ToonDB binaries:
321+
322+
- **toondb-bulk**: CLI tool for bulk data operations
323+
- **toondb-server**: Standalone database server
324+
- **toondb-grpc-server**: gRPC server implementation
325+
- **libtoondb_storage**: Native storage FFI library
326+
- **libtoondb_index**: Native indexing FFI library
327+
328+
### Supported Platforms
329+
330+
- ✅ **Linux x86_64** (manylinux_2_17_x86_64)
331+
- ✅ **macOS ARM64** (Apple Silicon, macosx_11_0_arm64)
332+
- ✅ **Windows x64** (win_amd64)
333+
334+
### Python Version Support
335+
336+
- Python 3.9
337+
- Python 3.10
338+
- Python 3.11
339+
- Python 3.12
340+
- Python 3.13
341+
342+
### Package Contents
343+
344+
Each wheel includes:
345+
- Python SDK code (`toondb` package)
346+
- Platform-specific binaries in `_bin/<platform>/`
347+
- Shared libraries in `lib/<platform>/`
348+
349+
### Source Distribution
350+
351+
The source distribution (`.tar.gz`) is also available for custom builds, though binaries are not included and would need to be obtained separately from the main ToonDB repository.
352+
353+
---
354+
355+
**Binary Source**: These Python wheels bundle pre-compiled binaries from the [ToonDB main repository](https://github.com/toondb/toondb) release v${{ inputs.toondb_version || inputs.version }}.
356+
EOF
357+
358+
- name: Create GitHub Release
359+
if: ${{ inputs.dry_run != true }}
360+
uses: softprops/action-gh-release@v1
361+
with:
362+
tag_name: v${{ inputs.version }}
363+
name: Release v${{ inputs.version }}
364+
body_path: release_notes.md
365+
draft: false
366+
prerelease: false
367+
files: dist/*
368+
env:
369+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
370+
371+
- name: Dry run - show what would be released
372+
if: ${{ inputs.dry_run == true }}
373+
run: |
374+
echo "🔍 Dry run mode - GitHub release would be created with:"
375+
echo ""
376+
cat release_notes.md
377+
echo ""
378+
echo "=== Files to attach ==="
379+
ls -lh dist/
380+
381+
# ===========================================================================
382+
# Publish to PyPI (parallel with GitHub release creation)
231383
# ===========================================================================
232384
publish:
233385
name: Publish to PyPI
@@ -284,36 +436,88 @@ jobs:
284436
summary:
285437
name: Release Summary
286438
runs-on: ubuntu-latest
287-
needs: [publish]
439+
needs: [build-wheel, build-sdist, create-release, publish]
288440
if: always()
289441
steps:
290-
- name: Summary
442+
- name: Download all artifacts
443+
uses: actions/download-artifact@v4
444+
with:
445+
path: dist/
446+
merge-multiple: true
447+
448+
- name: Generate comprehensive summary
291449
run: |
292-
echo "## 🐍 ToonDB Python SDK v${{ inputs.version }}" >> $GITHUB_STEP_SUMMARY
450+
echo "## 🎉 ToonDB Python SDK v${{ inputs.version }}" >> $GITHUB_STEP_SUMMARY
293451
echo "" >> $GITHUB_STEP_SUMMARY
294452
295453
if [ "${{ inputs.dry_run }}" = "true" ]; then
296-
echo "**Mode:** Dry Run (no packages published)" >> $GITHUB_STEP_SUMMARY
454+
echo "**Mode:** 🧪 Dry Run (validation only - nothing published)" >> $GITHUB_STEP_SUMMARY
297455
else
298-
echo "**Mode:** Production Release" >> $GITHUB_STEP_SUMMARY
456+
echo "**Mode:** 🚀 Production Release" >> $GITHUB_STEP_SUMMARY
299457
fi
300458
301459
echo "" >> $GITHUB_STEP_SUMMARY
302-
echo "### Installation" >> $GITHUB_STEP_SUMMARY
460+
echo "### 📦 Binary Source" >> $GITHUB_STEP_SUMMARY
461+
echo "" >> $GITHUB_STEP_SUMMARY
462+
TOONDB_VERSION="${{ inputs.toondb_version }}"
463+
if [ -z "$TOONDB_VERSION" ]; then
464+
TOONDB_VERSION="${{ inputs.version }}"
465+
fi
466+
echo "**Binaries pulled from:** [toondb/toondb v${TOONDB_VERSION}](https://github.com/${{ env.TOONDB_REPO }}/releases/tag/v${TOONDB_VERSION})" >> $GITHUB_STEP_SUMMARY
467+
echo "" >> $GITHUB_STEP_SUMMARY
468+
echo "This Python SDK wraps pre-compiled binaries from the main ToonDB repository." >> $GITHUB_STEP_SUMMARY
469+
echo "Each wheel contains platform-specific:" >> $GITHUB_STEP_SUMMARY
470+
echo "- \`toondb-bulk\`, \`toondb-server\`, \`toondb-grpc-server\` (executables)" >> $GITHUB_STEP_SUMMARY
471+
echo "- \`libtoondb_storage.*\`, \`libtoondb_index.*\` (FFI libraries)" >> $GITHUB_STEP_SUMMARY
472+
473+
echo "" >> $GITHUB_STEP_SUMMARY
474+
echo "### 📥 Installation" >> $GITHUB_STEP_SUMMARY
303475
echo '```bash' >> $GITHUB_STEP_SUMMARY
304476
echo "pip install toondb-client==${{ inputs.version }}" >> $GITHUB_STEP_SUMMARY
305477
echo '```' >> $GITHUB_STEP_SUMMARY
306478
307479
echo "" >> $GITHUB_STEP_SUMMARY
308-
echo "### Platforms Bundled" >> $GITHUB_STEP_SUMMARY
309-
echo "- ✅ Linux x86_64 (manylinux_2_17)" >> $GITHUB_STEP_SUMMARY
310-
echo "- ✅ macOS ARM64 (Apple Silicon)" >> $GITHUB_STEP_SUMMARY
311-
echo "- ✅ Windows x64" >> $GITHUB_STEP_SUMMARY
480+
echo "### 🐍 Python Version Support" >> $GITHUB_STEP_SUMMARY
481+
echo "Python 3.9 | 3.10 | 3.11 | 3.12 | 3.13" >> $GITHUB_STEP_SUMMARY
482+
483+
echo "" >> $GITHUB_STEP_SUMMARY
484+
echo "### 💻 Platform Support" >> $GITHUB_STEP_SUMMARY
485+
echo "| Platform | Wheel Tag | Status |" >> $GITHUB_STEP_SUMMARY
486+
echo "|----------|-----------|--------|" >> $GITHUB_STEP_SUMMARY
487+
echo "| Linux x86_64 | manylinux_2_17_x86_64 | ✅ Built |" >> $GITHUB_STEP_SUMMARY
488+
echo "| macOS ARM64 | macosx_11_0_arm64 | ✅ Built |" >> $GITHUB_STEP_SUMMARY
489+
echo "| Windows x64 | win_amd64 | ✅ Built |" >> $GITHUB_STEP_SUMMARY
490+
491+
echo "" >> $GITHUB_STEP_SUMMARY
492+
echo "### 📦 Packages Created" >> $GITHUB_STEP_SUMMARY
493+
echo '```' >> $GITHUB_STEP_SUMMARY
494+
ls -lh dist/ | tail -n +2 | awk '{printf "%-8s %s\n", $5, $9}'
495+
echo '```' >> $GITHUB_STEP_SUMMARY
312496
313497
echo "" >> $GITHUB_STEP_SUMMARY
314-
echo "### Status" >> $GITHUB_STEP_SUMMARY
315-
echo "| Job | Status |" >> $GITHUB_STEP_SUMMARY
498+
echo "### ✅ Build Status" >> $GITHUB_STEP_SUMMARY
499+
echo "| Job | Result |" >> $GITHUB_STEP_SUMMARY
316500
echo "|-----|--------|" >> $GITHUB_STEP_SUMMARY
317-
echo "| Build Wheels | ${{ needs.build-wheel.result || 'N/A' }} |" >> $GITHUB_STEP_SUMMARY
318-
echo "| Build sdist | ${{ needs.build-sdist.result || 'N/A' }} |" >> $GITHUB_STEP_SUMMARY
319-
echo "| Publish | ${{ needs.publish.result }} |" >> $GITHUB_STEP_SUMMARY
501+
502+
function status_icon() {
503+
case "$1" in
504+
success) echo "✅ Success" ;;
505+
failure) echo "❌ Failed" ;;
506+
cancelled) echo "⏸️ Cancelled" ;;
507+
skipped) echo "⏭️ Skipped" ;;
508+
*) echo "❔ $1" ;;
509+
esac
510+
}
511+
512+
echo "| Build Wheels | $(status_icon '${{ needs.build-wheel.result }}') |" >> $GITHUB_STEP_SUMMARY
513+
echo "| Build Source Dist | $(status_icon '${{ needs.build-sdist.result }}') |" >> $GITHUB_STEP_SUMMARY
514+
echo "| Create GitHub Release | $(status_icon '${{ needs.create-release.result }}') |" >> $GITHUB_STEP_SUMMARY
515+
echo "| Publish to PyPI | $(status_icon '${{ needs.publish.result }}') |" >> $GITHUB_STEP_SUMMARY
516+
517+
if [ "${{ inputs.dry_run }}" != "true" ]; then
518+
echo "" >> $GITHUB_STEP_SUMMARY
519+
echo "### 🔗 Links" >> $GITHUB_STEP_SUMMARY
520+
echo "- 📦 [PyPI Package](https://pypi.org/project/toondb-client/${{ inputs.version }}/)" >> $GITHUB_STEP_SUMMARY
521+
echo "- 🏷️ [GitHub Release](https://github.com/${{ github.repository }}/releases/tag/v${{ inputs.version }})" >> $GITHUB_STEP_SUMMARY
522+
echo "- 🔧 [Source Binaries](https://github.com/${{ env.TOONDB_REPO }}/releases/tag/v${TOONDB_VERSION})" >> $GITHUB_STEP_SUMMARY
523+
fi

‎CHANGELOG.md‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,13 +19,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1919
- This allows for independent versioning and faster CI/CD pipelines.
2020

2121
### Infrastructure
22-
- **New Release Workflow**: Now pulls pre-built binaries directly from main ToonDB releases.
22+
- **New Release Workflow**: Now pulls pre-built binaries directly from [toondb/toondb](https://github.com/toondb/toondb) releases.
23+
- Supports Python 3.9 through 3.13
24+
- Automatically creates GitHub releases with all wheel packages attached
25+
- Each wheel bundles platform-specific binaries and FFI libraries
26+
- See [RELEASE.md](RELEASE.md) for detailed release process documentation
2327
- **Trusted Publishing**: Configured PyPI Trusted Publisher (OIDC) security.
2428
- **Platform Bundles**:
25-
- Linux x86_64
29+
- Linux x86_64 (manylinux_2_17)
2630
- macOS ARM64 (Apple Silicon)
2731
- Windows x64
2832

33+
### Documentation
34+
- Added comprehensive [RELEASE.md](RELEASE.md) explaining how binaries are sourced from toondb/toondb
35+
- Updated README with binary source information
36+
- Enhanced release workflow with detailed summaries and status reporting
37+
2938
## [0.2.9] - 2026-01-02
3039

3140
### Added

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,8 @@ pip install toondb-client
4141
4242
**No Rust toolchain required!**
4343
44+
> ℹ️ **About the Binaries**: This Python SDK packages pre-compiled binaries from the [main ToonDB repository](https://github.com/toondb/toondb). Each wheel contains platform-specific executables (`toondb-bulk`, `toondb-server`, `toondb-grpc-server`) and native FFI libraries. See [RELEASE.md](RELEASE.md) for details on the release process.
45+
4446
## What's New in Latest Release
4547
4648
### 🎯 Namespace Isolation

0 commit comments

Comments
 (0)