|
6 | 6 |
|
7 | 7 | ## Summary |
8 | 8 |
|
9 | | -## Motivation |
| 9 | +This RFC intends to propose a cohesive architecture for the project's CI and workflow system. There have been many conversations (both public and private) on this topic, |
| 10 | +and I would like to try and propose a holistic vision that can be discussed and iterated in a public forum. |
| 11 | + |
| 12 | +## Current state |
| 13 | + |
| 14 | +The current solution includes (but is not limited) to CI, publishing and benchmarking. It is fully implemented using Github Actions workflows and apps, and relies on |
| 15 | +a mix [runs-on](https://runs-on.com/) powered self-hosted runners and GitHub hosted runners. |
| 16 | + |
| 17 | +### Infrastructure |
| 18 | + |
| 19 | +Our self-hosted runners use a custom AMI, that we build using a private repo only accessible to members of the GitHub organization. |
| 20 | + |
| 21 | +### Permissions |
| 22 | + |
| 23 | +There are four different scopes of permissions: |
| 24 | + |
| 25 | +1. Admins - very limited group of users that can manage the organization-wide settings, and the core repo settings. |
| 26 | +2. Maintainers - All TSC members, can push branches and open pull requests directly from within the core repo, and approve and merge pull requests. |
| 27 | + CI runs automatically for them, and they can trigger CI for external contributions. |
| 28 | +3. Members - Non-TSC members that are explicitly trusted. They have the same development experience a maintainer has, aside from the ability to approve and merge pull requests. |
| 29 | +4. Everyone else - have to open PRs using a fork in a classic GH flow. Requires a maintainer to run any actions-based workflow, label their PR etc. |
| 30 | + |
| 31 | +### Benchmarking |
| 32 | + |
| 33 | +TODO |
| 34 | + |
| 35 | +### Testing and CI |
| 36 | + |
| 37 | +TODO |
| 38 | + |
| 39 | +### Versioning and publishing |
| 40 | + |
| 41 | +Every pull request must include a label indicating the sort of change it includes (feature, fix, chore, performance, ci or skip). |
| 42 | +Those changes are accumulated into a draft release. What that draft is published, a workflow is triggers that gradually publishes all rust crates |
| 43 | +and all language or framework bindings in order. Those include: |
| 44 | + |
| 45 | +1. Rust crates (including integrations like `vortex-datafusion`). |
| 46 | +2. Python package |
| 47 | +3. JNI bindings |
| 48 | +4. Spark bindings |
10 | 49 |
|
11 | 50 | ## Design |
12 | 51 |
|
|
0 commit comments