Skip to content

Commit 8e59116

Browse files
authored
docs: update README for pixi-based build workflow (#180)
Replace the miniforge + conda constructor bootstrap with `pixi run build-installer`, list all four published installers, clarify that the `version` field in construct.yaml is rolling bookkeeping rewritten at release time, and fix stale `master` references plus a few typos.
1 parent 7f1d17b commit 8e59116

1 file changed

Lines changed: 22 additions & 22 deletions

File tree

README.md

Lines changed: 22 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,14 @@ Python 3 DIRACOS installer built using [conda constructor](https://github.com/co
1616

1717
## Installing DIRACOS2
1818

19-
These instructions will install the latest release of DIRACOS in a folder named `diracos`. To install a specific version replace `/latest/download/` in the URL with a version like `/download/2.0/`.
19+
These instructions will install the latest release of DIRACOS in a folder named `diracos`. Installers are published for the following platforms:
20+
21+
* `DIRACOS-Linux-x86_64.sh`
22+
* `DIRACOS-Linux-aarch64.sh`
23+
* `DIRACOS-Darwin-x86_64.sh` (client-only)
24+
* `DIRACOS-Darwin-arm64.sh` (client-only)
25+
26+
To install a specific version, replace `/latest/download/` in the URL with `/download/<version>/`. The full list of releases is available on the [releases page](https://github.com/DIRACGrid/DIRACOS2/releases).
2027

2128
```bash
2229
curl -LO https://github.com/DIRACGrid/DIRACOS2/releases/latest/download/DIRACOS-Linux-x86_64.sh
@@ -30,42 +37,35 @@ Further usage instructions are shown after installation.
3037

3138
### Building the installer
3239

33-
The DIRACOS installer is a self-extracting shell script that is generated using [conda constructor](https://github.com/conda/constructor). This can be installed using any `conda` installation. If you don't have a local conda installation, you can use the following steps on Linux:
40+
The DIRACOS installer is a self-extracting shell script generated using [conda constructor](https://github.com/conda/constructor). The build environment is managed with [pixi](https://pixi.sh):
3441

3542
```bash
36-
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
37-
bash Miniforge3-Linux-x86_64.sh -b -p $PWD/miniforge
38-
# This line is needed every time you wish to use miniforge in a new shell
39-
eval "$($PWD/miniforge/bin/conda shell.bash hook)"
40-
conda install constructor
43+
pixi install
44+
pixi run build-installer
4145
```
4246

43-
Once you have `constructor` installed, a new installer can be generating from the `construct.yaml` using:
44-
45-
```bash
46-
constructor . --platform=linux-64
47-
```
47+
This produces `DIRACOS-<version>-<platform>.sh` in the current directory, targeting the host platform. Cross-platform builds are handled in CI, where each of `linux-64`, `linux-aarch64`, `osx-64`, and `osx-arm64` is built on a matching runner. The `CONDA_OVERRIDE_GLIBC` / `CONDA_OVERRIDE_OSX` values for each target are set in `pixi.toml`.
4848

49-
The packages included are defined in the `construct.yaml` file, see the [upstream documentation](https://github.com/conda/constructor/blob/master/CONSTRUCT.md) for more details.
49+
The packages included are defined in `construct.yaml`; see the [upstream documentation](https://github.com/conda/constructor/blob/master/CONSTRUCT.md) for more details. For faster local iteration, `transmute_file_type` in `construct.yaml` can be disabled at the cost of a larger installer.
5050

5151
### Testing the installer
5252

53-
Basic tests of the installer can be ran using an arbitrary docker image by running:
53+
Basic tests of the installer can be run against an arbitrary docker image with:
5454

5555
```bash
5656
scripts/run_basic_tests.sh DOCKER_IMAGE DIRACOS_INSTALLER_FILENAME
5757
```
5858

5959
### Making a release
6060

61-
To ensure reproducibility, releases are made from build artifacts from previous pipelines and are tagged using GitHub actions by triggering the [Create release](https://github.com/DIRACGrid/DIRACOS2/actions/workflows/release.yaml?query=workflow%3A%22Create+release%22) workflow. This workflow runs [`scripts/make_release.py`](https://github.com/DIRACGrid/DIRACOS2/blob/main/scripts/make_release.py) and has the following optional parameters:
61+
To ensure reproducibility, releases are made from build artifacts of previous pipelines and tagged via GitHub Actions by triggering the [Create release](https://github.com/DIRACGrid/DIRACOS2/actions/workflows/release.yaml?query=workflow%3A%22Create+release%22) workflow. This workflow runs [`scripts/make_release.py`](https://github.com/DIRACGrid/DIRACOS2/blob/main/scripts/make_release.py) and has the following optional parameters:
6262

63-
* **Run ID**: The GitHub Actions workflow run ID. If not given, defaults to the most recent build of the `main` branch.
64-
* **Version number**: A [PEP-440](https://www.python.org/dev/peps/pep-0440/) compliant version number. If not given, defaults to the contents the contents of version field in the `construct.yaml` rounded to the next full release (i.e. `2.4a5` becomes `2.4` and `2.1` remains unchanged). If a pre-release is explicitly give, it will be marked as a pre-release in GitHub and won't affect the `latest` alias.
63+
* **Run ID**: The GitHub Actions workflow run ID whose artifacts will be released. If not given, defaults to the most recent build of the `main` branch.
64+
* **Version number**: A [PEP-440](https://www.python.org/dev/peps/pep-0440/) compliant version number. If not given, defaults to the `version` field in `construct.yaml` rounded to the next full release (e.g. `2.4a5` becomes `2.4`, `2.1` stays `2.1`). Passing an explicit pre-release marks the release as a pre-release in GitHub and does not affect the `latest` alias.
6565

66-
If the release process fails, a draft release might be left in GitHub. After the issue has been fixed this can be safely deleted before rerunning the CI.
66+
The `version` field in `construct.yaml` is rolling bookkeeping rather than something that is hand-edited per release: after each release, a commit is pushed to `main` bumping it to the next alpha (skipped if that would be a downgrade). The binary installer's embedded version is rewritten at release time by `make_release.py`.
6767

68-
After the release is made a commit will be pushed to master to bump the DIRACOS release number in `construct.yaml` to the next alpha release. If the next alpha release is older than the current contents of `construct.yaml`, this step is skipped.
68+
If the release process fails, a draft release may be left in GitHub. After the issue has been fixed it can be safely deleted before rerunning the CI.
6969

7070
### Building customised packages
7171

@@ -75,9 +75,9 @@ See [`management/conda-recipes`](https://github.com/DIRACGrid/management/tree/ma
7575

7676
DIRACOS2 is a smaller wrapper around [conda constructor](https://github.com/conda/constructor) which generates a self-extracting installer that is suitable for DIRAC's needs based on packages which are distributed by [conda-forge](https://conda-forge.org/). As such, most components are documented by their respective projects. The following is a list of remarks that are specific DIRACOS2:
7777

78-
* The build-and-test workflow is ran on every push event as well as once per day. GitHub Actions limitations will cause the nightly build to be disabled if there is no activity in the repository for 60 days. If this happens, it can be easily re-enabled by pushing an empty commit to master.
79-
* The DIRACOS installer follows the common pattern of being a shell script that exits followed by an arbitrary binary blob. In principle it is possible to create other kinds of installer with `constructor` however these are no deemed to be of interest at the current time.
80-
* Similarly to the original DIRACOS, releases are made by assigning a version number to a previous CI build. As `constructor` embeds the version number into the installer at build time, this means the version of the installed has to be edited for the binary artifact. Fortunately this is only embedded in the shell script at the start of the file, so it's a relatively simple string manipulation to modify it. This is done using in the `main()` function of [`scripts/make_release.py`](https://github.com/DIRACGrid/DIRACOS2/blob/main/scripts/make_release.py). As this is not an officially supported feature of constructor it is prone to changing at anytime.
78+
* The build-and-test workflow runs on every push event as well as once per day. GitHub Actions limitations will cause the nightly build to be disabled if there is no activity in the repository for 60 days. If this happens, it can be easily re-enabled by pushing an empty commit to `main`.
79+
* The DIRACOS installer follows the common pattern of being a shell script that exits followed by an arbitrary binary blob. In principle it is possible to create other kinds of installer with `constructor`, however these are not deemed to be of interest at the current time.
80+
* Similarly to the original DIRACOS, releases are made by assigning a version number to a previous CI build. As `constructor` embeds the version number into the installer at build time, this means the version has to be edited in the binary artifact. Fortunately this is only embedded in the shell script at the start of the file, so it's a relatively simple string manipulation, performed in the `main()` function of [`scripts/make_release.py`](https://github.com/DIRACGrid/DIRACOS2/blob/main/scripts/make_release.py). As this is not an officially supported feature of constructor it is prone to changing at any time.
8181

8282
## Troubleshooting
8383

0 commit comments

Comments
 (0)