-</code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">permissions</code> needs <code class="language-plaintext highlighter-rouge">contents: write</code> and <code class="language-plaintext highlighter-rouge">pull-requests: write</code> here (not just <code class="language-plaintext highlighter-rouge">contents: read</code>) since the docs step pushes a branch and opens a PR with the default <code class="language-plaintext highlighter-rouge">GITHUB_TOKEN</code>.</p> <p>Other workflows need to follow the same process, modifying <code class="language-plaintext highlighter-rouge">artifact-pattern</code> to match their own artifact naming scheme and otherwise reusing <code class="language-plaintext highlighter-rouge">publish-wheels</code> like the example. The <code class="language-plaintext highlighter-rouge">publish-to-gitlab</code> Action should only be used directly if a workflow needs the upload step without the docs PR side effect.</p> <h2 id="testing-a-new-workflow"> <a href="#testing-a-new-workflow" class="anchor-heading" aria-labelledby="testing-a-new-workflow"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> Testing a New Workflow </h2> <p>Open a new draft PR with the workflow(s) included, and include a <code class="language-plaintext highlighter-rouge">Trigger:</code> line in the PR description with a version for each package version you want to build, like so:</p> <p><code class="language-plaintext highlighter-rouge">Trigger: numpy:v2.5.0</code> <code class="language-plaintext highlighter-rouge">Trigger: numpy:v2.5.1</code></p> <p>The repository’s automation logic will pick up on and trigger the appropriate build workflows for each version. Achieving a passing (green) build may require several attempts including rework and possible patches, depending on the nature of the failure.</p> <h3 id="skipping-musl-builds"> <a href="#skipping-musl-builds" class="anchor-heading" aria-labelledby="skipping-musl-builds"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> Skipping musl Builds </h3> <p>While building for both glibc and musl (in cibuildwheel terms, <code class="language-plaintext highlighter-rouge">manylinux</code> and <code class="language-plaintext highlighter-rouge">musllinux</code>) is desirable, some of the projects we target do not build for musllinux (or they do, but run into various issues on riscv64 specifically), and so dependent packages cannot rely on musl versions of the packages either. If the musl builds fail without an obvious solution, strip those jobs from the workflow and retry, while opening an issue to track the musl incompatibility.</p> <h3 id="patching-a-project"> <a href="#patching-a-project" class="anchor-heading" aria-labelledby="patching-a-project"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> Patching a Project </h3> <p>Some workflows may fail consistently when building or testing a module, despite following the guidelines above. When this occurs, consider whether the failure meets one of the following criteria:</p> <ol> <li>The failure exercises a narrow part of the module’s functionality, or relies on external resources (e.g. large downloads over the network)</li> <li>The failure is due to reliance on some other software unavailable on riscv64</li> <li>The failure is a consequence of an artificial test limitation, e.g. a maximum timeout</li> <li>The project’s build scripts use host tooling which isn’t available on the runners or in the riscv64 manylinux images (e.g. <code class="language-plaintext highlighter-rouge">apt</code> vs <code class="language-plaintext highlighter-rouge">dnf</code>)</li> </ol> <p>Additionally, some packages may not fully incorporate all of the LICENSE files found in their repositories or as required by their dependencies. See the <a href="#licensing">Licensing</a> section for more info on this topic. In these cases, it may be justified to add one or more patch files to remove these cases from the workflow. In this scenario, follow these steps:</p> <ol> <li>Any such patches should be placed in a <code class="language-plaintext highlighter-rouge">patches/<package_name>/<version_tag></code> path inside <code class="language-plaintext highlighter-rouge">python-wheels</code>.</li> <li>Each patch should include an <code class="language-plaintext highlighter-rouge">Upstream-Status</code> tag. See the <a href="#upstream-status-tags">Upstream Status Tags</a> section for details on valid types.</li> <li>An extra step should be added to the build/test workflows before execution to use <code class="language-plaintext highlighter-rouge">git apply</code> to make necessary modifications to the project source.</li> <li>The change should be documented for the package, so that users are aware of modifications made.</li> </ol> <p><strong>Note: Patching should be performed and reviewed on a case-by-case basis - as much functionality as possible should be tested by our system to ensure a smooth user experience when consuming wheels from RISE’s package registry.</strong></p> <h2 id="releasing-a-wheel"> <a href="#releasing-a-wheel" class="anchor-heading" aria-labelledby="releasing-a-wheel"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> Releasing a Wheel </h2> <p>The <code class="language-plaintext highlighter-rouge">publish-to-gitlab</code> action does not run unless the workflow is triggered from main. This is intentional, and is meant to ensure that only those workflows which have been fully tested, reviewed, and merged are used to build and push packages. Following the merge of a PR, the workflow(s) must be re-triggered from the <code class="language-plaintext highlighter-rouge">main</code> branch in order to release the wheels to the package registry.</p> <h2 id="other-workflow-tips-and-tricks"> <a href="#other-workflow-tips-and-tricks" class="anchor-heading" aria-labelledby="other-workflow-tips-and-tricks"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> Other Workflow Tips and Tricks </h2> <h3 id="licensing"> <a href="#licensing" class="anchor-heading" aria-labelledby="licensing"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> Licensing </h3> <p>The wheels built by the <code class="language-plaintext highlighter-rouge">python-wheels</code> project use a variety of open-source licenses. Since RISE is the distributor of riscv64 wheels in the corresponding package registry, we must ensure that the wheels adhere to each project’s licensing requirements. More specifically, check:</p> <ol> <li>The built wheel contains one or more <code class="language-plaintext highlighter-rouge">LICENSE</code> files corresponding to those contained in the upstream project source.</li> <li>If the wheel ships any statically- or dynamically-linked libraries from other projects, the licensing requirements for those projects are also correctly addressed.</li> </ol> <p>If either point is not met, we should follow the <a href="#patching-a-project">Patching a Project</a> process for patching our build, and submit an issue and/or PR upstream to help them comply with license requirements as well.</p> <h4 id="publishing-gpl-sources"> <a href="#publishing-gpl-sources" class="anchor-heading" aria-labelledby="publishing-gpl-sources"><svg viewBox="0 0 16 16" aria-hidden="true"><use xlink:href="#svg-link"></use></svg></a> Publishing GPL Sources </h4> <p>Some builds statically or dynamically link against GPL-licensed components that aren’t part of the upstream project itself, but come from our build environment - most commonly the toolchain baked into the manylinux_riscv64 container (e.g. <code class="language-plaintext highlighter-rouge">gcc</code>). As the distributor of the resulting wheel, we need to make the corresponding source available permanently, not just for as long as a CI job’s artifacts happen to be retained.</p> <p>Use the <code class="language-plaintext highlighter-rouge">collect-gpl-sources</code> action in a job alongside <code class="language-plaintext highlighter-rouge">build_wheels</code> to pull the same container image and bundle the source RPM(s) for one or more installed packages into a <code class="language-plaintext highlighter-rouge">gpl-sources.tar</code> artifact:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gpl_sources:
0 commit comments