Move composites into stacks-core and refactor workflows by wileyj · Pull Request #7166 · stacks-network/stacks-core

wileyj · 2026-04-23T06:56:30Z

Apologies for the large PR 😭 .
The diff is big but it's mostly moving things around, copying over external composite actions and tightening up how scripts and workflows are formatted/used.

The main changes here:

updated all external composites to use recent version (will perform another pass to ensure we're up to date. i think nix-check is still on nodejs20 and will need an update before this may merge)
moved all used composites from stacks-network/actions into the ./github/actions/ folder
moved all scripts and created new scripts to be called by workflows in .github/scripts folder
release notes now are auto generated via template file in ./github/actions/release/create-github/release/release.template (and populated from ./gjtub/scripts/draft_release.sh). it follows the same form we've been using, but automates it pretty close to what we will want to publish.
removed ./net-test folder and other unused legacy files where i found them

in addiitional to many minor changes:

default permissions for all workflow are intentionally set to contents:read, and elevated where needed.
set default permissions for all scripts at 0644 so they are not executable (all should be run intentionally)
concurrency is updated to ensure cancelling a job will cancell all downstream jobs
use outputs as opposed to env vars for passing variables between steps/jobs
scripts are runnable locally (some do require env vars being set manually on the command line, but they will work nearly identically to a runner)
- scripts are uniform in output by using a logging.sh script - no more echo "something", we may now use `info "something"; error "something broke"
since we're utilizing local composites and scripts, it does require more checkouts, but we can use a sparse checkout for .github to call the scripts/composites.
all test workflows are run able to run manually and will create the caches needed if they don't exist (compared to the removed standalone-test workflow). on a PR, these steps are skipped
more standarized formatting so the workflows follow the same pattern. human readable is the goal vs dense text.

some sample workflow runs:

manual trigger (with caches already existing): https://github.com/wileyj/stacks-core/actions/runs/24705323620
manual trigger with cache built: https://github.com/wileyj/stacks-core/actions/runs/24706032973
PR trigger: https://github.com/wileyj/stacks-core/actions/runs/24730874928
Commit to an open PR: https://github.com/wileyj/stacks-core/actions/runs/24743694825?pr=264
release builds:

stacks-node (with signer): https://github.com/wileyj/stacks-core/actions/runs/24788491544
stacks-signer: https://github.com/wileyj/stacks-core/actions/runs/24790670859
releases created from the above:

coveralls · 2026-04-23T07:48:04Z

Coverage Report for CI Build 25584487095

Warning

Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes.
Quick fix: rebase this PR. Learn more →

Coverage decreased (-0.1%) to 85.6%

Details

Coverage decreased (-0.1%) from the base build.
Patch coverage: No coverable lines changed in this PR.
3263 coverage regressions across 86 files.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

3263 previously-covered lines in 86 files lost coverage.

Top 10 Files by Coverage Loss	Lines Losing Coverage	Coverage
stackslib/src/chainstate/stacks/db/transactions.rs	208	97.07%
stackslib/src/net/inv/epoch2x.rs	208	80.24%
stackslib/src/net/chat.rs	201	93.01%
stackslib/src/chainstate/stacks/miner.rs	196	83.16%
stacks-node/src/nakamoto_node/miner.rs	149	87.34%
stackslib/src/chainstate/stacks/db/mod.rs	135	86.21%
clarity/src/vm/costs/mod.rs	125	83.57%
stackslib/src/net/api/postblock_proposal.rs	125	80.14%
stacks-node/src/nakamoto_node/relayer.rs	107	86.64%
stackslib/src/config/mod.rs	101	68.84%

Coverage Stats


Relevant Lines:	218521
Covered Lines:	187053
Line Coverage:	85.6%
Coverage Strength:	16603804.77 hits per line

💛 - Coveralls

federico-stacks · 2026-04-23T10:48:34Z

+info()  { echo "${COLGREEN}INFO:${COLRESET}    $*"; }
+warn()  { echo "${COLYELLOW}WARN:${COLRESET}    $*"; }
+error() { echo "${COLRED}ERROR:${COLRESET}   $*" >&2; [[ -n "${GITHUB_STEP_SUMMARY:-}" ]] && echo "**ERROR:** $(strip_ansi "$*")" >> "${GITHUB_STEP_SUMMARY}"; }
+hl()    { printf '%s' "${COLYELLOW}$*${COLRESET}"; }  # highlight an inline value


maybe we could redirect all the logging to stderr just in case we need sh scripts to return some value on the stdout

I can see the appeal of that, and i don't have any concerns.
would you rather this was done in this PR or a followup? there are still several things i want to address in a later PR, but felt it was better to open what i have now since it's functioning the way i want (minus a few bugs you have already noted here).

I don’t have a strong preference either way. I'm fine with addressing it in this PR or as a follow-up, whatever you think fits best with your current scope.

Sanity check on this commit: all logs are now being written to the run’s Step Summary. Should we limit this behavior to error logs only? Otherwise, there’s a risk of cluttering the Step Summary with too much information.

Co-authored-by: Federico De Felici <234774522+federico-stacks@users.noreply.github.com>

…ocker_tag_rc/

wileyj · 2026-04-23T21:42:16Z

+    if: >-
+      github.repository == 'stacks-network/stacks-core' &&
+      (inputs.node_tag != '' || inputs.signer_tag != '')


this is a footgun and needs to be fixed.

with this in place, no fork can run the release workflows which is not the desired outcome. will have to find a different way to resolve this.

cc @dhaney-stacks

… when running a script locally

federico-stacks · 2026-05-04T11:04:01Z

+        printf '%s' "${cargo_output}" \
+            | sed $'s/\033\\[[0-9;]*[A-Za-z]//g' \
+            | sed $'s/\033.[A-G]//g' \
+            | tr "\n" "\r" \
+            | sed -E 's#Diff in ([^\r]*?) at line ([[:digit:]]+):\r((:?[ +-][^\r]*\r)+)#<details>\n<summary>\1:\2</summary>\n\n```diff\n\3```\n\n</details>\n\n#g' \
+            | tr "\r" "\n" >> "${GITHUB_STEP_SUMMARY}"
+    fi
+fi
+
+# Print the original cargo output to the terminal in case of fmt failures
+[[ -n "${cargo_output}" ]] && info "${cargo_output}"


Considering the current info implementation (#7166 (comment)) , which writes to the Step Summary, we end up duplicating the Cargo output in GITHUB_STEP_SUMMARY.

I'm not sure i see the duplication here. the intent is to log the output in 2 separate places though: in GITHUB_STEP_SUMMARY which has been the default location for a while, as well as in the log output of the step running the fmt command.
I've also not been able to reproduce the duplicated output - am i missing something here?

i see it now - will have to think about this one before i commit a fix.
i noticed it also got pulled into the check release job, where it now prints a warn to GITHUB_STEP_SUMMARY where it's not useful information (e.g. WARN: Branch 7166/merge does not match a release pattern. Skipping.).

incidentally, i don't see the duplication in rust format job, but i can see how it could happen based on the logging function(s).

federico-stacks · 2026-05-04T11:08:19Z

+#   - Branch does not match             → exits 0 (all outputs empty/false; downstream
+#                                         jobs guard themselves with is_node/signer_release checks)
+# Outputs:
+#   GITHUB_OUTPUT  - Path to the GitHub Actions output file (set by runner); prints to stdout if unset


nit: here and other similar scripts: prints to stdout if unset is not really true as they managed by info, which use stderr

good catch - it was a late change when i changed it to use that logging script vs copying those functions to all scripts. i'll adjust the comments throughout

federico-stacks · 2026-05-04T11:10:17Z

+    steps:
+        # Downloads the artifacts built in `create-source-binary`
+        - name: Download Artifacts
+          uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0


nit: this seems the only one using v6.0.0, while all the others use v8.0.1

…ME's

…release body template

…ases

wileyj added 13 commits April 22, 2026 14:51

Removing legacy bitcoin integration test files

8a6086b

Delete legacy dockerfiles for release and docsgen

36b3119

Add dockerfiles for release workflow

49a84ff

remove unnecessary workflows and scripts

e79455f

Updating main ci workflow

fc2dbcc

Update test workflows

516003d

Update release workflows

52f4f20

Update cache workflow

1b7c6b6

Update secondary workflows

9046c58

Update standalone docker image(nightly) workflow

d42d1c0

Adding composite actions updated from stacks-network/actions

027b196

Adding script for workflow call

8cd7a97

remove some testing triggers and fix a typo

2f1fa66

wileyj added the no changelog label Apr 23, 2026

wileyj closed this Apr 23, 2026

wileyj reopened this Apr 23, 2026

federico-stacks reviewed Apr 23, 2026

View reviewed changes

wileyj and others added 7 commits April 23, 2026 09:41

Apply suggestions from code review

c29998f

Co-authored-by: Federico De Felici <234774522+federico-stacks@users.noreply.github.com>

Remove net-test folder

b22e594

remove duplicate check of required binary

9af8a6e

fix var name when checking for an rc docker tag

402f9fd

fix broken conditional in docker tag for latest: s/artifact_pattern/d…

004ca25

…ocker_tag_rc/

fix concurrency rules for create-cache

6bc1fa9

remove scalar conditonal

fbaa57b

wileyj commented Apr 23, 2026

View reviewed changes

wileyj added 4 commits April 23, 2026 14:46

remove check for public repo

863999c

Add exclusion for non-public repos

2671d7e

Update versions of all composites to latest released hash

aedce72

echo to stderr

92f4a5a

wileyj added 2 commits April 27, 2026 13:51

modify stderr logging functions and better handle GITHUB_STEP_SUMMARY…

e1771c2

… when running a script locally

fix path for changelog fragments variable

5a0f9cb

wileyj mentioned this pull request Apr 29, 2026

Update versions.toml and changelog entries for 3.4.0.0.2 #7175

Merged

federico-stacks reviewed May 4, 2026

View reviewed changes

wileyj added 6 commits May 4, 2026 12:26

Update download-artifact composite and modify changelog fragment READ…

0cb8355

…ME's

Update workflow script comments

e9ae633

fix logging output from workflow steps and fix incorrect link in the …

2cd4a32

…release body template

Update release template with ghcr details and links to immutable images

30c81c2

Housekeeping - fixing comments in all workflow/scripts/composites

647dbc1

fix draft_release to create correct companion link for linked rc rele…

0c313b8

…ases

Conversation

wileyj commented Apr 23, 2026

Uh oh!

coveralls commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report for CI Build 25584487095

Coverage decreased (-0.1%) to 85.6%

Details

Uncovered Changes

Coverage Regressions

Coverage Stats

💛 - Coveralls

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coveralls commented Apr 23, 2026 •

edited

Loading