Skip to content

build: prepare for subtree split#276

Draft
Sjors wants to merge 2 commits into
bitcoin-core:masterfrom
Sjors:2026/05/subtree-split
Draft

build: prepare for subtree split#276
Sjors wants to merge 2 commits into
bitcoin-core:masterfrom
Sjors:2026/05/subtree-split

Conversation

@Sjors
Copy link
Copy Markdown
Member

@Sjors Sjors commented May 12, 2026

We can use a subtree split to export only a subset of this repository to downstream projects. The only thing we need to do is publish and maintain a branch, e.g. lib, generated as follows:

git subtree split --prefix=lib -b lib

I published such a branch here: https://github.com/Sjors/libmultiprocess/commits/lib/

Commits that don't touch the prefix directory are skipped entirely. Others are stripped of content outside the prefix. This of course breaks PGP signatures, but the lib branch commits are deterministically generated. On the Bitcoin Core side I plan to update the subtree linter to verify this.

Because we can only export one directory, this PR moves the following library sources to lib/:

  • CMakeLists.txt
  • cmake
  • include
  • pkgconfig
  • src
  • test

This leaves examples, documentation and CI outside the export. Tests could be moved out as well, but it seems useful to offer downstream projects an easy way to run these tests as part of their own CI.

A new top-level CMakeLists.txt wrapper preserves 'cmake -S .' for standalone builds; this is also the only way to build the example/ subdirectory. See comments in CMakeLists.txt and lib/CMakeLists.txt for details on how lib/ adapts between the two build modes. This can use some CMake expertise eyes. Most of the existing CMakeList.txt content is simply moved, so it's best reviewed with --color-moved=dimmed-zebra.

Commit 66d7b48 can be avoided if we drop examples from cmake/libmultiprocess.cmake in Bitcoin Core in a separate PR first.

Bitcoin Core side: Sjors/bitcoin#113 (will upstream later)

Fixes #232

@DrahtBot
Copy link
Copy Markdown

DrahtBot commented May 12, 2026

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
Concept ACK ryanofsky

If your review is incorrectly listed, please copy-paste <!--meta-tag:bot-skip--> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #277 (Add std::unordered_set support and a helper BuildList to dedup list build handlers by ViniciusCestarii)
  • #274 (Add nonunix platform support by ryanofsky)
  • #272 (ci: add Windows job by Sjors)
  • #269 (proxy: add local connection limit to ListenConnections by enirox001)
  • #238 (cmake, ci: updates for recent nixpkgs by ryanofsky)
  • #231 (Add windows support by ryanofsky)
  • #212 (ci: add newdeps job testing newer versions of cmake and capnproto by ryanofsky)
  • #209 (cmake: Increase cmake policy version by ryanofsky)
  • #204 (cmake: support default build and test workflow by purpleKarrot)
  • #175 (Set cmake_minimum_required(VERSION 3.22) by maflcko)
  • #163 (build: set cmake policy version to 3.31 by purpleKarrot)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

  • Capn'Proto -> Cap'n Proto [misspelled project name; the intended name is clearly “Cap'n Proto”]

2026-05-12 09:59:26

Move the library sources (CMakeLists.txt, cmake/, include/, pkgconfig/,
src/, test/) under a single lib/ prefix so that

    git subtree split --prefix=lib -b lib

produces a 'lib' branch containing only the files needed for a full
library build and install. Downstream projects consuming libmultiprocess
as a git subtree can pull from this branch and avoid churn from changes
to CI scripts, README, doc/, examples, shell.nix and other files that
are not part of the library itself.

A new top-level CMakeLists.txt wrapper preserves 'cmake -S .' for
standalone builds; this is also the only way to build the example/
subdirectory, which lives outside lib/ since downstream subtree
consumers do not need it. See comments in CMakeLists.txt and
lib/CMakeLists.txt for details on how lib/ adapts between the two
build modes.

The bulk of this commit is a pure file move (renames only). Outside
the move itself, only minimal touch-ups were necessary in
lib/CMakeLists.txt and ci/scripts/{ci,bitcoin_core_ci}.sh.
@Sjors Sjors force-pushed the 2026/05/subtree-split branch from 0e96999 to d8364e5 Compare May 12, 2026 09:53
Bitcoin Core's cmake/libmultiprocess.cmake currently references the
mpcalculator, mpprinter and mpexample targets, which no longer exist
in the subtree (the example/ directory lives outside the 'lib' branch).

Strip the offending line as part of replace_subtree so the bitcoin-core
CI job configures cleanly. Remove this workaround once a matching
Bitcoin Core PR has landed.
@ryanofsky
Copy link
Copy Markdown
Collaborator

ryanofsky commented Jun 2, 2026

Concept ACK. It would be great if only needed files were imported into bitcoin core and things like ci scripts, documentation, and examples were excluded, since this could speed up review in both repositories.

The approach here also seems workable although I think I'd probably still prefer the idea of moving unneeded files to a support branch as described #232 (comment) instead of using a partial subtree. That approach seems nice because it wouldn't require any changes to subtree import steps, or to the verifier, and wouldn't impact hashes, and wouldn't require any files except the support files to move. I also like the property of that approach where individual PRs only update the support branch or master branch, changes to master can't sneak in a support PR, and review effort can be focused on the right places (relatively more scrunity on master branch, and more freedom to experiment with CI improvements on the support branch).

@DrahtBot
Copy link
Copy Markdown

DrahtBot commented Jun 2, 2026

🐙 This pull request conflicts with the target branch and needs rebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

libmultiprocess-ext repo

3 participants