ci: validate and publish conan recipes automatically

PR validation: build.yml now exports every in-repo recipe into the local
conan cache before configuring, so 'conan install --build=missing' builds
recipes changed by the PR from source and downloads the rest from the
remote. Recipe and engine changes are validated atomically in one PR, and
master stays buildable during the publish window after a merge.

Publishing: a new workflow builds and uploads recipes on pushes to master
that touch ThirdParty/ConanRecipes (plus manual workflow_dispatch). It is
guarded to the canonical repo so forks don't run it, and pins cppstd to
17/gnu17 to match the binaries already on the remote.

build_recipes.py gains an --export-only mode that exports every version of
every recipe without building or platform filtering.

Author: FlyAndNotDown

.github/workflows/build.yml

@@ -46,6 +46,11 @@ jobs:
         with:
           node-version: 24
 
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+
       - name: Setup CMake
         uses: jwlawson/actions-setup-cmake@v2
         with:
@@ -57,6 +62,14 @@ jobs:
       - name: Config Conan Remote
         run: conan remote add explosion https://conan.kindem.online/artifactory/api/conan/conan
 
+      # Register the in-repo recipes in the local cache so the engine's 'conan install --build=missing'
+      # resolves them from this checkout: unchanged recipes hash to the revision already published on the
+      # remote (binaries are downloaded), changed ones get built locally, making recipe PRs self-contained.
+      - name: Export Conan Recipes
+        run: |
+          pip install pyyaml
+          python ThirdParty/ConanRecipes/build_recipes.py --export-only
+
       - name: Configure CMake
         run: cmake -B ${{github.workspace}}/build -G=Ninja -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DCI=ON
 

.github/workflows/publish-conan-recipes.yml

@@ -0,0 +1,82 @@
+name: Publish Conan Recipes
+
+on:
+  push:
+    branches: [master]
+    paths:
+      - 'ThirdParty/ConanRecipes/**'
+      - '.github/workflows/publish-conan-recipes.yml'
+  workflow_dispatch:
+
+concurrency:
+  group: publish-conan-recipes
+
+env:
+  CONAN_REMOTE_NAME: explosion
+  CONAN_REMOTE_URL: https://conan.kindem.online/artifactory/api/conan/conan
+  CMAKE_VERSION: '4.1.2'
+
+jobs:
+  publish:
+    # Guard against forks: pushes to a fork's master must not run (and could not upload anyway,
+    # since secrets are only configured on the canonical repository).
+    if: github.repository == 'ExplosionEngine/Explosion'
+
+    strategy:
+      # Platforms publish independently: a failure on one OS must not cancel the other mid-upload.
+      fail-fast: false
+      matrix:
+        # cppstd is pinned per platform to match the binaries already on the remote; consumers on a
+        # higher standard (the engine uses C++20) still match via Conan's default compatibility plugin.
+        include:
+          - os: windows-latest
+            cppstd: '17'
+          - os: macOS-latest
+            cppstd: gnu17
+
+    runs-on: ${{ matrix.os }}
+
+    steps:
+      - name: Set XCode Version
+        run: sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
+        if: runner.os == 'macOS'
+
+      - name: Setup MSVC
+        uses: ilammy/msvc-dev-cmd@v1
+        if: runner.os == 'Windows'
+
+      - name: Checkout Repo
+        uses: actions/checkout@v4
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+
+      - name: Setup CMake
+        uses: jwlawson/actions-setup-cmake@v2
+        with:
+          cmake-version: ${{env.CMAKE_VERSION}}
+
+      - name: Setup Conan
+        uses: conan-io/setup-conan@v1
+
+      - name: Detect Conan Profile
+        run: conan profile detect --force
+
+      - name: Config Conan Remote
+        run: conan remote add ${{ env.CONAN_REMOTE_NAME }} ${{ env.CONAN_REMOTE_URL }}
+
+      - name: Install Python Dependencies
+        run: pip install pyyaml
+
+      # --build=missing (the script default) downloads binaries the remote already has for the same
+      # recipe revision, so only recipes actually changed by the triggering push get rebuilt.
+      - name: Build And Upload Recipes
+        run: >-
+          python ThirdParty/ConanRecipes/build_recipes.py
+          --conan-arg=--settings=compiler.cppstd=${{ matrix.cppstd }}
+          --upload
+          --remote ${{ env.CONAN_REMOTE_NAME }}
+          --remote-user ${{ secrets.CONAN_REMOTE_USER }}
+          --remote-password ${{ secrets.CONAN_REMOTE_PASSWORD }}

ThirdParty/ConanRecipes/README.md

@@ -45,4 +45,28 @@ python build_recipes.py --upload \
     --remote <remote> \
     --remote-url <remote-url> \
     --remote-user <user> --remote-password <password>
+
+# just register every recipe version in the local conan cache (no build);
+# a later 'conan install --build=missing' then builds whatever the remote
+# has no binaries for
+python build_recipes.py --export-only
 ```
+
+## CI
+
+Recipe changes are validated and published automatically:
+
+- Every pull request runs the engine build workflow, which first runs
+  `build_recipes.py --export-only`. Unchanged recipes hash to the same
+  revision already published on the remote so their binaries are simply
+  downloaded, while recipes changed by the PR are built from source inside
+  the job. A PR can therefore change recipes and engine code together and
+  be validated atomically.
+- After a push to `master` that touches `ThirdParty/ConanRecipes`, the
+  `Publish Conan Recipes` workflow builds the changed recipes on Windows
+  and macOS and uploads them to the remote (credentials come from the
+  `CONAN_REMOTE_USER` / `CONAN_REMOTE_PASSWORD` repository secrets). It can
+  also be re-run manually via `workflow_dispatch` if an upload failed.
+
+To keep remote revisions in sync with git, avoid uploading from local
+machines; let the publish workflow be the only writer.

ThirdParty/ConanRecipes/build_recipes.py

@@ -5,6 +5,11 @@
 top entry in conandata.yml), filtered by the recipe's `platforms` list. The
 first failure stops the run and prints a summary. Uploading is opt-in and only
 runs once every recipe has built successfully.
+
+With --export-only the script just runs 'conan export' for every version of
+every recipe, so a subsequent 'conan install --build=missing' resolves recipes
+from the local cache and builds whatever the remote has no binaries for. CI
+uses this to validate recipe changes together with the engine build.
 """
 
 from __future__ import annotations
@@ -45,7 +50,8 @@ def __init__(self, path: Path):
 
         data = yaml.safe_load(self.conandata.read_text(encoding="utf-8")) or {}
         self.name = self._parse_name() or self.dir_name
-        self.version = latest_version(data)
+        self.versions = list_versions(data)
+        self.version = self.versions[0] if self.versions else None
         self.platforms = data.get("platforms") or []
         self.requires = data.get("requires") or []
 
@@ -62,14 +68,14 @@ def reference(self) -> str:
         return f"{self.name}/{self.version}"
 
 
-def latest_version(data: dict) -> str | None:
+def list_versions(data: dict) -> list[str]:
     sources = data.get("sources")
     if isinstance(sources, dict) and sources:
-        return next(iter(sources))
+        return [str(v) for v in sources]
     versions = data.get("versions")
     if isinstance(versions, list) and versions:
-        return versions[0]
-    return None
+        return [str(v) for v in versions]
+    return []
 
 
 def discover_recipes(root: Path) -> list[Recipe]:
@@ -104,8 +110,9 @@ def visit(recipe: Recipe):
     return ordered
 
 
-def run(cmd: list[str], cwd: Path | None = None) -> int:
-    print(f"\n$ {' '.join(cmd)}", flush=True)
+def run(cmd: list[str], cwd: Path | None = None, redact: set[str] | None = None) -> int:
+    shown = " ".join("***" if redact and arg in redact else arg for arg in cmd)
+    print(f"\n$ {shown}", flush=True)
     return subprocess.run(cmd, cwd=str(cwd) if cwd else None).returncode
 
 
@@ -140,6 +147,11 @@ def parse_args() -> argparse.Namespace:
     parser.add_argument(
         "--skip", action="append", default=[], help="skip these recipe names (repeatable)"
     )
+    parser.add_argument(
+        "--export-only",
+        action="store_true",
+        help="only 'conan export' every version of every recipe; no build, no platform filter",
+    )
 
     upload = parser.add_argument_group("upload")
     upload.add_argument(
@@ -157,9 +169,24 @@ def parse_args() -> argparse.Namespace:
     args = parser.parse_args()
     if args.upload and not args.remote:
         parser.error("--upload requires --remote")
+    if args.export_only and args.upload:
+        parser.error("--export-only cannot be combined with --upload")
     return args
 
 
+def export_all(args: argparse.Namespace, recipes: list[Recipe]):
+    count = 0
+    for recipe in recipes:
+        if not recipe.versions:
+            sys.exit(f"error: could not determine versions from {recipe.conandata}")
+        for version in recipe.versions:
+            cmd = [args.conan, "export", f"{recipe.dir_name}/conanfile.py", "--version", version]
+            if run(cmd, cwd=args.recipes_root) != 0:
+                sys.exit(f"error: failed to export {recipe.name}/{version}")
+            count += 1
+    print(f"\nExported {count} recipe version(s).", flush=True)
+
+
 def build_all(args: argparse.Namespace, recipes: list[Recipe], host: str):
     built: list[Recipe] = []
     skipped: list[tuple[Recipe, str]] = []
@@ -215,9 +242,11 @@ def upload_all(args: argparse.Namespace, built: list[Recipe]):
 
     if args.remote_user is not None:
         login = [args.conan, "remote", "login", args.remote, args.remote_user]
+        redact: set[str] = set()
         if args.remote_password is not None:
             login += ["-p", args.remote_password]
-        if run(login) != 0:
+            redact.add(args.remote_password)
+        if run(login, redact=redact) != 0:
             sys.exit("error: failed to log in to remote")
 
     for recipe in built:
@@ -250,9 +279,6 @@ def main():
     if not root.is_dir():
         sys.exit(f"error: recipes root not found: {root}")
 
-    host = current_platform()
-    print(f"Host platform: {host}", flush=True)
-
     recipes = discover_recipes(root)
     if args.only:
         recipes = [r for r in recipes if r.name in args.only or r.dir_name in args.only]
@@ -261,6 +287,13 @@ def main():
     if not recipes:
         sys.exit("error: no recipes to build")
 
+    if args.export_only:
+        export_all(args, recipes)
+        return
+
+    host = current_platform()
+    print(f"Host platform: {host}", flush=True)
+
     recipes = order_by_dependencies(recipes)
     print("Build order: " + ", ".join(r.name for r in recipes), flush=True)