CI: test scikit-learn downstream

[MarcoGorelli]

Never thought I'd see scikit-learn/scikit-learn#31127 land, but here we are 🙀

I think we need to run their tests now

[MarcoGorelli]

once i've done

uv pip install -U ninja meson-python numpy cython joblib threadpoolctl scipy
uv pip install --editable . --verbose --no-build-isolation --config-settings editable-verbose=true
uv pip install -U pandas polars pyarrow

then, locally, pytest -n 8 takes ~2 minutes. I didn't time how long compilation took

for now i'm ok with just running it locally before releases

[FBruzzesi]

I think the best way to do this is:

• Add a Github Action that is manual trigger (workflow_dispatch) and workflow_call.
• "Gate" the release to this such action and other unit tests and even downstream tests (not having this such blocker is something that was bugging me for some time).
• I am not sure how to avoid running every job twice. This needs to be investigated (maybe it's enough to add a condition such as if: github.head_ref != 'bump-version'?)

Example to require passing the jobs in pytest.yml

diff --git a/.github/workflows/publish_to_pypi.yml b/.github/workflows/publish_to_pypi.yml
index a2a1d0711..bb8974d3a 100644
--- a/.github/workflows/publish_to_pypi.yml
+++ b/.github/workflows/publish_to_pypi.yml
@@ -14,8 +14,15 @@ on:
 permissions: {}
 
 jobs:
+  tests:
+    name: Run tests
+    uses: ./.github/workflows/pytest.yml
+    permissions:
+      contents: read
+
   build:
     name: Build distribution 📦
+    needs: [tests]
     runs-on: ubuntu-latest
     permissions:
       contents: read
diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml
index 9da8bf293..1e32f9c8d 100644
--- a/.github/workflows/pytest.yml
+++ b/.github/workflows/pytest.yml
@@ -2,6 +2,7 @@ name: PyTest
 
 on:
   pull_request:
+  workflow_call:

WDYT @MarcoGorelli ?

[MarcoGorelli]

"Gate" the release to this such action and other unit tests and even downstream tests (not having this such blocker is something that was bugging me for some time).

same 😄

sounds good!