ci: Add scikit-learn to downstream tests

[FBruzzesi]

Description

Opening as draft and in downstram-tests (not slow version) to see how long it does take. From a local attempt it's still below 5 mins

What type of PR is this? (check all applicable)

• 💾 Refactor
• ✨ Feature
• 🐛 Bug Fix
• 🔧 Optimization
• 📝 Documentation
• ✅ Test
• 🐳 Other

Related issues

• Closes CI: test scikit-learn downstream #3602
• ci: Run workflows only if changes involves them #3680

[MarcoGorelli]

10 mins in ci, that's fine with me!

[FBruzzesi]

Thanks Marco, I will add a comment in scikit-learn PR asking if there is a better way to do this, and/or if we can use a pytest marker

[dangotbanned]

Might it make sense to reintroduce ¹ this workflow for scikit-learn?

• ci: remove downstream_tests_slow #3629

To me it looks like pointblank (7m 21s) and scikit-learn (11m 16s) are pretty big outliers here.
Conceptually, it is odd to group them with Test Downstream Libraries - Fast 😄

I wonder if we could speed up some runs by doing things like (#2910)?
For example:

• we don't need to run the workflow if a PR only touches docs/**
• if they test a specific set of backends, could we look for changes in those packages to trigger?

Footnotes

1. I'm not sure why vegafusion was removed, but assume there was a reason? ↩

[FBruzzesi]

Thanks for taking a look @dangotbanned - According to scikit-learn/scikit-learn#31127 (comment) we might run a subset of the tests.

In general I am a bit concerned/skeptical in running hardcoded tests, I would prefer having pytest markers. Happy to hear your thoughts

[dangotbanned]

In general I am a bit concerned/skeptical in running hardcoded tests, I would prefer having pytest markers. Happy to hear your thoughts

I'm not sure we're talking about the same thing.
I was interested in using filters in the workflow(s).

• https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow#using-filters-to-target-specific-paths-for-pull-request-or-push-events

Is a set of globs considered hardcoding?
Polars documents it here for the same motivation https://docs.pola.rs/development/contributing/ci/#design

Only run checks when they need to be run. A change to the Rust code does not warrant a linting check of the Python code, for example.

[FBruzzesi]

I'm not sure we're talking about the same thing.

We are not 🤣

I was interested in using filters in the workflow(s).

We can have both approaches! Filters in workflows will allow to skip workflows given specific paths. What I mentioned will enable much faster runtime when it runs

[dangotbanned]

We can have both approaches!

Perfect!
Yeah both works for me

[FBruzzesi]

I know it's a partial solution, but it might be worth merging to start running (fast) tests against scikit-learn.

[dangotbanned]

Thanks @FBruzzesi, just one comment from me

[dangotbanned]

Do we want to use anything like #3680 here?

[FBruzzesi]

The settings introduces there are all at workflow level, nothing job specific. We should be fine for now