[Proposal] CI HuggingFace Call Reduction

[jlarson4]

Proposal

Reduce HuggingFace calls done during the test suite in CI (Compatibility checks for 3.10, 3.11, 3.12 and Full Code Coverage Checks)

Motivation

Currently if more than one PR, branch update, or release are running at one time, we hit a HuggingFace rate limit error 429, which causes CI to fail erroneously.

Pitch

There are multiple ways we could go about this:

1. Identifying situations where HF calls are made that could be promoted to test fixtures (one call reused for many tests), or removed altogether in favor of creating small synthetic models that can be tested for the same details without losing strength of coverage
2. Determining if it is possible to have GitHub queue these CI checks, to ensure more than one of them is not running at once across multiple action workflows. We can use things like cancel-in-progress or concurrency checks to reduce the frequency of runs or restrict them from running at the same time. The flaw in limiting concurrency is an elongated feedback loop for contributors, we'd like to avoid this if possible.

Additional context

GitHub concurrency docs

Checklist

• I have checked that there is no similar issue in the repo (required)

[ak91456]

I want to contribute to this issue please assign to me