Bump transformers from 4.57.6 to 5.0.0

[dependabot]

Bumps transformers from 4.57.6 to 5.0.0.

Release notes

Sourced from transformers's releases.

Transformers v5 release notes

• Highlights
• Significant API changes: dynamic weight loading, tokenization
• Backwards Incompatible Changes
• Bugfixes and improvements

We have a migration guide that will be continuously updated available on the main branch, please check it out in case you're facing issues: migration guide.

Highlights

We are excited to announce the initial release of Transformers v5. This is the first major release in five years, and the release is significant: 1200 commits have been pushed to main since the latest minor release. This release removes a lot of long-due deprecations, introduces several refactors that significantly simplify our APIs and internals, and comes with a large number of bug fixes.

We give an overview of our focus for this release in the following blogpost. In these release notes, we'll focus directly on the refactors and new APIs coming with v5.

This release is the full V5 release. It sets in motion something bigger: going forward, starting with v5, we'll now release minor releases every week, rather than every 5 weeks. Expect v5.1 to follow next week, then v5.2 the week that follows, etc.

We're moving forward with this change to ensure you have access to models as soon as they're supported in the library, rather than a few weeks after.

In order to install this release, please do so with the following:

pip install transformers

For us to deliver the best package possible, it is imperative that we have feedback on how the toolkit is currently working for you. Please try it out, and open an issue in case you're facing something inconsistent/a bug.

Transformers version 5 is a community endeavor, and we couldn't have shipped such a massive release without the help of the entire community.

Significant API changes

Dynamic weight loading

We introduce a new weight loading API in transformers, which significantly improves on the previous API. This weight loading API is designed to apply operations to the checkpoints loaded by transformers.

Instead of loading the checkpoint exactly as it is serialized within the model, these operations can reshape, merge, and split the layers according to how they're defined in this new API. These operations are often a necessity when working with quantization or parallelism algorithms.

This new API is centered around the new WeightConverter class:

class WeightConverter(WeightTransform):
    operations: list[ConversionOps]
    source_keys: Union[str, list[str]]
    target_keys: Union[str, list[str]]

... (truncated)

Commits

• 08810b1 v5.0.0
• dcd7a10 fix dependencies to make sure release runs
• 93b425d Revise MIGRATION_GUIDE_V5.md for version 5 updates (#43485)
• 1e8d129 bring back clean_up_tokenization_spaces to tokenizers backend (#43426)
• 4df39f4 Fix lm_head weight tying in Mistral3ForConditionalGeneration (#43429)
• 55d1d09 Fix Pixtral with flash attention (#43443)
• a30413b fix gguf recent conversion issues (#43465)
• 728d64e 43125: Ensure correct checkpoint saving behavior by simplifying Trainer.save_...
• f5722a5 [DeepSpeed] add weight_mapping to _load_state_dict_into_zero3_model (#43303)
• c76c197 fix(models): Add tie_word_embeddings parameter to AyaVisionConfig for proper ...
• Additional commits viewable in compare view

[dependabot]

Looks like transformers is up-to-date now, so this is no longer needed.