fix: downgrade transformers due to breaking change in loading from checkpoint.

[dushyantbehl]

Description of the change

HF recently made a patch available in transformers which looks to be a breaking change for anyone looking to load a model from checkpoint.

                        except RuntimeError as e:
>                           raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
E                           _pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options 
E                               (1) Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
E                               (2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message.
E                               WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray._reconstruct was not an allowed global by default. Please use `torch.serialization.add_safe_globals([_reconstruct])` to allowlist this global if you trust this class/function.
E                           
E                           Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.

.tox/py/lib/python3.12/site-packages/torch/serialization.py:1096: UnpicklingError

This change also has numerous test cases failing in our PRs and would also break anyone loading a model from checkpoint.
Someone noticed this in HuggingFace in their internal testing and have made a patch available . Unfortunately they have not merged the patch yet so we are left with no choice but to limit the transformers version to ensure no one is blocked to use the library till they merge.

Related issue number

How to verify the PR

Was the PR tested

• I have added >=1 unit test(s) for every new method I have added.
• I have ensured all unit tests pass

[github-actions]

Thanks for making a pull request! 😃
One of the maintainers will review and advise on the next steps.

[willmj]

LGTM