newest transformers broken MT

[p4w3l]

Hi,

The latest transformers 4.52.1 make some breaks. I have error importing:

from multimodal_transformers.data import load_data_from_folder

ImportError: cannot import name 'BERT_INPUTS_DOCSTRING' from 'transformers.models.bert.modeling_bert' (/opt/conda/lib/python3.11/site-packages/transformers/models/bert/modeling_bert.py)

Using transformers==4.51.3 fixes the issue.

regards
Paweł

[harsh-dialpad]

Yes, I notice this as well. Its because transformers >= 4.51.3 removed the docstrings from BERT. Can be fixed by removing the corresponding references here: https://github.com/georgian-io/Multimodal-Toolkit/blob/master/multimodal_transformers/model/tabular_transformers.py

[p4w3l]

Even worse: With the newest dependencies logits returns NaNs or huge loss. Finally I fixed it by downgrading a list of dependencies:

transformers==4.44.2
torch==2.3.0
datasets==2.20.0
scikit-learn==1.5.1