transformers version 3.0.0 installaion failing with Rust lang 1.73.0.

[junaruga]

Hello,
Thank you for this project.

I am trying to install the dependency software to run this repository's example. My environment is Fedora Linux 38, and the used programming languages are below.

$ cat /etc/fedora-release
Fedora release 38 (Thirty Eight)

$ gcc --version
gcc (GCC) 13.2.1 20230728 (Red Hat 13.2.1-1)
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ rustc --version
rustc 1.73.0 (cc66ad468 2023-10-03) (Fedora 1.73.0-1.fc38)

$ python3 --version
Python 3.11.5

I tried to install the transformers version 3.0.0 pip package written in the README.md on the virtualenv (venv) environment for clean installation.

$ https://github.com/dmis-lab/biobert-pytorch.git
$ cd biobert-pytorch
$ python3 -m venv ./venv3
$ source venv3/bin/activate

Then I got the following error. The full log is here.

(venv3) $ pip3 install transformers==3.0.0
...<snip>...
         Compiling lock_api v0.3.4
           Running `rustc --crate-name lock_api --edition=2018 /home/jaruga/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lock_api-0.3.4/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="nightly"' -C metadata=127ef18477241ef2 -C extra-filename=-127ef18477241ef2 --out-dir /tmp/pip-install-qw0eyiqb/tokenizers_58d3dcdd2cd04f52a9bdb95742bdac5f/target/release/deps -L dependency=/tmp/pip-install-qw0eyiqb/tokenizers_58d3dcdd2cd04f52a9bdb95742bdac5f/target/release/deps --extern scopeguard=/tmp/pip-install-qw0eyiqb/tokenizers_58d3dcdd2cd04f52a9bdb95742bdac5f/target/release/deps/libscopeguard-1d6dee74c8567c3a.rmeta --cap-lints allow`
         Compiling ansi_term v0.12.1
           Running `rustc --crate-name ansi_term /home/jaruga/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ansi_term-0.12.1/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=6096edb26cdacb30 -C extra-filename=-6096edb26cdacb30 --out-dir /tmp/pip-install-qw0eyiqb/tokenizers_58d3dcdd2cd04f52a9bdb95742bdac5f/target/release/deps -L dependency=/tmp/pip-install-qw0eyiqb/tokenizers_58d3dcdd2cd04f52a9bdb95742bdac5f/target/release/deps --cap-lints allow`
      error[E0557]: feature has been removed
        --> /home/jaruga/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lock_api-0.3.4/src/lib.rs:91:42
         |
      91 | #![cfg_attr(feature = "nightly", feature(const_fn))]
         |                                          ^^^^^^^^ feature has been removed
         |
         = note: split into finer-grained feature gates

      For more information about this error, try `rustc --explain E0557`.
      error: could not compile `lock_api` (lib) due to previous error

      Caused by:
        process didn't exit successfully: `rustc --crate-name lock_api --edition=2018 /home/jaruga/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lock_api-0.3.4/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="nightly"' -C metadata=127ef18477241ef2 -C extra-filename=-127ef18477241ef2 --out-dir /tmp/pip-install-qw0eyiqb/tokenizers_58d3dcdd2cd04f52a9bdb95742bdac5f/target/release/deps -L dependency=/tmp/pip-install-qw0eyiqb/tokenizers_58d3dcdd2cd04f52a9bdb95742bdac5f/target/release/deps --extern scopeguard=/tmp/pip-install-qw0eyiqb/tokenizers_58d3dcdd2cd04f52a9bdb95742bdac5f/target/release/deps/libscopeguard-1d6dee74c8567c3a.rmeta --cap-lints allow` (exit status: 1)
      warning: build failed, waiting for other jobs to finish...
      error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib --` failed with code 101
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

[notice] A new release of pip available: 22.3.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
(venv3) $

Anyone, did you see this error? Is the cause of the error perhaps the rustic version is too new for the transformers version 3.0.0?

[Lola-work]

I have the same error. Have you solved it?

[Chuanhee24]

Hi there, facing the same error when installing transformers=3.0.0.

Wondering if you have found a workaround for installation?