Commit 748ec4c
authored
Fix HFInferenceParams missing is_compileable for transformers 5.x compatibility (#1553)
## Problem
Nightly CI has been failing for 7 consecutive days (4/11–4/17) across
`models/mixtral`, `models/llama3`, `models/qwen`, and
`verify-recipe-tests`.
All failures share the same root cause:
```
AttributeError: 'HFInferenceParams' object has no attribute 'is_compileable'
```
The 26.03 PyTorch container update (#1540) brought
`transformers==5.5.4`, which added an `is_compileable` property check on
cache objects during `generate()`. The custom `HFInferenceParams` class
does not implement this property.
## Fix
Added `is_compileable` property returning `False` to `HFInferenceParams`
in all 6 affected files (4 model sources + 2 recipe copies). The
property returns `False` since this custom TE-based cache wrapper is not
compatible with `torch.compile`.
## Failing CI run
https://github.com/NVIDIA/bionemo-framework/actions/runs/24558420210
## Testing
- Pre-commit checks pass
- `check_copied_files.py` confirms all copies are in sync
- Fix is minimal and targeted (5 lines per file)
Signed-off-by: svc-bionemo <svc-bionemo@nvidia.com>
Signed-off-by: svc-bionemo <267129667+svc-bionemo@users.noreply.github.com>
Co-authored-by: svc-bionemo <267129667+svc-bionemo@users.noreply.github.com>1 parent b508876 commit 748ec4c
6 files changed
Lines changed: 30 additions & 0 deletions
File tree
- bionemo-recipes
- models
- llama3
- mixtral
- qwen
- recipes
- llama3_native_te
- opengenome2_llama_native_te
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
591 | 591 | | |
592 | 592 | | |
593 | 593 | | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
876 | 876 | | |
877 | 877 | | |
878 | 878 | | |
| 879 | + | |
| 880 | + | |
| 881 | + | |
| 882 | + | |
| 883 | + | |
879 | 884 | | |
880 | 885 | | |
881 | 886 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
576 | 576 | | |
577 | 577 | | |
578 | 578 | | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
586 | 586 | | |
587 | 587 | | |
588 | 588 | | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
Lines changed: 5 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
597 | 597 | | |
598 | 598 | | |
599 | 599 | | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
Lines changed: 5 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
597 | 597 | | |
598 | 598 | | |
599 | 599 | | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
0 commit comments