build: Upgrade TRL version from 0.14 to 0.16#527

Merged

dushyantbehl merged 16 commits intofoundation-model-stack:mainfrom

Abhishek-TAMU:upgrade_trl

Apr 14, 2025

Collaborator

Abhishek-TAMU commented Apr 10, 2025 •

edited by dushyantbehl

Loading

Description of the change

Upgrade TRL version from 0.14.0 to 0.15.2 for supporting Vision Model Support PR.
Undocumented Prompt tuning.
Replace Prompt Tuning with LoRA tuning in unit test case where features other than Prompt Tuning is tested and Prompt tuning gives error.
Commented out test cases specifically testing Prompt tuning where Prompt tuning gives error.

Related issue number

Issue: https://github.ibm.com/ai-foundation/watson-fm-stack-tracker/issues/1718

How to verify the PR

Was the PR tested

I have added >=1 unit test(s) for every new method I have added.
I have ensured all unit tests pass

Abhishek-TAMU and others added 2 commits

April 10, 2025 13:34


          Upgrade TRL version

7afd938

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>


          Merge branch 'foundation-model-stack:main' into upgrade_trl

7b9e48d

Abhishek-TAMU requested review from Ssukriti, aluu317, anhuong, fabianlim and kmehant as code owners

April 10, 2025 17:36

github-actions Bot commented Apr 10, 2025

Thanks for making a pull request! 😃
One of the maintainers will review and advise on the next steps.

github-actions Bot added the build label

Abhishek-TAMU requested review from dushyantbehl and removed request for aluu317, anhuong and fabianlim

April 10, 2025 17:37

Abhishek-TAMU added 3 commits

April 10, 2025 19:48


          Add attention mask in dataset

dee22f1

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>


          Merge remote-tracking branch 'upstream/main' into upgrade_trl

b9667cd


          Version increase to 0.16.1

92f7c39

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

Abhishek-TAMU changed the title ~~build: Upgrade TRL version from 0.14.0 to 0.15.2~~ build: Upgrade TRL version from 0.14 to 0.16

Abhishek-TAMU added 2 commits

April 10, 2025 21:48


          Prompt tuning arg assign num_virtual_tokens=0

d511caf

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>


          Prompt tuning arg assign num_virtual_tokens=0

6d6eb18

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

Abhishek-TAMU commented

View reviewed changes

tests/test_sft_trainer.py

dushyantbehl reviewed

View reviewed changes

Collaborator

dushyantbehl left a comment

Need to clear out the PT support with TRL if its not working.

Else LGTM.

tests/artifacts/predefined_data_configs/duplicate_columns.yaml Outdated

tests/test_sft_trainer.py

tuning/sft_trainer.py

Abhishek-TAMU added 5 commits

April 11, 2025 12:16


          undocumented Prompt tuning and commented its unit tests

54a6764

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>


          Merge remote-tracking branch 'upstream/main' into upgrade_trl

fc7304e


          Remove DataCollatorForSeq2Seq for tokenized dataset with packing

11214b5

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>


          Remove DataCollatorForSeq2Seq for tokenized dataset with packing

8e3373f

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>


          Skipped PT tests

11e4dfc

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

dushyantbehl reviewed

View reviewed changes

tests/build/test_launch_script.py Outdated

		_validate_training_output(checkpoint, "ft")


		@pytest.mark.skipif(True, reason="This test is always skipped")

Collaborator

dushyantbehl Apr 14, 2025

Suggested change

      
            @pytest.mark.skipif(True, reason="This test is always skipped")
          
            @pytest.mark.skipif(True, reason="This test is deprecated so always skipped")

dushyantbehl reviewed

View reviewed changes

tests/test_sft_trainer.py Outdated

		############################# Prompt Tuning Tests #############################


		@pytest.mark.skipif(True, reason="This test is always skipped")

Collaborator

dushyantbehl Apr 14, 2025

Suggested change

      
            @pytest.mark.skipif(True, reason="This test is always skipped")
          
            @pytest.mark.skipif(True, reason="This test is deprecated so always skipped")

dushyantbehl reviewed

View reviewed changes

tests/test_sft_trainer.py Outdated

		assert "### Text: @NortonSupport Thanks much.\n\n### Label:" in output_inference


		@pytest.mark.skipif(True, reason="This test is always skipped")

Collaborator

dushyantbehl Apr 14, 2025

Suggested change

      
            @pytest.mark.skipif(True, reason="This test is always skipped")
          
            @pytest.mark.skipif(True, reason="This test is deprecated so always skipped")

dushyantbehl reviewed

View reviewed changes

tests/test_sft_trainer.py Outdated

		assert "### Text: @NortonSupport Thanks much.\n\n### Label:" in output_inference


		@pytest.mark.skipif(True, reason="This test is always skipped")

Collaborator

dushyantbehl Apr 14, 2025

Suggested change

      
            @pytest.mark.skipif(True, reason="This test is always skipped")
          
            @pytest.mark.skipif(True, reason="This test is deprecated so always skipped")

dushyantbehl reviewed

View reviewed changes

tests/test_sft_trainer.py

                       tuning_config = peft_config.PromptTuningConfig(
                           prompt_tuning_init="TEXT",
                           prompt_tuning_init_text="hello",
+                          num_virtual_tokens=0,

Collaborator

dushyantbehl Apr 14, 2025

To the above as well

Suggested change

      
                        num_virtual_tokens=0,
          
            @pytest.mark.skipif(True, reason="This test is deprecated so always skipped")

dushyantbehl reviewed

View reviewed changes

tests/test_sft_trainer.py Outdated

		_validate_training(tempdir, check_eval=True)


		@pytest.mark.skipif(True, reason="This test is always skipped")

Collaborator

dushyantbehl Apr 14, 2025

Suggested change

      
            @pytest.mark.skipif(True, reason="This test is always skipped")
          
            @pytest.mark.skipif(True, reason="This test is deprecated so always skipped")

dushyantbehl reviewed

View reviewed changes

tuning/data/data_preprocessing_utils.py Outdated

-                          )
-                      # packing for non tokenized dataset doesn't require a collator with SFTrainer.
+                      # With SFTTrainer, packing for a tokenized dataset uses default Collator,

Collaborator

dushyantbehl Apr 14, 2025 •

edited

Loading

With SFTTrainer, packing for both tokenized and non tokenized dataset use default collator, DataCollatorForLanguageModeling, and we do not need to pass any explicit collator in that case.

dushyantbehl reviewed

View reviewed changes

Collaborator

dushyantbehl left a comment

Minor nits requested but looks okay.


          PR Changes

e6744d2

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

Collaborator Author

Abhishek-TAMU commented Apr 14, 2025

@dushyantbehl Fixed the PR changes.

Abhishek-TAMU added 3 commits

April 14, 2025 12:58


          PR Changes

e295cf3

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>


          Fix lint

84c5106

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>


          Remove enable_reduce_loss_sum and _is_peft_model check

fe3b5af

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

dushyantbehl approved these changes

View reviewed changes

Collaborator

dushyantbehl left a comment

LGTM.

dushyantbehl merged commit 5bb5489 into foundation-model-stack:main

9 checks passed

Abhishek-TAMU deleted the upgrade_trl branch

April 14, 2025 18:06

kmehant pushed a commit that referenced this pull request


          build: Upgrade TRL version from 0.14 to 0.16 (#527)

4f3f71f

* Upgrade TRL version

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Add attention mask in dataset

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Version increase to 0.16.1

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Prompt tuning arg assign num_virtual_tokens=0

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Prompt tuning arg assign num_virtual_tokens=0

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* undocumented Prompt tuning and commented its unit tests

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Remove DataCollatorForSeq2Seq for tokenized dataset with packing

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Remove DataCollatorForSeq2Seq for tokenized dataset with packing

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Skipped PT tests

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* PR Changes

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* PR Changes

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Fix lint

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Remove enable_reduce_loss_sum and _is_peft_model check

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

---------

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

dushyantbehl pushed a commit to dushyantbehl/fms-hf-tuning that referenced this pull request


          build: Upgrade TRL version from 0.14 to 0.16 (foundation-model-stack#527

5fb4495

)

* Upgrade TRL version

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Add attention mask in dataset

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Version increase to 0.16.1

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Prompt tuning arg assign num_virtual_tokens=0

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Prompt tuning arg assign num_virtual_tokens=0

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* undocumented Prompt tuning and commented its unit tests

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Remove DataCollatorForSeq2Seq for tokenized dataset with packing

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Remove DataCollatorForSeq2Seq for tokenized dataset with packing

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Skipped PT tests

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* PR Changes

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* PR Changes

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Fix lint

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

* Remove enable_reduce_loss_sum and _is_peft_model check

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

---------

Signed-off-by: Abhishek <maurya.abhishek@ibm.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels