Reproducing Results

[yasserkhalil93]

Hi,

Thank you for your interesting work and for sharing you code!

I was trying to reproduce your results for main_WeightAveraging_with_surgery using ViT-B-32, where you report an accuracy of "0.8000810002986893".

Every time I run the code I get different accuracies, yet they are close to 80%. For example, I get 79.4%, 79.2% and so on.

I only ran "python src/main_WeightAveraging_with_surgery.py" without changing any hyperparameter.

Please note that I am using your checkpoints and your processed data.

Why am I getting different accuracies for every run? How can I reproduce your exact results?

Thank you in advance.

[EnnengYang]

Hi,

It is normal for the results to vary slightly between runs, because a batch of data is randomly loaded when performing the optimization, and the parameters of the surgery module are also randomly initialized.

If you want to fix the results of each run, you need to fix the program's random seed.

Thank you.

[yasserkhalil93]

In order to be able reproduce your results we need the seed/s that you used to run the experiments. Would you please provide the seed used? or if you ran multiple runs what seeds you used?

Thank you.