Thank you so much for your implementation! I have enjoyed reading your code.
I'm playing around with the method, but unfortunately I don't have access to much compute at the moment. Since it seems you've done a good amount of experimentation, I figured that I would try asking you a few questions about the method. Thank you so much in advance!
- How long does it take to run a model on CIFAR-10 on a single GPU (e.g. a 2080TI)? From a quick estimates, it looks like it should take around 6 min/epoch * 1000 epochs = 6000 minutes = 4 days. Does this match what you observe?
- Do you know how long does it take to obtain a "reasonably-good" result (e.g. 91% for CIFAR-10@250 or 65% for CIFAR-100@2500?
- Based on your experimentation, how important is the EMA of model parameters? If you were to evaluate without the EMA, do you know (approximately) what accuracy you would get?
- Based on your experimentation, do you have a sense of how sensitive the model is to hyperparameter changes? Is it relatively robust, or will performance drop drastically if a few are changed?
Thank you so much for your implementation! I have enjoyed reading your code.
I'm playing around with the method, but unfortunately I don't have access to much compute at the moment. Since it seems you've done a good amount of experimentation, I figured that I would try asking you a few questions about the method. Thank you so much in advance!