Hey @jcjohnson, first of all thank you for these, eternally thankful...
The issue-
4th paragraph Pytorch:Autograd -
"for example we usually don't want to backpropagate through the weight update steps when training a neural network"
This should be done when the network is being evaluated too right? At that time we don't want extra memory to be used(to keep track) if we aren't going to update the weights
Hey @jcjohnson, first of all thank you for these, eternally thankful...
The issue-
4th paragraph Pytorch:Autograd -
"for example we usually don't want to backpropagate through the weight update steps when training a neural network"
This should be done when the network is being evaluated too right? At that time we don't want extra memory to be used(to keep track) if we aren't going to update the weights