Remove duplicated optimizer.zero_grad() lines

Author: ValerianRey

docs/source/examples/amp.rst

@@ -53,7 +53,6 @@ following example shows the resulting code for a multi-task learning use-case.
         scaler.step(optimizer)
         scaler.update()
         optimizer.zero_grad()
-        optimizer.zero_grad()
 
 .. hint::
     Within the ``torch.autocast`` context, some operations may be done in ``float16`` type. For

docs/source/examples/iwrm.rst

@@ -69,7 +69,6 @@ batch of data. When minimizing per-instance losses (IWRM), we use either autojac
 
                 optimizer.step()
                 optimizer.zero_grad()
-                optimizer.zero_grad()
 
         In this baseline example, the update may negatively affect the loss of some elements of the
         batch.
@@ -105,7 +104,6 @@ batch of data. When minimizing per-instance losses (IWRM), we use either autojac
 
                 optimizer.step()
                 optimizer.zero_grad()
-                optimizer.zero_grad()
 
         Here, we compute the Jacobian of the per-sample losses with respect to the model parameters
         and use it to update the model such that no loss from the batch is (locally) increased.
@@ -141,7 +139,6 @@ batch of data. When minimizing per-instance losses (IWRM), we use either autojac
                 losses.backward(weights)
                 optimizer.step()
                 optimizer.zero_grad()
-                optimizer.zero_grad()
 
         Here, the per-sample gradients are never fully stored in memory, leading to large
         improvements in memory usage and speed compared to autojac, in most practical cases. The

tests/doc/test_rst.py

@@ -47,7 +47,6 @@ def test_amp():
         scaler.step(optimizer)
         scaler.update()
         optimizer.zero_grad()
-        optimizer.zero_grad()
 
 
 def test_basic_usage():
@@ -122,7 +121,6 @@ def test_iwmtl():
         losses.backward(weights)
         optimizer.step()
         optimizer.zero_grad()
-        optimizer.zero_grad()
 
 
 def test_iwrm():
@@ -146,7 +144,6 @@ def test_autograd():
             loss.backward()
             optimizer.step()
             optimizer.zero_grad()
-            optimizer.zero_grad()
 
     def test_autojac():
         import torch
@@ -201,7 +198,6 @@ def test_autogram():
             losses.backward(weights)
             optimizer.step()
             optimizer.zero_grad()
-            optimizer.zero_grad()
 
     test_autograd()
     test_autojac()
@@ -399,7 +395,6 @@ def test_partial_jd():
         losses.backward(weights)
         optimizer.step()
         optimizer.zero_grad()
-        optimizer.zero_grad()
 
 
 def test_rnn():