Skip to content

Commit cc4db21

Browse files
committed
Revamp autojac section
1 parent b217983 commit cc4db21

1 file changed

Lines changed: 14 additions & 5 deletions

File tree

README.md

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -126,11 +126,20 @@ Here is how to change a standard multi-task training loop to use Jacobian descen
126126

127127
### The `autojac` engine
128128

129-
The [`autojac` engine](https://torchjd.org/stable/docs/autojac/) provides fine-grained control
130-
over Jacobian computation and aggregation. It lets you compute Jacobians with respect to specific
131-
layers or activations (partial Jacobian descent), store them in `.jac` fields for inspection, and
132-
apply any aggregator independently. See the [autojac examples](https://torchjd.org/stable/examples/)
133-
for more details.
129+
The [`torchjd.autojac` engine](https://torchjd.org/stable/docs/autojac/) provides a way to compute
130+
Jacobians (generally of the losses with respect to the parameters). Its interface is very similar
131+
to that of [`torch.autograd`](https://docs.pytorch.org/docs/stable/autograd):
132+
[`autojac.jac`](https://torchjd.org/stable/docs/autojac/jac) is analog to
133+
[`autograd.grad`](https://docs.pytorch.org/docs/stable/generated/torch.autograd.grad) but
134+
returns Jacobians insteads of gradients, and
135+
[`autojac.backward`](https://torchjd.org/stable/docs/autojac/backward) is analog to
136+
[`autograd.backward`](https://docs.pytorch.org/docs/stable/generated/torch.autograd.backward)
137+
but accumulates Jacobians in the `.jac` fields of parameters instead of gradients in the `.grad`
138+
fields (these Jacobians can then be aggregated into gradients and moved to the `.grad` fields by
139+
calling [`autojac.jac_to_grad`](https://torchjd.org/stable/docs/autojac/jac_to_grad). Lastly, the
140+
[`mtl_backward`](https://torchjd.org/stable/docs/autojac/mtl_backward) function can be used for
141+
multi-task learning to compute and accumulate gradients with respect to task-specific parameters
142+
and Jacobians with respect to shared parameters.
134143

135144
### The `autogram` engine
136145

0 commit comments

Comments
 (0)