Skip to content

Commit c8e6afb

Browse files
committed
Add logit scale parameter options to ContrastivePretraining docstring
1 parent dc264de commit c8e6afb

1 file changed

Lines changed: 9 additions & 0 deletions

File tree

mmlearn/tasks/contrastive_pretraining.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,15 @@ class ContrastivePretraining(L.LightningModule):
112112
a `scheduler` key that specifies the scheduler and an optional `extras` key
113113
that specifies additional arguments to pass to the scheduler. If not provided,
114114
the learning rate will not be adjusted during training.
115+
init_logit_scale : float, optional, default=1 / 0.07
116+
The initial value of the logit scale parameter. This is the log of the scale
117+
factor applied to the logits before computing the contrastive loss.
118+
max_logit_scale : float, optional, default=100
119+
The maximum value of the logit scale parameter. The logit scale parameter
120+
is clamped to the range [0, log(max_logit_scale)].
121+
learnable_logit_scale : bool, optional, default=True
122+
Whether the logit scale parameter is learnable. If set to False, the logit
123+
scale parameter is treated as a constant.
115124
loss : CLIPLoss, optional, default=None
116125
The loss function to use.
117126
modality_loss_pairs : List[LossPairSpec], optional, default=None

0 commit comments

Comments
 (0)