File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -112,6 +112,15 @@ class ContrastivePretraining(L.LightningModule):
112112 a `scheduler` key that specifies the scheduler and an optional `extras` key
113113 that specifies additional arguments to pass to the scheduler. If not provided,
114114 the learning rate will not be adjusted during training.
115+ init_logit_scale : float, optional, default=1 / 0.07
116+ The initial value of the logit scale parameter. This is the log of the scale
117+ factor applied to the logits before computing the contrastive loss.
118+ max_logit_scale : float, optional, default=100
119+ The maximum value of the logit scale parameter. The logit scale parameter
120+ is clamped to the range [0, log(max_logit_scale)].
121+ learnable_logit_scale : bool, optional, default=True
122+ Whether the logit scale parameter is learnable. If set to False, the logit
123+ scale parameter is treated as a constant.
115124 loss : CLIPLoss, optional, default=None
116125 The loss function to use.
117126 modality_loss_pairs : List[LossPairSpec], optional, default=None
You can’t perform that action at this time.
0 commit comments