Add patience to reduce LR as CTM argument

[drodrigues0]

• Contextualized Topic Models version: 2.5.0
• Python version: 3.9.16
• Operating System: Ubuntu 20.04.6 LTS

Description

Currently, ReduceLROnPlateau function for CTM has a high hard-coded value for patience. So, whenever you want the LR to decrease during training we need to set the EarlyStopping patience to some value higher than 10, otherwise the train stops before the LR is actually decreased.

This is impractical because, waiting 10 epochs for the LR to decrease is a lot.

Code line

Propose to fix it

class CTM:

    # ...

    def __init__(
        self,
        # ...
        reduce_on_plateau=False,
        lr_patience: int = 3
        # ...

# init lr scheduler
if self.reduce_on_plateau:
    self.scheduler = ReduceLROnPlateau(self.optimizer, patience=sefl.lr_patience)

[vinid]

Hello! do you want to send a PR to add this?

…

On Mon, Aug 28, 2023 at 8:40 AM Diogo Rodrigues ***@***.***> wrote: - Contextualized Topic Models version: 2.5.0 - Python version: 3.9.16 - Operating System: Ubuntu 20.04.6 LTS Description Currently, ReduceLROnPlateau function for CTM has a high hard-coded value for patience. So, whenever you want the LR to decrease during training we need to set the EarlyStopping patience to some value higher than 10, otherwise the train stops before the LR is actually decreased. This is impractical because, waiting 10 epochs for the LR to decrease is a lot. Code line <https://github.com/MilaNLProc/contextualized-topic-models/blob/5dd3238dd3a49e8a792163b02d4c3f3b5deaae9f/contextualized_topic_models/models/ctm.py#L154C13-L154C76> Propose to fix it class CTM: # ... def __init__( self, # ... reduce_on_plateau=False, lr_patience: int = 3 # ... — Reply to this email directly, view it on GitHub <#141>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARBSSZZGUSBZREYG5VLTY3XXQ4PJANCNFSM6AAAAAA4A57AEA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>