Many RL techniques involve some kind of weight averaging. For example, SDFT's continual learning uses Exponential Moving Average (EMA) for the teacher.
While Tinker doesn't provide weight manipulations (eg polyak averaging, EMA), many RL algorithms are impossible or severely worsened, forcing users back to running their own training setups. Could you please include them?
Many RL techniques involve some kind of weight averaging. For example, SDFT's continual learning uses Exponential Moving Average (EMA) for the teacher.
While Tinker doesn't provide weight manipulations (eg polyak averaging, EMA), many RL algorithms are impossible or severely worsened, forcing users back to running their own training setups. Could you please include them?