Describe the task. It can be a feature, documentation, etc.
We want to assess the impact of different flash attention versions on throughput and memory consumption for different token size configurations.
The hope is for us to be able to do this in a sandboxed script, shielding much of WeatherGenerator's complexity. At the same time, our tests must reflect on the token configurations distributions in WeatherGenerator models or, if not, this limitation must be kept in mind.
Sandboxed environments in which we abstract away parts of the WG codebase would also be helpful for other performance-related investigations, e.g., testing other model sharding strategies.
Hedgedoc URL, if you are keeping notes, plots, logs in hedgedoc.
No response
Area
Describe the task. It can be a feature, documentation, etc.
We want to assess the impact of different flash attention versions on throughput and memory consumption for different token size configurations.
The hope is for us to be able to do this in a sandboxed script, shielding much of WeatherGenerator's complexity. At the same time, our tests must reflect on the token configurations distributions in WeatherGenerator models or, if not, this limitation must be kept in mind.
Sandboxed environments in which we abstract away parts of the WG codebase would also be helpful for other performance-related investigations, e.g., testing other model sharding strategies.
Hedgedoc URL, if you are keeping notes, plots, logs in hedgedoc.
No response
Area