Skip to content

[feat] Upstream Attn-QAT Video Diffusion Code#1225

Open
RandNMR73 wants to merge 203 commits intomainfrom
sync-branch
Open

[feat] Upstream Attn-QAT Video Diffusion Code#1225
RandNMR73 wants to merge 203 commits intomainfrom
sync-branch

Conversation

@RandNMR73
Copy link
Copy Markdown
Collaborator

Summary

  • Add Attn-QAT training kernels
  • Add flashinfer NVFP4 linear layers (currently hardcoded for Wan-2.1 arch)
  • Add Modified SageAttention3 kernels in the fastvideo-kernel package
  • Add Attn-QAT video model training scripts

Remove explicit_package_bases setting from mypy configuration
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope: attention Attention backends (VSA, STA, Flash, etc.) scope: data Data preprocessing, datasets scope: distributed SP, FSDP, USP, multi-node scope: docs Documentation scope: inference Inference pipeline, serving, CLI scope: infra CI, tests, Docker, build scope: kernel CUDA kernels, fastvideo-kernel scope: model Model architecture (DiTs, encoders, VAEs) scope: training Training pipeline, methods, configs scope: ui Job Runner UI type: feat New feature or capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.