Skip to content

[Perf] Improve FP8 dispatch storage release#43

Open
A-nnonymous wants to merge 2 commits into
PFCCLab:paddlefrom
A-nnonymous:panzhaowu/fp8-dispatch-storage-release
Open

[Perf] Improve FP8 dispatch storage release#43
A-nnonymous wants to merge 2 commits into
PFCCLab:paddlefrom
A-nnonymous:panzhaowu/fp8-dispatch-storage-release

Conversation

@A-nnonymous

Copy link
Copy Markdown
Collaborator

Summary

  • enable fused y1 quant handoff in the FP8 gated path
  • decouple down-projection backward from BF16 w2 storage in FP8 aligned mode
  • update ernie compat FP8 weight attributes and call signatures to match functional APIs

Validation

  • python -m py_compile sonicmoe/functional/__init__.py sonicmoe/ernie_compat/mlp_node_v2.py
  • PaddleFleet import smoke: paddlefleet_ops.is_sonic_moe_available() == True, bare sonicmoe evicted from sys.modules
  • 20-step PaddleFormers SonicMoE FP8 smoke completed on 8 GPUs with no Traceback/OOM
  • 300-step run before final cleanup completed with stable current memory (~16.986 GiB) and global peak 79.953 GiB vs BF16 85.731 GiB

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@A-nnonymous

Copy link
Copy Markdown
Collaborator Author

Added a minimal correctness fix in _DownProjection: restore ctx._topk_scores_needs_grad = True so Paddle torch-proxy does not silently drop router/topk score gradients.

Validation on Yichen-aligned 200-step configs (stopped after step75/80 as requested):

  • BF16 Sonic step75 loss: 4.87523794 vs Yichen BF16 baseline 4.91020107
  • FP8 Sonic offload step75 loss: 4.8631115
  • Perf sanity: BF16 step75 runtime 3.4606s; FP8 step75 runtime 3.5747s with max_memory_allocated 111.0GB, so no material perf regression observed.

This intentionally keeps the diff to one source file / one logic change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant