feat(training): wire additional WGAN-GP + diffusion consumers#1848
Merged
ooples merged 1 commit intoJul 10, 2026
Merged
Conversation
… fused primitives Extends PR #1847 with three additional consumer wire-ups discovered during a comprehensive audit against the primitives added in PR #1843. Each consumer attempts the fused path first via the existing IFusedOptimizerSpec / TryMapToFusedOptimizerConfig plumbing and falls back to eager tape training when the fused path can't engage. ## WGAN-GP consumers (additional beyond #1845) * NeuralNetworks/WGANGP.cs — standalone WGAN-GP class. Previously used the legacy flat-vector round-trip (three Critic.Predict calls per step, GetParameterGradients + host-side vector combine + UpdateCriticWithOptimizer). TrainCriticBatchWithGP now attempts WganGpFusedStep.TryStep first, using Critic.Layers as the parameter source and a per-batch ε ~ U(0, 1) sampler matching each critic's local ComputeGradientPenalty behavior. Falls back to the legacy path when the critic's optimizer has no fused-kernel mapping. ## Diffusion consumers (additional beyond #1846) * NeuralNetworks/SyntheticData/FinDiffGenerator.cs — per-row DDPM training (Sattarov et al. 2023). TrainBatch caches a MultiSlotFusedStep across rows so the compiled plan is built once and replayed via slot-data refresh for subsequent rows. Slots: (packedDenoiserInput, targetNoise). Falls back to the existing Train(input, targetNoise) call on miss. * NeuralNetworks/SyntheticData/AutoDiffTabGenerator.cs — per-row DDPM training with a custom TapeStepOver optimizer step. Same MultiSlotFusedStep caching pattern as FinDiff. Slots: (denoiserInput, targetNoise). Falls back to the existing tape-based TapeStepOver path on miss. ## Explicitly not wired in this PR (documented) * NeuralNetworks/GenerativeAdversarialNetwork.cs — uses BCE loss with optional GP regularization (a separate auxiliary optimizer step), NOT Wasserstein + GP. Wiring WganGpFusedStep would change training semantics from BCE to Wasserstein. Requires a separate design decision. * Finance/Forecasting/Foundation/{CCDM,MGTSD,TSDiff,TimeDiff,TimeGrad}.cs and Finance/Probabilistic/DiffusionTS.cs — each uses .Data.Span host-side packs for the denoising input (same trace-freeze issue TFC/CSDI hit in PR #1843). Each needs a TFC/CSDI-scale traceable rewrite BEFORE fused wiring is safe. Deferred to individual per-consumer PRs. * MetaLearning/Algorithms/{MetaDDPMAlgorithm,MetaDMAlgorithm,MetaDiffAlgorithm}.cs — hand-rolled Vector<T> params with index arithmetic, NOT the ILayer/Engine/tape infrastructure. Would need full rewrite to fit MultiSlotFusedStep. * DiffusionModelBase subclasses (DDPMModel, DiffWaveModel, LatentDiffusionModelBase) — the SupportsFusedDenoising opt-in hook (added in PR #1847) is available, but flipping it safely requires per-class audit of PredictNoise → UNet traceability. Left as follow-up work; the mechanism is in place. ## Verification * net8.0, net471, net10.0 all build clean. * No API surface changes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
ooples
added a commit
that referenced
this pull request
Jul 11, 2026
…for fused paths) 0.113.0 publishes Tensors PR #763 — this branch's gating dependency. #763's key fix (4C): the compiled/persistent backward now honors createGraph=true (GradientTape. ComputeGradients previously gated the compiled path on !createGraph), so the WGAN-GP gradient penalty's inner backward differentiates into the disc weights through the fused compiled plan instead of silently returning zeros (issue #1844). Against 0.111.2 the fused GPU-resident WGAN-GP path would have silently degraded to plain WGAN. The src/Training fused-primitive mirrors (WganGpFusedStep, MultiSlotFusedStep, DpSgdFusedStep) stay as the consumer-side primitives that #1847/#1848 centralize every consumer through; they call the public engine API, so the bump alone routes them onto #763's fixed compiled backward. Also picks up #765 (compiled ReduceMax axis fill) and #764 (resident-param fused-Adam fix). Native OneDNN/OpenBLAS/CLBlast bumped in lockstep (all published). Builds green on net8.0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to PR #1847. A comprehensive audit surfaced additional WGAN-GP and diffusion consumers beyond the four named in issues #1845/#1846. This PR wires the ones that fit the primitives cleanly and documents the ones that need TFC/CSDI-scale rework first.
Summary
Wired (3 new consumers):
src/NeuralNetworks/WGANGP.cs— standalone WGAN-GP class. Was using the legacy flat-vector round-trip (GetParameterGradients+ host-side combine +UpdateCriticWithOptimizer). Now attemptsWganGpFusedStep.TryStepfirst withCritic.Layersas the parameter source.src/NeuralNetworks/SyntheticData/FinDiffGenerator.cs— per-row DDPM training (Sattarov et al. 2023).TrainBatchnow caches aMultiSlotFusedStepacross rows.src/NeuralNetworks/SyntheticData/AutoDiffTabGenerator.cs— same per-row pattern as FinDiff.TrainBatchcachesMultiSlotFusedStepslots(denoiserInput, targetNoise).Explicitly not wired (documented in commit message):
GenerativeAdversarialNetwork.cs— uses BCE + optional GP regularization, NOT Wasserstein + GP. Wiring WganGpFusedStep would change training semantics..Data.Spanhost-side packs (same trace-freeze issue TFC/CSDI had in feat(training): GPU-resident fused step for non-TS single-net models #1843). Each needs a TFC/CSDI-scale traceable rewrite BEFORE fused wiring is safe.Vector<T>params with index arithmetic, not ILayer/Engine/tape.SupportsFusedDenoisingopt-in hook from feat(training): centralize WGAN-GP + diffusion consumers through fused primitives #1847 is available; flipping it requires per-class audit ofPredictNoise→ UNet traceability.Test plan
dotnet build -f net8.0— cleandotnet build -f net471— cleandotnet build -f net10.0— cleanBase branch
Targets
feat/gpu-resident-nonts(PR #1843's branch, which already contains #1847's merge). Once #1843 merges to master, this PR will retarget to master.🤖 Generated with Claude Code