Skip to content

Commit f74108c

Browse files
committed
fix: return mu (not mu+std) in reparameterize at eval time
In VarAutoEncoder and VarFullyConnectedNet, the reparameterize method computed and always returned . At training time, was correctly replaced with random noise scaled by the standard deviation. But at eval time, the raw standard deviation was still added to the mean — giving instead of just . The reparameterization trick (Kingma & Welling, 2014) defines: - Training: z = μ + ε · σ (ε ~ N(0,1)) - Inference: z = μ (no stochastic component) This fix restructures the method to return directly when not training, avoiding the unnecessary computation and eliminating the incorrect result at inference. Fixes #8413 Signed-off-by: Raphael Malikian <rtmalikian@gmail.com>
1 parent 15f5073 commit f74108c

2 files changed

Lines changed: 8 additions & 10 deletions

File tree

monai/networks/nets/fullyconnectednet.py

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -172,12 +172,11 @@ def decode_forward(self, z: torch.Tensor, use_sigmoid: bool = True) -> torch.Ten
172172
return x
173173

174174
def reparameterize(self, mu: torch.Tensor, logvar: torch.Tensor) -> torch.Tensor:
175-
std = torch.exp(0.5 * logvar)
175+
if self.training: # reparameterization trick only during training
176+
std = torch.exp(0.5 * logvar)
177+
return mu + torch.randn_like(std) * std
176178

177-
if self.training: # multiply random noise with std only during training
178-
std = torch.randn_like(std).mul(std)
179-
180-
return std.add_(mu)
179+
return mu
181180

182181
def forward(self, x: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]:
183182
mu, logvar = self.encode_forward(x)

monai/networks/nets/varautoencoder.py

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -142,12 +142,11 @@ def decode_forward(self, z: torch.Tensor, use_sigmoid: bool = True) -> torch.Ten
142142
return x
143143

144144
def reparameterize(self, mu: torch.Tensor, logvar: torch.Tensor) -> torch.Tensor:
145-
std = torch.exp(0.5 * logvar)
145+
if self.training: # reparameterization trick only during training
146+
std = torch.exp(0.5 * logvar)
147+
return mu + torch.randn_like(std) * std
146148

147-
if self.training: # multiply random noise with std only during training
148-
std = torch.randn_like(std).mul(std)
149-
150-
return std.add_(mu)
149+
return mu
151150

152151
def forward(self, x: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]:
153152
mu, logvar = self.encode_forward(x)

0 commit comments

Comments
 (0)