fix: add 1/n_samples normalization to logistic regression gradient

[Mukller]

Bug Fix: Missing 1/n_samples normalization in logistic regression gradient

The gradient update for logistic regression is missing the 1/n_samples normalization factor.
This makes the effective learning rate scale with the dataset size, causing inconsistent
training behavior across different dataset sizes.

Before (wrong — gradient not normalized by batch size):

self.param -= self.learning_rate * -(y - y_pred).dot(X)

After (fixed):

n_samples = X.shape[0]
self.param -= self.learning_rate * (1 / n_samples) * -(y - y_pred).dot(X)

The correct binary cross-entropy gradient is (1/n) * X.T @ (y_pred - y).
Note: The same fix was already applied to Regression.py in the codebase.

Closes #124

[Mukller]

Code Review: add 1/n_samples normalization to logistic regression gradient

Summary

Adds the missing 1 / n_samples normalization factor to the logistic regression gradient update. Without it, the effective learning rate scales linearly with dataset size: a dataset with 1000 samples would produce gradients 1000× larger than one with 1 sample. This makes the learning rate a dataset-size-dependent hyperparameter rather than an intrinsic one.

Critical Issues

#	File	Line	Issue	Severity
1	logistic_regression.py	~40	Gradient -(y - y_pred).dot(X) is not normalized by n_samples, making the effective step size proportional to dataset size	🔴 Critical

What the Fix Does

# Before (unnormalized gradient)
self.param -= self.learning_rate * -(y - y_pred).dot(X)

# After (normalized by sample count)
n_samples = X.shape[0]
self.param -= self.learning_rate * (1 / n_samples) * -(y - y_pred).dot(X)

The gradient of cross-entropy loss for logistic regression is:
∇J = (1/m) * Xᵀ(ŷ - y)

The 1/m factor ensures the gradient magnitude is independent of dataset size, allowing the same learning rate to work across datasets of different sizes.

What Looks Good

• Clean, minimal addition of the standard normalization
• Consistent with how the regularized Newton's method branch likely handles this

Verdict

Request Changes → now fixed. This normalization bug would require manually re-tuning the learning rate for every dataset size change.