Skip to content

Fix for #2374#2379

Open
brownbaerchen wants to merge 4 commits into
mainfrom
bugs/2374-_Bug_ht_var_x_axis=0_ignores_ddof
Open

Fix for #2374#2379
brownbaerchen wants to merge 4 commits into
mainfrom
bugs/2374-_Bug_ht_var_x_axis=0_ignores_ddof

Conversation

@brownbaerchen

Copy link
Copy Markdown
Collaborator

See #2374 for the description of the problem reported by @maliesen.

Short recap: The function __moment_w_axis takes unbiased as input and forwards it to the respective torch function (var in this case). However, it only does so if unbiased is True. We need to forward unbiased=False as well, though.

The same happens with Fisher (see #2375) which may also be fixed by this PR. It remains to be checked, though.

Issue/s resolved: #2374

Copilot AI review requested due to automatic review settings July 13, 2026 12:33
@github-project-automation github-project-automation Bot moved this to Todo in Roadmap Jul 13, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a bug in axis-based moment computations where falsy boolean options (notably unbiased=False) were not forwarded to the underlying torch implementation, causing ht.var(..., axis=..., ddof=0) to behave as if Bessel correction was enabled. It also adjusts the same forwarding logic for Fischer in kurtosis/skew paths and adds a regression test for the reported variance edge case.

Changes:

  • Forward unbiased and Fischer through __moment_w_axis whenever they are provided (including False).
  • Add a regression test for ht.var(..., axis=0, ddof=0) on a single-element input (issue #2374).
  • Fix a minor docstring typo (“implicitely” → “implicitly”).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
heat/core/statistics.py Fixes boolean kwarg forwarding for axis-based moment helpers (unbiased, Fischer) and corrects a docstring typo.
tests/core/test_statistics.py Adds a regression test covering the ddof=0 + axis edge case from #2374.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread heat/core/statistics.py Outdated
Comment on lines 1304 to 1308
kwargs = {"dim": axis}
if unbiased:
if unbiased is not None:
kwargs["unbiased"] = unbiased
if Fischer:
if Fischer is not None:
kwargs["Fischer"] = Fischer
Comment thread heat/core/statistics.py Outdated
Comment on lines 1305 to 1308
if unbiased is not None:
kwargs["unbiased"] = unbiased
if Fischer:
if Fischer is not None:
kwargs["Fischer"] = Fischer
@codecov

codecov Bot commented Jul 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.74%. Comparing base (5d42619) to head (18639de).
⚠️ Report is 14 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2379   +/-   ##
=======================================
  Coverage   91.74%   91.74%           
=======================================
  Files          87       87           
  Lines       14120    14120           
=======================================
  Hits        12955    12955           
  Misses       1165     1165           
Flag Coverage Δ
unit 91.74% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Comment thread heat/core/statistics.py Outdated
Comment thread heat/core/statistics.py Outdated
# helper for calculating a statistical moment with a given axis
kwargs = {"dim": axis}
if unbiased:
if unbiased is not None:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unbiased is correction in torch.var since version 2.0 and works with integers. We will have to look what it is for the other functions.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also called correction in numpy. See the docs.
Scipy kurtosis and skew have bias, see docs.

I switched to correction internally, but left the interface of ht.skew and ht.kurtosis to keep unbiased. What do you think?

@mtar mtar Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should revert correction back for now. It is only in var and accepts ints. Maybe, change it to bias and flip the values.
Nonetheless, we should use the same parameters as in the other APIs for our public functions and update the calls to torch.var

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correction is the closest to the torch and numpy APIs, so why would be do something yet completely different?

@mtar mtar Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I don't see it close to numpys API.

Torch/NumPy:

  • correction is only used in var
  • correction takes ints and floats and is a array api compatible name for ddof.
  • skew and kurtosis use bias in scipy and are bools

PR:

  • correction is used everywhere
  • correction takes bools in var and it has a different logic underneath
  • skew and kurtosis also have the correction parameter name

Right now in the PR, it is nothing alike but the name in a single place.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will keep it for now.

@github-project-automation github-project-automation Bot moved this from Todo to In Progress in Roadmap Jul 13, 2026
@maliesen

maliesen commented Jul 13, 2026

Copy link
Copy Markdown

I don't know if this is the right place for it, but upon merging the fix we can now omit the four atol=1e-1 argument values in the assertions in tests/naive_bayes/test_gaussiannb.py. This is to reflect the change in the numerical values of ht.naive_bayes.GaussianNB:

self.assertTrue(ht.isclose(gnb_heat.sigma_, sklearn_sigma, atol=1e-1).all())
self.assertTrue(ht.isclose(y_pred_proba_sklearn, y_pred_proba_local, atol=1e-1).all())

self.assertTrue(ht.isclose(gnb_heat.sigma_, sklearn_sigma, atol=1e-1).all())

# self.assertTrue(ht.isclose(gnb_heat.sigma_, sklearn_sigma, atol=1e-1).all())

@brownbaerchen

Copy link
Copy Markdown
Collaborator Author

@maliesen, thanks for pointing this out! You are solving all of our bugs! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

[Bug]: ht.var(x, axis=0) ignores ddof

4 participants