Skip to content

New Multi-Closure test analysis module#2272

Merged
scarlehoff merged 33 commits into
masterfrom
factorise_internal_mct_loader
Nov 11, 2025
Merged

New Multi-Closure test analysis module#2272
scarlehoff merged 33 commits into
masterfrom
factorise_internal_mct_loader

Conversation

@comane
Copy link
Copy Markdown
Member

@comane comane commented Feb 4, 2025 •
edited
Loading

addresses #2159

The main goals of this PR are:

  • multiclosure analysis modules are reviewed and rewritten based on the new findings of the inconsistent closure tests paper.
  • All functions / helpers used for computation of bias with exp. covmat are removed.
  • All functions / helpers for computation of variance are removed.
  • Removed vp-comparefits --closure

@comane comane changed the title [WIP] Polish Closure test module New Multi-Closure test analysis module Feb 9, 2025
@comane comane requested review from RoyStegeman and scarlehoff March 9, 2025 21:31
@RoyStegeman RoyStegeman requested a review from jacoterh March 10, 2025 09:17
@scarlehoff
Copy link
Copy Markdown
Member

scarlehoff commented Apr 8, 2025

@jacoterh will you review this one?

@jacoterh
Copy link
Copy Markdown
Collaborator

jacoterh commented Apr 8, 2025

Yes, sorry about the delay - I will take a closer look

@jacoterh
Copy link
Copy Markdown
Collaborator

jacoterh commented Apr 9, 2025

Hi @comane, many thanks for this. Would you mind updating the corresponding documentation on the website as well? Specifically, could you make sure what is written on https://github.com/NNPDF/nnpdf/blob/3556b8f13f8284a6ba06fd82f1b0541080672370/doc/sphinx/source/tutorials/closureestimators.rst is still up to date? For example, the --closure option was removed, while the documentation mentions it.

@scarlehoff
Copy link
Copy Markdown
Member

scarlehoff commented Apr 28, 2025

What is the status of this one?

@comane
Copy link
Copy Markdown
Member Author

comane commented Apr 28, 2025

What is the status of this one?

I wanted #2258 to be merged first. Now will have a final look at it. However, it should be basically ready to go.

@comane comane force-pushed the factorise_internal_mct_loader branch from 5ac750b to 61c33e8 Compare June 12, 2025 14:40
@scarlehoff
Copy link
Copy Markdown
Member

scarlehoff commented Oct 7, 2025

What is the status of this one?

What is the status of this one?

comane added 18 commits November 10, 2025 15:49
@comane @scarlehoff
first pass for removal of unused functions
23c18c1
@comane @scarlehoff
.
b7dfddb
@comane @scarlehoff
added t0set to loader for it to be produced and used in closure checks
4efbb57
@comane @scarlehoff
added typing to multiclosure loaders
6e16d42
@comane @scarlehoff
Renamed PCAInternalMulticlosureLoader to RegularizedMulticlosureLoader
54e269b
@comane @scarlehoff
renamed internal_multiclosure_data_loader to multiclosure_data_loader
47630a0
@comane @scarlehoff
renamed internal_multiclosure_data_loader_pca to regularized_multiclo…
c38dff1 
…sure_data_loader
@comane @scarlehoff
RegularizedMulticlosureLoader inherits from MulticlosureLoader
3dce735
@comane @scarlehoff
compute mean covmat from replicas inside the multiclosure dataset loa…
9fc4bdd 
…der function
@comane @scarlehoff
avoid performing cholesky decomposition on a diagonal matrix
5f7cc68
@comane @scarlehoff
added diagonal entries attribute to RegularizeMulticlosureLoader to a…
def1316 
…llow for proper regularization of covariance matrix in the full-dataset case
@comane @scarlehoff
added new function to compute bias and apposite module for bootstrap …
1436c12 
…related stuff
@comane @scarlehoff
added mean covmat function
30fda80
@comane @scarlehoff
moved bootstrap functions to resp module and added todos
ba7a051
@comane @scarlehoff
filled bootstrap module
08cdb4e
@comane @scarlehoff
added multiclosure_bootstrap to init file
098d970
@comane @scarlehoff
renamed and corrected functions in multiclosure_inconsistent_output
d199544
@comane @scarlehoff
changed the report structure
4d8dded
comane and others added 15 commits November 10, 2025 15:49
@comane @scarlehoff
changed names in example runcard
c8083a9
@comane @scarlehoff
moved inconsistent closure test module to closure tests
8a15a2a
@comane @scarlehoff
moved some of the functions from multiclosure inconsistent to multicl…
69e7c90 
…osure output
@comane @scarlehoff
added docstrings to multiclosure
cea0a8a
@comane @scarlehoff
remove unused runcard param
21d46fc
@comane @scarlehoff
fixed docstrings and removed unused variables
b876822
@comane @scarlehoff
fixed buggy behaviour for n_comp = 1
26644f0
@comane @scarlehoff
polished closure results, plots and reports from all closure estimato…
57b63af 
…rs (bias and variance) that were computed using the experimental covmat rather than the regularized PDF covmat
@comane @scarlehoff
removed compareclosure test
5086ca3
@scarlehoff
polished single data point plotting tools
79cc8a8
@scarlehoff
removed import ipdb
460a70f
@scarlehoff
changed labels in single data point plots
570f1ee
@comane @scarlehoff
removed --closure from vp-comparefits parser
874747e
@comane @scarlehoff
plot Rb vs 1-lambda
7111720
@comane @scarlehoff
adjusted patch in inconsistent_ct
035d799
@scarlehoff scarlehoff force-pushed the factorise_internal_mct_loader branch from a9729fb to 035d799 Compare November 10, 2025 14:49
scarlehoff
scarlehoff approved these changes Nov 11, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@scarlehoff scarlehoff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging this and hoping for the best.

@scarlehoff scarlehoff merged commit 0001fcc into master Nov 11, 2025
12 of 13 checks passed
@scarlehoff scarlehoff deleted the factorise_internal_mct_loader branch November 11, 2025 08:08
@scarlehoff scarlehoff mentioned this pull request Nov 11, 2025
Merged
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@scarlehoff scarlehoff scarlehoff approved these changes

@RoyStegeman RoyStegeman Awaiting requested review from RoyStegeman

@jacoterh jacoterh Awaiting requested review from jacoterh

Assignees

No one assigned

Labels

closure tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@comane @scarlehoff @jacoterh