PolySHAP approximator#559
Open
ThrashLion wants to merge 30 commits into
Open
Conversation
Forward-looking spec for the 3 new SV approximators (LeverageSHAP, PolySHAP, OddSHAP). Approximator classes are looked up dynamically by name, so the file auto-skips classes that have not yet been registered in shapiq.approximator. As each implementation lands, the corresponding parametrizations activate. - Interface conformance (always required): index='SV', n_players, max_order/min_order, values shape and dtype, interaction_lookup. - Numerical convergence vs ExactComputer (xfail strict=False): atol schedule by budget percentage. - Determinism: same (n, random_state, budget, game) -> bit-identical output. 75 tests, all currently SKIP on main. Will activate as classes land.
Honors the cross-method testing platform promised to the tutor:
unified harness covering every SV approximator in shapiq (the
existing 11 — KernelSHAP, SVARM, Permutation*, ProxySPEX, ... — and
the 3 new ones from this project) instead of only the new line-up.
Approximator list is sourced dynamically from
shapiq.approximator.SV_APPROXIMATORS (canonical registry) plus the
3 new project names, deduplicated. Future shapiq additions land in
the harness automatically.
Split into two scopes:
* test_interface_conformance — strict shape/dtype/index/lookup
contract from the API spec. Applied ONLY to the 3 new
approximators (the contract is ours; existing methods have
different default output conventions like ProxySPEX defaulting
to FBII and max_order=n).
* test_numerical_convergence_vs_exact + test_determinism — apply
to ALL SV approximators. Cross-method comparison against
ExactComputer ground truth on identical SOUM games. xfail with
strict=False so methods that do converge surface as XPASS;
methods still under development surface as XFAIL.
Two robustness helpers:
* _construct_or_skip — tries (n=, index='SV', max_order=1,
random_state=) first (covers multi-index methods like SPEX,
ProxySPEX, ProxySHAP, MSRBiased, kADDSHAP), then falls back to
minimal signature for SV-only methods (KernelSHAP, OwenSamplingSV).
* _safe_approximate — skips on ValueError raised by approximators
that explicitly refuse a regime (e.g. SPEX 'Insufficient budget
to compute the transform' at low budgets).
Results: 10 passed, 95 skipped, 90 xfailed, 23 xpassed. The 23
xpassed are existing shapiq SV methods that converge cleanly at
full budget on small SOUM — a useful baseline for the upcoming
benchmark report.
refactored PolySHAP+ExplanationFrontierGenerator into subclasses requiring no generator; registered all three subclasses; removed unexpected parameters; cleaner variable naming; included unedited efficient_sampling for now
Drop-in framework that any teammate can merge into their feature branch
to run head-to-head benchmarks against ExactComputer across every SV
approximator in shapiq, then plot the standard SHAP-literature metric
curves. No source files are modified — adds a top-level benchmark/
package, a single test file, and a small in-place test-helper sys.path
hook. Does not touch pyproject.toml or any other upstream config.
Files added:
* benchmark/__init__.py: makes the runner a proper Python package so
invocation is 'python -m benchmark.performance'.
* benchmark/_discovery.py: single source of truth for SV approximator
discovery + SV-mode construction. Holds:
- PROJECT_APPROXIMATOR_NAMES: LeverageSHAP, PolySHAP,
PolySHAPKAdd / Partial / Prior, OddSHAP.
- _SV_CONSTRUCT_OVERRIDES: per-class kwargs for non-standard
constructors (PolySHAP variants need max_order /
n_explanation_terms / q_prior).
- construct_for_sv(): three-stage construction (override ->
explicit SV signature -> minimal signature), returning
(estimator, exc) so the caller can report the most informative
exception. A ValueError from inside a matched signature wins
over a TypeError from a signature mismatch.
- safe_approximate(): catches ValueError and RuntimeError so
sparse approximators that refuse a budget regime (SPEX,
ProxySPEX, ...) skip the cell cleanly instead of crashing.
* benchmark/performance.py: CLI runner that consumes _discovery,
sweeps (method, game, budget, seed), records every cell in a
long-format CSV, and emits one PNG per (game, metric) plus a
runtime PNG. Seven metrics chosen from the union of LeverageSHAP,
PolySHAP, OddSHAP and shapiq.benchmark.metrics literature:
MSE / MAE / SSE / SAE / Precision@5 / Precision@10 / KendallTau.
Includes a '--check' interface-probe mode that prints a
constructibility table without running a sweep.
* benchmark/README.md: usage doc covering merge workflow, --check,
sweep CLI, output layout, CSV format, metric definitions, plot
conventions, and notes on the multi-index approximators that need
explicit (index='SV', max_order=1).
Files modified:
* tests/shapiq/tests_unit/tests_approximators/test_approximators_vs_exact.py:
now imports the shared helpers from benchmark._discovery via a
tightly-scoped sys.path hook at the top of the file. Picks up the
ValueError-priority construction and the RuntimeError-catch that
the test file previously did not have. Interface conformance is
now applied to the project's six new approximator names
(LeverageSHAP, PolySHAP + 3 variants, OddSHAP), so Matthias's
PolySHAP variants are no longer silently skipped by the contract
check.
Verified locally:
* pytest test_approximators_vs_exact.py: 10 passed, 170 skipped,
87 xfailed, 26 xpassed. No failures.
* python -m benchmark.performance --check: surfaces all 17 method
names (11 existing on main + 6 project additions) correctly.
* Drop-in compatibility verified by temporary merge into all three
feature branches (oddshap_approximator, leverageSHAP, PolySHAP) —
clean merge in each, --check picks up the local approximator.
Removed for lack of generalization
Owner
|
There are some issues with the Changelog and the code-quality. So the CI will not run through for evaluating this PR at this point. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
mmschlk
requested changes
Jul 14, 2026
mmschlk
left a comment
mmschlk
left a comment
Owner
There was a problem hiding this comment.
Thank you for your work on PolySHAP. I do not see any issues with the core implementation here (aka. PolySHAP). The only thing that I am not 100% happy about is the creation of specialized classes for variants of PolySHAP. I think it would have been a bit better to specify the variant of PolySHAP with a parameter in the classes constructor. This parameter could take literals ("prior", "partial", "kadd") and potentially other parameters as well that then offer the specialized behavior. Note, however that this point is rather minor.
Author
|
The variants are now unified in a single PolySHAP class, placed alongside the other regression approximators in regression/polyshap.py; only PolySHAP is exported and registered. The new constructor is well explained and should be easy to use. The mode follows from which argument is passed: max_order for the k-additive frontier (max_order=1 recovers KernelSHAP), max_terms for the budget-controlled partial frontier, and prior_frontier for an explicit frontier. For the unbounded behavior of former PolySHAPPartial set max_order=n. Each mode is documented in the constructor docstring, and passing prior_frontier together with max_order, max_terms, or sizes_to_exclude raises a ValueError. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Adds PolySHAP (Fumagalli, Witter & Musco, PolySHAP: Extending KernelSHAP with Interaction-Informed Polynomial Regression, ICLR 2026, arXiv:2601.18608) as a first-class Shapley-value approximator in
shapiq.PolySHAP generalizes KernelSHAP: instead of fitting a purely linear surrogate, it fits a k-additive polynomial surrogate of the game (capturing interactions up to a chosen order) and reads the Shapley values off it.
max_order=1recovers KernelSHAP exactly; higher orders improve estimation quality when the game has real interactions. A singlePolySHAPclass provides three frontier schemes, selected by which constructor argument is passed:max_order— full k-additive frontier up to that order.max_terms— budget-controlled partial frontier.prior_frontier— user-supplied set of interaction terms.The integration is purely additive: a self-contained
shapiq.approximator.regression.polyshapmodule plus registration. No shared library internals are modified (interaction_values.py,indices.py, regressionbase.py,owen.pyare byte-identical tomain).Public API Changes
Additive only — one new class exported from
shapiq.approximator:PolySHAP, registered inSV_APPROXIMATORS. No existing signatures or behavior change. The constructor takes its options as keyword-only arguments (max_order,max_terms,sizes_to_exclude,prior_frontier,pairing_trick,sampling_weights,random_state), consistent with the other approximators.How Has This Been Tested?
tests/shapiq/tests_unit/tests_approximators/test_approximator_polyshap.py(68 tests): frontier construction for all three modes,ValueErrorvalidation paths, exact Shapley recovery againstExactComputeronDummyGame, determinism/reproducibility, a 50-seed variance and convergence sweep, and full-enumeration exactness.tests/shapiqsuite passes (1319 passed, 12 skipped, 54 xfailed, 0 failed).ruff checkandruff formatare clean on all added/changed files.examples/approximators/plot_polyshap.pyexecutes end-to-end.Checklist
CHANGELOG.md(if relevant for users).Notes