Standardize _get_scores() return type across all query strategies#197
Merged
Conversation
CoolJosh0221
added a commit
to CoolJosh0221/libact
that referenced
this pull request
Jul 7, 2026
HintSVM.make_query selected the unlabeled point FARTHEST from the hinted
decision boundary (argmax|f(x)|), the opposite of what the algorithm intends.
Li, Ferng & Lin, "Active Learning with Hinted Support Vector Machine" (ACML
2012, Algorithm 1 Query step / §4.2) query the instance CLOSEST to the query
boundary, i.e. min|f(x)| — standard uncertainty sampling against the hinted
classifier. The old orientation also contradicted the _get_scores contract
("higher = more informative").
The bug dates to the original 2015 port (12305e6); PR ntucllab#197 (cac4c1e) only
repackaged make_query into _get_scores and, misleadingly, documented the
inverted orientation as intended.
Fix: negate the score (scores = -abs(decision_value)) so higher = closer to
the boundary = more informative; make_query keeps argmax and therefore now
selects min|f(x)|. This is an intentional query-behavior change and also
affects strategies that build on HintSVM (ActiveLearningByLearning as a
sub-strategy; DensityWeightedMeta composition).
Tests:
- test_hintsvm.py: add a skip guard so the module collects without the C
extension; assert argmax(scores) == make_query()'s pick (RNG re-seeded so
both calls see the same hint pool); assert direction — the on-boundary point
out-scores deep-in-cluster points and scores are <= 0 (p=0 isolates the
score direction from the hint-sampling mechanism). Real labels are -1/+1
because the C extension reserves label 0 as a hint marker.
- test_realdata.py: update the HintSVM golden query sequence, which encoded
the old (farthest-first) behavior, to the corrected deterministic sequence.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Standardize
_get_scores()to return(np.ndarray, np.ndarray)across all strategiesMotivation
_get_scores()had inconsistent return types across strategies —zipiterators,list(zip(...)), or not implemented at all. This blocks future plans for supporting batch-mode querying and ALBL soft-advice vectors, which need a uniform interface to consume scores from any strategy.Changes
Every
_get_scores()now returns(entry_ids: np.ndarray, scores: np.ndarray)where higher = more informative. Empty pools return(empty_int_array, empty_float_array). The base class default raisesNotImplementedError._get_scores()implementations (QBC, QUIRE, RandomSampling)NotImplementedError(VarianceReduction, ALBL)Backward compatibility
No public API changes.
make_query()still returnsint.return_score=Truestill returns(ask_id, list(zip(...))).Tests
24 new contract tests in
test_get_scores.pycovering return types, empty pools, andreturn_scorecompatibility. 3 existing test files updated for the new tuple format. 90/90 pass (11 pre-existing sklearn 1.8multi_classfailures are unrelated).