Implement TextImputer with Flexible Players, Perturbations and Target Callables

[samoger]

Motivation and Context

This PR implements a generalized TextImputer that integrates naturally into the existing shapiq imputation pipeline and enables coalition-based explanations for a broad range of NLP models through a modular architecture.

The implementation is built around three independent components:

• Player strategies — define the granularity of explanation (subwords, words, named entities, syntactic chunks, and sentences).
• Perturbation strategies — specify how missing players are represented, including [MASK], [PAD], removal, neutral replacement, WordNet-based replacement, MLM infilling, and attention masking.
• Target callables — provide model-specific scoring interfaces for encoder classifiers, causal language models, and encoder-decoder (Seq2Seq) models.

In addition, the implementation includes batched coalition evaluation for efficient model inference and integrates directly with HuggingFace transformers.

Compared with a model-specific implementation, this design cleanly separates player definitions, perturbation methods, and model architectures, making the TextImputer extensible to future NLP models and perturbation strategies while remaining compatible with the existing shapiq workflow.

---

Public API Changes

• Yes, Public API changes (backwards compatible)
  TextImputer follows the same interface as other shapiq imputers: it is initialized once with a model and an input text, then evaluated by passing coalition matrices directly (imputer(coalitions)).

Constructor

from shapiq.imputer.text_imputer import TextImputer
imputer = TextImputer(
    model=model,
    tokenizer=tokenizer,
    text=text,
    player_level="word",
    perturbation_type="mask",
    model_type="encoder_classifier",
)

##Supported player definitions

player_level="subword"
player_level="word"
player_level="named_entity"
player_level="chunk"
player_level="sentence"

##Supported perturbation strategies

perturbation_type="mask"
perturbation_type="pad"
perturbation_type="removal"
perturbation_type="neutral"
perturbation_type="wordnet_neutral"
perturbation_type="mlm_infilling"
perturbation_type="attention_mask"

##Supported model interfaces

model_type="encoder_classifier"
model_type="causal_lm"
model_type="seq2seq"

---

##Optional configuration

# Encoder classifiers
class_idx=1
output_type="logit"

# Causal LMs
target_label="good"
prompt_template="Review: {text}\n\nSentiment:"

# Seq2Seq
normalize_target_logprob=True

# MLM infilling
mlm_model_name="bert-base-uncased"
mlm_num_samples=100

How Has This Been Tested?

Implemented three dedicated test suites covering the TextImputer components:

• test_text_imputer.py

  • Player strategies
  • Perturbation strategies
  • Target callables
  • TextImputer orchestration
  • Factory functions
  • Error handling

• test_text_imputer_seq2seq.py

  • Seq2Seq target callable
  • Target-sequence scoring
  • Teacher-forcing evaluation

• test_text_imputer_attention.py

  • Attention-mask perturbation
  • Attention-mask input construction
  • Encoder-classifier support
  • Causal-LM support
  • Seq2Seq support

---

Checklist

• The changes have been tested locally.
• Documentation has been updated (if the public API or usage changes).
• An entry has been added to CHANGELOG.md (if relevant for users).
• The code follows the project's style guidelines.
• I have considered the impact of these changes on the public API.

---

[samoger]

Hi, we have now addressed all of the review comments and pushed the corresponding changes to the PR. We also replied to each comment individually to briefly explain the changes we made.

The PR should now be ready for another review. Thank you.

[samoger]

Hi, We have fixed the CI issue from the previous run, so the import errors should now be resolved and the CI tests can be run again. Thank you!

[XWTX42]

Hi! The NLTK WordNet issue has been fixed. The CI tests should pass now, so could you please rerun them? Thanks!

[codecov]

Codecov Report

❌ Patch coverage is 84.90566% with 112 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/shapiq/imputer/text/perturbations.py	86.30%	23 Missing ⚠️
src/shapiq/imputer/text/tensor_perturbation.py	80.70%	22 Missing ⚠️
src/shapiq/imputer/text/imputer.py	84.44%	21 Missing ⚠️
src/shapiq/imputer/text/callables.py	87.87%	20 Missing ⚠️
src/shapiq/imputer/text/players.py	87.32%	18 Missing ⚠️
src/shapiq/imputer/__init__.py	16.66%	5 Missing ⚠️
src/shapiq/imputer/text/_error.py	0.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[samoger]

Hi, we have fixed the issues from the previous CI test. Could you please run the tests again? Thanks!