Improve Feature Importance Support in Sklearn Khiops Estimators

[popescu-v]

Description

Currently, KhiopsSupervisedEstimator provides the feature_evaluated_importances_ and feature_used_importances_ attributes, which do not match the expected vanilla Scikit-learn estimators' feature_importances_ attribute.

Questions/Ideas

• A solution consists in adding this attribute and set it to an array to the variables' importance attribute for each of the selected variables in the model.
  • Also, the feature_evaluated_importances_ and feature_used_importances_ with the level and weight should be removed, as level and weight are specific to the analysis process, rather than to the model per se.
  • Also, the documentation should be updated to explain that, in v11, the importances are calculated as the average of the exact Shapley values of each feature on the training data.

[popescu-v]

After further investigation and trials, it results that:

• as Khiops uses part of the features in the input dataset (after feature selection in the preprocessing phase), plus the constructed features (feature pairs, trees, multi-table features or features derived via rules), importances (as averaged Shapley values over the training dataset) are computed for these used features, which only partially overlap with the input features.
• as a result, it is impossible to provide, in practice and in general, a feature_importances_ estimator attribute that abides by the expectations of Scikit-learn, i.e. contain importances of all the input features and only those.
• consequently, providing such an attribute would only make sense in very particular cases:
  • using only monotable training datasets (which would preclude the application of multi-table specific feature construction rules);
  • forbidding the construction of variable pairs, trees, and text features; while this is possible, this would deny much of the strength of the Khiops models, resulting in potentially subpar predictors (with respect to the achievable potential);
• as this would seem very limiting and rather confusing, it seems better not to provide the feature_importances_ attribute and to rely on the model_report_ attribute instead, in order to retrieve the importances of the variables that we wish.

Hence the way forward seems to be to:

• remove feature_evaluated_importances_, feature_used_importances_ attributes (along with feature_evaluated_names_ and feature_used_names_) which are not expected by Scikit-learn;
• add the code used to compute feature_used_importances_ as a new sample in the sklearn samples (khiops.samples.samples_sklearn) which would illustrate the usage of the model_report_ attribute.

[folmos-at-orange]

I think this argument should be summarized somewhere in the docs/docstrings.

[popescu-v]

Latest updates:

• all feature_*_importances_ are dropped;
• all evaluated and used feature names, numbers and (levels, weights) are dropped;
• documentation is updated and fixed where applicable;
• sklearn estimator's model_report_ attribute usage would be illustrated in 2 Q&As (separate from this issue):
  1. Using the Core API to use train_recoder in order to "flatten" a multi-table dataset, then train_predictor to build an SNB predictor, then to set as "unused" in the encoder model the variables that are "unused" variables in the predictor model.
  2. Using the flattened dataset representation (via the encoder - see point 1 above) as input to a custom subclass of KhiopsPredictor which provides the feature_importances_ and feature_names_in_ attributes; the relevant features are the input features, and their importances are all non-zero because the input only uses "used" features in the predictor.