Added the material for XGBoost optimization#30
Open
bbhattar wants to merge 8 commits into
Open
Conversation
rsiyer-intel
requested changes
May 18, 2026
Collaborator
|
Since the latest changes still have perf data, it cannot be approved till we get perf claim pre-requisites fulfilled. |
razdoburdin
reviewed
May 26, 2026
razdoburdin
reviewed
May 26, 2026
david-cortes-intel
suggested changes
May 26, 2026
david-cortes-intel
left a comment
david-cortes-intel
left a comment
Contributor
There was a problem hiding this comment.
General comment: this guide says 'xgboost', but it is limited to predictions/inference, while a similar guide could also be done for training, covering details like threading, hyperparameters to try, and similar.
razdoburdin
reviewed
May 26, 2026
razdoburdin
reviewed
May 26, 2026
razdoburdin
left a comment
razdoburdin
left a comment
There was a problem hiding this comment.
please update installation instructions and consider switching to the actual versions of the software.
napetrov
reviewed
May 27, 2026
…-learn, removing memory allocator section, and clarifying the scope to include all 3 methods
Author
@rsiyer-intel Updated the doc with PDT approved data |
bbhattar
requested review from
david-cortes-intel,
napetrov,
razdoburdin and
rsiyer-intel
Author
Done and done |
adgubrud
reviewed
Jun 30, 2026
adgubrud
reviewed
Jun 30, 2026
| For multiclass classification, default XGBoost, LightGBM, and daal4py all use one tree per class. CatBoost, on the other hand, uses vectorized trees. This means all other approaches end up processing `num_classes x` more trees compared to CatBoost, e.g., 7,000 vs 1,000 for Covtype. For smaller `num_estimators` like `100`, `daal4py` outperforms CatBoost, but as `num_estimators` gets larger, CatBoost provides better inference latency. | ||
| For multiclass classification, XGBoost, LightGBM, and daal4py (with default settings as of the tested versions) use one tree per class, while CatBoost uses symmetric (oblivious) trees that handle all classes in a single tree. This means daal4py ends up processing `num_classes × num_estimators` trees compared to CatBoost's `num_estimators` trees (e.g., 7,000 vs 1,000 for Covtype with 7 classes). As a result, CatBoost can provide better inference latency for multiclass tasks with many classes and large ensembles. | ||
|
|
||
| > **Note:** XGBoost is moving towards multi-output trees (via `multi_strategy="multi_output_tree"`) which would reduce this gap by handling all classes in a single tree, similar to CatBoost. Check the [XGBoost documentation](https://xgboost.readthedocs.io/en/latest/tutorials/multioutput.html) for the latest defaults. |
Contributor
There was a problem hiding this comment.
This tutorial doesn't show what the defaults are.
…e mention of undefined default, removed unnecessary symmetric tree mention
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Added the materials for XGBoost optimization. Please review and give me your feedback.