XGBoosting Home | About | Contact | Examples

What is a Decision Tree

Decision trees are a type of supervised learning algorithm used for both classification and regression tasks.

They are called “decision trees” because they resemble a tree structure, with a root node, internal nodes, and leaf nodes.

XGBoost is an ensemble of decision trees, so understanding how individual trees work is essential.

Structure of a Decision Tree

A decision tree consists of nodes and edges:

How Decision Trees Make Predictions

To make a prediction, the tree is traversed from the root node to a leaf node:

Splitting Criteria

At each internal node, the feature and split point are chosen to maximize the information gain or minimize the impurity:

Advantages of Decision Trees

Decision trees offer several advantages:

Limitations of Decision Trees

Decision trees also have some limitations:

Role in XGBoost

XGBoost leverages decision trees in the following ways:

See Also