XGBoosting Home | About | Contact | Examples

Configure XGBoost "grow_policy" Parameter

The grow_policy parameter in XGBoost determines how the trees are grown during the training process.

By setting this parameter, you can influence the structure of the resulting trees and potentially improve the model’s performance.

The grow_policy parameter requires that the tree_method parameter be set to 'approx' or 'dist', e.g. not 'exact'.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier

# Generate synthetic data
X, y = make_classification(n_samples=1000, n_features=20, n_informative=2, n_redundant=10, random_state=42)

# Split the dataset into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Configure the XGBoost model with a specific grow_policy
model = XGBClassifier(grow_policy='lossguide', tree_method='approx', eval_metric='logloss')

# Fit the model
model.fit(X_train, y_train)

# Make predictions
predictions = model.predict(X_test)

Understanding the “grow_policy” Parameter

The grow_policy parameter accepts two possible values: “depthwise” and “lossguide”.

Choosing the Right “grow_policy” Value

When deciding between “depthwise” and “lossguide” growth policies, consider the following:

Keep in mind that the optimal grow_policy value may depend on the specific problem and dataset you are working with.

Practical Tips

See Also