XGBoosting Home | About | Contact | Examples

Configure XGBoost "max_delta_step" Parameter

The max_delta_step parameter in XGBoost controls the maximum delta step allowed for each tree’s weight estimation. By adjusting max_delta_step, you can influence the model’s sensitivity to extreme weights and outliers.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier

# Generate synthetic data
X, y = make_classification(n_samples=1000, n_features=20, n_informative=2, n_redundant=10, random_state=42)

# Split the dataset into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize the XGBoost classifier with a max_delta_step value
model = XGBClassifier(max_delta_step=1, eval_metric='logloss')

# Fit the model
model.fit(X_train, y_train)

# Make predictions
predictions = model.predict(X_test)

Understanding the “max_delta_step” Parameter

The max_delta_step parameter determines the maximum delta step allowed for each tree’s weight estimation during the model’s training process. It can help make the model more conservative and robust to outliers by limiting the weight updates. max_delta_step accepts non-negative values, and the default value in XGBoost is 0, which means there is no constraint on the weight updates.

Choosing the Right “max_delta_step” Value

The value of max_delta_step affects the model’s sensitivity to extreme weights and outliers:

When setting max_delta_step, consider the trade-off between model sensitivity and stability:

Practical Tips

See Also