XGBoosting Home | About | Contact | Examples

Configure XGBoost "learning_rate" Parameter

The learning_rate parameter in XGBoost is an alias for the eta parameter, which controls the step size at each boosting iteration.

By adjusting the learning rate, you can influence the model’s performance and training time.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier

# Generate synthetic data
X, y = make_classification(n_samples=1000, n_features=20, n_informative=2, n_redundant=10, random_state=42)

# Split the dataset into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize the XGBoost classifier with a lower learning rate (eta)
model = XGBClassifier(eta=0.01, eval_metric='logloss')

# Fit the model
model.fit(X_train, y_train)

# Make predictions
predictions = model.predict(X_test)

As discussed in the tip on configuring the eta parameter, the learning rate determines the contribution of each tree to the final outcome by scaling the weights of the features.

A lower learning rate can lead to better generalization and reduced overfitting, while a higher learning rate may result in faster learning but suboptimal solutions.

To recap, the key points when configuring the learning_rate parameter are:

For practical guidance on choosing the right learning rate value, refer to the tip on configuring the eta parameter.

See Also