The evals_result() method in XGBoost allows you to access the evaluation metrics computed during training.
This is particularly useful when using early stopping or monitoring the model’s performance on a validation set.
This example demonstrates how to retrieve and visualize the learning curve for a chosen metric using evals_result() with scikit-learn’s API.
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier
import matplotlib.pyplot as plt
# Load the dataset
data = load_breast_cancer()
X, y = data.data, data.target
# Split the data into train, validation, and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.25, random_state=42)
# Define the XGBoost model
model = XGBClassifier(
objective='binary:logistic',
eval_metric='logloss',
early_stopping_rounds=10,
seed=42,
n_estimators=100
)
# Train the model
model.fit(X_train, y_train, eval_set=[(X_val, y_val)], verbose=False)
# Access the evaluation results
evals_result = model.evals_result()
# Plot the learning curve
plt.plot(evals_result['validation_0']['logloss'], label='Validation')
plt.xlabel('Boosting Round')
plt.ylabel('Log Loss')
plt.title('Learning Curve')
plt.legend()
plt.show()
When using scikit-learn’s API for XGBoost, you can define the model using the XGBClassifier or XGBRegressor class, depending on your task. The model hyperparameters, such as the objective function, evaluation metric, and early stopping rounds, can be specified during model initialization.
To train the model, use the fit() method and provide the training data (X_train, y_train). To track the evaluation metrics on a validation set during training, pass the validation data as a list of tuples to the eval_set parameter.
After training, you can access the evaluation results using the evals_result() method of the trained model object. The resulting dictionary has the following structure:
{
'validation_0': {
'metric_name': [metric_values_at_each_boosting_round]
}
}
In this example, we plot the learning curve for the log loss metric on the validation set. The learning curve visualizes how the metric evolves over the boosting rounds, providing insights into the model’s performance and convergence.
By monitoring the learning curve, you can assess whether the chosen number of boosting rounds is appropriate or if early stopping should be applied to prevent overfitting.
Using scikit-learn’s API for XGBoost provides a consistent and familiar interface for model training and evaluation. The evals_result() method allows you to easily access and analyze the evaluation metrics, enabling you to make informed decisions about hyperparameter tuning and model selection.