XGBoost uses different default evaluation metrics depending on the objective specified during training.
Understanding these default metrics can help you interpret the model’s performance without explicitly setting the eval_metric parameter.
This example demonstrates the default eval_metric used by XGBoost for binary classification, multi-class classification, and regression objectives.
We’ll train models with different objectives using the scikit-learn API and confirm the default metrics by plotting the evaluation results.
from sklearn.datasets import make_classification, make_regression
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier, XGBRegressor
import matplotlib.pyplot as plt
# Binary classification
X_binary, y_binary = make_classification(n_samples=1000, n_classes=2, random_state=42)
X_train_binary, X_test_binary, y_train_binary, y_test_binary = train_test_split(X_binary, y_binary, test_size=0.2, random_state=42)
# Multi-class classification
X_multi, y_multi = make_classification(n_samples=1000, n_classes=3, n_informative=5, random_state=42)
X_train_multi, X_test_multi, y_train_multi, y_test_multi = train_test_split(X_multi, y_multi, test_size=0.2, random_state=42)
# Regression
X_reg, y_reg = make_regression(n_samples=1000, n_features=5, noise=0.1, random_state=42)
X_train_reg, X_test_reg, y_train_reg, y_test_reg = train_test_split(X_reg, y_reg, test_size=0.2, random_state=42)
# Train XGBoost models with different objectives
objectives = ['binary:logistic', 'multi:softmax', 'reg:squarederror']
datasets = [(X_train_binary, y_train_binary, X_test_binary, y_test_binary),
(X_train_multi, y_train_multi, X_test_multi, y_test_multi),
(X_train_reg, y_train_reg, X_test_reg, y_test_reg)]
for objective, dataset in zip(objectives, datasets):
if objective.startswith('reg'):
model = XGBRegressor(objective=objective, n_estimators=100, random_state=42)
else:
model = XGBClassifier(objective=objective, n_estimators=100, random_state=42)
model.fit(dataset[0], dataset[1], eval_set=[(dataset[2], dataset[3])])
results = model.evals_result()
print(f"Objective: {objective}")
print(f"Default eval_metric: {list(results['validation_0'].keys())[0]}")
plt.figure(figsize=(8, 4))
plt.plot(results['validation_0'][list(results['validation_0'].keys())[0]])
plt.xlabel('Iteration')
plt.ylabel('Metric Value')
plt.title(f'Objective: {objective}')
plt.show()
In this example, we use the scikit-learn API to train XGBoost models. We create instances of XGBClassifier for binary and multi-class classification objectives and XGBRegressor for the regression objective.
We set the objective parameter to the corresponding objective string and specify the number of estimators (n_estimators) and random state (random_state) for reproducibility. For the multi-class classification objective.
We then fit the models using the fit() method, passing the training data, evaluation set, and setting verbose=False to suppress the training output.
After training each model, we retrieve the evaluation metric results using model.evals_result() and print the default metric used for each objective. Finally, we plot the evaluation metric results for each objective to visualize the model’s performance.
The output will show the default evaluation metrics used by XGBoost for each objective:
- Binary classification:
logloss - Multi-class classification:
mlogloss - Regression:
rmse
By using the scikit-learn API, you can easily train XGBoost models and access the evaluation results in a familiar and consistent manner. The default evaluation metrics remain the same as in the native XGBoost API, providing a convenient way to interpret the model’s performance without explicitly setting the eval_metric parameter.