When training an XGBoost model for regression tasks, the Root Mean Squared Error (RMSE) is a commonly used evaluation metric.
RMSE measures the average magnitude of the residuals (prediction errors) and penalizes large errors more heavily than small ones.
By setting eval_metric='rmse', you can monitor your model’s performance during training and enable early stopping to prevent overfitting.
Here’s an example of how to use RMSE as the evaluation metric with XGBoost and scikit-learn:
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from xgboost import XGBRegressor
import matplotlib.pyplot as plt
# Generate a synthetic regression dataset
X, y = make_regression(n_samples=1000, n_features=10, noise=0.1, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Create an XGBRegressor with RMSE as the evaluation metric
model = XGBRegressor(n_estimators=100, eval_metric='rmse', early_stopping_rounds=10, random_state=42)
# Train the model with early stopping
model.fit(X_train, y_train, eval_set=[(X_test, y_test)])
# Retrieve the RMSE values from the training process
results = model.evals_result()
epochs = len(results['validation_0']['rmse'])
x_axis = range(0, epochs)
# Plot the RMSE values
plt.figure()
plt.plot(x_axis, results['validation_0']['rmse'], label='Test')
plt.legend()
plt.xlabel('Number of Boosting Rounds')
plt.ylabel('RMSE')
plt.title('XGBoost RMSE Performance')
plt.show()
In this example, we generate a synthetic regression dataset using scikit-learn’s make_regression function. We then split the data into training and testing sets.
We create an instance of XGBRegressor and set eval_metric='rmse' to specify RMSE as the evaluation metric. We also set early_stopping_rounds=10 to enable early stopping if the RMSE doesn’t improve for 10 consecutive rounds.
During training, we pass the testing set as the eval_set to monitor the model’s performance on unseen data. After training, we retrieve the RMSE values using the evals_result() method.
Finally, we plot the RMSE values against the number of boosting rounds to visualize the model’s performance during training. This plot helps us assess whether the model is overfitting or underfitting and determines the optimal number of boosting rounds.
By using RMSE as the evaluation metric, we can effectively monitor the model’s regression performance, prevent overfitting through early stopping, and select the best model based on the lowest RMSE value.