XGBoost Configure fit() "eval_metric" Parameter

Specifying the eval_metric evaluation metric parameter in XGBoost’s fit() method is deprecated and will result in a warning:

UserWarning: `eval_metric` in `fit` method is deprecated for better compatibility
with scikit-learn, use `eval_metric` in constructor or`set_params` instead.

Specifying eval_metric as both a parameter to the model and to the fit() method will result in an error:

ValueError: 2 different `eval_metric` are provided.  Use the one in constructor or `set_params` instead.

Instead, the evaluation metric should only be specified as a model parameter when initializing the XGBoost model.

Here’s an example of how to correctly specify the evaluation metric:

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier
from sklearn.metrics import accuracy_score

# Generate a synthetic classification dataset
X, y = make_classification(n_samples=1000, n_classes=2, random_state=42)

# Split data into train, validation, and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.25, random_state=42)

# Initialize XGBClassifier with the evaluation metric specified as a model parameter
model = XGBClassifier(objective='binary:logistic', eval_metric='error', random_state=42)

# Train the model
model.fit(X_train, y_train, eval_set=[(X_val, y_val)], verbose=False)

# Make predictions and evaluate accuracy
y_pred = model.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.2f}")

In this example:

1. We generate a synthetic binary classification dataset using make_classification.

2. We initialize an XGBClassifier with objective='binary:logistic' for binary classification and specify the evaluation metric eval_metric='error' as a model parameter. This is the correct way to specify the metric.

3. We train the model using the fit() method.

4. Finally, we make predictions on the test set and evaluate the model’s accuracy.

Note that if we were to specify the evaluation metric in the fit() method like this: model.fit(X_train, y_train, eval_metric='error'), it would result in a deprecation warning. The correct approach is to specify it when initializing the model.

By specifying the evaluation metric correctly, we ensure that XGBoost uses the desired metric to evaluate the model’s performance during training without any warnings.