The “objective” parameter in XGBoost specifies the type of problem being solved, such as regression, binary classification, or multi-class classification.
Setting the correct “objective” is crucial for optimal model performance, as it determines the loss function used during training.
from sklearn.datasets import make_regression, make_classification
from xgboost import XGBRegressor, XGBClassifier
# Regression
X_reg, y_reg = make_regression(n_samples=100, n_features=10, noise=0.1, random_state=42)
reg_model = XGBRegressor(objective="reg:squarederror", n_estimators=100, random_state=42)
reg_model.fit(X_reg, y_reg)
# Binary classification
X_bin, y_bin = make_classification(n_samples=100, n_features=10, n_classes=2, random_state=42)
bin_model = XGBClassifier(objective="binary:logistic", n_estimators=100, random_state=42)
bin_model.fit(X_bin, y_bin)
# Multi-class classification
X_multi, y_multi = make_classification(n_samples=100, n_features=10, n_clusters_per_class=1, n_classes=3, random_state=42)
multi_model = XGBClassifier(objective="multi:softmax", n_estimators=100, random_state=42)
multi_model.fit(X_multi, y_multi)
The default “objective” values for common problem types are:
- Regression: “reg:squarederror” (formally “reg:linear”)
- Binary classification: “binary:logistic”
- Multi-class classification: “multi:softmax” or “multi:softprob”
Using an incorrect “objective” can lead to suboptimal model performance. For example, using a regression objective for a classification problem will result in the model trying to predict continuous values instead of class labels.
Advanced users can create custom objective functions for specific use cases, but for most common problems, the default objectives suffice.
When working with XGBoost, it’s essential to identify the problem type before setting the “objective”. If the model’s performance is unexpectedly poor, verify that the “objective” is set correctly. For multi-class problems, consider using “multi:softprob” instead of “multi:softmax” if class probabilities are needed.
Consult the XGBoost documentation for a complete list of available “objective” values and their use cases to ensure you’re using the most appropriate setting for your problem.