XGBoosting Home | About | Contact | Examples

Configure XGBoost Objective "reg:logistic" vs "binary:logistic"

This example contrasts two XGBoost objectives: "reg:logistic" for regression tasks where the target is a probability (between 0 and 1) and "binary:logistic" for binary classification tasks.

We’ll demonstrate when to use each objective and provide a complete code example showcasing their implementation and key differences.

from sklearn.datasets import make_classification, make_regression
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier, XGBRegressor
from sklearn.metrics import accuracy_score, mean_squared_error

# Generate a synthetic binary classification dataset
X_bin, y_bin = make_classification(n_samples=1000, n_classes=2, n_features=10, random_state=42)
X_train_bin, X_test_bin, y_train_bin, y_test_bin = train_test_split(X_bin, y_bin, test_size=0.2, random_state=42)
model_bin = XGBClassifier(objective="binary:logistic", n_estimators=100, learning_rate=0.1)
model_bin.fit(X_train_bin, y_train_bin)
y_pred_bin = model_bin.predict(X_test_bin)
accuracy_bin = accuracy_score(y_test_bin, y_pred_bin)

# Generate a synthetic regression dataset where the target is a probability
X_reg, y_reg = make_regression(n_samples=1000, n_features=10, random_state=42)
y_reg = (y_reg - y_reg.min()) / (y_reg.max() - y_reg.min())  # Normalize to 0-1 range
X_train_reg, X_test_reg, y_train_reg, y_test_reg = train_test_split(X_reg, y_reg, test_size=0.2, random_state=42)
model_reg = XGBRegressor(objective="reg:logistic", n_estimators=100, learning_rate=0.1)
model_reg.fit(X_train_reg, y_train_reg)
y_pred_reg = model_reg.predict(X_test_reg)
mse_reg = mean_squared_error(y_test_reg, y_pred_reg)

print(f"Binary Classification Accuracy: {accuracy_bin:.4f}")
print(f"Regression MSE: {mse_reg:.4f}")

Analysis of outcomes:

Best practices and tips:

By understanding the distinctions and appropriate applications of these two objectives, you can more effectively utilize XGBoost in your machine learning projects to tackle a wide range of predictive modeling challenges.

See Also