XGBoosting Home | About | Contact | Examples

Configure XGBoost "reg:absoluteerror" Objective (mean absolute error)

The "reg:absoluteerror" objective in XGBoost is used for regression tasks when the target variable is continuous.

It minimizes the mean absolute error (MAE) between the predicted and actual values, making it less sensitive to outliers compared to the "reg:squarederror" objective.

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from xgboost import XGBRegressor
from sklearn.metrics import mean_absolute_error

# Generate a synthetic dataset for regression
X, y = make_regression(n_samples=1000, n_features=10, noise=0.1, random_state=42)

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize an XGBRegressor with the "reg:absoluteerror" objective
model = XGBRegressor(objective="reg:absoluteerror", n_estimators=100, learning_rate=0.1)

# Fit the model on the training data
model.fit(X_train, y_train)

# Make predictions on the test set
y_pred = model.predict(X_test)

# Calculate the mean absolute error of the predictions
mae = mean_absolute_error(y_test, y_pred)
print(f"Mean Absolute Error: {mae:.4f}")

The "reg:absoluteerror" objective minimizes the absolute difference between the predicted and actual values, which is equivalent to the mean absolute error (MAE) loss function.

This objective is suitable when the target variable is continuous, and the goal is to minimize the average absolute difference between predictions and actual values. It is less sensitive to outliers compared to the "reg:squarederror" objective, which minimizes the squared error.

When using the "reg:absoluteerror" objective, consider the following tips:

  1. Ensure that the target variable is continuous and not categorical or binary.
  2. Scale the input features to a similar range to improve convergence and model performance.
  3. Use appropriate evaluation metrics for regression, such as MAE, MSE, or RMSE, to assess the model’s performance.
  4. Tune hyperparameters like learning_rate, max_depth, and n_estimators to optimize performance.

Generally using MAE will be slow as this objective does not provide the derivatives required for the optimization algorithm used within XGBoost. Instead, an alternative non-derivative based algorithm (e.g. a line search) is used.



See Also