The "reg:gamma" objective in XGBoost is used for regression tasks when the target variable follows a gamma distribution.
This objective is suitable for non-negative target variables with skewed distributions, such as insurance claims or rainfall amounts.
import numpy as np
from sklearn.model_selection import train_test_split
from xgboost import XGBRegressor
from sklearn.metrics import mean_poisson_deviance
# Generate a synthetic dataset with gamma-distributed target variable
np.random.seed(42)
X = np.random.rand(1000, 10)
y = np.random.gamma(shape=1, scale=1, size=1000)
# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Initialize an XGBRegressor with the "reg:gamma" objective
model = XGBRegressor(objective="reg:gamma", n_estimators=100, learning_rate=0.1)
# Fit the model on the training data
model.fit(X_train, y_train)
# Make predictions on the test set
y_pred = model.predict(X_test)
# Calculate the mean Poisson deviance
mpd = mean_poisson_deviance(y_test, y_pred)
print(f"Mean Poisson Deviance: {mpd:.4f}")
The "reg:gamma" objective minimizes the negative log-likelihood of the gamma distribution.
When using the "reg:gamma" objective, consider tuning key hyperparameters such as max_depth, learning_rate, and n_estimators to optimize performance. If the "reg:gamma" objective does not provide satisfactory results, consider trying other objectives like "reg:squarederror" or "reg:tweedie".
It is important to choose an appropriate evaluation metric for gamma regression, such as mean Poisson deviance, which measures the difference between the predicted and actual gamma distributions.