XGBoost Configure "interval-regression-accuracy" Eval Metric

The “interval-regression-accuracy” is an evaluation metric used in XGBoost for survival analysis models when dealing with interval-censored data. It measures the accuracy of the model’s predictions for the lower and upper bounds of the survival time intervals.

This metric should be used when the XGBoost model is configured with the “survival:aft” objective, which is designed for survival time prediction with interval-censored data. Using “interval-regression-accuracy” allows you to assess how well the model predicts the correct interval for each sample.

Here’s an example of how to use the “interval-regression-accuracy” metric in XGBoost’s native Python API:

import numpy as np
from sklearn.model_selection import train_test_split
import xgboost as xgb

# Generate a synthetic interval-censored survival dataset
np.random.seed(42)
n_samples = 1000
X = np.random.rand(n_samples, 5)
true_coef = np.array([1, -1, 2, -2, 1])
y_lower = np.exp(-(X @ true_coef + np.random.normal(size=n_samples)))
y_upper = y_lower + np.random.exponential(scale=0.1, size=n_samples)

# Split the data into training and testing sets
X_train, X_test, y_lower_train, y_lower_test, y_upper_train, y_upper_test = train_test_split(X, y_lower, y_upper, test_size=0.2, random_state=42)

# Convert data into DMatrix, specifying the label_lower_bound and label_upper_bound
dtrain = xgb.DMatrix(X_train, label_lower_bound=y_lower_train, label_upper_bound=y_upper_train)
dtest = xgb.DMatrix(X_test, label_lower_bound=y_lower_test, label_upper_bound=y_upper_test)

# Configure XGBoost parameters
params = {
    'objective': 'survival:aft',
    'eval_metric': 'interval-regression-accuracy',
    'aft_loss_distribution': 'normal',
    'aft_loss_distribution_scale': 1.0,
    'tree_method': 'hist',
    'learning_rate': 0.05,
    'max_depth': 3,
    'n_estimators': 100,
}

# Train the model
model = xgb.train(params, dtrain, num_boost_round=100, evals=[(dtest, 'test')])

# Make predictions on test set
y_pred_lower, y_pred_upper = model.predict(dtest, output_margin=True).transpose()

# Print evaluation metric on test set
print(f"Test interval-regression-accuracy: {model.eval(dtest, 'test').split(':')[-1]}")

In this example, we generate a synthetic interval-censored survival dataset where the lower bound of the survival time follows an exponential distribution based on a linear combination of the features, and the upper bound is obtained by adding an exponentially distributed value to the lower bound.

We split the data, convert to DMatrix format specifying the lower and upper bounds, and configure the XGBoost parameters. Note how we set 'objective': 'survival:aft' and 'eval_metric': 'interval-regression-accuracy'.

During training, we track the “interval-regression-accuracy” metric on the test set. After training, we make predictions for both the lower and upper bounds on the test set and print the final accuracy.

The “interval-regression-accuracy” ranges from 0 to 1, with higher values indicating better performance. It represents the fraction of samples for which the predicted interval contains the true survival time.

Some tips when using “interval-regression-accuracy”:

Use it alongside other survival metrics like “aft-nloglik” and concordance index to get a comprehensive view of model performance.

See Also