XGBoost Benchmark Model Training Time

Measuring the training time of an XGBoost model can help you optimize your machine learning pipeline and make informed decisions about resource allocation.

In this example, we’ll demonstrate how to use Python’s time.perf_counter() to benchmark the training time of an XGBoost model with different numbers of threads, showcasing the impact of multithreading on performance.

import numpy as np
from sklearn.datasets import make_classification
import xgboost as xgb
import time

# Generate a synthetic dataset
X, y = make_classification(n_samples=100000, n_features=100, n_informative=80,
                           n_redundant=10, n_classes=2, random_state=42)

# Create DMatrix for XGBoost
dtrain = xgb.DMatrix(X, label=y)

# Define a function to train an XGBoost model with a specified number of threads
def train_model(nthread):
    params = {'objective': 'binary:logistic', 'nthread': nthread, 'eval_metric': 'error',
              'max_depth': 6, 'eta': 0.1}
    start_time = time.perf_counter()
    xgb.train(params, dtrain, num_boost_round=100)
    end_time = time.perf_counter()
    return end_time - start_time

# Benchmark training time with different numbers of threads
num_threads = [1, 2, 4, 8]
training_times = []

for nt in num_threads:
    elapsed_time = train_model(nt)
    training_times.append(elapsed_time)
    print(f"Training with {nt} thread(s) took {elapsed_time:.2f} seconds.")

# Print results in a table
print("\nBenchmark Results:")
print("Threads | Training Time (s)")
print("--------|------------------")
for nt, tt in zip(num_threads, training_times):
    print(f"{nt:7d} | {tt:17.2f}")

Results may look as follows:

Training with 1 thread(s) took 6.94 seconds.
Training with 2 thread(s) took 3.33 seconds.
Training with 4 thread(s) took 2.16 seconds.
Training with 8 thread(s) took 2.35 seconds.

Benchmark Results:
Threads | Training Time (s)
--------|------------------
      1 |              6.94
      2 |              3.33
      4 |              2.16
      8 |              2.35

In this example:

We generate a synthetic dataset using sklearn.datasets.make_classification() with 100,000 samples, 100 features, and 2 classes.
We create a DMatrix object for XGBoost using the generated dataset.
We define a function train_model() that takes the number of threads as an argument, sets up the XGBoost parameters, trains the model, and returns the elapsed time using time.perf_counter().
We benchmark the training time with different numbers of threads (1, 2, 4, and 8) by calling the train_model() function in a loop.
We print the results in a formatted table showing the number of threads and corresponding training times.

By running this example, you can observe how the number of threads affects the training time of an XGBoost model. This information can help you make decisions about the optimal number of threads to use based on your system’s resources and the scale of your machine learning tasks.

See Also