XGBoosting Home | About | Contact | Examples

XGBoost Evaluate Model for Time Series using Walk-Forward Validation

This example demonstrates how to evaluate an XGBoost model for time series forecasting using walk-forward validation, a technique that assesses the model’s performance on unseen data by iteratively splitting the data into train and test sets. We’ll use a synthetic time series dataset to illustrate the process.

Walk-forward validation is crucial for time series forecasting because it mimics the real-world scenario where models are trained on historical data and used to make predictions on future, unseen data.

By using this validation method, we can get a more realistic estimate of the model’s performance.

# XGBoosting.com
# Evaluate XGBoost for Time Series Forecasting Using Walk-Forward Validation
import numpy as np
import pandas as pd
from xgboost import XGBRegressor
from sklearn.metrics import mean_squared_error

# Generate a synthetic time series dataset
series = np.sin(0.1 * np.arange(200)) + np.random.randn(200) * 0.1

# Prepare data for supervised learning
df = pd.DataFrame(series, columns=['value'])
for i in range(1, 4):
    df[f'lag_{i}'] = df['value'].shift(i)
df = df.dropna()

X = df.drop(columns=['value']).values
y = df['value'].values

# Define the number of lags and the test size for each iteration
n_lags = 3
n_test = 1

# Initialize lists to store predictions and actual values
predictions = []
actual = []

# Perform walk-forward validation
for i in range(len(X) - n_lags - n_test + 1):
    X_train, X_test = X[i:i+n_lags], X[i+n_lags:i+n_lags+n_test]
    y_train, y_test = y[i:i+n_lags], y[i+n_lags:i+n_lags+n_test]

    model = XGBRegressor(n_estimators=100, learning_rate=0.1, random_state=42)
    model.fit(X_train, y_train)

    y_pred = model.predict(X_test)

# Calculate the Mean Squared Error
mse = mean_squared_error(actual, predictions)
print(f"Mean Squared Error: {mse:.4f}")

In this example, we:

  1. Generate a synthetic time series dataset using a sine wave with added noise.
  2. Prepare the data for supervised learning by creating lagged features.
  3. Define the number of lags (n_lags) and the test size (n_test) for each iteration of the walk-forward validation.
  4. Initialize lists to store the predictions and actual values.
  5. Perform walk-forward validation:
    • Split the data into train and test sets for each iteration
    • Train the XGBoost model on the training set
    • Make one-step predictions on the test set
    • Store the predictions and actual values
  6. Calculate the Mean Squared Error (MSE) on the collected predictions and actual values.

By using walk-forward validation, we can assess how well the XGBoost model generalizes to unseen data and get a more realistic estimate of its performance for time series forecasting tasks. This example can be easily adapted to work with real-world time series datasets and extended to include additional evaluation metrics or visualization of the results.

See Also