XGBoosting Home | About | Contact | Examples

Configure XGBoost "monotone_constraints" Parameter

The monotone_constraints parameter in XGBoost allows you to incorporate domain knowledge into your model by specifying monotonic constraints for each feature.

This parameter enforces the model to be monotonically increasing or decreasing with respect to certain features.

import numpy as np
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from xgboost import XGBRegressor

# Generate synthetic data with a monotonically increasing relationship between the first feature and the target
X, y = make_regression(n_samples=1000, n_features=5, random_state=42, noise=0.1)
X[:, 0] = np.sort(X[:, 0])
y = 0.5 * X[:, 0] + y

# Split the dataset into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize the XGBoost regressor with monotone_constraints
model = XGBRegressor(monotone_constraints=(1, 0, 0, 0, 0))

# Fit the model
model.fit(X_train, y_train)

# Make predictions
predictions = model.predict(X_test)

Understanding the “monotone_constraints” Parameter

The monotone_constraints parameter is a tuple that specifies the monotonic constraints for each feature in your dataset. The possible values are:

The length of the monotone_constraints list must match the number of input features in your dataset. In the example above, monotone_constraints=(1, 0, 0, 0, 0) means that the first feature is constrained to be monotonically increasing, while the remaining features have no constraints.

Using “monotone_constraints” in Practice

Monotonic constraints are useful when you have prior domain knowledge about the relationship between features and the target variable. By incorporating this knowledge into your model, you can improve its interpretability and make its behavior more predictable and easier to explain.

Some situations where monotonic constraints might be beneficial include:

However, it’s important to note that misspecifying monotonic constraints can lead to poor model performance if the actual relationship between the feature and the target does not match the specified constraint.

Practical Tips

It’s worth noting that the impact of monotonic constraints on the model’s training time and resource requirements is not well-documented in the XGBoost documentation.

See Also