BayesSearchCV fit_params not working for XGBoost

[SG-391]

Hello,

I am trying to pass an 'eval_set' to an XGBoost model within BayesSearchCV through the fit_params parameter, in order to allow for early stopping. However, any value provided within fit_params seems to be ignored as I systematically get the following error: "ValueError: Must have at least 1 validation dataset for early stopping."

I attach part of the code used.

Any suggestion? Thanks

from xgboost import XGBClassifier

train_batch = df_train[my_cols].to_numpy()
test_batch = df_test[my_cols].to_numpy()

labels = df_train[col_y].to_numpy()
lab_test = df_test[col_y].to_numpy()

eval_set = [(test_batch, lab_test.reshape(-1))]

search_space = {
   'max_depth': Integer(3, 9, transform='identity'),
   'learning_rate': Real(0.01, 0.3, prior='log-uniform', transform='identity'),
   'reg_alpha': Real(0.0001,0.1, transform='identity'),
   'reg_lambda': Real(0.0001,0.1, transform='identity'),
   'subsample': Real(0.3, 1.0, transform='identity'),
   'colsample_bytree': Real(0.3, 1.0, transform='identity')
}

XGBmodel = XGBClassifier(objective='multi:sofprob',
   num_class=3,
   n_estimators=500,
   tree_method='hist',
   device='cuda',
   booster='gbtree',
   early_stopping_rounds=20,
   random_state=42)

bayes_search = BayesSearchCV(estimator=XGBmodel, search_spaces=search_space,fit_params={'eval_set': eval_set},n_points=1, n_iter=15,cv=3, scoring='average_precision', n_jobs=-1, verbose=2, random_state=42)

bayes_search.fit(train_batch, labels.reshape(-1))

print(f"Best parameters: {bayes_search.best_params_}")
print(f"Best score: {bayes_search.best_score_}")

print(f"Test score: {bayes_search.score(test_batch,lab_test.reshape(-1))}")

Environment:

skopt 0.10.2
numpy 1.26.4
scipy 1.13.1
sklearn 1.5.1
Windows-11-10.0.22631-SP0
Python 3.12.3 | packaged by conda-forge | (main, Apr 15 2024, 18:20:11) [MSC v.1938 64 bit (AMD64)]