LightGBMTunerCV error when using custom objective function

[pmandiola]

Expected behavior

LightGBMTunerCV should work without problems when using a custom objective function. LightGBM cv() docs specify that a custom function can be passed to the "objective" parameter.

Environment

• Optuna version: 4.1.0
• Optuna Integration version: 4.1.0
• Python version: 3.11.5
• OS: macOS-13.4-arm64-arm-64bit

Error messages, stack traces, or logs

Traceback (most recent call last):
  File "/Users/pmandiola/Documents/Lidz/data-exploration/.venv/lib/python3.11/site-packages/marimo/_runtime/executor.py", line 157, in execute_cell
    exec(cell.body, glbls)
  Cell marimo://1_train_model.py#cell=cell-10, line 24, in <module>
    tuner.run()
  File "/Users/pmandiola/Documents/Lidz/data-exploration/.venv/lib/python3.11/site-packages/optuna_integration/_lightgbm_tuner/optimize.py", line 475, in run
    self.tune_feature_fraction()
  File "/Users/pmandiola/Documents/Lidz/data-exploration/.venv/lib/python3.11/site-packages/optuna_integration/_lightgbm_tuner/optimize.py", line 500, in tune_feature_fraction
    self._tune_params([param_name], len(param_values), sampler, "feature_fraction")
  File "/Users/pmandiola/Documents/Lidz/data-exploration/.venv/lib/python3.11/site-packages/optuna_integration/_lightgbm_tuner/optimize.py", line 583, in _tune_params
    study.optimize(
  File "/Users/pmandiola/Documents/Lidz/data-exploration/.venv/lib/python3.11/site-packages/optuna/study/study.py", line 475, in optimize
    _optimize(
  File "/Users/pmandiola/Documents/Lidz/data-exploration/.venv/lib/python3.11/site-packages/optuna/study/_optimize.py", line 63, in _optimize
    _optimize_sequential(
  File "/Users/pmandiola/Documents/Lidz/data-exploration/.venv/lib/python3.11/site-packages/optuna/study/_optimize.py", line 160, in _optimize_sequential
    frozen_trial = _run_trial(study, func, catch)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pmandiola/Documents/Lidz/data-exploration/.venv/lib/python3.11/site-packages/optuna/study/_optimize.py", line 248, in _run_trial
    raise func_err
  File "/Users/pmandiola/Documents/Lidz/data-exploration/.venv/lib/python3.11/site-packages/optuna/study/_optimize.py", line 197, in _run_trial
    value_or_values = func(trial)
                      ^^^^^^^^^^^
  File "/Users/pmandiola/Documents/Lidz/data-exploration/.venv/lib/python3.11/site-packages/optuna_integration/_lightgbm_tuner/optimize.py", line 340, in __call__
    self._postprocess(trial, elapsed_secs, average_iteration_time)
  File "/Users/pmandiola/Documents/Lidz/data-exploration/.venv/lib/python3.11/site-packages/optuna_integration/_lightgbm_tuner/optimize.py", line 271, in _postprocess
    trial._trial_id, _LGBM_PARAMS_KEY, json.dumps(self.lgbm_params)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py", line 258, in iterencode
    return _iterencode(o, 0)
           ^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type function is not JSON serializable

Steps to reproduce

1. Create a custom objective and eval function (I'm using focal loss as defined here)
2. Pass the custom function in the "objective" parameter for LightGBMTunerCV
3. Run the tuner

dtrain = lgb.Dataset(X_train, y_train)

params = {
    "objective": focal_loss,
    "metric": "focal_loss",
    "verbosity": -1,
    "boosting_type": "gbdt"
}

tuner = lgbo.LightGBMTunerCV(
    params,
    dtrain,
    callbacks=[early_stopping(100), log_evaluation(100)],
    feval = eval_error
)

tuner.run()

Additional context (optional)

The error seems to occur when the lgbm_params are stored as json in the Trial object after running each trial, as the custom function is not json serializable. Happy to contribute with a fix if possible.