Skip to main content

Common Mistakes & Best Practices for LightGBM

1. Using removed early-stopping syntax​

Current LightGBM uses callbacks=[lightgbm.early_stopping(...)] in fit. The old early_stopping_rounds= fit keyword is not a LightGBM 4.7 API. Version 4.7 also deprecates eval_set; provide validation features and targets through eval_X and eval_y. The library does not reserve them automatically.

The stopping set is model-selection data, not final test data. With cross-validation, fixed global eval_X and eval_y are not fold-local. Either tune a fixed round count inside the search, or implement a nested procedure that creates a stopping subset within each training fold.

2. Letting leaf-wise trees grow without support​

max_depth alone does not characterize a leaf-wise tree. Tune num_leaves with minimum leaf observations/Hessian and split-gain constraints. Small leaves on small datasets can memorize noise even when nominal depth appears reasonable.

3. Breaking categorical semantics​

LightGBM casts categorical values to int32; negative codes are treated as missing. Preserve pandas categorical dtype or a documented stable mapping. Do not ordinal-encode categories and then forget to mark them categorical, and never compute target encodings before a split.

High-cardinality identifiers can still overfit. Compare native handling with fold-local one-hot/other appropriate representations and validate unseen-category behavior.

4. Scaling and feature selection by habit​

Tree thresholds generally do not require standardization. Global scaling or feature selection before cross-validation leaks fold information and rarely helps the tree itself. Put any learned transformation inside the training fold because other pipeline components may need it.

5. Treating class weights as a metric solution​

Class weights and scale_pos_weight change training. They do not select a deployment threshold or guarantee calibrated probabilities. Evaluate average precision or ROC AUC for ranking, log loss/Brier score for probabilities, and a validation-selected threshold for decisions.

6. Unfair speed and quality claims​

Fix threads, hardware, data representation, search/stopping budget, and metric. Include dataset construction and tuning time. GPU transfer can dominate small tasks, and unconstrained CPU parallelism can oversubscribe a parallel outer search.

7. Reading split importance as truth​

Split and gain importances can favor features with many candidate splits and divide credit unpredictably among correlated variables. Confirm findings with held-out permutation importance and stability across resamples; predictive importance is not causality.

Checklist​

  • Pin LightGBM 4.7.0 and record CPU/GPU build details.
  • Use callbacks and a training-only validation set for stopping.
  • Constrain leaves with data support, not depth alone.
  • Preserve categorical codes, feature order, and missing-value policy.
  • Isolate test data until the full procedure is fixed.