Classical Machine Learning

Gradient Boosting

An ensemble technique that builds models in sequence, each correcting the errors of the last, behind top performers like XGBoost.

Gradient boosting builds a series of simple models, usually small decision trees, where each new one focuses on the mistakes the previous ones made. Added together they form a single strong predictor. Libraries like XGBoost and LightGBM made it the go-to method for tabular data competitions and many real-world prediction systems.

Reference: Wikipedia: Gradient Boosting