Classical Machine Learning
Gradient Boosting
An ensemble technique that builds models in sequence, each correcting the errors of the last, behind top performers like XGBoost.
Gradient boosting builds a series of simple models, usually small decision trees, where each new one focuses on the mistakes the previous ones made. Added together they form a single strong predictor. Libraries like XGBoost and LightGBM made it the go-to method for tabular data competitions and many real-world prediction systems.
Reference: Wikipedia: Gradient Boosting
Related terms
Decision Tree
A model that makes predictions by following a branching series of yes or no questions, like a flowchart.
Random Forest
An ensemble model that combines many decision trees and lets them vote, far more accurate and stable than one tree.
Loss Function
The formula that scores how wrong a model is, giving training a single number to push downward.
Gradient Descent
The optimization method that gradually tweaks a model to lower its error, like walking downhill to the lowest point.