Gradient Boosting

While bagging (L10) fits trees independently and averages them, boosting fits trees sequentially — each one correcting the errors of the previous ensemble. This lecture covers the boosting intuition, the gradient boosting algorithm, the learning rate tradeoff, and production implementations (XGBoost, LightGBM). Gradient boosting often achieves the best performance on tabular data but requires careful tuning.