Ensemble Methods

Single decision trees are transparent but overfit easily. This lecture shows how combining many trees into an ensemble dramatically improves performance. We cover decision tree basics (splits, impurity, pruning), then bagging (bootstrap aggregating) and random forests (bagging + random feature selection). Key topics include feature importance, out-of-bag error, and the key hyperparameters to tune.