Support Vector Machines

SVMs take a different approach: instead of averaging many models, find the single best boundary between classes by maximising the margin. This lecture covers the maximum-margin idea, soft margins (the C parameter), and the kernel trick for nonlinear boundaries. We compare SVMs with random forests and discuss when each approach is more appropriate.

Optional: technical supplement

For students who want the machinery: where the ½‖w‖² objective comes from, the dual problem that proves only support vectors matter, why kernels never need coordinates (with the rings lift verified by algebra and by code), and the hinge-loss view that makes the SVM a cousin of logistic regression.

Open the supplementary slides ↗ — optional and self-contained; nothing in it is required for the project.