Neural Networks

This is a conceptual bridge lecture, not a deep learning course. We connect logistic regression to the single neuron, then build up to hidden layers and activation functions. The lecture focuses on when neural networks add value (large data, structured inputs like images and text) and when classical ML remains the better choice (tabular data, small samples, interpretability requirements).

If you want the mathematical version of the bridge — a linear model drawn and trained as a one-neuron network — see the optional Lecture 4 supplement.

Optional: technical supplement

For students who want the chain rule on paper: backpropagation derived on a one-hidden-layer network small enough to write every term, why sigmoid’s ≤ ¼ derivative causes vanishing gradients while ReLU’s doesn’t, why alpha in MLPClassifier is exactly ridge’s penalty (weight decay), and a numerical gradient check that verifies all of it to 1e-10.

Open the supplementary slides ↗ — optional and self-contained; nothing in it is required for the project.