Skip to content

Chapter 7 of 8

Deep Learning Foundations

A single neuron computes \(a = \sigma(\mathbf{w}^\top\mathbf{x} + b)\) where \(\mathbf{w}\) are weights, \(b\) is bias, and \(\sigma\) is a nonlinear activation. A layer in matrix form: \(\mathbf{z} = W\mathbf{x} + \mathbf{b}\), \(\mathbf{a} = \sigma(\mathbf{z})\), with \(W \in \mathbb{R}^{d_{\text{out}}\times d_{\text{in}}}\) containing \(d_{\text{in}} \times d_{\text{out}} + d_{\text{out}}\) parameters. The forward pass of a multi-layer network stacks such layers: \(\mathbf{a}^{(0)} = \mathbf{x}\), \(\mathbf{z}^{(l)} = W^{(l)}\mathbf{a}^{(l-1)}+\mathbf{b}^{(l)}\), \(\mathbf{a}^{(l)} = \sigma(\mathbf{z}^{(l)})\). Depth allows hierarchical feature extraction while width controls per-layer capacity. Activation functions inject nonlinearity—without them, stacking layers is equivalent to one linear layer. Sigmoid \(\sigma(x) = 1/(1+e^{-x})\) squashes to (0,1) but saturates at ±∞ causing vanishing gradients (max derivative 0.25 at x=0). Tanh \((e^x-e^{-x})/(e^x+e^{-x})\) is zero-centered (better gradient flow) but still saturates; derivative \(1 - \tanh^2(x)\). ReLU \(\max(0,x)\) is the default for hidden layers—no saturation for positive inputs (gradient = 1), cheap to compute—but suffers from dying ReLU if pre-activation stays negative. Leaky ReLU uses slope \(\alpha\) for negative inputs (\(\alpha \approx 0.01\)); PReLU learns \(\alpha\). ELU \(\alpha(e^x-1)\) for \(x \leq 0\) saturates smoothly. GELU \(x\Phi(x)\) weights input by probability of being positive under a Gaussian (used in BERT, GPT). Swish/SiLU \(x\sigma(x)\) is smooth, non-monotonic; SwiGLU \(\mathrm{Swish}(\mathbf{x}_1) \odot \mathbf{x}_2\) is used in Llama's FFN. Softmax \(\sigma(\mathbf{z})_i = e^{z_i}/\sum_j e^{z_j}\) produces a probability distribution (numerically stable version subtracts \(\max_j z_j\) before exponentiation); temperature scaling \(\sigma(\mathbf{z}/T)\) sharpens or flattens.

Backpropagation efficiently computes gradients via the chain rule. The error signal propagates as \(\delta^{(l)} = ((W^{(l+1)})^\top \delta^{(l+1)}) \odot \sigma'(z^{(l)})\), and weight gradients are outer products \(\partial\mathcal{L}/\partial W^{(l)} = \delta^{(l)}(a^{(l-1)})^\top\); bias gradients are simply \(\delta^{(l)}\). Backward cost \(\approx 2\times\) forward cost. Key gradient identities: \(\partial\|x\|_2^2/\partial x = 2x\), \(\partial\mathcal{L}/\partial z = \hat y - y\) for both MSE \(\mathcal{L}_{\mathrm{MSE}} = \frac{1}{N}\sum_i (y_i - \hat y_i)^2\) and cross-entropy + softmax \(\partial z = \hat y - y\). Loss functions include MAE \(\frac{1}{N}\sum |y_i - \hat y_i|\) (robust to outliers, non-differentiable at 0), Huber (quadratic for small residuals, linear for large), and hinge \(\max(0, 1-y\hat f)\) for SVMs. The vanishing gradient problem arises because \(\partial\mathcal{L}/\partial W^{(1)} = \prod_{l=1}^{L-1}(W^{(l)})^\top \cdot \delta^{(L)}\)—if each factor has norm \(<1\), products decay exponentially. Exploding gradients occur analogously when norms exceed 1. Solutions include residual connections \(\mathbf{y} = F(\mathbf{x},\{W_i\}) + \mathbf{x}\) (gradient flows through identity shortcut \(\partial\mathcal{L}/\partial\mathbf{x} = \partial\mathcal{L}/\partial\mathbf{y}(1 + \partial F/\partial\mathbf{x})\)), proper initialization, and normalization layers.

Weight initialization matters for signal propagation. Xavier (Glorot) initialization \(W \sim \mathcal{N}(0, 2/(n_{\text{in}}+n_{\text{out}}))\) preserves variance for tanh/sigmoid activations. He initialization \(W \sim \mathcal{N}(0, 2/n_{\text{in}})\) accounts for ReLU killing half the activations. Normalization layers further stabilize training. Batch normalization normalizes over the batch dimension \(\hat x_i = (x_i - \mu_{\mathcal{B}})/\sqrt{\sigma_{\mathcal{B}}^2+\epsilon}\), then rescales with learned \(\gamma, \beta\); at inference, uses running exponential moving averages \(\mu_{\mathrm{running}} = (1-m)\mu_{\mathrm{running}} + m\mu_{\mathcal{B}}\). Layer normalization normalizes over features within each sample (batch-independent, preferred for transformers). Group normalization divides channels into groups; instance normalization is per-sample per-channel (used in style transfer). Spectral normalization \(\bar W = W/\sigma_{\max}(W)\) enforces 1-Lipschitz layers (stabilizes GANs). Regularization prevents overfitting: L2 (weight decay) adds \(\frac{\lambda}{2}\sum w_j^2\) with gradient \(\lambda\theta\) (equivalent to Gaussian prior in MAP); L1 adds \(\lambda\sum|w_j|\) (Laplace prior, encourages sparsity). Dropout randomly zeroes activations \(\tilde a_i = a_i m_i/(1-p)\) with Bernoulli mask \(m_i\); inverted dropout scaling \(1/(1-p)\) makes \(\mathbb{E}[\tilde a] = a\), so no rescaling needed at test time—acts as ensemble training.

Convolutional networks introduce strong inductive biases. A 2D convolution slides a kernel over the input: \((I*K)[i,j] = \sum_{mn} I[i+m,j+n]K[m,n]\)—local connectivity and weight sharing (translation equivariance, parameter-independent of spatial size). Pooling reduces spatial dimensions: max pooling \(\max_{(m,n)} X_{im,jn}\) takes the maximum in each window, average pooling takes the mean. Receptive field grows linearly with depth (kernel size \(k\) gives \(k + (k-1)(l-1)\) at layer \(l\)). Residual (skip) connections \(\mathbf{y} = F(\mathbf{x},\{W_i\}) + \mathbf{x}\) let layers learn residuals; the identity shortcut guarantees non-vanishing gradients. Pre-activation design moves BN+ReLU before weights, giving a clean identity path through the skip connection. The universal approximation theorem guarantees that a single hidden layer with enough neurons can approximate any continuous function \(\sup|\hat f - f| < \epsilon\) to arbitrary precision, but doesn't address learnability or parameter efficiency—deep networks with ReLU create \(O((N/k)^{(k-1)d})\) linear regions, exponentially more than wide shallow networks with the same parameter count. Transposed convolutions provide learnable upsampling in decoders. Spectral norm regularization, knowledge distillation \(\mathcal{L}_{\mathrm{KD}} = T^2 D_{\mathrm{KL}}(\text{softmax}(z_T/T)\|\text{softmax}(z_S/T))\) with temperature \(T\), and the LSTM cell state \(C_t = f_t \odot C_{t-1} + i_t \odot \tilde C_t\) (forget, input, output gates \(f_t, i_t, o_t = \sigma(W[h,x]+b)\) with candidate \(\tilde C_t = \tanh(W_C[h,x]+b_C)\)) are additional architectural primitives. Sharpness-aware minimization (SAM) \(\min_\theta \max_{\|\epsilon\|\leq\rho} \mathcal{L}(\theta+\epsilon)\) explicitly seeks flat minima for better generalization.

All chapters
  1. 1Linear Algebra Foundations
  2. 2Matrix Decompositions and Subspaces
  3. 3Differential Calculus
  4. 4Optimization Theory and Methods
  5. 5Probability and Statistical Inference
  6. 6Information Theory
  7. 7Deep Learning Foundations
  8. 8Transformer Architecture

Drill it

Reading is not remembering. These come from the AI Math deck:

Q

What is a vector in the context of machine learning?

\[\mathbf{x} = \begin{bmatrix}x_1 \\ x_2 \\ \vdots \\ x_n\end{bmatrix} \in \mathbb{R}^n\]Symbols: \(x_i\) = i-th component; \(\mathbb{R}^n\) = n-dimensional rea...

Q

How do you add two vectors?

\[\mathbf{a} + \mathbf{b} = \begin{bmatrix}a_1+b_1 \\ a_2+b_2 \\ \vdots \\ a_n+b_n\end{bmatrix}\]Symbols: \(a_i, b_i\) = corresponding components of vectors a a...

Q

How do you multiply a vector by a scalar?

\[c\mathbf{x} = \begin{bmatrix}cx_1 \\ cx_2 \\ \vdots \\ cx_n\end{bmatrix}\]Symbols: \(c \in \mathbb{R}\) = scalar; \(x_i\) = i-th component.Intuition: Scales e...

Q

What is the L2 (Euclidean) norm of a vector?

\[\|\mathbf{x}\|_2 = \sqrt{\sum_{i=1}^n x_i^2}\]Symbols: \(\|\cdot\|_2\) = L2 norm; \(x_i\) = i-th component; \(n\) = dimension.Intuition: The straight-line (Eu...