Calculus is built on the idea of a limit. The notation \(\lim_{x \to a} f(x) = L\) means that as \(x\) gets arbitrarily close to \(a\), the values \(f(x)\) get arbitrarily close to \(L\). A function is continuous at \(a\) when the limit exists and equals \(f(a)\); otherwise it has a removable discontinuity (a hole), a jump discontinuity (two different one-sided limits), or an infinite discontinuity (a vertical asymptote). Limit laws let you break complicated limits into simpler pieces: the limit of a sum is the sum of the limits, the limit of a product is the product of the limits, and a constant can be factored out. Special limits like \(\lim_{x \to 0} \sin(x)/x = 1\) and \(\lim_{x \to \infty} (1+1/n)^n = e\) anchor many derivations.
The derivative measures instantaneous rate of change. Formally, \(f'(a) = \lim_{h \to 0} [f(a+h) - f(a)]/h\), which geometrically is the slope of the tangent line at \((a, f(a))\) and physically is velocity when \(f\) is position. A handful of rules handle most differentiable functions: the power rule \(d/dx[x^n] = nx^{n-1}\), the product rule, the quotient rule, the chain rule \(d/dx[f(g(x))] = f'(g(x)) g'(x)\), and the linearity of differentiation. The chain rule is the most important for machine learning because it lets gradients flow through nested functions, which is exactly the structure of a neural network where each layer is a function of the previous layer's output. The exponential function is its own derivative: \(d/dx[e^x] = e^x\), and the natural logarithm differentiates to \(1/x\).
Critical points occur where the derivative is zero or undefined, and they are candidates for local maxima, local minima, or saddle points. The first derivative test examines the sign change of \(f'\), and the second derivative test uses the concavity (encoded in \(f''\)) to classify critical points. The Mean Value Theorem guarantees that somewhere between any two points on a differentiable curve, the tangent slope equals the average slope. Implicit differentiation extends the chain rule to equations that mix \(x\) and \(y\) without solving for \(y\) explicitly. Activation functions used in neural networks have well-known derivatives: \(\sigma'(x) = \sigma(x)(1-\sigma(x))\) for the sigmoid, \(\tanh'(x) = 1 - \tanh^2(x)\), and \(\text{ReLU}'(x) = \mathbf{1}[x>0]\).
Integration is the reverse of differentiation. An antiderivative \(F\) satisfies \(F' = f\), and the indefinite integral \(\int f(x)\,dx\) represents the family of all antiderivatives plus an arbitrary constant \(C\). The definite integral \(\int_a^b f(x)\,dx\) computes the net signed area under \(f\) between \(a\) and \(b\). The Fundamental Theorem of Calculus links the two ideas: \(\int_a^b f = F(b) - F(a)\) where \(F\) is any antiderivative. Integration techniques reverse the differentiation rules: substitution undoes the chain rule, integration by parts undoes the product rule, and the power rule for integration adds one to the exponent and divides. In probability, integrals compute expectations and tail probabilities from continuous densities, and in machine learning they appear in loss derivations, variational inference, and the Gaussian integral \(\int_{-\infty}^{\infty} e^{-x^2} dx = \sqrt{\pi}\).