AI Math Beginner

This deck introduces the foundational ideas of exponents and roots, starting with simple questions like what a "power" really means and moving up to the rules that govern how expressions with exponents behave. You'll explore concepts such as bases, exponents, negative powers, the power-of-a-power rule, and fractional exponents, as well as what square roots and cube roots actually represent. Together, these ideas form the building blocks for nearly every mathematical formula you'll meet later on.

It's designed for learners who are just beginning their journey into the math behind artificial intelligence and machine learning, especially those who want a clear, gentle refresher on topics they may have seen in school but not used in a while. If you're preparing to study more advanced material such as logarithms, algebra, or calculus, the rules in this deck will show up again and again, so getting comfortable with them now will save you a lot of effort down the road.

Because these rules are short but easy to mix up, the best way to study this deck is to work through a small set of cards at a time and return to them over several days rather than trying to memorize everything in one sitting. When a rule feels abstract, try writing out a tiny example by hand, like rewriting \(2^5 \cdot 2^3\) as \(2^8\), so your brain links the rule to a concrete calculation. With steady, spaced-out review, the patterns here will quickly start to feel like second nature.

The most basic operations in mathematics, after addition and multiplication, are powers and roots. An exponent tells you how many times to multiply a base by itself: \(a^n\) means \(a \times a \times \cdots \times a\) repeated \(n\) times. Two facts anchor all the rules that follow. Any nonzero number raised to the zero power equals one, because \(a^n / a^n = a^0 = 1\), and a negative exponent means the reciprocal of the corresponding positive power, so \(x^{-n} = 1/x^n\). These definitions preserve the elegance of the exponent rules across all integers and, ultimately, all real numbers.

The rules for combining powers follow directly from how multiplication behaves. Multiplying two powers with the same base adds their exponents: \(x^a \cdot x^b = x^{a+b}\). Dividing subtracts them: \(x^a / x^b = x^{a-b}\). Raising a power to another power multiplies them: \((x^a)^b = x^{ab}\). And when an exponent applies to a product or a fraction, it distributes to every factor: \((xy)^n = x^n y^n\) and \((x/y)^n = x^n / y^n\). Together these rules let you simplify almost any expression involving powers.

Roots and fractional exponents are two ways of describing the same operation. The square root of \(a\) is the non-negative number whose square is \(a\), while the \(n\)th root \(\sqrt[n]{x}\) is the number whose \(n\)th power is \(x\). Fractional exponents combine a root and a power: \(x^{m/n} = \left(\sqrt[n]{x}\right)^m\). Square roots obey the same distribution rules as exponents: \(\sqrt{ab} = \sqrt{a}\sqrt{b}\) and \(\sqrt{a/b} = \sqrt{a}/\sqrt{b}\). When a radical appears in a denominator, multiplying by a clever form of one (rationalizing) clears it cleanly.

Logarithms answer the inverse question: given a base and a result, what exponent produced it? The definition \(\log_b(x) = y \iff b^y = x\) makes clear that logs and exponentials are two sides of the same coin. Special cases include the common log base 10 and the natural log base \(e \approx 2.718\). Three rules govern how logs behave: \(\log_b(xy) = \log_b x + \log_b y\), \(\log_b(x/y) = \log_b x - \log_b y\), and \(\log_b(x^n) = n \log_b x\). The change-of-base formula lets you rewrite any logarithm in any convenient base. These properties are essential throughout machine learning because they convert multiplication into addition (which is what makes log-likelihoods tractable) and because they tame the exponential growth of quantities like probabilities and information content, where \(-\log_2(p)\) measures the surprise of an event with probability \(p\).

Algebra is the language of unknowns. A variable, usually a letter like \(x\) or \(\theta\), stands in for a number we may or may not know, and an expression combines variables with the standard operations. Evaluating an expression means substituting known values for variables and simplifying using the order of operations (parentheses, exponents, multiplication and division from left to right, then addition and subtraction). Like terms share the same variable part, so \(3x^2\) and \(-7x^2\) combine freely while \(3x^2\) and \(3x\) do not; combining means adding or subtracting coefficients, never exponents.

The distributive property, \(a(b+c) = ab + ac\), is the engine that powers both expansion and factoring. To expand, you distribute a factor across every term inside parentheses; to factor, you reverse the process, pulling out the greatest common factor. The FOIL pattern (First, Outer, Inner, Last) is a specialized distributive technique for multiplying two binomials. Several factoring identities appear constantly: \(a^2 - b^2 = (a+b)(a-b)\) (difference of squares), \((a \pm b)^2 = a^2 \pm 2ab + b^2\) (perfect square trinomials), and \(a^3 \pm b^3 = (a \pm b)(a^2 \mp ab + b^2)\) (sum or difference of cubes). For general trinomials \(ax^2 + bx + c\), you find two numbers whose product is \(ac\) and whose sum is \(b\), then split the middle term and factor by grouping.

Solving an equation means isolating the variable using inverse operations while keeping both sides balanced. Linear equations collapse to a single value through simple arithmetic; systems of two linear equations can be solved by substitution or by adding the equations after multiplying so that one variable cancels. Quadratic equations can be solved by factoring (and applying the zero product property, which says that if a product equals zero then at least one factor must be zero), by completing the square (adding \((b/2)^2\) to form a perfect square), or by the quadratic formula \(x = (-b \pm \sqrt{b^2-4ac})/(2a)\). The discriminant \(b^2 - 4ac\) tells you at a glance whether there are two real roots, one repeated root, or two complex roots.

Rational expressions, which are ratios of polynomials, behave much like numerical fractions. You multiply them by multiplying numerators and denominators, divide by flipping the second fraction, and add or subtract them by finding a common denominator. Simplifying means factoring and canceling, while remembering that values making the denominator zero are excluded from the domain. Beyond solving, algebra also studies polynomial structure: a polynomial's degree is its highest power, the Fundamental Theorem of Algebra guarantees \(n\) roots for a degree-\(n\) polynomial (counting multiplicity), and polynomials can be added, subtracted, multiplied, and divided using long division. These tools build up from arithmetic and form the basis for nearly every higher idea in mathematics.

A function is a rule that assigns exactly one output to each input, written \(y = f(x)\) and read \(f\) of \(x\). The domain is the set of valid inputs and the range is the set of actual outputs produced. The vertical line test identifies graphs that represent functions, and the horizontal line test identifies which functions are one-to-one and therefore invertible. Common families include linear functions \(f(x) = mx + b\), quadratic functions \(f(x) = ax^2 + bx + c\), exponential functions, logarithmic functions, polynomial functions, and rational functions. Each family has its own characteristic shape, asymptotes, and growth behavior.

Linear functions describe any constant rate of change. The slope-intercept form \(y = mx + b\) makes the slope \(m\) (rise over run) and y-intercept \(b\) immediately visible; the point-slope form \(y - y_1 = m(x - x_1)\) is convenient when you know one point and the slope. Two lines are parallel when their slopes are equal and perpendicular when their slopes are negative reciprocals. Quadratic functions trace parabolas whose vertex is at \((-b/(2a), f(-b/(2a)))\), the axis of symmetry \(x = -b/(2a)\) being a vertical line through the vertex. Functions can be shifted horizontally or vertically, reflected across the axes, or stretched/compressed by multiplying the input or output by constants.

The Cartesian coordinate system anchors geometry to algebra by representing points as ordered pairs \((x, y)\) on perpendicular axes. The four quadrants correspond to the four sign combinations of the coordinates. The distance formula \(d = \sqrt{(x_2-x_1)^2 + (y_2-y_1)^2}\) extends the Pythagorean theorem to any two points, and the midpoint formula averages each coordinate. Three-dimensional coordinates add a third axis \(z\), with distances computed by summing the squared differences across all three dimensions.

Vectors encode both magnitude and direction, and they support operations tailored to those two aspects. Geometrically, a 2D vector is an arrow; algebraically, it is a pair of components \((v_x, v_y)\). Addition places vectors head-to-tail, scalar multiplication stretches or flips them, and the dot product \(\mathbf{a} \cdot \mathbf{b} = a_x b_x + a_y b_y\) equals \(\|\mathbf{a}\|\|\mathbf{b}\|\cos\theta\), encoding both magnitude and the angle between them. Two vectors are orthogonal precisely when their dot product is zero. The dot product therefore measures directional similarity and underlies everything from projections to cosine similarity to neural attention. The cross product, defined in three dimensions, produces a vector perpendicular to both inputs with magnitude equal to the area of the parallelogram they span.

A sequence is an ordered list of numbers indexed by the positive integers, \(a_1, a_2, a_3, \ldots\), with a general term \(a_n\) given by some formula. The simplest families are arithmetic sequences, where each term differs from the previous by a fixed common difference \(d\) (so \(a_n = a_1 + (n-1)d\)), and geometric sequences, where each term is obtained by multiplying by a fixed common ratio \(r\) (so \(a_n = a_1 \cdot r^{n-1}\)). The sums of arithmetic and geometric series both have closed forms: \(S_n = n(a_1+a_n)/2\) for arithmetic, and \(S_n = a_1(1-r^n)/(1-r)\) for geometric with \(r \neq 1\).

Sigma notation \(\sum_{i=m}^{n} a_i\) compresses a sum into a compact form, with \(i\) being the index running from the lower limit \(m\) to the upper limit \(n\). Two properties make summation flexible: a constant factor can be pulled outside the sum, and a sum of sums splits into separate sums. More usefully, sums can be re-indexed (shifting the index by a constant adjusts both limits) and split at intermediate points. A double summation \(\sum_i \sum_j a_{ij}\) sums over a two-dimensional grid, which is exactly the structure of matrix multiplication and the layer-by-layer operations in a neural network.

The behavior of an infinite series depends on whether its sequence of partial sums approaches a finite limit. A geometric series with \(|r|<1\) converges to \(a/(1-r)\); one with \(|r| \geq 1\) diverges. The ratio test gives a general criterion: if \(\lim |a_{n+1}/a_n| < 1\) the series converges absolutely, if greater than 1 it diverges. A necessary condition is that \(a_n \to 0\), but not sufficient: the harmonic series \(\sum 1/n\) diverges despite its terms shrinking to zero. Telescoping series offer an elegant exception, where consecutive terms cancel in pairs so only the boundary values survive.

Two specific formulas appear constantly. First, the sum of the first \(n\) positive integers is \(n(n+1)/2\) (Gauss's famous result), and the sum of the first \(n\) squares is \(n(n+1)(2n+1)/6\). Second, factorials \(n!\) count arrangements, binomial coefficients \(\binom{n}{k} = n!/(k!(n-k)!)\) count unordered selections, and Pascal's identity \(\binom{n}{k} = \binom{n-1}{k-1} + \binom{n-1}{k}\) gives a recursive way to build them. The binomial theorem \((x+y)^n = \sum_{k=0}^n \binom{n}{k} x^{n-k} y^k\) packages all this into a single expansion formula. In machine learning, summation notation is essential for expressing expectations, dot products, mean squared error, cross-entropy, sample variance, and the matrix multiplications that drive every forward pass through a network.

Calculus is built on the idea of a limit. The notation \(\lim_{x \to a} f(x) = L\) means that as \(x\) gets arbitrarily close to \(a\), the values \(f(x)\) get arbitrarily close to \(L\). A function is continuous at \(a\) when the limit exists and equals \(f(a)\); otherwise it has a removable discontinuity (a hole), a jump discontinuity (two different one-sided limits), or an infinite discontinuity (a vertical asymptote). Limit laws let you break complicated limits into simpler pieces: the limit of a sum is the sum of the limits, the limit of a product is the product of the limits, and a constant can be factored out. Special limits like \(\lim_{x \to 0} \sin(x)/x = 1\) and \(\lim_{x \to \infty} (1+1/n)^n = e\) anchor many derivations.

The derivative measures instantaneous rate of change. Formally, \(f'(a) = \lim_{h \to 0} [f(a+h) - f(a)]/h\), which geometrically is the slope of the tangent line at \((a, f(a))\) and physically is velocity when \(f\) is position. A handful of rules handle most differentiable functions: the power rule \(d/dx[x^n] = nx^{n-1}\), the product rule, the quotient rule, the chain rule \(d/dx[f(g(x))] = f'(g(x)) g'(x)\), and the linearity of differentiation. The chain rule is the most important for machine learning because it lets gradients flow through nested functions, which is exactly the structure of a neural network where each layer is a function of the previous layer's output. The exponential function is its own derivative: \(d/dx[e^x] = e^x\), and the natural logarithm differentiates to \(1/x\).

Critical points occur where the derivative is zero or undefined, and they are candidates for local maxima, local minima, or saddle points. The first derivative test examines the sign change of \(f'\), and the second derivative test uses the concavity (encoded in \(f''\)) to classify critical points. The Mean Value Theorem guarantees that somewhere between any two points on a differentiable curve, the tangent slope equals the average slope. Implicit differentiation extends the chain rule to equations that mix \(x\) and \(y\) without solving for \(y\) explicitly. Activation functions used in neural networks have well-known derivatives: \(\sigma'(x) = \sigma(x)(1-\sigma(x))\) for the sigmoid, \(\tanh'(x) = 1 - \tanh^2(x)\), and \(\text{ReLU}'(x) = \mathbf{1}[x>0]\).

Integration is the reverse of differentiation. An antiderivative \(F\) satisfies \(F' = f\), and the indefinite integral \(\int f(x)\,dx\) represents the family of all antiderivatives plus an arbitrary constant \(C\). The definite integral \(\int_a^b f(x)\,dx\) computes the net signed area under \(f\) between \(a\) and \(b\). The Fundamental Theorem of Calculus links the two ideas: \(\int_a^b f = F(b) - F(a)\) where \(F\) is any antiderivative. Integration techniques reverse the differentiation rules: substitution undoes the chain rule, integration by parts undoes the product rule, and the power rule for integration adds one to the exponent and divides. In probability, integrals compute expectations and tail probabilities from continuous densities, and in machine learning they appear in loss derivations, variational inference, and the Gaussian integral \(\int_{-\infty}^{\infty} e^{-x^2} dx = \sqrt{\pi}\).

Multivariable calculus extends differentiation to functions of many variables. The partial derivative \(\partial f/\partial x_i\) measures how \(f\) changes when only \(x_i\) moves; the gradient \(\nabla f\) collects all partial derivatives into a vector that points in the direction of steepest ascent, with magnitude equal to the rate of increase. To minimize a function, you move in the opposite direction, which is exactly why gradient descent updates parameters as \(\theta_{t+1} = \theta_t - \eta \nabla L(\theta_t)\). The Hessian matrix collects all second partial derivatives and encodes the local curvature: at a critical point, a positive-definite Hessian indicates a local minimum, a negative-definite Hessian indicates a local maximum, and mixed eigenvalues signal a saddle point.

The chain rule generalizes to several variables through the Jacobian matrix, whose entries are the first-order partial derivatives of a vector-valued function. Composing functions multiplies their Jacobians. In backpropagation, this chain is unrolled layer by layer, multiplying local derivatives from the loss all the way back to each weight. Automatic differentiation implements this automatically by tracking operations in a computational graph, computing exact derivatives up to machine precision in either forward or reverse mode, with reverse mode being the natural choice for backpropagation since it handles many parameters efficiently.

algebra_essentials">Linear algebra supplies the language of vectors, matrices, and the transformations between them. A matrix encodes a linear map, and matrix multiplication composes such maps. Special matrices include the identity, diagonal, triangular, symmetric, orthogonal, and positive-definite matrices, each with useful properties. The determinant measures how a transformation scales volumes, and a matrix is invertible precisely when its determinant is nonzero. Eigenvalues and eigenvectors characterize how a matrix acts along its principal directions: \(A\mathbf{v} = \lambda \mathbf{v}\). Diagonalization writes a matrix as \(A = PDP^{-1}\), and the spectral theorem guarantees that every symmetric matrix can be orthogonally diagonalized with real eigenvalues.

The singular value decomposition (SVD) factors any matrix \(A = U\Sigma V^\top\) and generalizes eigendecomposition to non-square matrices. Truncating the SVD yields the best low-rank approximation in the Frobenius norm, which underlies principal component analysis (PCA), recommender systems, and matrix factorization. Optimization theory builds on convexity: a convex function has the property that every local minimum is also a global minimum, so gradient descent is guaranteed to find the best solution. For non-convex problems common in deep learning, variants like SGD, momentum, Adam, and learning rate schedules (warmup, cosine annealing) help navigate complex loss landscapes. Regularization adds a penalty term to the loss, with L1 promoting sparsity, L2 promoting small weights, and dropout providing implicit regularization through stochastic network perturbations.

Probability theory formalizes uncertainty. A sample space \(\Omega\) contains all possible outcomes, and events are subsets of \(\Omega\). For a uniform sample space, the probability of an event equals the number of favorable outcomes divided by the total. The complement rule \(P(A^c) = 1 - P(A)\), the union rule \(P(A \cup B) = P(A) + P(B) - P(A \cap B)\), and conditional probability \(P(A \mid B) = P(A \cap B)/P(B)\) are the basic tools. Independence means \(P(A \cap B) = P(A)P(B)\), and Bayes' theorem \(P(A \mid B) = P(B \mid A)P(A)/P(B)\) updates beliefs about a hypothesis in light of new evidence.

Random variables assign numerical values to outcomes. Discrete variables have probability mass functions (PMFs) listing \(P(X = x)\) for each value, while continuous variables have probability density functions (PDFs) \(f(x)\) where probabilities are areas \(\int_a^b f(x) dx\). The cumulative distribution function \(F(x) = P(X \leq x)\) is the integral of the PDF. The expected value \(E[X] = \sum x P(X=x)\) (or \(\int x f(x) dx\) in the continuous case) is the probability-weighted average, while variance \(\text{Var}(X) = E[(X-\mu)^2]\) measures spread around the mean. Important distributions include Bernoulli (one trial), binomial (\(n\) trials), Poisson (rare events), the normal/Gaussian (ubiquitous from the Central Limit Theorem), and the exponential (waiting times).

Statistics summarizes data and infers properties of populations. The sample mean \(\bar{x}\) and sample variance \(s^2\) (dividing by \(n-1\) for an unbiased estimate) are the foundational descriptive statistics; the median and mode provide alternative measures of center that are robust to outliers. Covariance and correlation measure how two variables move together, with the Pearson correlation \(r\) normalized to \([-1, 1]\). Hypothesis testing evaluates whether data is consistent with a null hypothesis, with the p-value quantifying the probability of observing data this extreme under that null.

In machine learning, these foundations combine to give us the building blocks of models and training. Maximum likelihood estimation finds parameters maximizing \(P(\text{data} \mid \theta)\), while MAP estimation adds a prior \(P(\theta)\) to encode beliefs. Minimizing mean squared error is equivalent to maximum likelihood under Gaussian noise, and minimizing cross-entropy is equivalent to maximum likelihood under a categorical or Bernoulli model. Entropy \(H(X) = -\sum P(x)\log P(x)\) and KL divergence \(D_{KL}(P \| Q) = \sum P(x)\log(P(x)/Q(x))\) measure uncertainty and distributional difference, respectively, with cross-entropy equal to entropy plus KL divergence. Numerical stability matters as much as mathematical correctness: the log-sum-exp trick, log-space likelihood computation, and mixed precision training prevent overflow and underflow in deep networks. The theoretical insights from convexity, gradient descent, regularization, and the bias-variance tradeoff all converge in the practice of designing and training modern learning systems.