AI Math Beginner Practice Exam

Question 1 AI Math Beginner

Why is the derivative zero at a local extremum?

Factor out the largest perfect square: \(\sqrt{50} = \sqrt{25 \cdot 2} = 5\sqrt{2}\).

\[\sqrt{50} = \sqrt{25 \cdot 2} = 5\sqrt{2}\] At a local maximum or minimum the function stops increasing and starts decreasing (or vice versa), so the tangent line is horizontal and its slope is zero.

\[f \text{ has local extremum at } c \implies f'(c) = 0 \text{ (if } f'(c) \text{ exists)}\] For any positive exponent \(n\), the value \(x^n\) approaches 0 as \(x\) approaches 0.

\[\lim_{x \to 0} x^n = 0 \quad (n > 0)\] In vector form, the parameter vector is updated by subtracting the learning rate times the gradient vector of the loss.

\[\boldsymbol{\theta}_{t+1} = \boldsymbol{\theta}_t - \eta \nabla_{\boldsymbol{\theta}} L(\boldsymbol{\theta}_t)\]

Question 2 AI Math Beginner

How do you write the L1 norm using summation?

The L1 norm of a vector is the sum of the absolute values of its components.

\[\lVert \mathbf{x} \rVert_1 = \sum_{i=1}^{n} \lvert x_i \rvert\] Approximately 68% of values fall within 1 standard deviation of the mean, 95% within 2, and 99.7% within 3.

\[P(\mu - k\sigma \leq X \leq \mu + k\sigma) \approx \begin{cases} 0.68 & k=1 \\ 0.95 & k=2 \\ 0.997 & k=3 \end{cases}\] A subsequence is formed by selecting an infinite collection of terms from a sequence while preserving their original order. It is written \(\{a_{n_k}\}\) where the indices satisfy \(n_1 < n_2 < n_3 < \cdots\).

\[ a_{n_1},\; a_{n_2},\; a_{n_3},\; \ldots \quad (n_1 < n_2 < n_3 < \cdots) \] Two expressions are equivalent if they produce the same value for every substitution of the variable(s).

\[2(x+3) \equiv 2x+6 \text{ for all } x\]

Question 3 AI Math Beginner

What is the negative binomial distribution?

The negative binomial counts the number of trials needed to achieve \(r\) successes, generalizing the geometric distribution.

\[P(X = k) = \binom{k-1}{r-1} p^r (1-p)^{k-r}\] The absolute value of any real number is always greater than or equal to zero, equaling zero only when the number itself is zero.

\[\lvert x \rvert \geq 0, \text{ and } \lvert x \rvert = 0 \iff x = 0\] A metric space is a set with a distance function satisfying non-negativity, identity of indiscernibles, symmetry, and the triangle inequality.

\[d(x, y) \geq 0, \quad d(x,y) = 0 \iff x = y, \quad d(x,y) = d(y,x), \quad d(x,z) \leq d(x,y) + d(y,z)\] The common ratio \(r\) is the constant factor between consecutive terms. It equals any term divided by the immediately preceding term.

\[ r = \frac{a_{n+1}}{a_n} \]

Question 4 AI Math Beginner

What is log_2(1)?

It equals 0 because 2 raised to the power 0 equals 1.

\[\log_2(1) = 0 \text{ because } 2^0 = 1\] When two functions are multiplied, the derivative is the first times the derivative of the second plus the second times the derivative of the first.

\[\frac{d}{dx}[f(x) g(x)] = f(x) g'(x) + f'(x) g(x)\] A positive slope means the line rises from left to right; a negative slope means the line falls from left to right.

\[m > 0: \text{ rising line},\quad m < 0: \text{ falling line}\] It is the length of the perpendicular dropped from the point to the line, given by a formula using the line's coefficients.

\[ d = \frac{\lvert ax_0 + by_0 + c \rvert}{\sqrt{a^2 + b^2}} \]

Question 5 AI Math Beginner

What are irrational numbers?

The common ratio \(r\) is the constant factor between consecutive terms. It equals any term divided by the immediately preceding term.

\[ r = \frac{a_{n+1}}{a_n} \] The \(i\)-th component of the product \(A\mathbf{x}\) is the dot product of the \(i\)-th row of \(A\) with the vector \(\mathbf{x}\).

\[(A\mathbf{x})_i = \sum_{j=1}^{n} A_{ij}\,x_j\] The cross product produces a vector perpendicular to both inputs with magnitude equal to the area of the parallelogram they span.

\[\mathbf{a} \times \mathbf{b} = \begin{vmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\ a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \end{vmatrix}\] Irrational numbers are real numbers that cannot be written as a ratio of two integers; their decimal expansions never terminate or repeat, e.g. \(\sqrt{2}\) and \(\pi\).

\[ \mathbb{R} \setminus \mathbb{Q} = \{x \in \mathbb{R} \mid x \notin \mathbb{Q}\} \]

Question 6 AI Math Beginner

How does SVD relate to PCA?

Find two numbers that multiply to \(c\) and add to \(b\), then write the trinomial as a product of two binomials.

\[x^2 + 5x + 6 = (x+2)(x+3)\] PCA can be computed via SVD of the data matrix: the right singular vectors give the principal components and the singular values squared (divided by \(n-1\)) give the eigenvalues of the covariance matrix.

\[X = U\Sigma V^\top \implies \text{PCA directions} = \text{columns of } V\] A sum telescopes when consecutive terms partially cancel so that intermediate values disappear and only the first and last boundary terms survive.

\[\sum_{i=1}^{n}(f(i) - f(i-1)) = f(n) - f(0)\] At a critical point, if the Hessian is positive definite it is a local minimum, if negative definite it is a local maximum, and if indefinite (mixed signs) it is a saddle point.

\[H \succ 0 \implies \text{min}; \quad H \prec 0 \implies \text{max}; \quad \text{mixed} \implies \text{saddle}\]

Question 7 AI Math Beginner

How do you simplify the square root of a fraction?

A difference of two perfect squares factors into the product of a sum and a difference.

\[a^2 - b^2 = (a+b)(a-b)\] This is the geometric series, converging for \(|x| < 1\).

\[\frac{1}{1-x} = \sum_{n=0}^{\infty} x^n = 1 + x + x^2 + x^3 + \cdots \quad (|x| < 1)\] The square root of a fraction equals the square root of the numerator divided by the square root of the denominator.

\[\sqrt{\frac{a}{b}} = \frac{\sqrt{a}}{\sqrt{b}} \quad (a \geq 0,\, b > 0)\] Leibniz notation uses \(\frac{df}{dx}\), Lagrange notation uses \(f'(x)\), and Newton notation uses \(\dot{f}\); all mean the same thing.

\[\frac{df}{dx} = f'(x) = \dot{f}\]

Question 8 AI Math Beginner

What is the score function?

An indeterminate form like \(0/0\) or \(\infty/\infty\) means the limit cannot be determined by direct substitution and requires further analysis.

\[\text{Common forms: } \frac{0}{0},\; \frac{\infty}{\infty},\; 0 \cdot \infty,\; \infty - \infty,\; 0^0,\; 1^\infty,\; \infty^0\] The score function is the gradient of the log-likelihood with respect to parameters; its expected value is zero, and its variance is the Fisher information.

\[s(\theta) = \nabla_\theta \log P(x \mid \theta), \quad E[s(\theta)] = 0\] The expected value is the probability-weighted sum of all possible values of the random variable.

\[E[X] = \sum_{i} x_i \, P(X = x_i)\] The natural log goes to negative infinity as its argument approaches zero from the right.

\[\lim_{x \to 0^+} \ln(x) = -\infty\]

Question 9 AI Math Beginner

What is log_b(x) + log_b(y) simplified?

Coordinate descent minimizes a function by optimizing one variable at a time while holding all others fixed; it is simple but can be slow for tightly coupled variables.

\[\text{Update } x_i \leftarrow \arg\min_{x_i} f(x_1, \ldots, x_i, \ldots, x_n) \text{ for each } i\] The log decreases without bound, approaching negative infinity as x gets arbitrarily close to 0 from the positive side.

\[\lim_{x \to 0^+} \log(x) = -\infty\] An affine transformation is a linear transformation followed by a translation; neural network layers perform affine transformations before applying activations.

\[\mathbf{y} = W\mathbf{x} + \mathbf{b}\] By the product rule, the sum of two logs with the same base equals the log of their product.

\[\log_b(x) + \log_b(y) = \log_b(x \cdot y)\]

Question 10 AI Math Beginner

What is the Gaussian error function (erf)?

The optimal weights minimizing MSE can be found directly using the normal equation, without iterative optimization.

\[\mathbf{w}^* = (X^\top X)^{-1} X^\top \mathbf{y}\] Two expressions are equivalent if they produce the same value for every substitution of the variable(s).

\[2(x+3) \equiv 2x+6 \text{ for all } x\] At a critical point where \(f'(c) = 0\): if \(f''(c) > 0\) it is a local minimum, if \(f''(c) < 0\) it is a local maximum, and if \(f''(c) = 0\) the test is inconclusive.

\[f'(c) = 0 \text{ and } f''(c) > 0 \implies \text{local min}; \quad f''(c) < 0 \implies \text{local max}\] The error function is related to the CDF of the standard normal and appears in the GELU activation and probability calculations.

\[\text{erf}(x) = \frac{2}{\sqrt{\pi}} \int_0^x e^{-t^2}\, dt\]

Question 11 AI Math Beginner

What is the exponential family of distributions?

The absolute value of a negative number is its positive counterpart — you negate it to get the distance from zero.

\[\lvert -7 \rvert = -(-7) = 7\] When \(\lvert r \rvert < 1\), the infinite series sums to the first term divided by one minus the common ratio.

\[ S = \frac{a_1}{1 - r} \] The exponential family is a broad class of distributions (Gaussian, Bernoulli, Poisson, etc.) that share a common mathematical form with natural parameters and sufficient statistics.

\[P(x \mid \eta) = h(x) \exp(\eta^\top T(x) - A(\eta))\] Approximately 68% of values fall within 1 standard deviation of the mean, 95% within 2, and 99.7% within 3.

\[P(\mu - k\sigma \leq X \leq \mu + k\sigma) \approx \begin{cases} 0.68 & k=1 \\ 0.95 & k=2 \\ 0.997 & k=3 \end{cases}\]

Question 12 AI Math Beginner

What is a saddle point in multivariable calculus?

At a local maximum or minimum the function stops increasing and starts decreasing (or vice versa), so the tangent line is horizontal and its slope is zero.

\[f \text{ has local extremum at } c \implies f'(c) = 0 \text{ (if } f'(c) \text{ exists)}\] This integral gives the arctangent function, which is the inverse of tangent.

\[\int \frac{1}{1+x^2}\, dx = \arctan(x) + C\] A saddle point is a critical point where the gradient is zero but the point is neither a local max nor a local min; the function curves up in some directions and down in others.

\[\nabla f = \mathbf{0} \text{ but } H \text{ has both positive and negative eigenvalues}\] Approximately 68% of values fall within 1 standard deviation of the mean, 95% within 2, and 99.7% within 3.

\[P(\mu - k\sigma \leq X \leq \mu + k\sigma) \approx \begin{cases} 0.68 & k=1 \\ 0.95 & k=2 \\ 0.997 & k=3 \end{cases}\]

Question 13 AI Math Beginner

What does the notation f: A to B mean?

The sum of \(n\) terms of a geometric series with first term \(a_1\) and ratio \(r \neq 1\) uses the formula below. When \(r = 1\) the sum is simply \(n a_1\).

\[ S_n = a_1 \cdot \frac{1 - r^n}{1 - r} \] This notation declares that \(f\) is a function with domain \(A\) and codomain \(B\), so every input from \(A\) produces an output in \(B\).

\[f: A \to B \;\text{ means each } a \in A \text{ maps to some } f(a) \in B\] Differentiate with respect to each variable: \(\partial f/\partial x = 2x\) and \(\partial f/\partial y = 2y\), so the gradient is \((2x, 2y)\).

\[\nabla f = \begin{bmatrix} 2x \\ 2y \end{bmatrix}\] Most intermediate terms cancel because each positive piece is annulled by the negative piece of the next term, so the total collapses to the boundary values.

\[\sum_{i=1}^{n}(a_i - a_{i-1}) = (a_1 - a_0) + (a_2 - a_1) + \cdots + (a_n - a_{n-1}) = a_n - a_0\]

Question 14 AI Math Beginner

What is the additivity property of definite integrals?

A definite integral over \([a, c]\) can be split into two integrals over \([a, b]\) and \([b, c]\).

\[\int_a^c f(x)\, dx = \int_a^b f(x)\, dx + \int_b^c f(x)\, dx\] Diagonalization makes matrix powers easy: \(A^k = PD^kP^{-1}\), turning repeated matrix multiplication into simple exponentiation of eigenvalues.

\[A^k = P \text{diag}(\lambda_1^k, \ldots, \lambda_n^k) P^{-1}\] Reflecting across the x-axis keeps the x-coordinate unchanged and negates the y-coordinate.

\[ (x,\, y) \mapsto (x,\, -y) \] The complement of set A, relative to a universal set U, is the collection of all elements in U that are not in A.

\[ A^c = \{x \mid x \in U \text{ and } x \notin A\} \]

Question 15 AI Math Beginner

What is the midpoint formula between two points?

A negative-one exponent flips the fraction, so \((x/y)^{-1} = y/x\).

\[\left(\frac{x}{y}\right)^{-1} = \frac{y}{x} \quad (x \neq 0,\, y \neq 0)\] Set up two cases: the expression inside equals \(a\) or equals \(-a\), then solve each.

\[\lvert x \rvert = a \implies x = a \text{ or } x = -a \quad (a \geq 0)\] The Kronecker delta equals 1 when its two indices are equal and 0 otherwise; it appears in identity matrices and orthonormality conditions.

\[\delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases}\] The midpoint is found by averaging the x-coordinates and averaging the y-coordinates of the two points.

\[ M = \left(\frac{x_1+x_2}{2},\, \frac{y_1+y_2}{2}\right) \]

Question 16 AI Math Beginner

What is the AM-GM inequality?

The arithmetic mean is always at least as large as the geometric mean for non-negative numbers.

\[\frac{a + b}{2} \geq \sqrt{ab} \quad (a, b \geq 0)\] The \(i\)-th component of the product \(A\mathbf{x}\) is the dot product of the \(i\)-th row of \(A\) with the vector \(\mathbf{x}\).

\[(A\mathbf{x})_i = \sum_{j=1}^{n} A_{ij}\,x_j\] For any norm on a vector space, the norm of a sum is at most the sum of the norms, generalizing the real-number triangle inequality.

\[\lVert \mathbf{u} + \mathbf{v} \rVert \leq \lVert \mathbf{u} \rVert + \lVert \mathbf{v} \rVert\] Dropout randomly sets a fraction of neuron outputs to zero during training, preventing co-adaptation and reducing overfitting.

\[\text{Each neuron kept with probability } p, \text{ zeroed with probability } 1-p\]

Question 17 AI Math Beginner

What is the Fisher information matrix?

The Fisher information measures how much information an observable random variable carries about an unknown parameter; it is the expected Hessian of the negative log-likelihood.

\[I(\theta) = E\left[-\frac{\partial^2 \log P(X \mid \theta)}{\partial \theta^2}\right]\] The beta distribution is defined on \([0, 1]\) and is parameterized by two shape parameters; it is the conjugate prior for the Bernoulli and binomial distributions.

\[f(x; \alpha, \beta) = \frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha, \beta)}, \quad x \in [0, 1]\] The spectral theorem states that every real symmetric matrix can be diagonalized by an orthogonal matrix, with all eigenvalues real.

\[A = A^\top \implies A = Q\Lambda Q^\top, \quad Q \text{ orthogonal}, \Lambda \text{ real diagonal}\] The axes divide the plane into four regions: Q1 (+,+), Q2 (−,+), Q3 (−,−), Q4 (+,−), numbered counterclockwise.

\[ \text{QI: } x>0,y>0;\quad \text{QII: } x<0,y>0;\quad \text{QIII: } x<0,y<0;\quad \text{QIV: } x>0,y<0 \]

Question 18 AI Math Beginner

What is the relationship between cross-entropy loss and Bernoulli likelihood?

Minimizing binary cross-entropy is equivalent to maximizing the log-likelihood of a Bernoulli model for binary classification.

\[\min \text{BCE} = \max \sum_i [y_i \log \hat{y}_i + (1-y_i)\log(1-\hat{y}_i)]\] The expected value is the probability-weighted sum of all possible values of the random variable.

\[E[X] = \sum_{i} x_i \, P(X = x_i)\] Reflecting across the y-axis negates the x-coordinate while keeping the y-coordinate unchanged.

\[ (x,\, y) \mapsto (-x,\, y) \] A level curve is the set of all points where a function of two variables takes a constant value; gradients are perpendicular to level curves.

\[\{(x, y) : f(x, y) = c\} \quad \text{(level curve for constant } c\text{)}\]

Question 19 AI Math Beginner

What is a steep slope vs a shallow slope?

The Frobenius norm squared is the sum of the squares of every matrix entry, computed via a double sum over rows and columns.

\[\lVert A \rVert_F^2 = \sum_{i=1}^{m}\sum_{j=1}^{n} A_{ij}^2\] Factor out the largest perfect square: \(\sqrt{50} = \sqrt{25 \cdot 2} = 5\sqrt{2}\).

\[\sqrt{50} = \sqrt{25 \cdot 2} = 5\sqrt{2}\] A steep slope has a large absolute value of m so the line is nearly vertical, while a shallow slope has m near zero so the line is nearly horizontal.

\[ \lvert m \rvert \gg 1 \text{ (steep)},\quad \lvert m \rvert \approx 0 \text{ (shallow)} \] The natural numbers are the non-negative whole numbers used for counting; conventions differ on whether 0 is included, but in most modern usage \(\mathbb{N}\) includes 0.

\[ \mathbb{N} = \{0, 1, 2, 3, \ldots\} \]

Question 20 AI Math Beginner

What is log_b(x) - log_b(y) simplified?

A constant factor can be pulled outside the integral sign, just as with derivatives.

\[\int c \cdot f(x)\, dx = c \int f(x)\, dx\] Each data sample is represented as a point in a high-dimensional feature space where coordinates are feature values, and geometric notions like distance and angle measure similarity.

\[ \mathbf{x} = (x_1, x_2, \ldots, x_n) \in \mathbb{R}^n \] By the quotient rule, the difference of two logs with the same base equals the log of their quotient.

\[\log_b(x) - \log_b(y) = \log_b\!\left(\frac{x}{y}\right)\] If one task can be done in \(m\) ways and a mutually exclusive second task in \(n\) ways, then either task can be done in \(m + n\) ways.

\[\text{Total ways} = m_1 + m_2 + \cdots + m_k\]

Question 21 AI Math Beginner

Why do you flip the inequality sign when multiplying or dividing by a negative number?

The result is always negative because one negative sign remains after all pairs cancel.

\[(-a)^{2k+1} = -a^{2k+1}\] Multiplying or dividing by a negative number reverses the order of the number line, so the direction of the comparison must be reversed to keep the statement true.

\[-2x < 6 \implies x > -3\] If one task can be done in \(m\) ways and an independent second task in \(n\) ways, then both tasks together can be done in \(m \times n\) ways.

\[\text{Total ways} = m_1 \times m_2 \times \cdots \times m_k\] \(2^{10} = 1024\) and \(10^3 = 1000\), so \(2^{10}\) is slightly larger; this is why a kilobyte is 1024 bytes.

\[2^{10} = 1024 > 1000 = 10^3\]

Question 22 AI Math Beginner

What is the matrix trace's linearity property?

The trace is linear: the trace of a sum equals the sum of the traces, and a scalar can be factored out.

\[\text{tr}(A + B) = \text{tr}(A) + \text{tr}(B), \quad \text{tr}(cA) = c \cdot \text{tr}(A)\] The axes divide the plane into four regions: Q1 (+,+), Q2 (−,+), Q3 (−,−), Q4 (+,−), numbered counterclockwise.

\[ \text{QI: } x>0,y>0;\quad \text{QII: } x<0,y>0;\quad \text{QIII: } x<0,y<0;\quad \text{QIV: } x>0,y<0 \] A set is an unordered collection of distinct objects called elements; order does not matter and each element appears only once.

\[ S = \{1, 2, 3\} \] Most intermediate terms cancel because each positive piece is annulled by the negative piece of the next term, so the total collapses to the boundary values.

\[\sum_{i=1}^{n}(a_i - a_{i-1}) = (a_1 - a_0) + (a_2 - a_1) + \cdots + (a_n - a_{n-1}) = a_n - a_0\]

Question 23 AI Math Beginner

What does it mean for two lines to be perpendicular?

The second derivative is the derivative of the first derivative; it measures how the rate of change itself is changing.

\[f''(x) = \frac{d^2 f}{dx^2} = \frac{d}{dx}\left[\frac{df}{dx}\right]\] Polynomial long division divides one polynomial by another of lower or equal degree, producing a quotient and remainder.

\[\frac{x^2+3x+2}{x+1} = x+2\] Two lines are perpendicular if their slopes are negative reciprocals of each other, making them meet at a 90 degree angle.

\[ m_1 \cdot m_2 = -1 \] The square root of \(x^2\) always returns the non-negative value, which is exactly the definition of absolute value.

\[\sqrt{x^2} = \lvert x \rvert\]

Question 24 AI Math Beginner

What is an outlier (statistical definition)?

Adding like terms gives \(x^2 + x^2 = 2x^2\); you add the coefficients, not the exponents.

\[x^2 + x^2 = 2x^2 \neq 4x^2 = (x+x)^2\] An arithmetic-geometric sequence has terms formed by multiplying the corresponding terms of an arithmetic sequence and a geometric sequence. Its sum is found by the multiply-and-subtract technique used for geometric series.

\[ a_n = (a + (n-1)d)\, r^{n-1} \] An outlier is a data point that falls more than 1.5 times the IQR below \(Q_1\) or above \(Q_3\).

\[\text{Outlier if } x < Q_1 - 1.5 \cdot \text{IQR} \text{ or } x > Q_3 + 1.5 \cdot \text{IQR}\] Matrix inversion has the same asymptotic complexity as matrix multiplication: \(O(n^3)\) for an \(n \times n\) matrix, making it expensive for large models.

\[A^{-1} \text{ costs } O(n^3)\]

Question 25 AI Math Beginner

How do you find the derivative of e^(x^2)?

Two sets A and B are disjoint if they share no elements in common, that is, their intersection is the empty set.

\[ A \cap B = \emptyset \] Exponential growth means a quantity multiplies by a fixed factor greater than 1 during each equal time period, so the increase speeds up over time.

\[Q(t) = Q_0 \cdot a^t, \quad a > 1\] By the chain rule with outer function \(e^u\) and inner function \(u = x^2\): the derivative is \(e^{x^2} \cdot 2x\).

\[\frac{d}{dx}[e^{x^2}] = 2x \, e^{x^2}\] Regularization adds a penalty term to the loss function to discourage overly complex models and reduce overfitting.

\[L_{\text{reg}}(\theta) = L(\theta) + \lambda R(\theta)\]

Question 26 AI Math Beginner

What is the relationship between log of a product and a sum?

A set is an unordered collection of distinct objects called elements; order does not matter and each element appears only once.

\[ S = \{1, 2, 3\} \] MAE is the average of absolute differences between predictions and true values; it is less sensitive to outliers than MSE.

\[\text{MAE} = \frac{1}{n}\sum_{i=1}^n |y_i - \hat{y}_i|\] Taking the logarithm of a product converts it into a sum of logarithms, which is why log-likelihood in ML is a sum rather than a product.

\[\log \prod_{i=1}^{n} a_i = \sum_{i=1}^{n} \log a_i\] A Lagrange multiplier \(\lambda\) is introduced to convert a constrained optimization problem into an unconstrained one by combining the objective and constraint.

\[\mathcal{L}(\mathbf{x}, \lambda) = f(\mathbf{x}) - \lambda g(\mathbf{x})\]

Question 27 AI Math Beginner

What is the least squares solution to an overdetermined system?

The derivative of tanh equals 1 minus tanh squared, analogous to how the sigmoid derivative involves sigmoid times one minus sigmoid.

\[\frac{d}{dx}[\tanh(x)] = 1 - \tanh^2(x)\] An even function is symmetric about the y-axis: replacing \(x\) with \(-x\) leaves the function value unchanged.

\[f(-x) = f(x) \text{ for all } x \text{ in the domain}\] The intersection of A and B is the set of all elements that belong to both A and B simultaneously.

\[ A \cap B = \{x \mid x \in A \text{ and } x \in B\} \] When \(A\mathbf{x} = \mathbf{b}\) has no exact solution (more equations than unknowns), the least squares solution minimizes the squared error.

\[\hat{\mathbf{x}} = \arg\min_\mathbf{x} \|A\mathbf{x} - \mathbf{b}\|_2^2 = (A^\top A)^{-1} A^\top \mathbf{b}\]

Question 28 AI Math Beginner

What is the maximum likelihood estimation (MLE) principle?

The signed distance is positive on one side of the hyperplane and negative on the other, with its magnitude equal to the perpendicular distance.

\[ d = \frac{\mathbf{w} \cdot \mathbf{x}_0 + b}{\lVert \mathbf{w} \rVert} \] An affine transformation is a linear transformation followed by a translation; neural network layers perform affine transformations before applying activations.

\[\mathbf{y} = W\mathbf{x} + \mathbf{b}\] A is a subset of B if every element of A is also an element of B, written \(A \subseteq B\); A may equal B.

\[ A \subseteq B \iff \forall x,\; x \in A \Rightarrow x \in B \] MLE finds the parameter values that maximize the probability (likelihood) of observing the given data under the model.

\[\hat{\theta}_{MLE} = \arg\max_\theta \prod_{i=1}^n P(x_i \mid \theta) = \arg\max_\theta \sum_{i=1}^n \log P(x_i \mid \theta)\]

Question 29 AI Math Beginner

What is the categorical distribution?

The softmax denominator is the sum of exponentiated logits over all \(K\) classes, normalising the output into a probability distribution.

\[Z = \sum_{k=1}^{K} e^{z_k}\] Doubling time is the constant period required for a growing quantity to double, regardless of the current amount, and is a defining feature of true exponential growth.

\[Q(t) = Q_0 \cdot 2^{t/T_d}, \quad T_d = \text{doubling time}\] The categorical distribution generalizes the Bernoulli to \(K\) outcomes, each with its own probability; a single trial produces exactly one category.

\[P(X = k) = p_k, \quad \sum_{k=1}^K p_k = 1\] Apply the rule \(\sqrt{x^4} = (x^4)^{1/2} = x^{4/2} = x^2\) for \(x \geq 0\).

\[\sqrt{x^4} = x^2 \quad (x \geq 0)\]

Question 30 AI Math Beginner

What is interval notation for a closed interval?

Attention computes a weighted sum of value vectors, where the weights come from the similarity (dot product) between a query and key vectors.

\[\text{Attention}(Q, K, V) = \text{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V\] The Mahalanobis distance accounts for correlations between variables using the covariance matrix, measuring how many standard deviations a point is from the mean.

\[d_M(\mathbf{x}, \boldsymbol{\mu}) = \sqrt{(\mathbf{x} - \boldsymbol{\mu})^\top \Sigma^{-1} (\mathbf{x} - \boldsymbol{\mu})}\] A closed interval includes both endpoints and is written with square brackets.

\[[a, b] = \{x \mid a \leq x \leq b\}\] Move all variable terms to one side and all constant terms to the other, then solve for the variable.

\[3x + 2 = x + 10 \implies 2x = 8 \implies x = 4\]

Question 31 AI Math Beginner

What is the interquartile range (IQR)?

The IQR is the range between the 25th and 75th percentiles; it measures the spread of the middle 50% of the data and is robust to outliers.

\[\text{IQR} = Q_3 - Q_1\] The dot product of two vectors is the sum of the element-wise products of their components.

\[\mathbf{a} \cdot \mathbf{b} = \sum_{i=1}^{n} a_i b_i\] A metric space is a set with a distance function satisfying non-negativity, identity of indiscernibles, symmetry, and the triangle inequality.

\[d(x, y) \geq 0, \quad d(x,y) = 0 \iff x = y, \quad d(x,y) = d(y,x), \quad d(x,z) \leq d(x,y) + d(y,z)\] The median is the middle value when data is sorted; it splits the dataset so that half the values are below and half are above.

\[\text{Median} = \text{middle value of sorted data}\]

Question 32 AI Math Beginner

How does sequence convergence relate to gradient descent convergence?

Gradient descent produces a sequence of parameter vectors; the algorithm is said to converge when that sequence converges to a fixed point where the gradient is zero. Convergence rate depends on the learning rate and the curvature of the loss surface.

\[ \theta_{t+1} = \theta_t - \eta \nabla_\theta \mathcal{L}(\theta_t) \] The probability that at least one of two events occurs equals the sum of their individual probabilities minus the probability of both occurring simultaneously, to avoid double-counting.

\[P(A \cup B) = P(A) + P(B) - P(A \cap B)\] The indefinite integral represents the family of all antiderivatives of a function, written with a constant of integration \(C\).

\[\int f(x)\, dx = F(x) + C \quad \text{where } F'(x) = f(x)\] A series converges if its sequence of partial sums approaches a finite limit as the number of terms grows.

\[\sum_{n=1}^{\infty} a_n = L \iff \lim_{N \to \infty} \sum_{n=1}^{N} a_n = L\]

Question 33 AI Math Beginner

What is the cross product in 3D (brief intro)?

Concavity describes whether a curve opens upward (like a cup) or downward (like a cap), determined by the sign of the second derivative.

\[\text{Concave up: } f'' > 0 \quad \text{Concave down: } f'' < 0\] The sum of the first \(n\) cubes equals the square of the \(n\)-th triangular number, a neat identity connecting cubes to squares of sums.

\[ \sum_{k=1}^{n} k^3 = \left(\frac{n(n+1)}{2}\right)^2 \] The cross product of two 3D vectors produces a third vector perpendicular to both, with magnitude equal to the area of the parallelogram they span.

\[ \mathbf{a} \times \mathbf{b} = \begin{pmatrix}a_y b_z - a_z b_y \\ a_z b_x - a_x b_z \\ a_x b_y - a_y b_x\end{pmatrix} \] The Mahalanobis distance accounts for correlations between variables using the covariance matrix, measuring how many standard deviations a point is from the mean.

\[d_M(\mathbf{x}, \boldsymbol{\mu}) = \sqrt{(\mathbf{x} - \boldsymbol{\mu})^\top \Sigma^{-1} (\mathbf{x} - \boldsymbol{\mu})}\]

Question 34 AI Math Beginner

What is the set difference A minus B?

A logarithmic function is the inverse of an exponential function, returning the exponent to which the base must be raised to produce a given value.

\[f(x) = \log_b(x) \iff b^{f(x)} = x,\quad b > 0,\; b \neq 1\] The set difference A minus B contains every element that is in A but not in B; it removes B's elements from A.

\[ A \setminus B = \{x \mid x \in A \text{ and } x \notin B\} \] Divide the natural log of x by the natural log of 3 using the change-of-base formula.

\[\log_3(x) = \frac{\ln(x)}{\ln(3)}\] The sum of \(n\) terms of a geometric series with first term \(a_1\) and ratio \(r \neq 1\) uses the formula below. When \(r = 1\) the sum is simply \(n a_1\).

\[ S_n = a_1 \cdot \frac{1 - r^n}{1 - r} \]

Question 35 AI Math Beginner

What is a unit vector?

Exponential growth means a quantity multiplies by a fixed factor greater than 1 during each equal time period, so the increase speeds up over time.

\[Q(t) = Q_0 \cdot a^t, \quad a > 1\] A unit vector has magnitude exactly 1 and points in a specific direction, used to represent pure direction.

\[ \lVert \hat{u} \rVert = 1 \] In continuous probability, the probability that a random variable falls in an interval equals the integral of its density function over that interval.

\[P(a \leq X \leq b) = \int_a^b f_X(x)\, dx\] The sum of the cubes of the first \(n\) positive integers equals the square of the triangular number \(n(n+1)/2\).

\[\sum_{i=1}^{n} i^3 = \left(\frac{n(n+1)}{2}\right)^2\]

Question 36 AI Math Beginner

What is an event in probability?

The sum of squares of the integers 0 through \(n-1\) equals \((n-1)n(2n-1)/6\).

\[\sum_{i=0}^{n-1} i^2 = \frac{(n-1)\,n\,(2n-1)}{6}\] An event is any subset of the sample space; its probability is a number in \([0,1]\) representing how likely that collection of outcomes is to occur.

\[A \subseteq \Omega, \quad P(A) \in [0, 1]\] The size of the union of two sets equals the sum of their individual sizes minus the size of their intersection, correcting for double-counting overlapping elements.

\[\lvert A \cup B \rvert = \lvert A \rvert + \lvert B \rvert - \lvert A \cap B \rvert\] A logarithmic function is the inverse of an exponential function, returning the exponent to which the base must be raised to produce a given value.

\[f(x) = \log_b(x) \iff b^{f(x)} = x,\quad b > 0,\; b \neq 1\]

Question 37 AI Math Beginner

Why do we maximize log-likelihood instead of likelihood?

Adding like terms gives \(x^2 + x^2 = 2x^2\); you add the coefficients, not the exponents.

\[x^2 + x^2 = 2x^2 \neq 4x^2 = (x+x)^2\] In ML, derivatives tell us how the loss changes with respect to each parameter; gradient descent uses this information to iteratively reduce the loss.

\[\theta_{\text{new}} = \theta_{\text{old}} - \eta \frac{\partial L}{\partial \theta}\] Taking the log converts products into sums (easier to differentiate and more numerically stable), and does not change the location of the maximum since log is monotonically increasing.

\[\arg\max_\theta \prod_i P(x_i \mid \theta) = \arg\max_\theta \sum_i \log P(x_i \mid \theta)\] The spectral theorem states that every real symmetric matrix can be diagonalized by an orthogonal matrix, with all eigenvalues real.

\[A = A^\top \implies A = Q\Lambda Q^\top, \quad Q \text{ orthogonal}, \Lambda \text{ real diagonal}\]

Question 38 AI Math Beginner

What is the triplet loss?

The result is always negative because one negative sign remains after all pairs cancel.

\[(-a)^{2k+1} = -a^{2k+1}\] Triplet loss pushes an anchor closer to a positive example and farther from a negative example by at least a margin \(\alpha\).

\[L = \max(0, d(a, p) - d(a, n) + \alpha)\] It equals 0 because any nonzero base raised to the power 0 gives 1.

\[\log_b(1) = 0 \text{ because } b^0 = 1\] At a local maximum or minimum the function stops increasing and starts decreasing (or vice versa), so the tangent line is horizontal and its slope is zero.

\[f \text{ has local extremum at } c \implies f'(c) = 0 \text{ (if } f'(c) \text{ exists)}\]

Question 39 AI Math Beginner

What is the Cauchy-Schwarz inequality?

In ML, derivatives tell us how the loss changes with respect to each parameter; gradient descent uses this information to iteratively reduce the loss.

\[\theta_{\text{new}} = \theta_{\text{old}} - \eta \frac{\partial L}{\partial \theta}\] The distribution of the sum of two independent random variables is the convolution of their individual distributions.

\[f_{X+Y}(z) = \int f_X(x) f_Y(z - x)\, dx = (f_X * f_Y)(z)\] The absolute value of the dot product of two vectors is at most the product of their lengths, with equality when they point in the same direction.

\[\lvert \mathbf{u} \cdot \mathbf{v} \rvert \leq \lVert \mathbf{u} \rVert \, \lVert \mathbf{v} \rVert\] The spectral theorem states that every real symmetric matrix can be diagonalized by an orthogonal matrix, with all eigenvalues real.

\[A = A^\top \implies A = Q\Lambda Q^\top, \quad Q \text{ orthogonal}, \Lambda \text{ real diagonal}\]

Question 40 AI Math Beginner

What is the difference between supervised and unsupervised learning?

A subsequence is formed by selecting an infinite collection of terms from a sequence while preserving their original order. It is written \(\{a_{n_k}\}\) where the indices satisfy \(n_1 < n_2 < n_3 < \cdots\).

\[ a_{n_1},\; a_{n_2},\; a_{n_3},\; \ldots \quad (n_1 < n_2 < n_3 < \cdots) \] Supervised learning uses labeled data (input-output pairs) to learn a mapping; unsupervised learning finds patterns in unlabeled data without target outputs.

\[\text{Supervised: } (x_i, y_i) \to f(x) \quad \text{Unsupervised: } x_i \to \text{structure}\] The indicator function returns 1 if a condition is true and 0 otherwise; it is used to write piecewise expressions compactly.

\[\mathbb{1}_A(x) = \begin{cases} 1 & x \in A \\ 0 & x \notin A \end{cases}\] The median is the middle value when data is sorted; it splits the dataset so that half the values are below and half are above.

\[\text{Median} = \text{middle value of sorted data}\]

Question 41 AI Math Beginner

How do you represent the real numbers on a number line?

Machine epsilon is the smallest positive number \(\epsilon\) such that \(1 + \epsilon \neq 1\) in floating-point arithmetic; it measures the precision limit.

\[\epsilon_{\text{mach}} \approx 2.2 \times 10^{-16} \quad \text{(64-bit double)}\] Every real number corresponds bijectively to exactly one point on an infinite straight line, with negative numbers to the left of zero and positive numbers to the right.

\[ \mathbb{R} \longleftrightarrow \text{the number line from } -\infty \text{ to } +\infty \] The square root of \(x^2\) always returns the non-negative value, which is exactly the definition of absolute value.

\[\sqrt{x^2} = \lvert x \rvert\] An affine transformation is a linear transformation followed by a translation; neural network layers perform affine transformations before applying activations.

\[\mathbf{y} = W\mathbf{x} + \mathbf{b}\]

Question 42 AI Math Beginner

What is machine epsilon?

Differentiate with respect to each variable: \(\partial f/\partial x = 2x\) and \(\partial f/\partial y = 2y\), so the gradient is \((2x, 2y)\).

\[\nabla f = \begin{bmatrix} 2x \\ 2y \end{bmatrix}\] This integral gives the arctangent function, which is the inverse of tangent.

\[\int \frac{1}{1+x^2}\, dx = \arctan(x) + C\] Machine epsilon is the smallest positive number \(\epsilon\) such that \(1 + \epsilon \neq 1\) in floating-point arithmetic; it measures the precision limit.

\[\epsilon_{\text{mach}} \approx 2.2 \times 10^{-16} \quad \text{(64-bit double)}\] A definite integral is the limit of a Riemann sum as the number of rectangles grows to infinity and their widths shrink to zero; summation is the discrete analogue of integration.

\[\int_a^b f(x)\,dx \approx \sum_{i=1}^{n} f(x_i)\,\Delta x\]

Question 43 AI Math Beginner

What is the product notation (capital Greek letter pi)?

A function is differentiable at a point if its derivative exists there; the graph has a well-defined, non-vertical tangent line at that point.

\[f'(a) = \lim_{h \to 0} \frac{f(a+h)-f(a)}{h} \text{ exists}\] The Kronecker delta equals 1 when its two indices are equal and 0 otherwise; it appears in identity matrices and orthonormality conditions.

\[\delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases}\] The capital-pi notation \(\Pi\) represents the product of a sequence of terms in the same way that \(\Sigma\) represents a sum.

\[\prod_{i=m}^{n} a_i = a_m \cdot a_{m+1} \cdots a_n\] Catastrophic cancellation occurs when subtracting two nearly equal floating-point numbers, causing a large relative error because significant digits cancel.

\[a \approx b \implies a - b \text{ loses precision}\]

Question 44 AI Math Beginner

What are the mean and variance of the normal distribution?

The exponential distribution models the time between events in a Poisson process; it is memoryless, meaning the remaining wait time does not depend on how long you have already waited.

\[f(x) = \lambda e^{-\lambda x},\; x \geq 0, \quad E[X] = \frac{1}{\lambda}\] The log decreases without bound, approaching negative infinity as x gets arbitrarily close to 0 from the positive side.

\[\lim_{x \to 0^+} \log(x) = -\infty\] Convergence is extremely slow because each step makes only a tiny improvement, and training may get stuck in shallow local minima.

\[\eta \text{ too small} \implies \text{very slow convergence}\] The mean \(\mu\) is the center of the bell curve and the variance \(\sigma^2\) controls its width; about 68% of values fall within one standard deviation of the mean.

\[E[X] = \mu, \quad \text{Var}(X) = \sigma^2\]

Question 45 AI Math Beginner

Why is a constant of integration C added?

Because the derivative of any constant is zero, infinitely many functions differ by a constant and have the same derivative; \(C\) accounts for all of them.

\[\frac{d}{dx}[F(x) + C] = F'(x) = f(x) \quad \text{for any constant } C\] The Fisher information measures how much information an observable random variable carries about an unknown parameter; it is the expected Hessian of the negative log-likelihood.

\[I(\theta) = E\left[-\frac{\partial^2 \log P(X \mid \theta)}{\partial \theta^2}\right]\] Swish is a smooth, non-monotonic activation function that often outperforms ReLU; it is the input times its sigmoid.

\[\text{Swish}(x) = x \cdot \sigma(x) = \frac{x}{1 + e^{-x}}\] An even function is symmetric about the y-axis: replacing \(x\) with \(-x\) leaves the function value unchanged.

\[f(-x) = f(x) \text{ for all } x \text{ in the domain}\]

Question 46 AI Math Beginner

What is a discrete random variable?

A discrete random variable takes on a finite or countably infinite set of values, each with a specific probability.

\[P(X = x_i) = p_i, \quad \sum_i p_i = 1\] The distribution of the sum of two independent random variables is the convolution of their individual distributions.

\[f_{X+Y}(z) = \int f_X(x) f_Y(z - x)\, dx = (f_X * f_Y)(z)\] An exponential function has a constant base raised to a variable exponent; it models growth or decay at a rate proportional to the current value.

\[f(x) = a \cdot b^x,\quad b > 0,\; b \neq 1\] Pascal's identity states that each binomial coefficient equals the sum of the two binomial coefficients directly above it in Pascal's triangle.

\[\binom{n}{k} = \binom{n-1}{k-1} + \binom{n-1}{k}\]

Question 47 AI Math Beginner

How do you write the Frobenius norm squared using double summation?

A diagonal matrix has nonzero entries only on the main diagonal; multiplying by it scales each dimension independently.

\[D = \text{diag}(d_1, \ldots, d_n), \quad D\mathbf{x} = (d_1 x_1, \ldots, d_n x_n)\] The Frobenius norm squared is the sum of the squares of every matrix entry, computed via a double sum over rows and columns.

\[\lVert A \rVert_F^2 = \sum_{i=1}^{m}\sum_{j=1}^{n} A_{ij}^2\] The Bayes optimal classifier assigns each input to the class with the highest posterior probability, achieving the lowest possible error rate for the given data distribution.

\[\hat{y} = \arg\max_c P(c \mid \mathbf{x})\] It is the length of the perpendicular dropped from the point to the line, given by a formula using the line's coefficients.

\[ d = \frac{\lvert ax_0 + by_0 + c \rvert}{\sqrt{a^2 + b^2}} \]

Question 48 AI Math Beginner

What is the graphical relationship between f and its inverse?

The graph of \(f^{-1}\) is the mirror image of the graph of \(f\) reflected across the line \(y = x\).

\[\text{Graph of } f^{-1} = \text{reflection of graph of } f \text{ across } y = x\] The Boltzmann distribution assigns higher probability to lower-energy states; it appears in softmax (with temperature) and energy-based models.

\[P(x) = \frac{e^{-E(x)/T}}{\sum_{x'} e^{-E(x')/T}}\] The outer product of column vectors \(\mathbf{a}\) and \(\mathbf{b}\) produces a matrix; it is used in gradient updates and covariance computations.

\[\mathbf{a}\mathbf{b}^\top \in \mathbb{R}^{m \times n}, \quad (\mathbf{a}\mathbf{b}^\top)_{ij} = a_i b_j\] The exponential distribution models the time between events in a Poisson process; it is memoryless, meaning the remaining wait time does not depend on how long you have already waited.

\[f(x) = \lambda e^{-\lambda x},\; x \geq 0, \quad E[X] = \frac{1}{\lambda}\]

Question 49 AI Math Beginner

What is the Sherman-Morrison formula?

This series converges for \(-1 < x \leq 1\) and has alternating signs with increasing denominators.

\[\ln(1+x) = \sum_{n=1}^{\infty} \frac{(-1)^{n+1} x^n}{n} = x - \frac{x^2}{2} + \frac{x^3}{3} - \cdots\] The Sherman-Morrison formula is a special case of Woodbury for rank-1 updates; it updates the inverse when a single outer product is added.

\[(A + \mathbf{u}\mathbf{v}^\top)^{-1} = A^{-1} - \frac{A^{-1}\mathbf{u}\mathbf{v}^\top A^{-1}}{1 + \mathbf{v}^\top A^{-1}\mathbf{u}}\] A set is convex if the line segment between any two points in the set lies entirely within the set.

\[\lambda \mathbf{x} + (1-\lambda)\mathbf{y} \in S \quad \forall\, \mathbf{x}, \mathbf{y} \in S, \;\lambda \in [0,1]\] The dual problem provides a lower bound on the optimal value of the primal problem; under strong duality (e.g., for convex problems), the primal and dual optima are equal.

\[\max_\lambda \min_\mathbf{x} \mathcal{L}(\mathbf{x}, \lambda) \leq \min_\mathbf{x} \max_\lambda \mathcal{L}(\mathbf{x}, \lambda)\]

Question 50 AI Math Beginner

How do you complete the square for x^2 + bx?

Apply the rule \(\sqrt{x^4} = (x^4)^{1/2} = x^{4/2} = x^2\) for \(x \geq 0\).

\[\sqrt{x^4} = x^2 \quad (x \geq 0)\] The integral of \(e^{-x^2}\) over the entire real line equals \(\sqrt{\pi}\); this result is fundamental to the normal distribution.

\[\int_{-\infty}^{\infty} e^{-x^2}\, dx = \sqrt{\pi}\] Add the square of half the coefficient of \(x\) to form a perfect square trinomial.

\[x^2 + bx + \left(\tfrac{b}{2}\right)^2 = \left(x + \tfrac{b}{2}\right)^2\] Polynomial long division divides one polynomial by another of lower or equal degree, producing a quotient and remainder.

\[\frac{x^2+3x+2}{x+1} = x+2\]