Deploying machine learning at scale requires careful systems engineering. Retrieval-Augmented Generation (RAG) augments a language model by retrieving relevant external documents and including them in the prompt. RAG grounds responses in up-to-date, verifiable sources, reducing hallucination, and is cheaper and more dynamic than fine-tuning. A typical RAG pipeline chunks documents, embeds them with a model like a sentence-transformer, indexes the embeddings in a vector database, retrieves the most relevant chunks for a query, and feeds them to the generator. Vector databases such as FAISS, Pinecone, Weaviate, Milvus, and Qdrant store and index high-dimensional embeddings for fast similarity search. At the heart of vector search is approximate nearest neighbor (ANN) retrieval, which trades exactness for massive speedups; algorithms like HNSW (Hierarchical Navigable Small World), IVF, and LSH support billion-scale retrieval in milliseconds. Cosine similarity \( \cos(A, B) = (A \cdot B) / (\|A\| \|B\|) \) is widely used to compare embeddings because it measures directional alignment independent of magnitude. Embedding fine-tuning adjusts pre-trained embeddings on domain-specific data with contrastive losses, often improving retrieval quality where off-the-shelf embeddings miss nuance.
Fine-tuning specializes a pre-trained model for a task. Full fine-tuning updates all weights, which is expensive for large models. LoRA (Low-Rank Adaptation) freezes the original weights and injects trainable low-rank matrices into transformer layers, reducing trainable parameters by 10× to 10000× while matching full fine-tuning quality; multiple LoRA adapters can be swapped for different tasks. Knowledge distillation trains a small student model to mimic a larger teacher, using the teacher's soft probability outputs as additional supervision. Quantization reduces numerical precision (e.g., float32 to int8), shrinking models and speeding inference with minimal accuracy loss; quantization-aware training simulates low precision during training for better results. Pruning removes unimportant weights or entire filters, often combined with fine-tuning to recover accuracy. ONNX provides an open format for representing models, enabling interoperability across frameworks and runtimes like ONNX Runtime and TensorRT.
Recommendation systems predict user preferences and suggest items. Collaborative filtering relies on patterns in user–item interactions without needing item features, using user-based similarity, item-based similarity, or matrix factorization. Matrix factorization decomposes the interaction matrix into low-rank user and item matrices whose dot products approximate observed ratings \[ R \approx U V^\top \] a technique popularized by the Netflix Prize that captures latent preferences and enables efficient recommendations. Content-based methods use item features, and hybrid systems combine both. Modern deep approaches like two-tower neural networks underlie production recommenders. A major challenge is the cold-start problem for new users or items with no interaction history.
Time series forecasting predicts future values from past observations. Classical methods include ARIMA, which combines autoregression, differencing, and moving average components, and Exponential Smoothing. Modern approaches use RNNs, LSTMs, Temporal Convolutional Networks, and transformer-based models such as Informer and PatchTST. Meta's Prophet library is designed for business time series with daily seasonality, holidays, and trend changes, and is robust to missing data and outliers. Stationarity—constant statistical properties over time—is an assumption of many models, while seasonality is a regular periodic pattern to model or remove. The autocorrelation function (ACF) measures correlation between a series and its lagged version, helping identify seasonality and inform model orders. Anomaly detection identifies points that significantly deviate from the rest of the distribution; Isolation Forest isolates anomalies quickly via random splits in trees, and One-class SVM learns a boundary around normal data using a kernel such as RBF. Operationalizing ML introduces challenges like data drift and concept drift, where input distributions or feature–target relationships change over time, requiring monitoring and retraining. MLflow manages the ML lifecycle, including experiment tracking, model packaging, and registries, while feature stores like Feast ensure consistency between offline and online feature pipelines. A/B testing compares model variants by randomly assigning users and measuring business metrics—the gold standard for validating that a new model actually improves outcomes in production.