Scaling is the practice of increasing a system's capacity to handle growing demand. The two fundamental approaches are vertical scaling, which adds more power (CPU, RAM, disk) to an existing machine, and horizontal scaling, which adds more machines to the resource pool. Vertical scaling is simpler because it avoids distributed coordination and requires no code changes, but it hits a hardware ceiling, creates a single point of failure, and often involves downtime during upgrades. Horizontal scaling, by contrast, offers near-infinite scalability, better fault tolerance since one server failing does not take down the system, and the ability to use commodity hardware. The trade-off is increased complexity in data consistency and distributed coordination.
When a system runs across multiple servers, a load balancer becomes essential. A load balancer distributes incoming network traffic across backend servers so that no single server is overwhelmed. Sitting between clients and servers, it improves availability by routing around failed servers, increases throughput by enabling parallel processing, and reduces latency by directing traffic to the least-loaded server. Common implementations include NGINX, HAProxy, and AWS ALB or NLB.
Load balancers use various algorithms to decide where to send each request. Round robin distributes requests sequentially across servers, while weighted round robin assigns more requests to higher-capacity machines. Least connections routes to the server with the fewest active connections, IP hash provides session stickiness by routing based on client IP, and least response time sends traffic to the fastest-responding server. Random selection is occasionally used when simplicity matters most.
Load balancers also differ in the OSI layer at which they operate. Layer 4 (transport) load balancers route based on IP addresses and TCP or UDP ports without inspecting packet contents, making them fast but less flexible. Layer 7 (application) load balancers inspect HTTP headers, cookies, and URLs to make smarter routing decisions, enabling content-based routing, SSL termination, and request modification. To know which servers are healthy, load balancers perform periodic probes through health checks, which can be active (sending HTTP or TCP requests at intervals) or passive (monitoring real traffic for errors). Configurable parameters include interval, timeout, unhealthy threshold, and healthy threshold.