The short answer to what is a load balancer: it is a network component that distributes incoming connections or requests across a pool of backend servers. It uses a scheduling algorithm to pick a target and health checks to remove failed servers, so no single machine is overloaded. Layer 4 load balancers route by IP address and port. Layer 7 load balancers route by HTTP content such as host, path and headers.
Detection speed is the number that matters most in practice. With HAProxy's default health check settings (a probe every 2 seconds, 3 consecutive failures to mark a server down), a dead backend can keep receiving new traffic for roughly 6 seconds plus the probe timeout before it is ejected. This is why production setups in 2026 pair active health checks with passive outlier detection.
A load balancer exposes one or more virtual IP addresses (VIPs) to clients and holds a pool of real servers behind them. For each new connection or request it does three things: it checks which backends are currently healthy, applies an algorithm to choose one, and then forwards the traffic. The forwarding method depends on the layer it operates at.
State is the hidden cost. A layer 4 balancer keeps a connection table keyed on the 5-tuple (source IP, source port, destination IP, destination port, protocol), so every packet of a flow reaches the same backend. A layer 7 balancer terminates the client connection, often including TLS. It parses the request and opens or reuses a separate connection to the backend, so it holds two sockets per active client plus buffers.
Active checks probe each backend on an interval. Passive checks, called outlier detection in Envoy, watch real traffic and eject a server after consecutive errors or timeouts. The worst-case detection window for active checks is roughly the interval multiplied by the failure threshold, plus the probe timeout.
Run the arithmetic on your own settings. Take a 5-second interval, a fall threshold of 3 and a 2-second timeout. That gives up to 17 seconds of misrouted requests. At 2,000 requests per second spread over 10 backends, about 3,400 requests hit the dead node in that window. Passive detection on 5 consecutive 5xx responses cuts the exposure to a handful of requests.
The hashing numbers explain why caches use consistent hashing. With naive modulo hashing, growing a pool from 10 to 11 servers remaps about 10/11, or 91%, of keys. Consistent hashing remaps about 1/11, or 9%. For a cache tier, that is the difference between a cold-cache stampede and a minor dip in hit ratio.
Layer 4 vs layer 7 load balancing is a trade between throughput and visibility. Layer 4 forwards packets or TCP and UDP flows without reading payloads, so it adds microseconds and scales to millions of packets per second. Layer 7 reads HTTP, which enables path routing, retries and header rewrites, but it costs TLS termination CPU and adds latency per request.
| Property | Layer 4 (network load balancing) | Layer 7 (application load balancing) |
|---|---|---|
| Unit of balancing | Connection or flow | Individual HTTP request or stream |
| TLS | Passed through to backends | Usually terminated at the balancer |
| Routing inputs | IPs, ports, protocol | Host, path, headers, cookies, method |
| Return path | Can use direct server return (DSR) | Always through the balancer |
| Typical failure mode | Imbalance from long-lived multiplexed connections | CPU saturation from TLS and parsing |
Layer 4 balances connections, so any protocol that multiplexes many requests over one connection needs layer 7 to spread actual load.
The classic trap is gRPC or HTTP/2 behind a layer 4 balancer. A client opens one connection and sends thousands of streams over it, so one backend takes all of them while the others idle. Rebalancing only happens when the connection closes, which is why teams either move to a layer 7 proxy or cap connection age on the server.
Real deployments stack several tiers. Global server load balancing (GSLB) picks a region, through DNS answers or Anycast routing. A layer 4 tier such as Linux IPVS or Google's Maglev spreads flows across a fleet of layer 7 proxies using ECMP and consistent hashing. The layer 7 tier, such as Envoy, HAProxy or NGINX, then routes requests to services.
DNS-based GSLB is limited by TTLs and resolver location: clients may cache an answer for the full TTL, often 30 to 300 seconds, after a region fails. Anycast moves that decision into BGP, so withdrawing a route shifts traffic as fast as the routing tables converge. A CDN combines both ideas into a distributed load balancer for content, steering users to a nearby edge and shielding the origin. BlazingCDN, for example, uses Anycast routing with built-in failover and can run as a second CDN behind DNS with a weighted traffic split, which is global load balancing applied across CDN providers.
Load balancer vs. reverse proxy: a reverse proxy terminates client connections on behalf of servers. A layer 7 load balancer is a reverse proxy whose main job is distributing requests across a pool. A layer 4 load balancer is not a reverse proxy at all, because it never terminates the application connection.
Load balancer vs. API gateway: an API gateway adds authentication, quotas, request transformation and API versioning on top of layer 7 routing. Many gateways embed a load balancer, but buying a gateway for distribution alone adds latency and configuration you do not need.
Load balancer vs. CDN: a CDN caches and serves content from edge servers, so most requests never reach the backend pool. A load balancer distributes every request it receives. CDNs use load balancing internally, but caching is what reduces origin work.
A defensible HAProxy backend for a variable-latency API combines four settings, described here directive by directive:
Add slowstart on recovered servers so a re-admitted node with cold caches ramps up over 30 to 60 seconds instead of absorbing its full share instantly.
Round robin means even load. It means an even request count. A pool where 1% of requests take 100 times longer will skew badly under round robin.
Layer 7 is always better. For raw TCP, UDP, game traffic or very high packet rates, layer 4 with DSR is cheaper and faster, and the right answer is usually both tiers.
A load balancer removes the single point of failure. A single balancer is one. Production designs run pairs with VRRP, ECMP fan-out or Anycast, and they keep state small enough to survive a failover.
Sticky sessions are harmless. They defeat least-connections balancing and concentrate load after deploys. Move session state to a shared store and keep affinity for cache locality only.
A layer 4 load balancer routes TCP or UDP flows by IP address and port without reading payloads, while a layer 7 load balancer parses HTTP and routes each request by host, path or headers. Layer 4 is faster and protocol-agnostic. Layer 7 enables content routing, retries and per-request balancing of multiplexed HTTP/2 and gRPC traffic.
Most load balancer health checks run every 2 to 30 seconds, and the right interval depends on how long you can tolerate misrouted traffic. Multiply the interval by the failure threshold and add the timeout to get the worst-case detection window. Pair active checks with passive outlier detection so real errors eject a backend within seconds.
Least outstanding requests or power of two random choices usually beat round robin for APIs with variable latency. Both route new work away from servers that are already busy. Round robin assumes every request costs the same, so a few slow calls can pile up on one backend and inflate p99 latency for the whole pool.
A CDN is not strictly a load balancer, but it performs global load balancing as part of content delivery. It steers users to edge servers through Anycast or DNS, balances load across those servers, and caches responses so most requests never reach the origin. A load balancer, by contrast, forwards every request it receives to a backend.
Pick one production pool and calculate its worst-case detection window from the health check interval, failure threshold and timeout. Then verify it: in staging, stop a backend under synthetic load and record how many requests fail and for how long, from the balancer's own error counters. If the measured window exceeds your error budget, shorten the interval, enable passive outlier detection, and check whether HTTP/2 or gRPC clients are pinning connections to single backends at layer 4.
If part of that traffic is heavy static content such as video segments, installers or game patches, BlazingCDN's Anycast CDN with built-in failover can take that delivery load off your own balancers and origin.