Load Balancer Diagram: Layer 4 vs Layer 7, Health Checks and Algorithms

"Put a load balancer in front of it" is the most common sentence in system design, and it hides a handful of decisions that matter in production: which layer it works at, how it picks a backend, how it knows a backend is healthy, and what happens to requests in flight when one is removed.

This post walks through those decisions and draws the two-tier setup that most large deployments end up with: a layer 4 balancer in front of a fleet of layer 7 proxies.

Layer 4 vs layer 7

The difference that bites: a layer 4 balancer balances connections, not requests. With HTTP/2 or gRPC, one client opens one long-lived connection and sends thousands of requests over it, all of which land on the same backend. Balancing gRPC well needs a layer 7 proxy or client-side load balancing.

Why two tiers

Large setups usually stack them. A layer 4 tier holds the public IP addresses, absorbs connection floods and spreads connections across a fleet of layer 7 proxies by hashing each flow. The proxies terminate TLS, apply routing rules and balance individual requests across backend pools.

Each tier scales on its own: the layer 4 tier on packets and connections, the proxy fleet on requests and TLS handshakes. Splitting them also limits blast radius, since a bad proxy config change hits the proxies behind one tier, not the addresses users connect to. And because proxies are stateless and share one config, adding one is just adding capacity.

Balancing algorithms

Least request with two random choices is a good default for HTTP services. Envoy's load balancer documentation describes it: pick two hosts at random (two by default), send the request to the one with fewer active requests. Research by Mitzenmacher and others shows this is nearly as good as checking every backend, without the cost. Consistent hashing, such as ring hash or the Maglev algorithm, keeps the same key on the same backend and moves only a small share of keys when backends change, which is what caches want.

Health checks

A balancer must stop sending traffic to backends that cannot serve it. Two mechanisms work together:

• Active checks: the balancer calls a health endpoint, for example GET /healthz every 5 seconds, and removes a backend after a number of consecutive failures.
• Passive checks (outlier detection): the balancer watches real responses and ejects a backend that returns errors or times out, then brings it back after a cooling period.

Make the health endpoint honest but cheap. If it checks every dependency, one slow database makes every backend unhealthy at once and the balancer has nowhere to send traffic. Check what this instance needs to serve requests, and leave shared dependencies to the application's own error handling.

Draining and stickiness

When a backend is removed for a deploy or scale-in, it should stop receiving new requests but finish the ones in flight. Configure a draining period (cloud balancers call it a deregistration delay) longer than your slowest normal request, and make the application stop accepting new work on shutdown.

Sticky sessions send a user to the same backend every time, usually with a cookie. They help when state lives in a server's memory, and they hurt everything else: uneven load, lost sessions when a backend dies, harder deploys. Move session state to a shared store and turn stickiness off where you can.

Beyond one region

Everything above happens inside one site. Spreading users across regions is a separate layer: DNS-based routing that answers with the nearest healthy region (latency or geolocation routing), or anycast, where many sites announce the same IP address and internet routing delivers each user to a nearby one. Global balancing picks the region; the two tiers in this post pick the server. Keep health signals flowing upward, so a region whose proxies cannot reach their pools stops receiving users instead of returning errors to them.

The architecture diagram

How to read it:

• Left: clients resolve one DNS name and open TCP connections on port 443 to the layer 4 load balancer.
• Middle: the layer 4 tier spreads connections across the proxy fleet by flow hash. The frame shows that every proxy runs the same config.
• Right: the proxies terminate TLS and route each request by path: /api/orders and /api/search to their pools, /static to a bucket.
• The Markdown block holds the routing table, the algorithm per pool and the health check settings, the details that usually live only in a config file.

What to label on a load balancer diagram

• The layer and protocol on each hop: TCP 443, HTTP/2, gRPC.
• Where TLS terminates. It decides what each tier can see and where certificates live.
• The algorithm per pool, and the hash key if you hash.
• Health check path, interval and threshold.
• Draining and stickiness settings, if any.

The API gateway architecture diagram covers the layer 7 features that sit on top of routing, the Kubernetes ingress diagram shows the same ideas inside a cluster, and the network topology diagram places load balancers in the wider network.

Build it in Archboard

1. Sign in with Google or GitHub and create a diagram.
2. Add a Person from the C4 Model pack for clients, and a DNS and a Load Balancer from the Network pack.
3. Add a Frame from the Core pack for the proxy fleet with two Container nodes inside it.
4. Add two Compute nodes and an Object Storage node from the Cloud pack for the pools and the bucket.
5. Label edges with protocols and routes, and add a Markdown block from the Core pack with the routing table and health checks.

The example uses 10 diagram nodes, within the Free plan.

FAQ

What is the difference between a layer 4 and a layer 7 load balancer? A layer 4 balancer distributes TCP or UDP connections without looking inside them. A layer 7 balancer understands HTTP, so it can route by path or host and balance individual requests.

Which load balancing algorithm should I use? Least request with two random choices is a strong default for HTTP. Use consistent hashing when the same key should reach the same backend, as with caches.

Why does my gRPC service send all traffic to one pod? Because gRPC uses long-lived HTTP/2 connections and a layer 4 balancer balances connections, not requests. Put a layer 7 proxy in between or use client-side load balancing.

What should a health check endpoint check? That this instance can serve requests: the process is up and its own critical resources are available. Avoid checking shared dependencies that would fail every backend at once.

Are sticky sessions a good idea? Only when state cannot leave the server. They cause uneven load and lost sessions on failures; a shared session store is usually better.

Start free — one seat, no credit card.