Service Mesh Architecture Diagram: Data Plane, Control Plane and mTLS

Once a system has dozens of services, the same networking code shows up in every one of them: retries, timeouts, TLS between services, metrics per call, traffic splitting for releases. A service mesh moves that code out of the services and into the network path, where one team can configure it for everyone.

This post explains how a mesh is built, what the control plane and data plane each do, how automatic mTLS works, and the costs that make a mesh the wrong choice for many teams. The diagram shows the classic sidecar design.

Data plane and control plane

The data plane is a set of proxies that sit in the path of every request between services. In the sidecar model, each pod gets its own proxy container. The app talks plain HTTP to its local proxy; the proxy handles everything on the wire: encryption, retries, load balancing, metrics. Istio uses Envoy as its sidecar; Linkerd uses its own purpose-built proxy, written in Rust.

The control plane never touches request traffic. It watches the cluster for services and endpoints, turns your policies into proxy configuration, pushes that configuration to every proxy, and issues the certificates proxies use to identify themselves. In Istio this is istiod; see the Istio architecture page. Linkerd's architecture reference describes the equivalent pieces.

Automatic mTLS and workload identity

Mutual TLS between every pair of services is the feature that convinces most teams. Each proxy gets a short-lived certificate from the control plane that names the workload, typically as a SPIFFE ID derived from the Kubernetes namespace and service account, such as spiffe://cluster.local/ns/shop/sa/orders. When two proxies connect, both present certificates, both verify them, and the connection is encrypted.

Two consequences matter:

• Identity replaces network location. A policy can say "payments accepts calls only from the orders service account", which is stronger than "from this subnet".
• Rotation is automatic. Certificates live for hours, and nobody files a ticket to renew them.

Turn on strict mode once everything is in the mesh, so plaintext connections are refused rather than silently accepted.

One request through the mesh

Follow a call from orders to payments:

1. The orders app sends a plain HTTP request to the payments service name. Traffic redirection sends it to the local sidecar instead of the network.
2. The orders sidecar looks up healthy payments endpoints from the configuration the control plane pushed, picks one with its load balancing algorithm and applies the timeout for this route.
3. It opens, or reuses, an mTLS connection to the payments sidecar, presenting the orders certificate.
4. The payments sidecar verifies the certificate, checks the authorization policy (is orders allowed to call payments?) and forwards the request to the payments app on localhost.
5. Both sidecars record the request, its status and its latency. If the call fails with a retryable status, the orders sidecar may retry within its budget.

Neither app knows any of this happened, which is the point and also the risk: behavior you cannot see in the code must be visible somewhere else, such as on the diagram.

Rolling out a mesh

Adopt it in steps. Install the control plane, add services one namespace at a time in permissive mode so plaintext still works, watch the metrics, then switch each namespace to strict mTLS. Add authorization policies last, starting with the services that hold sensitive data.

Traffic policy

Because every call passes a proxy, the mesh can enforce the same rules everywhere:

• Timeouts and retries, with budgets so retries do not multiply during an outage.
• Circuit breaking and outlier detection: stop sending traffic to instances that keep failing.
• Traffic splitting by weight or header, which is how canary releases are implemented in a mesh.
• Fault injection for testing: add delays or errors to a percentage of calls.

Be careful with retries in two places. If the app retries and the sidecar also retries, one failing call becomes many. Pick one owner for retries, and write it down.

Observability for free (almost)

Proxies report request counts, error rates and latency per source and destination, without changing the apps. They can also start and propagate trace spans, but the app must still forward the trace headers from incoming to outgoing requests, or traces break at every service.

Sidecars vs ambient mode

Sidecars have real costs: a proxy in every pod uses CPU and memory, adds latency to every hop, and makes pod startup and upgrades more involved. Istio's ambient mode removes the per-pod sidecar. A per-node proxy called ztunnel handles mTLS and layer 4 policy, and optional waypoint proxies handle layer 7 features only for the services that need them. The trade-off is a different operational model and fewer years of production history than sidecars.

The architecture diagram

How to read it:

• Left: the ingress gateway receives traffic from outside and opens an mTLS connection to the sidecar in the orders pod.
• Middle: inside each pod, the sidecar forwards to the app over localhost. The orders sidecar calls the payments and inventory sidecars over mTLS; the apps never handle certificates.
• Bottom: the control plane pushes configuration and certificates to the proxies (dashed edges). It is not in the request path.
• Right: sidecars report metrics and traces (dotted). The Markdown block shows a policy for payments: who may call it, timeouts and retries, outlier ejection and strict mTLS.

When not to use a mesh

• A handful of services, or a monolith with a few satellites. A library or an API gateway covers the needs at a fraction of the cost.
• No team to own it. A mesh is infrastructure that needs upgrades, debugging skills and on-call knowledge.
• When the main goal is encryption only. Many clouds and CNI plugins can encrypt pod traffic without a full mesh.

What to put on a mesh diagram

• Both planes, clearly separated, with the control plane off the request path.
• Where TLS starts and ends: proxy to proxy, and plain HTTP on localhost.
• Identities, at least for services with access policies.
• Policies that change behavior: timeouts, retries, circuit breaking.

The Kubernetes architecture diagram shows the cluster the mesh runs in, the zero trust architecture diagram explains why identity-based policies matter, and the microservices architecture diagram covers the services themselves.

Build it in Archboard

1. Sign in with Google or GitHub and create a diagram.
2. Add a Container from the C4 Model pack for the ingress gateway.
3. For each pod, add a Frame from the Core pack with a Component for the sidecar and a Container for the app inside it.
4. Add a Container for the control plane and Metrics from the DevOps pack.
5. Draw mTLS edges between sidecars, dashed edges from the control plane, dotted telemetry edges, and a Markdown block from the Core pack with one service's policy.

The example uses 13 diagram nodes, within the Free plan.

FAQ

What is the difference between a service mesh and an API gateway? An API gateway manages traffic entering the system from outside. A service mesh manages traffic between services inside it. Many systems use both.

Does a service mesh add latency? Yes, each proxy hop adds a small amount, usually well under a few milliseconds per hop, plus CPU and memory for the proxies. Measure it for your traffic.

Do I need to change my application code for a mesh? Mostly not. The exception is tracing: apps must forward trace headers for traces to connect across services.

What is ambient mode? An Istio data plane without sidecars: a per-node proxy handles mTLS and layer 4, and optional waypoint proxies handle layer 7 features where needed.

Who should own the mesh? A platform team. Service teams use its features through policies, but someone has to upgrade it, monitor it and debug it.

Start free — one seat, no credit card.