API Gateway Architecture Diagram: Routing, Auth, Rate Limits and BFFs

The API gateway is the box every architecture diagram has and few explain. It sits at the edge and collects responsibilities over time: routing, authentication, rate limits, request rewriting, and sometimes business rules it should never have had. When the diagram shows an arrow passing through a gateway icon, nobody can tell which of those it does, and the questions come up during an incident instead of in a review.

This post builds an API gateway architecture diagram for a common setup: a web app, mobile apps and partners behind one gateway, with a backend for each frontend. It covers what the gateway should do and what it should not, how token checks and rate limits work, and what to label so the diagram answers real questions.

What belongs in the gateway

A load balancer spreads requests across instances; it does not know who the caller is. A service mesh handles traffic between services inside the system. The gateway handles traffic entering from outside and understands clients: their tokens, their routes and their quotas. Many systems run all three.

Token checks at the edge

In the example, clients sign in with the identity provider and send a JSON Web Token (RFC 7519) with each request. The gateway verifies the signature with the provider's public keys, which it fetches as a JWKS document and caches, and checks the claims: the issuer (iss), the audience (aud) and the expiry (exp).

The JWT best current practices in RFC 8725 are worth following here. A library must let you specify the allowed algorithms and must not use any others, so configure the list instead of trusting the token's header. When one issuer creates tokens for several applications, the token must carry an audience, and each application must check that the audience is its own.

A valid token says who the caller is. It does not say what they may do. The gateway can reject a missing scope, but the service still decides whether this user may change this order.

Rate limits

Count per client, not per IP address. Many users can share one address behind a NAT, and one scraper can use many addresses. Key the limit on the API key, the OAuth client or the user.

Share the counters. With several gateway instances, each one counting on its own, a limit of 100 per minute becomes 100 per instance. A shared store such as Redis keeps one count per client across all instances.

Pick an algorithm and write it down. A fixed window resets every minute and allows bursts at the boundary. A sliding window smooths that out. A token bucket allows short bursts up to a set size while holding an average rate.

Answer with 429. RFC 6585 defines 429 Too Many Requests. The response should explain the condition and may include Retry-After, which RFC 9110 defines as either an HTTP date or a number of seconds to wait. RFC 6585 does not say how the server identifies the client or counts requests: that part is your design, so put it on the diagram. An IETF draft defines standard RateLimit and RateLimit-Policy header fields, but at the time of writing (October 2026) it is still a draft.

Decide what happens when the counter store is down. Failing open keeps the API up without limits; failing closed protects the services and rejects traffic. Either can be right, as long as it is a decision and not an accident.

Backends for frontends

Sam Newman's Backends For Frontends pattern replaces one general-purpose API backend with a server-side component for each user experience. Each BFF is tightly focused on a single UI and is usually maintained by the team that builds that UI, so the team can change the API as its screens change. A shared, general-purpose backend tends to become a bottleneck, and the article notes that aggregating edge services have a habit of growing bloated code.

How many BFFs? Newman credits Stewart Gleadow with a simple rule: one experience, one BFF. If two apps offer the same experience and one team owns both, they can share one; SoundCloud, for example, uses one BFF for its iOS and Android listener apps.

Partners are different. They need a stable, versioned API, not one shaped around a screen, so their routes go to the services' public API without a BFF.

The example: an online shop

The shop has a React web app, iOS and Android apps, and partners that create orders through an API. A WAF filters traffic before the gateway. The gateway verifies tokens, counts requests per client in Redis and routes by path prefix. The web and mobile teams each own a BFF, and both BFFs call the Orders, Catalog and Accounts services inside the private network. Partner calls go straight to the Orders service's versioned API.

How to read it:

• Left to right, one request. A client calls through the WAF to the gateway, which routes it to a BFF or to the partner API.
• Dashed edges are off the request path. The web app signs in with the identity provider, and the gateway fetches the provider's signing keys and caches them.
• Below the gateway, Redis holds the rate limit counters that every gateway instance shares.
• Inside the private network, each BFF calls only the services its screens need.
• The Markdown block is the gateway policy: what it checks, the limit, and what it leaves to the services.

What to label

• Routes for each backend, as path prefixes or hosts.
• Authentication on every entry: the token type, who issues it and what the gateway checks.
• Limits: the key, the number and window, and what the client gets back.
• Timeouts and retries at the gateway. Retries at the edge multiply load on a struggling service.
• Failure behavior when the identity provider, the counter store or a backend is unavailable.
• Ownership. The platform team often owns the gateway configuration; UI teams own their BFFs.

Common mistakes

• Business logic in the gateway. Rewrites and conditions grow into rules nobody can test. Keep the gateway thin.
• The gateway as the only security check. Services behind it should not trust a request just because it arrived. The zero trust architecture diagram shows how to draw checks on every hop.
• Limits per IP only, which punish users behind shared addresses and miss clients that rotate addresses.
• Local counters on each instance, which multiply the limit by the instance count.
• One BFF for everything, which is a general-purpose backend again.
• A 429 without details. Clients that do not know how long to wait guess, and they usually guess too short.

For what sits behind the gateway, see the microservices architecture diagram. On Kubernetes, the Kubernetes Ingress diagram covers the layer that often sits in front of the gateway or replaces parts of it.

Build it in Archboard

1. Sign in with Google or GitHub and create a diagram.
2. From the C4 Model pack, add a Container for the web app and one for the mobile app, and an External System for the partner.
3. From the Security pack, add a WAF and an Identity Provider.
4. From the Cloud pack, add an API Gateway, and from the Data pack a Redis node for the rate limit counters.
5. Add a Core Frame named Private network, with C4 Model Container nodes for the two BFFs and the three services.
6. Connect them left to right, label each route with its path, and make sign-in and key fetching dashed. Add a Core Markdown block with the gateway policy.

The example uses 14 diagram nodes, within the Free plan.

FAQ

Is an API gateway the same as a load balancer? No. A load balancer spreads requests across instances. A gateway understands API requests: routes, tokens and quotas. Many gateways run behind a load balancer.

Do I need a service mesh if I have an API gateway? They solve different problems. The gateway handles traffic from outside; a mesh handles calls between services, such as mutual TLS and retries. Draw them separately if you run both.

Should services validate the token if the gateway already did? Yes, for the claims they rely on. The gateway rejects bad tokens early, and each service still authorizes the action itself.

How many BFFs should I have? One per user experience. Two apps with the same experience and the same owning team can share one.

What should a 429 response contain? A short explanation and a Retry-After header with the number of seconds to wait, so clients back off instead of retrying immediately.

Start free — one seat, no credit card.