Platform Engineering · Lesson 22

Proxies, Reverse Proxies, and Load Balancers: Who Speaks for Whom

Three boxes sit between a person and a website, and they get mixed up all the time. A forward proxy works for the client: it sits in the office or the private network and goes out to the internet on everyone's behalf. A reverse proxy works for the server: it sits in front of the site, ends TLS, routes by path, caches, and hides the machines behind it. A load balancer is a reverse proxy with one main job, spreading requests across many copies of the same service and dropping the ones that fail their health checks. This lesson follows one request from an office laptop, out through the office proxy, across the internet, into the site's reverse proxy and load balancer, to a backend and back, and shows what each box does, what it hides, and how two of them together can lock a whole office out.

Difficulty: Intermediate · Know this first: Lesson F22, Lesson F20, Lesson 21

Run it as:

When it breaks, I check

One thing to remember

Where the time goes

This bar splits one request into its hops. It is a worked example, not a measurement. The two proxies cost a few milliseconds each; the backend is most of the bar. When something goes wrong at a proxy, the bar often gets shorter, not longer, because the request is stopped or rejected early.

Try it: how many people share one address?

An office proxy puts many people behind one public address. Move the sliders and see what a per-IP rate limit does to them, and what happens when the limit is keyed on the user instead.

requests from the office address = people × requests per person · rejected = (requests − limit) ÷ requests when the limit is keyed on the IP address.

4,000requests/min from one IP
85%rejected, keyed on IP
0%rejected, keyed on user

Keyed on IP, most of the office is locked out

Check yourself

Three questions. No score is saved anywhere — this is just you against the material.

Basic to master

Proxies grow in four levels: first which side each box works for, then configuring a reverse proxy and a balancer, then the client address, health, and timeouts at scale, and finally designing the whole chain from CDN to pod and debugging it hop by hop.

Basic — Which side does it work for?

At the basic level you can point at any box in a request path and say whose side it is on. A forward proxy is chosen by the client side, an office, a school, a private subnet, and goes out to the internet for them. A reverse proxy is run by the site and receives traffic for its servers. A load balancer is a reverse proxy that spreads requests across copies of one service.

Types:

  • Forward proxies: an office web proxy, a school filter, a cloud egress proxy, a VPN or privacy proxy.
  • Reverse proxies: Nginx, Caddy, Envoy, Traefik, HAProxy, a CDN edge like Cloudflare.
  • Load balancers: AWS ALB and NLB, Google Cloud Load Balancing, HAProxy, F5, kube-proxy inside Kubernetes.

Uses: a forward proxy to control and log what people or servers reach; a reverse proxy for one front door with TLS, caching, and routing; a load balancer when one copy of a service is not enough or must not be a single point of failure.

Where it breaks: mixing up the sides. A block page from the office proxy looks like the site is down, and the site team can never reproduce it.

Try it: Basic

A school installs a box that every student's laptop must go through, and it blocks gaming sites. What is it?

Now do it: Draw the path from your own laptop to a site you use at work. Mark every proxy and which side runs it.

Intermediate — Configure a reverse proxy and a pool

At the intermediate level you can stand up a reverse proxy in front of an app: terminate TLS, route paths to different pools, serve static files from a cache, and balance a pool with health checks.

Types:

  • Layer 4 balancing (TCP or UDP): fast, never reads the request, cannot route by path. AWS NLB, HAProxy in TCP mode.
  • Layer 7 balancing (HTTP): reads the request, routes by host, path, header, or cookie, and can end TLS. AWS ALB, Nginx, Envoy.
  • Algorithms: round robin, least connections, weighted, and consistent hashing on a user or session key.
  • Health checks: active (the balancer asks each backend) and passive (it notices failing responses).

Uses: one domain serving a frontend at / and an API at /api; zero-downtime deploys by draining one backend at a time.

Scale and failure: a health check that only asks "is the process up" keeps sending traffic to a backend that cannot reach its database. Check what the request needs.

Try it: Intermediate

You need to send /api to one pool and /images to another. Which balancer can do it?

Now do it: Write an Nginx config with two upstream pools and two location blocks, then run nginx -t to check it.

Advanced — Client address, timeouts, and the proxy as a single point of failure

At the advanced level you handle what a proxy changes about a request. The connection now comes from the proxy, so the client's address must travel in X-Forwarded-For or the PROXY protocol, and the app must trust it only from its own proxies. Every proxy adds its own timeouts and limits. And the proxy that removed the backends' single point of failure is now one itself.

Types:

  • Client address: X-Forwarded-For and Forwarded headers at Layer 7, the PROXY protocol at Layer 4, read with a trusted-proxy list.
  • Timeouts per hop: connect, read, idle, and keep-alive, each shorter than the one in front of it.
  • Proxy redundancy: an active and standby pair with a floating address, an active-active pair behind DNS or anycast, or a managed cloud balancer.

Uses: rate limiting, fraud checks, and audit logs that need the real client; long-lived WebSocket and streaming connections.

Scale and failure: trusting the leftmost X-Forwarded-For value lets anyone forge their address; take the rightmost address your own proxies did not add. A proxy idle timeout shorter than the backend's keep-alive gives random 502 errors under load.

Try it: Advanced

A request arrives with X-Forwarded-For: 1.2.3.4, 198.51.100.23, from your reverse proxy at 10.0.4.12. Your proxies are 10.0.0.0/8. What is the client's address?

Now do it: Run the client_ip.py script in Go deeper, then add a second trusted proxy to the chain and check the answer stays the same.

Master — Design the whole chain and debug it hop by hop

At the master level you design the full path, CDN, cloud load balancer, ingress, service mesh sidecar, pod, plus the egress path out, and you can say for each hop what it terminates, what it adds, what it hides, and what its timeouts and limits are. When something breaks, you find the hop from the error itself.

Types:

  • A production chain: browser, CDN edge, cloud L7 balancer, ingress controller, sidecar proxy, pod; outbound through a NAT gateway or egress proxy.
  • API gateways: a reverse proxy with authentication, quotas, and API routing.
  • Service mesh: a proxy next to every service, for mutual TLS, retries, and traffic splitting between services.

Uses: any service with users across the internet and dependencies outside its network.

Trade-off: every proxy buys control, TLS, caching, routing, or policy, and costs a hop, a config to keep right, a timeout to align, and one more place an error page can come from. Add a proxy for a job you can name; remove the ones whose job nobody can.

Try it: Master

Users see a 504 from your site. The ingress logs show nothing for those requests. Where is the problem most likely?

Now do it: List every proxy between the internet and one of your pods, with its idle timeout. Check each one is longer than the hop behind it.

Go deeper

The question every proxy setup has to answer correctly is "who is the client?" Here is the rule in code: walk X-Forwarded-For from the right, skip the hops you run yourself, and stop at the first one you do not.

Find the real client address behind your proxies

# python3 client_ip.py
import ipaddress

TRUSTED = [ipaddress.ip_network("10.0.0.0/8")]     # our own reverse proxies and balancers

def trusted(ip):
    return any(ipaddress.ip_address(ip) in net for net in TRUSTED)

def client_ip(peer, xff):
    """Walk X-Forwarded-For from the right and stop at the first hop we do not run."""
    chain = [h.strip() for h in xff.split(",") if h.strip()] + [peer]
    for hop in reversed(chain):
        if not trusted(hop):
            return hop
    return chain[0]

seen = [
    ("10.0.4.12", "198.51.100.23"),               # office user, through our reverse proxy
    ("10.0.4.12", "1.2.3.4, 198.51.100.23"),      # same user, with a forged header
    ("203.0.113.9", ""),                          # no proxy of ours in front: the peer is the client
]
for peer, xff in seen:
    print(f"peer {peer:<12} xff {xff or '-':<24} -> client {client_ip(peer, xff)}")

staff, per_user, limit = 2000, 2, 600             # people behind one office IP, requests/min, limit/min
sent = staff * per_user
print(f"keyed on IP:   {sent} req/min from one address, limit {limit}, rejected {(sent - limit) / sent:.0%}")
print(f"keyed on user: {per_user} req/min each, limit {limit}, rejected {max(0, per_user - limit) / per_user:.0%}")

Expected: all three requests resolve to the right client, 198.51.100.23, 198.51.100.23 (the forged 1.2.3.4 is ignored), and 203.0.113.9. Then keyed on IP: 4000 req/min from one address, limit 600, rejected 85% and keyed on user: 2 req/min each, limit 600, rejected 0%.

The same rule in Nginx

# reverse proxy: pass the client on
location / {
    proxy_pass http://shop_pool;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Host $host;
}

# a server behind it: believe the header only from our own proxies
set_real_ip_from 10.0.0.0/8;
real_ip_header   X-Forwarded-For;
real_ip_recursive on;

Do this: put the first block in a reverse proxy, the second in the backend's Nginx, and request a page. Expected: the backend's access log shows your public address, not 10.x.x.x. Remove set_real_ip_from and the log goes back to the proxy's address.

One time it broke

A retailer I worked with moved its site behind a new reverse proxy tier on a Tuesday afternoon. Everything passed its checks. On Wednesday morning their biggest business customer, one company with three thousand staff behind one office proxy, could not place orders.

Timeline

  • 09:05The business customer's staff start work. Orders fail with "Too many requests" after one or two clicks.
  • 09:20Support escalates. The site's dashboards are green: latency normal, backends healthy, error rate only slightly up.
  • 09:40An engineer notices the rate limiter's top key is not the customer's office address but 10.0.4.12, the new proxy. Every user on the site shares it; the office's morning rush pushed it over the limit.
  • 09:55The proxy config is fixed to set X-Forwarded-For, and the app is changed to read it from trusted proxies only. Orders recover.

What changed after

The proxy rollout checklist gained a line: log the client address the backend sees, from outside, before switching traffic. Rate limits for signed-in customers moved from the IP address to the account, because one office address can carry thousands of honest people. The old setting had been right by accident: the previous Layer 4 balancer passed the client address through untouched.

If you remember nothing else

Design exercise

Design the proxy layers for a company that runs a public web shop and a set of internal services: the shop must survive the loss of a server, and the internal services may reach only three outside APIs.

Clarifying questions to ask

Which traffic comes in, and which goes out? Does the shop need path-based routing, or is TCP balancing enough? Where should TLS end, and must traffic inside be encrypted too? How will the backends learn the client's address? Which outside APIs, and do they need fixed source addresses for allow lists?

Rough sizing

Say 3,000 requests a second at peak to the shop, six app copies at about 50 percent each, static files at 70 percent of requests, and a few hundred outbound calls a second from internal services.

Shape of the answer

Inbound: DNS to a CDN for static files, then a Layer 7 cloud balancer that terminates TLS and routes /api and /cart to an app pool with health checks on the real request path; X-Forwarded-For set at the edge and read with a trusted-proxy list. Outbound: an egress proxy with an allow list of three domains and fixed public addresses, logging every call. Both tiers in pairs or managed, across zones.

Trade-offs to name

Terminating TLS at the edge simplifies certificates and allows caching and routing, and means traffic inside must be protected another way. An egress proxy gives control and fixed addresses, and becomes a dependency every outbound call needs. A Layer 7 balancer can route and inspect, and costs a little latency and a more complex config than Layer 4.

Saved on this device, in this browser.
Next in this series · Lesson 23
Sharing the Work Evenly

One server cannot carry a real service. Sooner or later it fills up, or it dies at 2 AM, and every user finds out at the same time. The fix is not a bigger server.

Go to Lesson 23 →

Or browse the full course from the Lessons index.