Every working day starts the same way. Mail arrives. Pages load. Both feel instant, and both are the same shape underneath: a client asks, a lookup finds the right server, a gate checks the sender, a server answers. I have debugged both halves at 2 AM, and the failure is almost never where the user points. The mail is in the spam folder because the sender failed a check nobody set up. The page is slow because the proxy forwards to a backend that answers in 4 ms while the user waits 2 seconds. This lesson walks one mail and one page through the same nine hops, so you can see where each one breaks.
Difficulty: Intermediate · Know this first: Lesson 16, Lesson 63, Lesson 55, Lesson 24
By the end of this lesson you will have run a real mail conversation against a test server on your own machine, read real SPF and DMARC records with dig, and served a real page through Nginx and through Caddy as a reverse proxy. You will also know the four protocols by what they do, not by their initials: SMTP moves mail between servers, IMAP keeps mail on the server so every device sees the same inbox, POP3 downloads mail and usually deletes it from the server, and the three auth checks (SPF, DKIM, DMARC) answer one question: did this mail really come from the domain in the From line. You will know the difference between a forward proxy and a reverse proxy, and why a load balancer is not the same thing as a reverse proxy.
What you need: Python 3 for the mail lab, dig for the DNS checks, and a Caddy binary plus a compiled Nginx for the web server lab. The mail server and the web servers run on localhost only. No mail leaves your machine. No account is needed. The outputs shown are the outputs that ran on this machine, and they are marked that way. Your dates and message IDs will differ.
Mail and the web look like different systems. They are not. Both start with a client that asks. Both need a lookup to find the right server. Both pass a gate that checks the sender. Both end with a server that answers. The names change. The shape does not.
Mail starts when you hit send. Your mail client talks SMTP to your outgoing server. That is submission, usually port 587, usually with STARTTLS to lock the wire. The outgoing server asks DNS for the MX record of the destination domain. The MX record names the receiving server. The receiving server runs three checks before it accepts the mail. SPF asks if the sending server is allowed to send for that domain. DKIM asks if the signature on the mail matches the public key in DNS. DMARC asks what to do when either check fails, and it checks that the domain in the From line lines up with the domain that passed. Then the mail lands in a mailbox, and IMAP lets every device read that same mailbox. POP3 is the older way: it downloads the mail and usually deletes it from the server, so the mail lives on one device only.
The web half starts when you click a link in that mail. DNS finds the server. A reverse proxy answers first. It sits in front of the web server and speaks for it. It ends TLS, it caches static files, and it forwards the request to the backend that can answer it. The web server behind it serves the page. Nginx, Apache, and Caddy are all web servers that can do this job on their own. Nginx is the one I reach for when the config must be explicit and the proxy rules are long. Apache is the one with the longest history and the widest module list. Caddy is the one that ships TLS by default and needs the shortest config. Pick one and learn it well. The protocol does not care which one you picked.
A forward proxy sits next to the client and speaks for the client. Your browser asks the proxy, the proxy asks the internet. A reverse proxy sits next to the server and speaks for the server. The internet asks the proxy, the proxy asks the backend. Same box, opposite side. A load balancer is a reverse proxy with one extra job: it spreads the requests across more than one backend and it checks their health. Every load balancer is a reverse proxy. Not every reverse proxy is a load balancer. A reverse proxy with one backend and a cache is still a reverse proxy. It just has nothing to balance.
Run dig +short MX gmail.com and dig +short TXT gmail.com in your own terminal. The first line of the MX answer is the server your mail would go to. The TXT answer holds the SPF record. It reads v=spf1 redirect=_spf.google.com. That one line is the whole SPF policy for the domain. Then run dig +short TXT _dmarc.gmail.com. The answer reads v=DMARC1; p=none; sp=quarantine; rua=mailto:mailauth-reports@google.com. That is a DMARC policy that watches and reports but does not reject yet. Keep those three answers open. The lab repeats them, and the diagram walks through what each one does.
The diagram above follows one mail and one page through nine hops. These four paths are the same idea drawn four ways. One clean trip, mail to page. One forged sender, stopped at the gate. One size trail, counted in kilobytes. One quiet night, counted in percent. Run each path. One run takes 7.2 seconds across the path, so you can watch each hop light up in turn.
The return rule: the green dot only starts moving after the orange dot reaches the data hop. A return with no request is a lie. The diagram never draws one. The failure replay at the end of Run all four shows the other side of the rule. Send a forged mail, and the orange dot stops at the auth gate and never comes back.
One mail and one page, end to end. Client, SMTP relay, DNS MX lookup, auth gate, mailbox, reverse proxy, web server. Every hop answers, and the web server builds the page.
Not run yet.
Same mail, forged From line. SPF passes on the forger's envelope domain, but DMARC fails because the From line does not line up. The policy is p=reject. The mail stops at the auth gate and never reaches a mailbox.
Not run yet.
The same trip, counted in kilobytes at the simulator defaults: a 2 MB attachment, so each mail hop carries 2,056 KB, and a 180 KB page at the web server. The trail adds up to 8,405.2 KB. The DNS lookup and the auth check move almost nothing. The mail and the page move almost everything.
Not run yet.
The same trip at 3 AM. Each hop shows how busy it is. The relay runs at 12 percent. The proxy runs at 11 percent. The rest is capacity waiting for morning, and the mail queue is empty. Quiet nights are when a relay earns its keep by doing nothing.
Not run yet.
Every hop charges rent in milliseconds. The bar below is a worked example for one mail and one page, not a measurement from a live system, but the shape is honest. The auth gate is the expensive mail hop, because three checks run there. The web server is the expensive page hop, because it builds the answer. In the forged story the trip ends at the auth gate, because a rejected mail never reaches a mailbox. In the slow story the web server hop grows to 120 ms, because nothing is cached. Move a simulator slider and watch the web server hop and the total move with it.
The budget bar above shows three fixed stories. This simulator lets you write your own. It answers the trade-off at the heart of this lesson: mail volume fills the queue, and cache decides how fast the page is. You set how many mails arrive per minute, how big the attachments are, how often the cache hits, and how many proxy workers serve the page. Move a slider and watch the mail size, the queue delay, the web server time, and the budget total change. The numbers are a teaching example, built from the formulas below, not live measurements. The shape is the lesson: the queue grows with volume, and the page speed follows the cache.
How many mails the relay takes in each minute. 120 is a busy team. 1,000 is a mail storm, and the queue starts to grow.
How big the attachment on each mail is. 0 is a plain text mail. 25 MB is the usual provider limit, and the mail is 25,608 KB on the wire.
How often the proxy answers from cache instead of waking the backend. 70 percent is a healthy site. 0 percent is the slow story, where every byte is built from scratch.
How many workers serve the page. More workers answer faster, up to a point. One worker queues everything behind it.
Try this: (1) At the defaults, 120 mails a minute, 2 MB attachments, 70 percent cache hits, 4 workers, each mail is 2,056 KB, the queue delay is 0 ms, the web server hop counts 14 ms, and the budget total is 108 ms. (2) Push to 1,000 mails a minute, 25 MB attachments, 0 percent cache hits, 1 worker. Each mail is 25,608 KB, the queue delay is 76 ms, the web server hop counts 58 ms, and the budget total is 152 ms. The queue is the story now, not the page. (3) Drop to 10 mails a minute, no attachment, 100 percent cache hits, 8 workers. Each mail is 8 KB, the queue delay is 0 ms, the web server hop counts 1 ms, and the budget total is 95 ms. The cache did the work, and the workers had nothing to queue.
Formulas, in plain terms. Mail size in KB = 8 plus attachment MB times 1,024. Queue delay in ms = mails per minute minus 240, divided by 10, or 0 if the result is negative. The relay clears 240 mails a minute before the queue starts to grow. Serve time in ms = cache hits times 3 ms plus cache misses times 40 ms, then scaled by workers: each worker below 4 adds 15 percent, each worker above 4 cuts 15 percent. The budget total is the good-scenario hop sum with the web server hop replaced by the serve time. The numbers are a teaching model, not a benchmark. The shape will not change on a real relay.
Do this now, on your own machine. You will run a real SMTP conversation against a test server on localhost, read real SPF and DMARC records with dig, and serve a real page through Caddy and through Nginx as a reverse proxy. The Python parts and the server parts ran on this machine and their outputs are shown exactly as they ran. The Apache part runs on your machine, because no httpd binary is installed in this VM, and it is marked that way. No mail leaves your machine in any step.
Save this file as mail_server.py. It listens on 127.0.0.1:1025, answers EHLO, MAIL FROM, RCPT TO, DATA, and QUIT with real 250 codes, and saves the message it receives to received.txt. No TLS, no relay, no mail leaves this machine.
"""Minimal local SMTP test server. Standard library only.
Listens on 127.0.0.1:1025, answers EHLO/MAIL FROM/RCPT TO/DATA/QUIT
with real 250 codes, stores the message it receives. No TLS, no relay,
no mail leaves this machine. Run it, then run send_mail.py.
"""
import socketserver
class Handler(socketserver.StreamRequestHandler):
def handle(self):
self.wfile.write(b"220 localhost test SMTP ready\r\n")
data_lines = []
in_data = False
while True:
line = self.rfile.readline()
if not line:
break
text = line.decode("utf-8", "replace").strip()
if in_data:
if text == ".":
in_data = False
self.wfile.write(b"250 OK message accepted\r\n")
with open("received.txt", "w") as f:
f.write("\n".join(data_lines))
else:
data_lines.append(text)
continue
cmd = text.split(" ", 1)[0].upper()
if cmd == "EHLO":
self.wfile.write(b"250-localhost\r\n250 STARTTLS\r\n")
elif cmd in ("MAIL", "RCPT"):
self.wfile.write(b"250 OK\r\n")
elif cmd == "DATA":
in_data = True
self.wfile.write(b"354 End data with <CR><LF>.<CR><LF>\r\n")
elif cmd == "QUIT":
self.wfile.write(b"221 Bye\r\n")
break
else:
self.wfile.write(b"502 Command not implemented\r\n")
if __name__ == "__main__":
with socketserver.TCPServer(("127.0.0.1", 1025), Handler) as srv:
print("listening on 127.0.0.1:1025")
srv.serve_forever()
Run it in one terminal:
python3 mail_server.py
Then save this file as raw_smtp.py and run it in a second terminal. It opens a raw socket and prints every line both sides send.
"""Raw SMTP conversation with the local test server.
Prints every line both sides send, prefixed C: and S:. Run
mail_server.py first in another terminal. Standard library only.
"""
import socket
s = socket.create_connection(("127.0.0.1", 1025), timeout=10)
f = s.makefile("rwb")
def recv():
line = f.readline().decode().strip()
print("S:", line)
return line
def send(text):
print("C:", text)
f.write((text + "\r\n").encode())
f.flush()
recv()
send("EHLO localhost")
recv()
recv()
send("MAIL FROM:<reader@example.test>")
recv()
send("RCPT TO:<friend@example.test>")
recv()
send("DATA")
recv()
send("Subject: hello from the lab")
send("")
send("This is the body. One line, no attachment.")
send(".")
recv()
send("QUIT")
recv()
s.close()
python3 raw_smtp.py
Expected output, exactly as it ran on this machine:
S: 220 localhost test SMTP ready C: EHLO localhost S: 250-localhost S: 250 STARTTLS C: MAIL FROM:<reader@example.test> S: 250 OK C: RCPT TO:<friend@example.test> S: 250 OK C: DATA S: 354 End data with <CR><LF>.<CR><LF> C: Subject: hello from the lab C: C: This is the body. One line, no attachment. C: . S: 250 OK message accepted C: QUIT S: 221 Bye
Read the transcript top to bottom. The server greets with 220. The client says EHLO and the server answers 250 twice, once for itself and once to advertise STARTTLS. MAIL FROM and RCPT TO each get 250 OK. DATA gets 354, which means send the body now. The body ends with a line holding only a dot, and the server answers 250 OK, message accepted. QUIT gets 221 Bye. That is the whole SMTP conversation. Every mail you have ever sent did this, with more headers and a longer body.
These commands run against public DNS. They only read. Run them in your own terminal:
dig +short MX gmail.com dig +short TXT gmail.com dig +short TXT _dmarc.gmail.com
Expected output for the MX lookup, exactly as it ran on this machine. The order of the lines can vary between runs. The servers and the priority numbers do not.
40 alt4.gmail-smtp-in.l.google.com. 20 alt2.gmail-smtp-in.l.google.com. 5 gmail-smtp-in.l.google.com. 10 alt1.gmail-smtp-in.l.google.com. 30 alt3.gmail-smtp-in.l.google.com.
Expected output for the TXT lookup on gmail.com. The SPF record is the last line. The other lines are verification keys for other services, and they are not SPF.
"yahoo-verification-key=+eZwIPSgRxkAUzDtqT8Qhk+j4A1JF6V/wtGoyGTkELY=" "globalsign-smime-dv=CDYX+XFHUw2wml6/Gb8+59BsH31KzUr6c1l2BPvqKX8=" "_n365txknyxaxmmyyn7czzugzuiap479" "yahoo-verification-key=dKYwfVbaxatmcXiXy6LDAxMRirqpOq5tj98iJv9qWVk=" "v=spf1 redirect=_spf.google.com"
Expected output for the DMARC lookup:
"v=DMARC1; p=none; sp=quarantine; rua=mailto:mailauth-reports@google.com"
Read the two records plainly. The SPF record reads v=spf1 redirect=_spf.google.com. That one line is the whole SPF policy: look at _spf.google.com for the real list. The DMARC record reads v=DMARC1; p=none; sp=quarantine; rua=mailto:mailauth-reports@google.com. The policy is p=none, which means watch and report but do not reject yet. A domain that has finished its rollout changes p=none to p=reject, and then a forged mail is dropped at the gate instead of reported.
Download the Caddy binary for your machine from the Caddy releases page and put it on your PATH. On this machine the binary was Caddy v2.10.2, downloaded as a tarball and run from /tmp. Make a page first:
mkdir -p /tmp/webroot printf '<h1>Hello from Caddy</h1>\n<p>Served on localhost, port 2015.</p>\n' > /tmp/webroot/index.html
Then save this file as Caddyfile. It serves the page on port 2015 with TLS off, because this is localhost.
{
admin off
auto_https off
}
:2015 {
root * /tmp/webroot
file_server
}
caddy run --config /tmp/Caddyfile --adapter caddyfile
In a second terminal, curl it:
curl -s -i http://127.0.0.1:2015/
Expected output, exactly as it ran on this machine. Your Date and Etag values will differ.
HTTP/1.1 200 OK Accept-Ranges: bytes Content-Length: 65 Content-Type: text/html; charset=utf-8 Etag: "dn14xigm5g6q1t" Last-Modified: Sat, 21 Nov 2026 19:27:44 GMT Server: Caddy Vary: Accept-Encoding Date: Sat, 21 Nov 2026 19:27:46 GMT <h1>Hello from Caddy</h1> <p>Served on localhost, port 2015.</p>
Now the reverse proxy half. Start a plain backend on port 8001, then point Caddy at it with this file, saved as Caddyfile-proxy:
python3 -m http.server 8001 --directory /tmp/webroot
{
admin off
auto_https off
}
:2016 {
reverse_proxy 127.0.0.1:8001
}
caddy run --config /tmp/Caddyfile-proxy --adapter caddyfile
curl -s -i http://127.0.0.1:2016/
Expected output, exactly as it ran on this machine:
HTTP/1.1 200 OK Content-Length: 65 Content-Type: text/html Date: Sat, 21 Nov 2026 19:27:50 GMT Last-Modified: Sat, 21 Nov 2026 19:27:44 GMT Server: SimpleHTTP/0.6 Python/3.12.3 Via: 1.0 Caddy <h1>Hello from Caddy</h1> <p>Served on localhost, port 2015.</p>
Two headers prove the proxy. Server: SimpleHTTP/0.6 Python/3.12.3 is the backend answering. Via: 1.0 Caddy is the proxy saying it carried the request. The page is the same. The path is different. That is what a reverse proxy is: the same answer, carried by a middleman that can cache it, log it, and end TLS for it.
On this machine apt could not install Nginx, because the package lists were stale and another process held the apt lock, so Nginx 1.28.0 was compiled from the source tarball from nginx.org and installed to /tmp/nginx. On your machine, install it with your package manager if you can. The config below is the one that ran here, saved as nginx.conf. It serves the page on port 8080 and proxies port 8081 to the backend on port 8001. Your paths will differ. The shape will not.
worker_processes 1;
pid /tmp/nginx/logs/nginx.pid;
error_log /tmp/nginx/logs/error.log;
events { worker_connections 64; }
http {
access_log /tmp/nginx/logs/access.log;
server {
listen 8080;
server_name localhost;
root /tmp/nginx/html;
location / { index index.html; try_files $uri $uri/ =404; }
}
server {
listen 8081;
server_name localhost;
location / { proxy_pass http://127.0.0.1:8001; }
}
}
/tmp/nginx/sbin/nginx -t -c /tmp/nginx/conf/nginx.conf /tmp/nginx/sbin/nginx -c /tmp/nginx/conf/nginx.conf
Expected output from the config test, exactly as it ran on this machine:
nginx: the configuration file /tmp/nginx/conf/nginx.conf syntax is ok nginx: configuration file /tmp/nginx/conf/nginx.conf test is successful
Then curl the direct server:
curl -s -i http://127.0.0.1:8080/
Expected output, exactly as it ran on this machine. Your Date value will differ.
HTTP/1.1 200 OK Server: nginx/1.28.0 Date: Sat, 21 Nov 2026 19:29:45 GMT Content-Type: text/html Content-Length: 65 Last-Modified: Sat, 21 Nov 2026 19:29:28 GMT Connection: keep-alive ETag: "6b01f118-41" Accept-Ranges: bytes <h1>Hello from Nginx</h1> <p>Served on localhost, port 8080.</p>
One honest failure from this run, because it teaches a real lesson. The first direct curl answered 404, not 200. The error log said stat() "/tmp/nginx/html/" failed (13: Permission denied). The worker runs as nobody, and the directory was mode 750 owned by root. The fix was chmod 755 on the directory and chmod 644 on the file. Then the same curl answered 200. A 404 from Nginx is usually permissions or a wrong root, and the error log says which one in plain words. Read it before you change the config.
Then curl the proxy port, with the Python backend from step 2 still running on port 8001:
curl -s -i http://127.0.0.1:8081/
Expected output, exactly as it ran on this machine:
HTTP/1.1 200 OK Server: nginx/1.28.0 Date: Sat, 21 Nov 2026 19:29:30 GMT Content-Type: text/html Content-Length: 65 Connection: keep-alive Last-Modified: Sat, 21 Nov 2026 19:29:28 GMT <h1>Hello from Nginx</h1> <p>Served on localhost, port 8080.</p>
Same page, same backend, two front doors. Port 8080 serves the file directly. Port 8081 proxies to the backend. That is the difference between a web server and a reverse proxy, running on one machine with one config file.
This step runs on your machine, not in this VM. No httpd binary is installed here, so this config was not executed here. It is the same site in Apache shape, and it is the config to compare against the Nginx one above. Save it as a virtual host on a machine with Apache installed, enable it, and curl it the same way.
<VirtualHost *:80>
ServerName localhost
DocumentRoot /var/www/html
<Directory /var/www/html>
Options -Indexes +FollowSymLinks
AllowOverride None
Require all granted
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
Read it next to the Nginx server block. DocumentRoot is Nginx root. The Directory block is the permissions answer to the 404 in step 3: Apache says who may read the directory in the config, in plain words. Options -Indexes turns off directory listings, which is the safe default. The reverse proxy version of this host adds one line, ProxyPass / http://127.0.0.1:8001/, with ProxyPassReverse next to it. Same idea as the Nginx proxy_pass and the Caddy reverse_proxy. Three servers, one shape.
Stop the mail server and change one line in it: make the RCPT TO answer 550 No such user instead of 250 OK. Run the raw conversation again and watch smtplib report the refusal. Then put it back. A server you can break on purpose is a server you understand. Then change the Caddyfile to serve a different page and curl it again. The page changes, the 200 stays. That is the whole job of a web server.
Stop the mail server, the Caddy processes, the Nginx master, and the Python backends. Delete nothing else. The test mail never left your machine, and the pages only ever lived on localhost. The scripts and the configs stay in your work directory, and you can run them again on the next lesson.
Three questions. No score is saved anywhere. This is just you against the material.
Running mail and a web server is not one skill. It is four levels of the same job, and every team I have worked with sits on one of them. The levels are not about how many protocol names you can recite. They are about whether you can find the broken hop when something fails. Read them in order. Each level fixes the failure that hurt the most at the level below it.
At the basic level, you run the lab and you read the outputs. You send a mail to the test server and you read the transcript. You serve a page with Caddy and you curl it. You run dig on a domain you use and you read the SPF record. The failure that ends this level is the first real bounce. A mail that never arrives and no error anywhere is the moment you learn that mail fails silently.
Types:
Uses: Transactional mail from applications (password resets, receipts) and the web server in front of every service.
Scale and failure: Mail fails silently: a rejected or spam-foldered message produces no error in the sending application. Delivery must be measured from bounce and complaint reports, not assumed from a successful send.
Trade-off: Running your own mail server gives full control and makes deliverability your problem; a mail service handles reputation and costs per message.
Do this: run step 0 and step 1 of the lab. Send the test mail, read the transcript, then run the three dig commands on your own domain and write down the SPF and DMARC lines. Two outputs, one page.
Try it: Basic
My mail never arrives and there is no error anywhere. What did I just learn?
Now do it: Run step 0 and step 1 of the lab above: send the test mail, read the transcript, then run the three dig commands on your own domain and write down the SPF and DMARC lines.
At the intermediate level, you stop guessing and you start reading. A missing mail means reading the Authentication-Results header and the relay log, in that order. A slow page means reading the proxy access log and the upstream response time, in that order. You set up SPF, DKIM, and DMARC on a domain you control, and you move the policy from p=none to p=quarantine once the reports are clean. The failure that ends this level is the forged mail that passes SPF, because you learn that SPF alone was never enough.
Types:
Uses: Any domain that sends mail, and any service whose slowness must be located between the proxy and the backend.
Scale and failure: Moving DMARC straight to reject without reading reports blocks legitimate senders, such as a billing provider sending as your domain. Start at none, read a few weeks of reports, then tighten.
Trade-off: Strict authentication stops spoofing and can drop legitimate mail from senders you forgot. Reports are the cost of tightening safely.
Do this: take a mail in your own inbox, open its headers, and find the Authentication-Results line. Write down the result of each of the three checks. Then find one mail that failed a check and trace where it landed: inbox, spam, or quarantine.
Try it: Intermediate
A mail is missing. What do I read first?
Now do it: Take a mail in your own inbox, open its headers, and find the Authentication-Results line. Write down the result of each of the three checks.
At the advanced level, the reverse proxy is doing real work. It ends TLS, it caches static files, it rate limits the login page, and it forwards the rest to the right backend. The mail relay has a queue you watch, with an alert when the queue depth grows past the line you set. The DMARC policy is p=reject, and the rua reports arrive every day and get read. The failure that ends this level is the backend that dies behind a healthy proxy, because you learn that a proxy that answers 200 for a dead backend is a proxy that lies.
Types:
Uses: Every production web entry point, and any system that sends mail at volume.
Scale and failure: A proxy that answers 200 with an error page when the backend is down hides the outage from monitoring. Health checks must reflect the backend, and the proxy must return 502 or 503 quickly when it cannot reach one.
Trade-off: Doing more at the proxy (TLS, caching, limits) simplifies backends and makes the proxy a critical, more complex component that needs the same care as the services behind it.
Do this: add a cache rule for static files to the Nginx config from the lab, then curl the same asset twice and compare the response times. Then stop the backend and curl the proxy port. Write down what the user sees, and fix the proxy so a dead backend answers 502 fast instead of hanging.
Try it: Advanced
The proxy answers 200 but the backend is dead. What is wrong?
Now do it: Run the drill above: add a cache rule for static files to the Nginx config from the lab, curl the same asset twice, then stop the backend and curl the proxy port and write down what the user sees.
At the master level, mail and the web are one system with one runbook. The relay has a queue alert and a backpressure rule: when the queue passes the line, the relay slows intake instead of dropping mail silently. The proxy has a health check on every backend, and a dead backend leaves rotation on its own. The DMARC reports feed a dashboard, and a new failing domain gets a ticket the same day. The runbook for a missing mail and the runbook for a slow page are written down, and both start with the same line: read the log before you change the config.
Types:
Uses: Organisations where mail and web are customer-facing systems with SLOs, not side utilities.
Scale and failure: At volume, mail reputation is a shared resource: one bad campaign can push all transactional mail into spam folders. Separate sending domains or IPs for marketing and transactional mail keep one from hurting the other.
Trade-off: Separation and monitoring cost setup and ongoing attention; the alternative is discovering a deliverability problem from customers who never got their password reset.
Do this: write the two runbooks. One page for a missing mail: relay log, auth header, spam folder, queue depth. One page for a slow page: proxy log, upstream time, cache hit rate, backend health. Put the alert threshold on each page. A runbook with no threshold is a runbook nobody follows.
Try it: Master
The relay queue passes the line. What should the relay do?
Now do it: Write the two runbooks from the tier above. One page for a missing mail and one page for a slow page, each with its alert threshold on the page.
Nginx, Apache, and Caddy all serve the same page, and the config for each one says the same three things: listen on a port, serve files from a directory, and forward the rest to a backend. Nginx says it with a server block and a location. Apache says it with a VirtualHost and a Directory block. Caddy says it with a site address and two lines. The differences are real but small. Nginx wins when the proxy rules are long and explicit. Apache wins when the site needs a module that only Apache has. Caddy wins when you want TLS without thinking about it, because it fetches and renews the certificate on its own. I have run all three in production. The one I pick is the one the team already knows, because a server the team cannot debug is a server that will page them at 2 AM.
Here is the Nginx location block I would add to the lab config before any site goes live. It caches static files for an hour, it forwards the rest to the backend, and it sends the real client address through so the backend log stays honest.
location ~* \.(css|js|png|jpg|svg|ico)$ {
proxy_cache_valid 200 1h;
add_header Cache-Control "public, max-age=3600";
proxy_pass http://127.0.0.1:8001;
}
location / {
proxy_pass http://127.0.0.1:8001;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
Two lines in that block matter more than the rest. proxy_set_header Host $host keeps the backend seeing the real hostname, so virtual hosts and redirects work. X-Forwarded-For keeps the backend log showing the real client address instead of the proxy address. A backend log that only ever shows the proxy IP is a log that cannot tell you who hit it. I have debugged that exact log, and it is a long night.
Never start at p=reject. Start at p=none and read the rua reports for two weeks. The reports list every server that sends as your domain, including the ones you forgot: the marketing tool, the invoice tool, the old cron job. Add SPF and DKIM for each real sender. Then move to p=quarantine. Watch for another two weeks. Then move to p=reject. A team that jumps straight to reject blocks its own invoice mail on the first day, and the lesson they learn is to turn DMARC off. The rollout is the work. The policy is the last line.
The lab server has no TLS, on purpose. It is a test server on localhost. To check TLS on a real mail server, the command is openssl s_client -starttls smtp -connect mail.example.com:25, run from your own machine against a server you are allowed to test. It prints the certificate chain and the TLS version, and it tells you whether STARTTLS works at all. That flag is the only place the openssl command appears in this lesson, because the protocol teaching lives in the transcript, not in the TLS handshake. The TLS handshake itself is Lesson 55. Go there if the output surprises you.
At 10x the mail volume, the relay queue becomes the system. The simulator shows it: above 240 mails a minute the queue grows, and at 1,000 a minute the delay is 76 ms and climbing. The fix is not a bigger server. It is a queue you watch, a backpressure rule that slows intake, and a second relay. At 10x the web traffic, the cache becomes the system. The proxy serves the static files, the backend only sees the requests that miss, and the cache hit rate is the number on the dashboard. At 100x, mail and web split. The relay gets its own fleet with its own queue. The proxy gets more workers and a second backend. The auth checks do not change. SPF is still one DNS lookup. DMARC is still one policy line. The protocols scale because they were built simple. The systems around them are the part that grows.
One rule, no exceptions: a mail server must never relay mail for strangers. An open relay is a server that sends mail for anyone who asks, and spammers find it within hours. The test server in this lab only listens on localhost, which is why it is safe. A real relay checks authentication on submission, port 587, and it checks the recipient domain on relay, port 25. If you ever inherit a relay, the first command is the one that proves it refuses strangers. Everything else waits.
A forged invoice mail reached the finance inbox. It used the real company domain in the From line, the real invoice format, and a new bank account number. The mail looked right in every way except one: it was not from us. SPF passed, because the forger sent it from their own server with their own envelope domain. DMARC was p=none, so the receiving server noted the failure and delivered the mail anyway. Finance nearly paid it. The only reason they did not is that the amount was larger than usual, and someone called the supplier to check.
Three things. First, the DMARC policy moved from p=none to p=quarantine that week, and to p=reject two weeks later, after the reports were clean. Second, the rua reports now feed a dashboard, and a new sending domain gets a ticket the same day instead of sitting unread for three weeks. Third, finance got a rule that is not technical at all: a changed bank account number always gets a phone call, no matter how real the mail looks. The technical fix stops the next forgery. The phone call stops the one that gets through anyway.
Design mail and the web for a small shop. 5,000 customers. The site is a catalog with a checkout. The shop sends order confirmations, password resets, and a weekly mail to all 5,000 customers. The site must stay up when the checkout backend restarts. The weekly mail must not land in spam. Sketch the mail flow and the web flow, name the protocols and the checks, and say what breaks first when the shop doubles.
Start with the mail. Order confirmations and password resets go through the shop's own relay on port 587 with authentication, because they are transactional and they must arrive fast. The weekly mail to 5,000 people goes through a separate sender, because bulk mail has a different reputation and a different rate limit, and one bad bulk send must never delay a password reset. Both senders get SPF and DKIM. DMARC starts at p=none, moves to quarantine, then to reject, with the rua reports read every week. The weekly send is paced at 200 mails a minute, inside the relay's 240 a minute clearing rate from the simulator, so the queue never grows. At 200 a minute, 5,000 mails take 25 minutes. That is fine for a weekly mail. It is not fine for a password reset, which is why the two senders are separate.
Then the web. A reverse proxy answers first. It ends TLS, it caches the static catalog pages, and it forwards checkout requests to two backends with a health check. The checkout backend restarts one at a time, and the health check takes the restarting backend out of rotation, so the site stays up. The catalog pages cache for an hour, so the cache hit rate sits near 80 percent and the backend only sees the misses and the checkouts. The proxy log records the real client address with X-Forwarded-For, so the backend log stays honest.
Then the numbers. The weekly mail is 5,000 mails at 200 a minute, 25 minutes, queue delay near zero. The site at peak takes 300 requests a minute, and 80 percent hit the cache, so the backends see 60 requests a minute between two of them. Each backend answers in about 12 ms, so the web server hop stays fast. The budget for one page load is near the clean story: about 108 ms.
Trade-offs to name. Two senders cost more than one, and they buy separation: bulk reputation never touches transactional mail. Two backends cost more than one, and they buy the restart without downtime. The cache buys speed and costs freshness: a price change takes up to an hour to show, unless the shop purges the cache on deploy. DMARC p=reject buys safety and costs rollout time: two weeks at p=none, two weeks at quarantine, then reject. When the shop doubles, the weekly mail is what breaks first. 10,000 mails at 200 a minute take 50 minutes, and the queue is the first thing to watch. The second thing is the bulk sender's rate limit, which is a contract, not a config.
Saved in this browser, so the Lessons index can show your progress across the course.