The brief from the last lesson says what the parcel-tracking service must do, how fast, and how big. The high-level design says what it is made of: which components exist, what each one owns, which arrows are synchronous and which go through a queue, and where one failure stops spreading. This lesson follows one courier scan from the van to the customer's screen, hop by hop, and at every hop names the HLD decision that put it there. By the end you can draw an HLD that answers the three questions every reviewer asks first: what owns this data, what happens when this box is slow, and what happens when it is gone.
Difficulty: Intermediate · Know this first: Lesson 163, Lesson 48, Lesson 32
This bar is the freshness budget, not a request latency: the time from a scan in the van to the status on the customer's screen, against the 60-second requirement from the brief. It is a worked example, not a measurement, and the shape is the point: on a normal day the queue wait and the replica lag dominate, and both are deliberate.
Move the sliders. The processor pool is the box that sets the lag during a surge. Watch the queue hop in the freshness bar above follow your numbers.
surge rate = normal peak × multiplier · drain rate = workers × 250 scans/s · backlog growth/s = surge rate − drain rate (never below zero)
queue lag after a 2-minute surge = backlog ÷ drain rate · workers needed = surge rate ÷ 250, rounded up
The surge breaks the 60-second freshness budget
Three questions. No score is saved anywhere — this is just you against the material.
High-level design looks like drawing boxes. It is four levels of the same job: first boxes and arrows that match the requirements, then paths and boundaries that survive load, then failure domains and consistency choices that survive outages, then an HLD that keeps working as the company and the traffic grow around it.
At the basic level an HLD is a diagram where every box is a component you will deploy or depend on, and every arrow is a call or a message. The test is simple: each functional requirement from the brief can be traced as a path through the boxes, and each box has one sentence saying what it owns.
Types:
Uses: the first design review of any service. The win is a shared picture: everyone argues about the same boxes.
Where it breaks: boxes named after technologies, not responsibilities. "Postgres" says nothing about what the box owns; "parcel status, system of record" does.
Try it: Basic
Which label makes a better HLD box?
Now do it: Redraw one diagram you own with every box labelled by what it owns, not what it runs on.
At the intermediate level you separate the paths and mark every arrow. Reads and writes usually have different volumes, latency targets, and failure costs, so they get different paths. Every arrow is either synchronous (the caller waits) or asynchronous (the caller hands off to a queue or event), and the choice is written on the arrow.
Types:
Uses: any service where reads and writes differ by 10x or more, or where a client cannot wait. The courier app cannot wait for a database; the customer can wait 20 ms for a cache.
Scale and failure: a synchronous chain of five services at 99.9 percent each is about 99.5 percent available, and its latency is the sum. An async hop breaks both chains and costs you lag and at-least-once delivery, which means idempotency becomes mandatory.
Try it: Intermediate
Why does the ingest API answer 202 Accepted instead of 200 OK after processing?
Now do it: Mark every arrow on one of your diagrams sync or async. Any sync arrow to a slow or bursty component is a candidate for a queue.
At the advanced level the HLD answers what happens when each box is slow or gone. Every component gets a failure row: what breaks, what the user sees, and what takes over. Every copy gets a staleness number. Every queue gets a backpressure plan for when consumers fall behind.
Types:
Uses: services with an SLO, and anything other teams depend on. Reviewers at this level skip the happy path and read the failure table.
Scale and failure: at 10x the first thing to break is usually the shared dependency nobody drew, such as an auth service or a DNS resolver that every box calls. The cost is redundancy and complexity; the trade is between availability and the number of moving parts a person must understand at 2 AM.
Try it: Advanced
During a surge the scan queue grows for an hour. Nothing is lost. What should the HLD already contain for this case?
Now do it: Add a failure row for every box in one of your HLDs: what breaks, what the user sees, what takes over.
At the master level the HLD is designed for change: more traffic, more teams, more regions, and the migration from today's shape to next year's. You design cells and regions, you draw the cost model next to the component model, and you plan the path between versions so the system never needs a big-bang rewrite.
Types:
Uses: platform services, high-growth products, and any system whose outage makes the news. The habit at this level is to ask "what does this design look like at 10x, and how do we get there without stopping?"
Trade-off: cells and regions multiply operational load and make cross-cell queries hard. They are justified by the blast radius they remove, which the master-level HLD states as a number.
Try it: Master
Leadership wants the tracking service to survive a full region outage. What does a master-level HLD add first?
Now do it: For a system you know, write the blast radius of its biggest failure domain as a percentage of customers. Then sketch the cell or region split that would halve it.
The diagram is the start of an HLD, not the whole of it. I pair it with a component table, because a table can be checked and a diagram can only be admired. The table has one row per box and five columns: owns, called by, sync or async, staleness, and failure behaviour.
component owns in from kind stale if down courier app local scans, retries courier - - stores offline, retries ingest API validation, auth, accept courier app sync - app queues scans on phone scan queue ordered scans per parcel ingest API async ~2 s API rejects, phone retries processor status rule, dedupe scan queue async - queue grows, alarm on age primary DB scan_history, status (SoR) processor sync 0 processing stops, no loss read replica copy of status primary async ~2 s reads fail over to cache status cache copy, 30 s TTL read API sync 30 s reads go to replica read API customer status view customer sync - page shows last known
Every row has an owner and a failure behaviour. An empty cell in the last column is the design review's first question.
A synchronous path is only as available as the product of its parts. This short script makes that visible, and shows why the async boundary matters: the courier's write path only depends on the boxes before the queue.
# python3 path_availability.py
from math import prod
avail = {"gateway": 0.9995, "ingest_api": 0.999, "queue": 0.9999,
"processor": 0.999, "primary_db": 0.9995}
sync_write = ["gateway", "ingest_api", "primary_db"] # no queue
async_write = ["gateway", "ingest_api", "queue"] # courier waits for these only
for name, path in [("sync", sync_write), ("async", async_write)]:
a = prod(avail[c] for c in path)
print(f"{name:5} {a:.5f} ~{(1 - a) * 43200:.0f} min/month failing")
Expected: the sync path is about 99.80 percent, roughly 86 failing minutes a month from the courier's point of view; the async path is about 99.84 percent and, more importantly, no longer depends on the database at all. Add "processor" to the sync list and run it again to see the cost of one more synchronous hop.
A tracking service I reviewed wrote courier scans synchronously to the database. It had been fine for a year. Then a routine index rebuild slowed writes from 20 ms to four seconds for forty minutes on a Monday morning, the busiest scanning hour of the week.
The redesign put a queue behind the ingest API, moved the status rule into one processor, made scan id a unique key, and split the read path into its own deployment with its own connection pool. The next index rebuild slowed processing for an hour and nobody outside the team noticed. The HLD review checklist gained one line: every write path that a client retries must end at a durable buffer and an idempotency key.
Draw the HLD for a ride receipt service: when a ride ends, the rider gets a receipt in the app and by email within two minutes, and finance can query every receipt for seven years.
How many rides end per second at peak? Does the receipt need the final fare from the payment system, and how long can that take? Are receipts ever corrected after they are sent? What does finance query by: rider, date, city? Is email delivery part of the requirement or best effort?
Say 50 million rides a month, peaking at about 60 a second on a Friday night with a 3x surge: 180 receipts a second. At 3 KB each that is about 150 GB a month and 13 TB over seven years before compression, so the hot store and the archive are different boxes.
Ride-ended event onto a queue; a receipt builder that waits for the fare-finalised event and owns the receipt format; a receipts table as system of record; an email sender behind its own queue with retries; a read API for the app; nightly export to a columnar archive for finance. Two async boundaries: fare finalisation and email.
Waiting for the final fare delays the receipt but avoids sending corrections; sending immediately with "pending" is faster and noisier. One store for app and finance is simpler; a separate archive keeps the app fast and makes seven-year queries cheap. Email inside the builder is fewer boxes; email behind its own queue keeps a slow email provider from delaying in-app receipts.