Platform Engineering · Lesson 144

Docker Compose: A Whole App in One File

Lesson 143 wired two containers by hand, one command at a time. This lesson wires four at once, in one file: a web front, an API, a database, and a cache. The file is called compose.yaml. It names the services, the networks, and the volumes, and it says which service waits for which. By the end you have a whole app that starts with one command and stops with one command, in the right order.

Difficulty: Advanced · Know this first: Lesson 143 — Docker Networking: Containers Talking to Each Other

Run it as:

When it breaks, I check

One thing to remember

Where the time goes

Every hop charges rent in milliseconds. The bar below is a worked example for this lesson's scenario — not a measurement from a live system — but the shape is honest: on a cache hit the trip is almost free, on a miss the kitchen eats the budget. Step through the lesson or switch the scenario and watch the bar change.

Extra diagram · from scratch to master

The app, mapped: web, API, database, cache

Start with one file. Add one service at a time until four containers run as one app, in the right order.

4 services1 file1 missing condition
Compose file
1 min
Web
2 min
API
3 min
Database
5 min
Cache
2 min
Health gate
3 min
Starts too early
trip ends
Request / flowResponse · the way backFailure / dead end

Try it: change the inputs and watch the trip bend

The budget bar above shows three fixed stories. This simulator lets you write your own. Move a slider and watch the start, the wait, and the ready time change in the budget bar, in the hop timings, and in the total. The numbers are a teaching example, not live measurements. The shape is the lesson: more services and longer health waits mean a longer start, and the database is the long pole.

4

How many services the app has. Every service adds its own start time, and more services mean more waiting on the ones below them.

3

How many start-order edges the app has: web waits on API, API waits on database, and so on. Each edge is a wait for the service below it.

15

How long each service takes to report healthy. A slow database makes every service above it wait longer.

Try this: (1) Set services to 10, depends_on edges to 10, health wait to 60. The start is about 20 min, the wait about 20 min, the ready about 10 min, and the total is about 52 min. (2) Set services to 2, depends_on edges to 1, health wait to 5. The start is about 4 min, the wait about 0 min, the ready about 0 min, and the total is about 6 min. Same file shape. Only the inputs changed.

Check yourself

Three questions. No score is saved anywhere — this is just you against the material.

Try it: edit the pipeline and check it

Here is the full compose file from this lesson, ready to edit. It runs a web front, an API, a database, and a cache, with the API waiting for a healthy database. Change it, then run the check.

Try breaking it on purpose: delete the condition: service_healthy line. Run the check and watch it fail. Then press Reset and try a change that keeps it passing, like adding a second volume for the cache.

Press "Run the check" to see whether this pipeline does what the lesson asks.

Expected output when it passes: The file has four services, a health-gated depends_on, a database healthcheck, and named networks and volumes. This app would start in the right order.

Basic to master

I take the same file four times here. First I learn the four services and what each one does. Then I learn the order: who waits for whom, and what proves ready. Then I learn the data: which volumes survive, and which do not. Last I run the whole app as one thing.

Basic — Four services, one file

Definition: Compose is one YAML file that names every service in the app. A service is one container, named, with an image, and optional ports, environment, volumes, and networks. Compose creates one network for the app and puts every service on it, so the services find each other by name. Docker compose up starts the app. Docker compose down stops it. Docker compose ps lists it.

Types:

  • A service, which is one container in the app.
  • A network, which Compose makes for the app, so services find each other by name.
  • A volume, which holds the data that must survive a restart.
  • An environment block, which passes values into a service.

Uses: I use this to run a whole app on my machine with one command. I write the file, I validate it with docker compose config, and I start it with docker compose up -d. Try it now: edit the compose file in the Try it box so the web depends on the API, and run the check. Then practice the full app in the Docker labs.

docker compose up -d, flag by flag. up builds the stack and starts it. -d is the flag that gives me my terminal back.

  • up -d creates and starts the services. -d detaches, so the prompt comes back at once. Without -d the logs fill the terminal, and the stack stops when I close it.

Flags you will meet next: --build rebuilds the images before starting, and --force-recreate recreates the containers even when nothing changed.

Try it: run up -d, then run docker compose ps. The services should show as running. Then run up without -d in a scratch terminal and watch the logs fill it. Stop it with Ctrl-C and note the stack stops with you.

docker compose config, flag by flag. config prints what Compose really got, after every file and variable is layered. It is the fastest way to check a compose file without starting anything.

  • docker compose config prints the merged compose file. It is the fastest way to see what Compose really got, after all the files and variables are layered together.
  • --quiet only checks the file is valid and prints nothing. Without it config prints the whole merged YAML. I use --quiet in a check, and plain config when I want to read the merge.
  • --services prints only the service names, one per line. Without it config prints the whole merged file.
  • --volumes prints only the volume names, one per line. Without it config prints the whole merged file.

Flags you will meet next: config --models prints the model names, and --variables prints the variables and their defaults.

Try it: run config --quiet and check the exit code: 0 means the file is valid. Then run config --services and read the names. If a service is missing from that list, Compose never saw it.

Dimensions: At low traffic everything is nearly free. What breaks first is the YAML: a wrong indent, a missing colon, a service name that does not match. It costs one docker compose config before every up.

Intermediate — Order: who waits for whom

Definition: depends_on says a service starts after another one. With no condition, it waits only for the container to exist. With condition: service_healthy, it waits until the other service reports healthy. With condition: service_completed_successfully, it waits for a one-off job to finish and succeed. The database is the service everything waits on, because it is the slowest to become ready.

Types:

  • depends_on with no condition, which waits for the container to exist.
  • depends_on with service_healthy, which waits for the healthcheck to pass.
  • depends_on with service_completed_successfully, which waits for a job to finish.
  • A healthcheck, which proves a service is ready, not just started.

Uses: I use this when an app fails on its first lines and then works. I add condition: service_healthy to the database entry and run docker compose up again. Try it now: delete the condition line in the Try it box, run the check, and watch it fail. Then press Reset. Then practice the ordering in the Docker labs.

Dimensions: As the app grows, the start order grows, and a missing condition costs a flaky start that works on the second try. What breaks is the retry that hides the bug: the API retries until the database is ready, and nobody notices the order is wrong until the day the retry gives up.

Advanced — Data: what survives, and what does not

Definition: A named volume survives docker compose down, because Compose keeps volumes unless you pass -v. An anonymous volume is hard to find again, and a plain down can remove it. The database gets a named volume, because its data is the app. The cache gets none, because its data is disposable. The network is removed on down and remade on up, because the network holds no data, only wires.

docker compose down -v, flag by flag. down stops the stack. -v decides whether the data survives it. That one letter is the difference between a restart and a wipe.

  • docker compose down stops the services and removes their containers and networks. The volumes stay, so the data survives a down and a second up.
  • down -v -v also removes the named volumes. The data is gone. That is the difference between down and down -v, and it is the flag I check twice before I run it.

Flags you will meet next: -t 10 waits ten seconds for a graceful stop, and --rmi local also removes the images the stack built.

Try it: run down, then list the volumes: the data volume is still there. Run up -d again and the data is back. Then run down -v and list again. The volume is gone, and the next up starts empty.

Types:

  • A named volume, which survives a down and is easy to find.
  • An anonymous volume, which is hard to find and easy to lose.
  • A bind mount, which maps a host folder into a container, and is for dev, not for data.
  • A network, which is remade on every up, because it holds no data.

Uses: I use this when the data must survive a restart and a redeploy. I name the database volume in the file, and I test it: down, up, and read the data back. Try it now: add a volume to the cache service in the Try it box and run the check. The check still passes. The lesson text is what tells you the volume does not belong there.

Dimensions: At ten times the services, the volumes multiply. What breaks is the unnamed volume: a database whose data lives in a volume nobody can name, and a down -v that deletes it. It costs named volumes, written in the file, from the first day.

Validate the ordering before you start the stack. A depends_on without a condition starts the API at the same time as the database, and the API fails on its first connection. I check the compose file with a script that reads the YAML and lists every service, its dependencies, and the condition on each one. Any dependency with no condition is flagged. The fix is condition service_healthy with a real health check on the database, so the API starts when the database is ready, not when it merely exists.

  • Every depends_on needs a condition. Started is not ready.
  • The health check must test the real thing: pg_isready for the database, not a TCP connect.
  • Validate the file with compose config before you run it. A typo in YAML is a stack that never starts.

Try it: validate a sample compose file and find the missing condition.

# python3 compose-check.py
import yaml
text = """
services:
  api:
    depends_on:
      db:
        condition: service_healthy
  cache:
    depends_on:
      db: {}
  db:
    image: postgres:16
"""
data = yaml.safe_load(text)
for name, svc in data["services"].items():
    for dep, opts in (svc.get("depends_on") or {}).items():
        cond = (opts or {}).get("condition", "NONE")
        print(name, "depends on", dep, "- condition:", cond, "->", "OK" if cond == "service_healthy" else "FLAG: no health gate")

Expected: api is fine with service_healthy, cache is flagged with no condition. The cache starts with the database and fails the same way the API used to.

Master — The whole app as one thing

Definition: At this level the compose file stops being a dev tool and becomes the app: one file, versioned with the code, that starts the whole system on any machine with one command. The file names the services, the order, the networks, and the volumes. The secrets come from outside the file. The images are pinned by version. And I rehearse the worst day: the database is slow, the cache is empty, the API starts early, and the app still comes up clean.

Types:

  • One file, which is the app, versioned with the code.
  • Pinned images, which never move under the app.
  • Secrets from outside the file, which never land in git.
  • A rehearsed cold start, which proves the order holds when everything is slow.

Uses: I use this when more than one person runs the app, or when the app must start clean on a fresh machine. It is also how I review a setup I did not build: I read the compose file and ask what each service waits for, and what proves it is ready. Try it now: open any compose file you own and find the condition on each depends_on. If one has none, that is the first gap to close.

Dimensions: At a hundred times the services, the file is not the bottleneck, the blast radius is. What breaks is the single file every service depends on, or the secret that landed in it. It costs a split into overrides and a secret store, not more services in one file.

Down must not delete the data, unless I say so. A compose down that deletes the volumes deletes the database. That is the default only when I pass the flag, but the flag is one letter away from a habit. I check the volumes before and after a down, and I keep a backup before any down that touches a volume. The rule is simple: down keeps data, down -v deletes it, and down -v never runs on a stack with real data in it without a backup first.

  • Down keeps the data. Down -v deletes it. Know which one you typed.
  • Back up before any down -v on a stack with real data.
  • List the volumes after the down and confirm the data volume is still there.

Try it: list the commands that check the data survives a down.

# Run on a host with Docker Compose. Check the volume before and after.
docker compose ps
docker volume ls --filter name=app-data
docker compose down
docker volume ls --filter name=app-data
echo "volume still listed above means the data survived the down"

docker volume ls, flag by flag. A volume is invisible until it is listed. This is the command that proves the data survived.

  • docker volume ls lists the volumes on the host. Without it I cannot tell if the data survived, because the volume is invisible until it is listed.
  • --filter name=app-data shows only volumes whose name matches. Without it I get every volume on the host, and the one I care about hides in the list.

Flags you will meet next: docker volume create makes a volume by hand, docker volume inspect shows where it lives on disk, and docker volume prune deletes unused volumes, so I never run it blind.

Try it: run docker compose down, then docker volume ls --filter name=app-data. The volume is still listed. The data survived the down. Then run down -v and list again. It is gone.

docker compose ps, flag by flag. ps lists the containers Compose manages for this project. A container started by hand does not show up here.

  • docker compose ps lists the containers Compose manages for this project. A container started by hand does not show up here, because Compose only knows its own project.

Flags you will meet next: --services lists the service names, and -q prints only the container IDs.

Try it: run docker compose ps, then run docker ps. The first shows only this project. The second shows everything on the host. The gap between them is the containers Compose does not own.

Expected: the volume is listed both times. If it is gone after the down, the command was down -v, and the data is gone with it.

Go deeper

Start with the full file below. It is the app this lesson builds: web, API, database, and cache, with the API waiting for a healthy database. Do this: save it as compose.yaml, run docker compose config to validate it, then run docker compose up -d. Expected output: the config check prints the file back clean, and the up command starts four containers.

services:
  web:
    image: myapp-web:1.0.0
    ports:
      - "3000:3000"
    depends_on:
      - api
  api:
    image: myapp-api:1.0.0
    environment:
      DATABASE_URL: postgres://db:5432/app
      CACHE_URL: redis://cache:6379
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_started
  db:
    image: postgres:16
    volumes:
      - db-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5
  cache:
    image: redis:7
networks:
  app-net:
    driver: bridge
volumes:
  db-data:

What the snippet means: the web is the only service with a published port, and it waits on the API. The API connects to db and cache by name, and waits for a healthy database and a started cache. The database holds its data on a named volume and proves it is ready with pg_isready. The cache holds disposable data and needs no volume. One network carries the app.

depends_on without a condition waits for existence, not readiness. Do this: remove the condition: service_healthy line, run docker compose down and docker compose up -d, then run docker compose logs api. Expected output: the first log lines show connection refused, then the API connects. Put the condition back, repeat the down and up, and read the log again. Expected output: no refused lines. The API waited. That is the difference, and it is one line.

docker compose logs, flag by flag. Logs are per service here, not per container. The service name is the filter.

  • docker compose logs api shows the logs of one service. Without the service name I get every service mixed together, and the error I want hides in the middle.

Flags you will meet next: -f follows the logs, and --tail 50 shows the last fifty lines.

Try it: run docker compose logs api, then run it with no service name. The first shows one service. The second shows every service mixed together, and the error hides in the middle.

The data survives a down, and that is worth proving. Do this: write a row into the database, run docker compose down, then run docker compose up -d and read the row back. Expected output: the row is still there. Then run docker compose down -v in a scratch copy of the app only, and read again. Expected output: the row is gone. The -v flag is the difference, and it is forever. Practice the full app in the Docker labs.

At 10x the services, the shape holds and the numbers grow. At 100x, the design changes: the file splits into a base file and overrides, the secrets move out of the file, and the images are pinned by version. The services never stop being one file per app. The file just gets more readers. Lesson 145 is where the images get small.

Validate the whole file with one command

A compose file with a typo is a stack that fails in a confusing way, and the error rarely points at the typo. I validate the file before I run it, every time. Compose config prints the file back with everything resolved: the images, the networks, the volumes, the dependencies. I read that output and I check the four things that matter: the API depends on the database with a health gate, the database has its volume, the web is the only published port, and the network is the one I named. If any of those is wrong in the output, it is wrong in the file.

# Validate and inspect the resolved file. Run in the folder with compose.yaml.
docker compose config --quiet && echo "compose file is valid YAML and valid Compose"
docker compose config --services
docker compose config --volumes

Expected: the first line prints valid, the second lists web, api, db, and cache, the third lists the database volume. If the first line fails, I fix the YAML before I run anything.

One time it broke

What broke

An app failed on every deploy, then worked on the second try, and nobody knew why. The API started before the database was ready, failed its first connection, and retried until the database answered. On a fast machine the retry worked. On a slow machine the retry gave up first, and the deploy failed. The team called it flaky and moved on, twice. It was not flaky. It was an order bug with a retry hiding it.

Timeline

  • 11:02,the deploy started, and the API log showed connection refused on its first lines.
  • 11:03,the API retried, connected, and the deploy went green. The team called it a slow database.
  • 11:40,the next deploy ran on a slower host. The API retried, gave up, and the deploy went red.
  • 11:50,I opened the compose file and saw depends_on: [db] with no condition. It waited for the container to exist, not for the database to be ready.
  • 11:55,I added condition: service_healthy to the db entry and redeployed.
  • 12:00,the API started after the database reported healthy, and the log showed no refused lines.

What changed after

Two things. First, every depends_on in every compose file has a condition, and a depends_on with no condition is treated as broken in review. Second, I read the first lines of the API log after every deploy, not just the last ones. The first lines are where an order bug lives. Try it now: open any compose file you own and find the condition on each depends_on. If one has none, add it before the next deploy.

If you remember nothing else

Build it yourself

Build a working four-service app in one compose file: web, API, database, and cache, with the API waiting for a healthy database. It passes when all four services report healthy, the web answers hello, the data survives a restart, and the API log shows no refused connections.

Build it, step by step

  • Write a compose.yaml with four services: web on port 3000, api with DATABASE_URL and CACHE_URL, db on a named volume with a pg_isready healthcheck, and cache with no volume.
  • Add depends_on to the api service with condition: service_healthy for db and condition: service_started for cache, and validate the file with docker compose config.
  • Start the app with docker compose up -d, and confirm all four services show in docker compose ps with db marked healthy.
  • Write a row into the database, run docker compose down and docker compose up -d, and confirm the row is still there.
  • Read the API log with docker compose logs api and confirm the first lines show a clean connection, with no refused lines. Then stop the app with docker compose down.

What success looks like

Success looks like this: the config check prints the file back clean. All four services show in ps, with db healthy. Curl answers hello on port 3000. The row survives the restart. The API log has no refused lines. If the log shows refused lines, the condition is missing. If the row is gone, the volume is unnamed or the down passed -v.

If it fails, check this

If the config check fails, the YAML is broken. Read the error line. If the API log shows refused lines, the depends_on condition is missing. If the data is gone after a restart, the volume is anonymous or the down passed -v. If the web answers but the page is empty, the web started before the API. Check the order.

Next in this series
Lesson 145 — Multi-Stage Builds and BuildKit: Small, Fast Images