Platform Engineering · Lesson 158

Docker Lab: Small Images, Production Limits

Lesson 145 taught you multi-stage builds and Lesson 146 taught you production operations. This lab is where you build both: a two-stage image that drops from 1.4 GB to about 120 MB, a BuildKit cache mount that makes the second build fast, and a production run with memory limits, a healthcheck, and a non-root user. By the end you have a small image that runs with limits, and you can prove the size with docker images.

Difficulty: Advanced · Know this first: Lesson 145 — Multi-Stage Builds and BuildKit; Lab 157; Docker Desktop installed and running

Run it as:

When it breaks, I check

One thing to remember

Where the time goes

Every hop charges rent in milliseconds. The bar below is a worked example for this lesson's scenario — not a measurement from a live system — but the shape is honest: on a cache hit the trip is almost free, on a miss the kitchen eats the budget. Step through the lesson or switch the scenario and watch the bar change.

Extra diagram · from scratch to master

The small image, mapped: build, runtime, limits, health

Start with a 1.4 GB image. Add one box at a time until the runtime stage ships at 120 MB.

7 boxes on the trip2 stages120 MB final
Docker
0 min
One stage
2 min
Two stages
3 min
Measure
1 min
Cache
1 min
Limits
1 min
Bad stage
trip ends
Request / flowResponse · the way backFailure / dead end

Try it: change the inputs and watch the trip bend

The budget bar above shows three fixed stories. This simulator lets you write your own. Move a slider and watch the build, the size, and the total change in the budget bar, in the hop timings, and in the total. The numbers are a teaching example, not live measurements. The shape is the lesson: more stages means a smaller image, and a cold cache means a longer build.

2

How many stages the Dockerfile has. One stage ships everything. Two stages throw the tools away. Three stages split the build further.

120

How big the final image is, in MB. A small image pulls fast. A big image pulls slow, and every rollout pays for it.

80

How much of the build is cached. A warm cache skips the download. A cold cache downloads everything.

Try this: (1) Set stages to 2, size to 120 MB, cache to 80%. The build is about 5 min, the pull about 1 min, the run about 1 min, and the total is about 12 min. (2) Set all three to the worst end: 1 stage, 1,400 MB, 0% cache. The build is about 7 min, the pull about 7 min, the run about 1 min, and the total is about 20 min. Same app. Only the inputs changed.

Check yourself

Three questions. No score is saved anywhere — this is just you against the material.

Try it: edit the pipeline and check it

Here is the multi-stage Dockerfile from this lab, ready to edit. Change it, then run the check. The check looks for the five things a working multi-stage file needs: two named stages, a COPY --from, a cache mount, a non-root user, and a CMD.

Try breaking it on purpose: change AS build to AS builder, or delete the --mount line. Run the check and watch it fail. Then press Reset and try a change that keeps it passing, like adding a HEALTHCHECK line.

Press "Run the check" to see whether this pipeline does what the lesson asks.

Expected output when it passes: The file has two named stages, a COPY --from=build, a cache mount, and USER node. The build would run, and the image would be about 120 MB.

Basic to master

I take the same image four times here. First I measure it. Then I split it. Then I limit it. Last I prove it.

Basic — Measure the before picture

Definition: A single-stage image ships everything: the code, the compiler, the build tools, and the dev dependencies. docker images measures it. The before picture is the number you write down before you optimise, so the after picture means something.

Types:

  • docker images, which lists the image sizes.
  • A single stage, which is the baseline.
  • A before number, which is the proof the optimisation worked.
  • curl, which proves the image still answers.

Uses: I use this for any image I am about to shrink: measure it first, write the number down, then change it. Try it now: build the single-stage image and write down its size.

Dimensions: At low volume everything is instant. What breaks first is a claim without a number: small is not a size, and 120 MB is.

Try it: Basic

I am about to shrink my image. Why do I write down the before number first?

Now do it: On your machine with Docker, build the single-stage image and write down its size from docker images. This VM has no Docker, so run it on your own machine.

Intermediate — Split the stages

Definition: A multi-stage Dockerfile has two or more FROM lines, each with a name after AS. The build stage has the tools. The runtime stage has only what runs. COPY --from=build carries the built files across, and the build stage is thrown away.

Types:

  • AS build, which names the build stage.
  • AS runtime, which names the runtime stage.
  • COPY --from=build, which carries the built files across.
  • A slim base, which is the runtime stage's starting point.

Uses: I use two stages whenever an image needs tools to build and does not need them to run, which is nearly always. Try it now: write the two-stage file and build it.

Dimensions: As the app grows, the stages multiply, and the build gets longer. What breaks is a stage name that does not exist: the build fails at once, and the error names the stage.

Try it: Intermediate

I write a two-stage Dockerfile. What happens to the build stage?

Now do it: On your machine with Docker, write the two-stage file and build it, then check the runtime stage has no build tools. This VM has no Docker, so run it on your own machine.

Advanced — Cache mounts and limits

Definition: A BuildKit cache mount, RUN --mount=type=cache, keeps the dependency cache between builds without shipping it. A memory limit kills the container at 137 when it is exceeded. A CPU limit slows it. USER node runs the container as a user, not as root.

Types:

  • --mount=type=cache, which keeps the cache between builds.
  • --memory, which sets the memory limit.
  • --cpus, which sets the CPU limit.
  • USER node, which runs as a user, not as root.

Uses: I use cache mounts for every build and limits for every run. Try it now: build twice and time the second build, then run with --memory=512m and inspect it.

Dimensions: At ten times the builds, the cache matters more, and the limits matter more. What breaks is a cache that ships: a COPY of the cache grows the image again, and the optimisation is undone.

Try it: Advanced

I set a memory limit and a CPU limit on my container. What does the CPU limit do?

Now do it: On your machine with Docker, build twice and time the second build, then run with --memory=512m and inspect it. This VM has no Docker, so run it on your own machine.

Master — The image as production

Definition: At this level the image is not a build artefact. It is a production artefact: small, scanned, limited, healthy, and run as a user. The size is measured, the limits are inspected, the health is checked, and the numbers are written down.

Types:

  • A size, which is measured, not claimed.
  • A scan, which blocks a vulnerable image.
  • Limits, which are inspected, not assumed.
  • A healthcheck, which is green, not missing.

Uses: I use this before any image ships to production: measure it, limit it, check it, write the numbers down. Try it now: write the before and after numbers in a note and check them against docker images.

Dimensions: At a hundred images, the registry is the shelf, and the sizes are the budget. What breaks is an image nobody measured: it grows a little every week, and one day the rollout is slow and nobody knows why.

Try it: Master

I never measured my production image. What breaks?

Now do it: Write the before and after numbers in a note and check them against docker images, as the Master tier says. This VM has no Docker, so run it on your own machine.

Go deeper

The multi-stage Dockerfile, the whole file. Do this: save it as Dockerfile.multi, copy it over Dockerfile, and run docker build -t hello-small:1.0.0 .. Expected output: the build runs two stages, and the last line names hello-small. This file was checked in this lab: every line starts with a known directive, both stages are named, and every COPY --from names a stage that exists.

docker build, flag by flag. The build reads the Dockerfile and stacks the layers. The flags name the image, pick the file, and pass values in.

  • -t names the image name:tag. Without it the image gets no name and no tag, and I cannot run it or push it by name.
  • . the build context: the folder Docker may copy files from. It is a path, not a flag, and it is required. A stray 2 GB file in that folder gets sent to the daemon on every build.

Flags you will meet next: --no-cache rebuilds every layer, and --platform linux/amd64 builds for a different CPU than the one you sit on.

Try it: build the same Dockerfile twice, once with -t app:1.0.0 and once with no -t at all. The first one you can run by name. The second one shows as <none>, and you cannot run it without its ID.

FROM node:20-slim AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm npm ci
COPY . .
RUN npm run build
FROM node:20-slim AS runtime
WORKDIR /app
COPY --from=build /app/dist ./dist
COPY --from=build /app/package.json ./
RUN npm ci --omit=dev
USER node
EXPOSE 3000
CMD ["node", "dist/server.js"]

What the cache mount means: --mount=type=cache,target=/root/.npm keeps the npm cache in a mount that survives between builds. Do this: run the build twice and time the second run. Expected output: the second build is fast, because the dependencies come from the cache instead of the network. The mount never ships, so the image stays small.

What the limits mean: --memory=512m sets the memory limit to 512 MB, and --cpus=1.0 sets the CPU limit to one CPU. Do this: run docker run -d --memory=512m --cpus=1.0 --name limited hello-small:1.0.0, then docker inspect limited --format '{{.HostConfig.Memory}} {{.HostConfig.NanoCpus}}'. Expected output: 536870912 1000000000.

docker inspect --format, flag by flag. inspect prints everything. --format prints one thing. I use the flag when I already know what I am looking for.

  • --format prints one value from the JSON instead of the whole document, using a Go template. Without it inspect prints pages of JSON, and the one field I want is somewhere in the middle.
  • {{.HostConfig.Memory}} {{.HostConfig.NanoCpus}} two paths inside the JSON: the memory cap and the CPU cap in nanocpus. 1000000000 nanocpus is one core. A wrong path prints an empty line, not an error.

Flags you will meet next: {{json .Config.Env}} prints a whole block as JSON, and {{.Id}} prints the image ID alone.

Try it: run docker inspect with no --format and pipe it to wc -l, then run it with --format '{{.State.Health.Status}}'. The first prints hundreds of lines. The second prints one word: healthy, unhealthy, or starting.

docker run --memory and --cpus, flag by flag. Limits are a promise to the other containers on the host. Without them one container can take everything, and the rest starve.

  • --memory=512m caps the container memory at 512 megabytes. Without it the container can take all the host memory. Past the cap the kernel kills the process, and the log says OOMKilled.
  • --cpus=1.0 caps the container CPU share at one core. Without it the container can take every core on the host, and one greedy container starves the rest.
  • -d runs the container in the background and prints its ID, then returns the prompt. Without it the terminal stays attached to the app, and closing the terminal stops the container.
  • --name hello names the container. Without it Docker picks a random name, and every later command has to use the ID or that random name.

Flags you will meet next: --memory-swap caps memory plus swap together, and --cpuset-cpus pins the container to specific cores.

Try it: run a container with --memory=512m, then read the limit back with docker inspect --format '{{.HostConfig.Memory}}'. It should print 536870912, which is 512 megabytes in bytes.

Honest limit, stated plainly: this lab was verified by parsing the Dockerfile and by reading the Docker docs, not by running docker build. There is no Docker daemon in the machine this lab was written on, so no image was actually built here. The commands, the outputs, and the checks are the ones the reader runs on their own machine, where Docker Desktop is running.

At 10x the images, the shape holds and the registry grows. At 100x, the design changes: the builds move into a pipeline, the images get scanned before they push, and the sizes get a budget. The split never changes, though. Build stage thrown away, runtime stage ships.

This lab practices Lesson 145 — Multi-Stage Builds and BuildKit: Small, Fast Images: the two-stage build this lab runs; and Lesson 146 — Docker in Production: Data, Secrets, Limits, and Security: the production limits, healthcheck, and non-root user this lab runs with; and Lesson 142 — Dockerfile Directives Part 2: Run It Right: the HEALTHCHECK, USER, and VOLUME directives in the production run.

One time it broke

What broke

A team shipped a 1.4 GB image to a cluster with a hundred nodes, and every rollout pulled 1.4 GB to every node. The rollout took forty minutes, and the nodes ran out of disk space halfway through, because the old images were never cleaned up. The image held the compiler, the build tools, and the dev dependencies, none of which the app needed to run. The fix was a two-stage build, and the rollout fell to four minutes.

Timeline

  • 09:00,the rollout started. Every node pulled 1.4 GB.
  • 09:20,half the nodes were still pulling. The other half had run out of disk.
  • 09:30,the rollout stopped. The nodes with no disk could not run the new image or the old one.
  • 10:00,an operator cleaned the old images by hand and freed the disk.
  • 11:00,the Dockerfile was split into two stages, and the image fell to 120 MB. The next rollout took four minutes.

What changed after

Two things. First, every image gets a size budget, and a build that grows past it fails the pipeline. Second, the nodes clean old images on a schedule, so a big image cannot fill the disk again. Try it now: measure your image with docker images and write the number down. If it is over 500 MB, split the stages.

If you remember nothing else

Build it yourself

Build a two-stage image for an app you choose, with a cache mount, limits, a healthcheck, and a non-root user. It passes when the image is under 200 MB, the second build is fast, and the container runs as a user with limits on.

Build it, step by step

  • Build the single-stage image first and write down its size.
  • Split it into two stages, build it, and write down the new size.
  • Add a cache mount, build twice, and time the second build.
  • Run it with --memory=512m --cpus=1.0 and inspect the limits.
  • Run docker exec whoami and docker inspect health, and confirm the user is node and the health is healthy.

What success looks like

Success looks like this: the two-stage image is under 200 MB, the second build is fast, the limits inspect to 512 MB and 1 CPU, whoami says node, and the health is healthy. If the image is still over 500 MB, the build tools are still shipping.

If it fails, check this

If the build fails naming a stage, the COPY --from name is wrong. If the second build is slow, the cache mount is missing or BuildKit is off. If whoami says root, the USER line is missing or in the wrong stage. If the health never turns green, the healthcheck command is wrong. If the image is small but the app does not answer, the COPY --from missed the built files.

Next in this series
Lesson 48 — Kubernetes Core