Lesson 49 gave you the idea: describe the infrastructure you want, and let a tool make it real. This lesson gives you the tool most teams reach for first. Terraform reads your files, reads the state file that remembers what it built, compares both against the real cloud, and shows you a plan before it changes anything. One apply creates a network, two servers, and a load balancer. I learned Terraform the way I learn most tools in this job. I deleted a state file once, watched Terraform offer to rebuild a whole environment it had already built, and learned in one afternoon that the state file is the tool. This lesson walks one apply through Terraform, hop by hop, from a first HCL file to the deep end: remote state, workspaces, policy as code, drift detection, and importing infrastructure that already exists.
Every hop charges rent in milliseconds. The bar below is a worked example for one apply on a 40-resource stack, not a measurement from a live system, but the shape is honest: in most slow applies I have chased, one or two hops eat almost the whole budget. Step through the lesson or switch the scenario and watch the bar change. The locked apply costs three times the normal one and changes nothing. Same files, same plan. Only the lock changed, and the bill tripled with nothing to show for it.
I start from scratch: one HCL file describing one resource, one provider, local state on one laptop. Then I add one piece at a time until I reach the full picture: a full stack of network plus servers plus load balancer with dependency arrows, provider plugins, and the shared remote state backend the whole team reads from.
HCL files (network, two servers, load balancer) I start here. This is the simplest piece, and nothing else exists yet.
From scratch to master. Starts from scratch: the simplest wrong trip, a single hand-edited security group caught by refresh and fixed by apply. Builds to the Master-level picture: the lock trip (waiting apply, nothing written, old state preserved) and the split-state incident — two state files, two truths, a duplicated stack — the failure mode the whole lesson's state discipline exists to prevent.
Console edit: security group changed by hand. I start here. This is the simplest piece, and nothing else exists yet.
The budget bar above shows three fixed stories. This simulator lets you write your own. It models one apply on a stack you size yourself, with drift in the mix. Move a slider and watch the plan hop, the apply hop, and the total change in the budget bar, in the hop timings, and in the status line. The numbers are a teaching example, not live measurements. The shape is the lesson: the apply grows with the stack, drift grows the plan and the apply both, and the fixed hops stay fixed no matter how big the stack gets.
Each resource costs 60 ms to plan and 90 ms to apply. At 40 resources the base plan is 2,400 ms and the base apply is 3,600 ms, before drift.
Each drifted resource costs 400 ms extra to plan and 600 ms extra to apply. At 2 drifted that is 800 ms extra on the plan and 1,200 ms extra on the apply.
The state read and the state write each pay the backend latency once, so the fixed hops are 2,250 ms plus two times the backend latency.
Try this: (1) Leave the defaults: 40 resources, 2 drifted, 100 ms backend latency. The total is about 10,450 ms. (2) Push the stack to 200 resources with 20 drifted. The total is about 52,450 ms. Same tool, same files. Only the size changed. (3) Set everything back, then set the backend latency to 500 ms and watch the fixed hops rise by 800 ms. That is what a slow state backend costs on every single run.
Three questions. No score is saved anywhere — this is just you against the material.
Terraform looks like one big tool. It is not. It is four layers stacked on each other, and each layer only makes sense after the one below it. I learned it in this order, and I teach it in this order: write one file and apply it, then split it into modules and stop hardcoding, then protect the state and the plan, then run it for a whole organisation. Skip a layer and the next one reads like magic. Build them in order and nothing in this lesson is magic.
Terraform at the basic level is HCL files, a provider, and the plan and apply loop, with state stored on your own machine. You write a resource block, you run init, you run plan, you read the plan, you run apply.
Types:
Uses: when a team is learning Terraform or building something small for one person. One network, a few servers, a load balancer. The first win is that the infrastructure is written down, and a second person can read the file and know what exists.
Dimensions: Scale: holds up to a few dozen resources and one engineer. Failure modes: the state file on one laptop is a single point of failure, and deleting it means Terraform forgets everything it built; a teammate who applies from their own laptop with their own state file builds a second copy of the same infrastructure. Cost: learning time; Terraform itself is free, the cloud bill is whatever the resources cost. Trade-offs: local state is simple and it is also a trap. It works until a second person joins, and then it breaks silently, by building duplicates.
Reuse. A module is a folder of HCL files that builds one thing, like a network or a database, with inputs in and outputs out. You call the module instead of copying the blocks. Variables get types, defaults, and validation. Workspaces let one set of files build several copies of the same stack, one per workspace, usually one per environment.
Types:
Uses: when the same pattern shows up twice. Two environments that need the same network. Ten services that need the same database shape. A module turns a copy-paste into a call, and a variable turns a hardcoded value into an input.
Dimensions: Scale: holds to a few hundred resources and a small team; the plan slows down as the resource count grows, because the refresh is one API call per resource. Failure modes: a module change ripples into every caller at once, so an unversioned module edit is a change to every stack that uses it; a count over a list that changes order will destroy and recreate resources, because Terraform tracks count instances by position: remove the middle item of ["a", "b", "c"] and res[1] now holds "c", so Terraform destroys and recreates it and drops res[2]; a for_each over the same list is safe under reorder, because Terraform tracks each instance by a stable key: remove "b" and only res["b"] is destroyed, while res["a"] and res["c"] are untouched; a teammate who forgets which workspace they are in applies to the wrong environment. Cost: review time; modules need owners and versions, or they become shared code nobody maintains. Trade-offs: modules buy reuse and cost a versioning problem. Workspaces buy cheap environments and cost a footgun: one set of files, many states, and one wrong workspace switch away from applying to prod.
Protection. Remote state lives in a shared backend with locking, so two applies cannot write the same state. Policy as code checks every plan against written rules before it may apply. Drift detection compares the real cloud with the state on a schedule and reports the difference. Import brings infrastructure that already exists into Terraform's state.
Types:
Uses: when a team shares one Terraform project and the stakes are real. A state backend with a lock stops two applies from corrupting the state. A policy check stops the public bucket before it exists. Drift detection finds the console edit on Tuesday instead of at the next incident. Import is how a team adopts Terraform without rebuilding a live environment.
Dimensions: Scale: holds to thousands of resources, but the state file grows with them, and a single state file for a whole company becomes a bottleneck. Failure modes: a stale lock from a crashed apply blocks every later apply until someone releases it; a policy that is too strict blocks good plans, and a team that learns to skip the check has a check that does nothing; an import that misses a setting will show a diff on the next plan, and applying that diff changes the live resource. Cost: operational; the backend needs its own backup, the policies need owners, and the drift reports need someone to read them. Trade-offs: remote state buys safety and costs a backend to run and back up. Policy buys prevention and costs false positives that slow the team down. Drift detection buys visibility and costs a report nobody reads unless someone owns it.
The lock that never clears. An apply that is killed mid-run leaves its lock behind. The next plan fails with a lock error, and the error names a lock ID. The lock is doing its job. The mistake is clearing it without checking whether a run is still live.
Do this: start a plan, stop it with Ctrl-C, then run terraform plan again. Expected: the plan fails and the error prints a lock ID and the name of the run that holds it. Check no run is live, then clear it with the command in the snippet, using the ID from the error. Run the plan again and expect it to pass.
terraform force-unlock -force LOCK_ID
What the snippet means: force-unlock removes the lock record only. It never touches the cloud. Clear it while a run is live and two applies write the same state.
Read the flags: terraform force-unlock -force LOCK_ID
A lock that outlives its run blocks every later plan and apply. Force-unlock clears the lock record. It never touches the cloud.
force-unlock: This removes the lock from the state backend. It is surgery, not a normal step. I check no run is live before I run it, because two applies writing one state corrupt it.-force: This skips the yes or no prompt. Without it, Terraform asks first. In a script the prompt would hang forever, so -force is the only way the command works there.LOCK_ID: This is the lock ID from the error message, not a name I invent. The wrong ID clears nothing, and the error tells me the right one.Try it: start a plan, stop it with Ctrl-C, run terraform plan again, copy the lock ID from the error, and run force-unlock with it. Expect the next plan to pass.
A change that forces a replacement. Some arguments can change in place. Some cannot. Change the machine image on a running server and Terraform cannot edit it in place, so it destroys the server and builds a new one. The plan says so in plain words, and I read that line before I apply.
Do this: in a scratch config, change the image ID on a server resource and run terraform plan. Expected: the plan says the resource must be replaced, and it names the argument that forces it. Add the lifecycle block from the snippet to stop a replacement you did not mean, run the plan again, and expect an error instead of a destroy.
resource "aws_instance" "app" {
ami = "ami-0123456789abcdef0"
instance_type = "t3.micro"
lifecycle {
prevent_destroy = true
}
}
What the snippet means: prevent_destroy turns an accidental replacement into an error. I use it on databases and on anything I cannot rebuild from a backup in minutes.
Terraform as a platform. The state is split into many small state files, one per team or per layer, and stacks read each other's outputs through remote state data sources. Modules are versioned, tested, and published in a private registry. The plan and apply run in a pipeline, never on a laptop, with approvals and audit logs.
Types:
Uses: when Terraform runs a whole organisation. A platform team owns the modules and the policies, and product teams call them. The pipeline is the only place applies happen, so every change has a plan, an approval, and a log. Split state means one team's mistake cannot lock the whole company's infrastructure.
Dimensions: Scale: holds at tens of thousands of resources, because the state is split and the plan runs per stack, not per company. Failure modes: a split state means a change that spans two stacks needs two applies in the right order, and the order is a human decision; a module version pinned everywhere means a security fix needs a version bump in every caller; a pipeline that runs applies is a single point of failure for all infrastructure change. Cost: a platform team; modules, policies, registries, and pipelines all need owners, and the owners are people. Trade-offs: split state buys isolation and costs coordination. A pipeline buys auditability and costs a queue. Versioned modules buy safety and cost upgrade work. The master layer is not more Terraform. It is Terraform with the human problems handled on purpose instead of by accident.
One stack reads another stack. Split state means one stack cannot see another stack by accident. It reads the other stack through a remote state data source, and it reads only the outputs the other stack chose to publish. That boundary is the whole point of the split.
Do this: add the data source from the snippet to a second stack, pointing at the first stack, and run terraform plan. Expected: the plan shows the value from the first stack, and nothing else from it. Change an output in the first stack and the second stack sees the new value on its next plan.
data "terraform_remote_state" "network" {
backend = "s3"
config = {
bucket = "platform-tf-state"
key = "network/terraform.tfstate"
region = "us-east-1"
}
}
output "vpc_id" {
value = data.terraform_remote_state.network.outputs.vpc_id
}
What the snippet means: the data source is a read-only window into one state file. The second stack depends on the first stack, and the apply order is now a decision, not an accident.
The module bump, one caller first. A module version pinned everywhere means a security fix needs a bump in every caller. I bump one caller first, run its plan, and read the diff. If the diff is clean, I roll the bump out. If it is not, one caller is broken, not forty.
Do this: change the version on one caller module block, run terraform init and terraform plan in that stack only. Expected: the plan shows only the changes from the new module version, and the other callers are untouched. A bump that changes a resource name is a major version, and it gets a second review.
Read the flags: terraform init
Init installs what the config needs. Providers, modules, and the backend all come from init. Plan means nothing before init has run.
terraform init (no flags): With no flags, init downloads providers and modules and sets up the backend. I rerun it after any provider, module, or backend change.-upgrade (added): The lesson did not show this one yet. It installs the newest provider versions allowed by the constraints and rewrites the lock file. Without it, init keeps the locked versions.-reconfigure (added): The lesson did not show this one yet. It points init at a new backend without copying state. I use it when the backend moved and the old state stays where it is.Try it: change the version on one caller module block, run terraform init and terraform plan in that stack only. Expect the plan to show changes from the new module version and nothing from the other callers.
module "network" {
source = "registry.internal/platform/network/aws"
version = "2.5.0"
cidr_block = "10.20.0.0/16"
}
What the snippet means: the version pins the module the way a lock file pins a library. One caller proves the bump before the rest of the organisation takes it.
The lesson above is one apply through nine hops. Production is the rest of it. Here is what I actually spend my time on once the plan is read and the budget is written down.
Remote state is not just a file in a bucket. It is a file in a bucket with three properties, and all three matter. First, versioning is on, so a corrupted state can be rolled back to the last good version. Second, encryption is on, because the state file holds secrets in plain text, and a state file in a public bucket is a credential leak. Third, locking is on, so two applies cannot write the same state at the same time. I have seen a team skip the locking table because the bucket alone felt like enough. It was not. Two applies ran at once, the state file was written twice, and the next plan wanted to create resources that already existed. The bucket is storage. The lock is safety. You need both.
Workspaces and folders both give you several environments, and teams argue about which one to use. A workspace is one set of files and several states, one per workspace. A folder is one set of files per environment, copied or shared through modules. I pick folders when the environments are genuinely different, because a dev network and a prod network usually differ in more than one variable, and a folder makes the difference visible. I pick workspaces when the environments are the same shape with different values, and only then. The mistake is using workspaces for environments that drift apart. Once dev and prod need different resource blocks, the workspace has one set of files pretending to be two, and every change has to be safe for both. Split the files before that happens.
A policy check only works if the team cannot skip it and does not want to. The first part is plumbing: the check runs in the pipeline, and the pipeline blocks the apply on failure. The second part is harder. A rule that fails good plans teaches the team to route around it. I write rules narrow and specific: no public buckets, encryption on for databases, tags present. I do not write rules about taste. And I give the rule an owner and a way to request an exception, because a team with no exception path will build one in secret. A policy with no owner rots. A policy with no exception path gets bypassed.
Drift is not a Terraform problem. It is a people problem. Someone fixes a security group in the console during an incident, the incident ends, and nobody writes the fix back into the files. The next plan wants to undo the fix, and the team learns to fear the plan. Drift detection is a scheduled plan that reports instead of applying, usually once a day. The report is not the fix. The fix is a rule: every console change gets written back into the files within a day, or it gets reverted. I have seen teams run drift detection for a year and never act on the report. The report is a mirror. It only helps if someone looks in it.
At 10x the resources, the plan gets slow, because the refresh is one API call per resource and the state file is big. I split the state by layer: network, platform, and apps, each with its own state file. At 100x, the design changes. The modules are versioned and published, the applies run only in the pipeline, and the plan output is stored and audited. The state files are many and small, and one stack reads another stack's outputs instead of sharing a state file. And the drift report gets a budget line of its own, because at that scale the drift is not one console edit. It is a hundred small ones, and a hundred small ones is a second source of truth.
terraform {
required_version = ">= 1.5"
backend "s3" {
bucket = "platform-tf-state"
key = "network/terraform.tfstate"
region = "us-east-1"
encrypt = true
dynamodb_table = "platform-tf-locks"
}
}
module "network" {
source = "registry.internal/platform/network/aws"
version = "2.4.0"
cidr_block = "10.20.0.0/16"
environment = "prod"
}
import {
to = aws_security_group.app
id = "sg-0a1b2c3d4e5f6a7b8"
}
Three blocks, three jobs. The backend block says where the state lives and what holds the lock: a versioned, encrypted bucket and a lock table. Without the lock table, two applies can write the same state. The module block calls a versioned network module, and the version pin is the safety: an unpinned module changes under you when the registry updates. The import block brings an existing security group into the state, so Terraform manages what a human built by hand instead of trying to create a second one. Read them together and you can see the whole posture: state is shared and locked, reuse is versioned, and adoption does not mean rebuilding.
Import brings a hand-built resource under Terraform control. The import itself is the easy part. The proof is the plan after it. A clean import shows an empty plan. A diff after an import means the config is missing a setting the real resource already has.
Do this: write an import block for a resource that already exists, run terraform plan, and read the diff. Expected: the first plan after the import shows no changes. If it shows changes, add the missing settings to the config one at a time until the plan is empty. Only then is the resource really imported.
Read the flags: terraform plan after an import block
An import block adopts a live resource into Terraform. The first plan after the import should show no changes. Any diff means the config does not match reality yet.
terraform plan (after import): The plan compares the imported resource against the config. An empty plan means the adoption worked. A diff names the settings I still have to add to the config.terraform import (added): The lesson did not show this one yet. This is the older command form of the same job, run as terraform import aws_instance.app i-0123456789abcdef0. The import block is the newer form and lives in the code, so I prefer it.terraform show (added): The lesson did not show this one yet. It prints the imported resource from the state. I run it to see the real settings before I write the config to match them.Try it: write an import block for a resource that already exists, run terraform plan, and read the diff. Expect no changes on the first plan. Add missing settings one at a time until the plan is empty.
import {
to = aws_instance.app
id = "i-0123456789abcdef0"
}
resource "aws_instance" "app" {
ami = "ami-0123456789abcdef0"
instance_type = "t3.micro"
}
What the snippet means: the import block tells Terraform which real object the resource block owns. The empty plan is the test that the config describes the object truthfully.
This one stayed with me because two state files told two different stories, and both of them were right about their own world. The cloud had one truth. Terraform had two.
A team ran Terraform from two places. The pipeline used remote state in a shared backend. One engineer, debugging a production problem late at night, ran apply from his laptop with local state, because that was the setup he had used when the project started. His local state file was months old. Terraform read it, saw none of the resources the pipeline had built, and offered to create them all again. He approved the plan without reading the destroy count, because the plan said create, not destroy, and create felt safe. By morning there were two load balancers, two sets of servers, and a second network with the same name. Traffic still flowed, because the old resources were untouched, but the bill doubled and the new servers answered health checks nobody was watching. The worst part was the cleanup: deleting the duplicates meant working out which copy was the real one, resource by resource, by hand.
Three things. First, applies run only in the pipeline now. No laptop applies, no exceptions, and the pipeline is the only place with credentials that can write infrastructure. Second, the backend block is in the shared repo and the local backend config is deleted, so a fresh checkout cannot accidentally use local state. Third, every plan is stored and the destroy count is alerted on: any plan with a destroy above zero needs a second approver. The state file was never the bug in this incident. The bug was two sources of truth, and the fix was to have one.
This is the kind of question I would ask in an interview, and the kind I ask myself before I sign off on a real Terraform setup. Take a minute with it before you read my approach.
Design the Terraform setup for a platform with 3 environments (dev, staging, prod), 200 resources per environment, and 4 teams sharing the code. The network is shared, the apps are per team. Applies must never run from a laptop. No public storage buckets, ever. The latency budget is not the question here: the budget is blast radius. A bad apply must not take down more than one environment. Sketch the state layout, the module layout, the pipeline, and the policy checks, and say what you would measure.
Start with the budget from this lesson. One apply on 40 resources takes about 10,450 ms at the simulator defaults. At 200 resources with no drift it is about 2,250 ms fixed plus 12,000 ms of plan plus 18,000 ms of apply, which is about 32,250 ms. That is one stack. Three environments in one state file would be one plan of 600 resources, about 90,000 ms, and one bad apply could touch all three. So I split the state three ways, one per environment, and the network gets its own fourth state file, because the network is shared and the apps depend on it. The blast radius of one apply is one environment. The pipeline runs one stack at a time, plan first, approval second, apply third. At 4 teams and 3 environments, that is 12 stacks plus the network, and the module registry holds one versioned network module and one versioned app module per team. At 12 stacks with one apply each per day, the pipeline handles 12 applies a day, and a 32-second apply means the pipeline is busy for about 6 minutes a day. The queue is never the problem. The approval wait is, so approvals are per environment, not per team.
Split state buys isolation and costs coordination: a change that spans the network and an app needs two applies in the right order, and the order is a human decision. A pipeline-only apply buys auditability and costs a queue: every change waits its turn, and a down pipeline means no infrastructure change at all. Versioned modules buy safety and cost upgrade work: a security fix needs a version bump in every caller. Workspaces buy cheap environments and cost a footgun, so I use folders and modules instead when the environments differ in shape. And the target is blast radius, not speed. An apply that takes 30 seconds and can only hurt one environment beats an apply that takes 10 seconds and can hurt all three.
Saved in this browser, so the Lessons index can show your progress across the course.