Platform Engineering · Lesson 151

Linux Lab: Write Your First Bash Scripts

Lesson 72 taught you variables, loops, and functions. This lab is where you write the scripts and run them. You write a hello script, a backup script that refuses to run without its arguments, and you check every script with bash -n before it runs. By the end you have a backup script that works, and you know why set -euo pipefail is the first line of every script you will write from now on.

Difficulty: Beginner · Know this first: Lesson 72 — Bash Scripting; Lab 150

Run it as:

When it breaks, I check

One thing to remember

Where the time goes

Every hop charges rent in milliseconds. The bar below is a worked example for this lesson's scenario — not a measurement from a live system — but the shape is honest: on a cache hit the trip is almost free, on a miss the kitchen eats the budget. Step through the lesson or switch the scenario and watch the bar change.

Extra diagram · from scratch to master

The script lab, mapped: hello, check, loop, backup

Start with one file to back up. Add one box at a time until diff proves the copy matches.

7 boxes on the trip3 scripts1 refused call
Lab folder
0 min
Hello
1 min
bash -n
1 min
Variables
1 min
Loop
2 min
Backup
2 min
Refused
trip ends
Request / flowResponse · the way backFailure / dead end

Try it: change the inputs and watch the trip bend

The budget bar above shows three fixed stories. This simulator lets you write your own. Move a slider and watch the loop, the backup, and the total change in the budget bar, in the hop timings, and in the total. The numbers are a teaching example, not live measurements. The shape is the lesson: more folders means more backup runs, and a refused call costs a retry.

1

How many folders the loop backs up. Each folder is one run of the same script, so the backup hop grows with the count.

1

How many files each folder holds. More files means a longer copy, and a longer diff to prove it.

0

How many times the script is called with no arguments before someone reads the usage line. Each retry costs about 2 min.

Try this: (1) Set folders to 1, files to 1, retries to 0. The loop is about 1 min, the backup about 2 min, the proof about 1 min, and the total is about 9 min. (2) Set all three to the worst end: 10 folders, 100 files, 3 retries. The loop is about 10 min, the backup about 25 min, the proof about 7 min, and the total is about 47 min. Same script. Only the inputs changed.

Check yourself

Three questions. No score is saved anywhere — this is just you against the material.

Try it: edit the pipeline and check it

Here is the backup script from this lab, ready to edit. Change it, then run the check. The check looks for the four things a working backup script needs: a shebang, set -euo pipefail, an argument guard, and a copy command.

Try breaking it on purpose: delete the set -euo pipefail line, or remove the :? guard. Run the check and watch it fail. Then press Reset and try a change that keeps it passing, like adding an echo before the copy.

Press "Run the check" to see whether this pipeline does what the lesson asks.

Expected output when it passes: The script has a shebang, set -euo pipefail, a guarded argument, and a quoted copy. bash -n would be silent, and the backup would run.

Basic to master

I take the same scripts four times here. First I greet. Then I check. Then I loop. Last I back up and prove it.

Basic — A script that greets

Flags in the next block, letter by letter. I read every flag out loud before I run a command I copied. A flag I cannot explain is a flag I do not run yet.

  • #!/bin/bash is the shebang line. It names the interpreter that runs the file. Without it, the shell guesses, and the guess is not always bash. I put it on the first line of every script, with nothing before it.
  • chmod +x first.sh makes the script executable. The +x adds the execute bit. Without it, running the script by name fails with permission denied.
  • ./first.sh lab runs the script with one argument. The ./ means run the file in this directory, because the current directory is not on the command search path. The lab after it is the first argument, and the script reads it as $1.
  • $0 is the script name. $1 is the first argument. $# is how many arguments there were. None of these are flags. They are the shell variables every script reads.
  • ${1:-world} reads the first argument and falls back to world when it is missing or empty. The :- means use the default when the value is empty. Without it, a missing argument would print an empty greeting.
  • echo "Hello, $name" prints the greeting. No flags. The double quotes let the variable expand and keep the words together.

Check it: run ./first.sh lab and see Hello, lab. Then run ./first.sh with no argument and see Hello, world, because the default kicked in.

Definition: A script is a text file with a shebang line, #!/bin/bash, that names the interpreter. chmod +x makes it executable. ./ runs it. $1 is the first argument, $0 is the script name, $# is the count.

Types:

  • The shebang, which says which shell runs the file.
  • chmod +x, which makes the file executable.
  • $1 and $#, which carry the arguments in.
  • echo, which prints the result.

Uses: I use this for any task I run twice: greet, print, count. Try it now: run ./first.sh lab and check it prints Hello, lab.

Dimensions: At low volume everything is instant. What breaks first is the permission: a script that is not executable gives permission denied, not file not found.

Intermediate — Check before you run

Flags in the next block, letter by letter. I read every flag out loud before I run a command I copied. A flag I cannot explain is a flag I do not run yet.

  • bash -n first.sh checks the syntax without running anything. The -n means no execution, parse only. Silence means the syntax is clean. An error names the line. I run it before every run of a script I just edited.
  • set -e stops the script at the first command that fails. The -e means exit on error. Without it, a script keeps running after a failure and the later steps run on a broken state.
  • set -u stops the script when it uses a variable that was never set. The -u means unset variables are an error. Without it, a typo in a variable name silently becomes an empty string.
  • set -o pipefail makes a pipeline fail when any part of it fails. Without it, a pipeline reports only the exit code of its last command, so false | true exits 0 and the failure hides. I write set -euo pipefail as one line at the top of scripts I trust.
  • bash -x first.sh prints each command before it runs it, with a + in front and the variables already filled in. The -x means trace. I run it when a script does the wrong thing and I need to see which line did it.

Check it: break a quote in first.sh, run bash -n first.sh, and read the line it names. Then fix it, run bash -n again, and see silence.

Definition: bash -n checks the syntax without running anything. set -e stops on a failed command. set -u turns an empty variable into an error. set -o pipefail makes a failed command inside a pipe fail the whole pipe.

Types:

  • bash -n, which checks syntax and prints nothing when it passes.
  • set -e, which stops the script when a command fails.
  • set -u, which refuses empty variables.
  • set -o pipefail, which catches failures inside pipes.

Uses: I use bash -n before every run, and set -euo pipefail at the top of every script I write. Try it now: break a quote in first.sh, run bash -n, and watch it name the line.

Dimensions: As the script grows, the checks stay cheap. What breaks is a script written without them: it fails halfway, and the half that ran cannot be undone.

Advanced — Loops and functions

Flags in the next block, letter by letter. I read every flag out loud before I run a command I copied. A flag I cannot explain is a flag I do not run yet.

  • for d in srcdata srcdata2 srcdata3; do echo "backing up $d"; done runs the block once per item. The list after in is the items. The $d inside is the current item. Without the list, the loop would run once per argument the script received, which is a different list and a confusing bug.
  • local backup="..." makes a variable that lives only inside the function. The local keyword means function scope. Without it, the variable would leak into the rest of the script and could clobber a variable with the same name outside.
  • $1 inside a function is the function's own first argument, not the script's. The function has its own arguments, and they shadow the script's while the function runs.
  • cp -r "$src"/. "$dest"/ copies a whole directory. The -r means recursive, copy the directory and everything inside it. Without -r, cp refuses to copy a directory. The /. after the source means copy the contents, not the directory itself, so the files land directly in the destination.

Check it: write the three folder loop and run it. It should print three lines, one per folder. Then change the list to one folder and run it again. It should print one line.

Definition: A for loop runs a block once per item in a list. A function is a named block that runs only when it is called, and its arguments arrive as $1 and $2 inside it. A loop over folders with a function inside it is how one script does many jobs.

Types:

  • for d in list, which runs the block once per item.
  • A function, which names a repeated block.
  • $1 inside a function, which is the function's own first argument.
  • local, which keeps a function's variables inside the function.

Uses: I use loops when the same job repeats, and functions when the same block repeats. Try it now: write the three-folder loop from step 5 and check it prints three lines.

Dimensions: At ten times the folders, the loop still works, and the time grows with the count. What breaks is an unquoted variable inside the loop: one folder name with a space becomes two wrong folders.

Master — A backup you can trust

Flags in the next block, letter by letter. I read every flag out loud before I run a command I copied. A flag I cannot explain is a flag I do not run yet.

  • ${1:?usage: nightly-backup.sh SOURCE DEST} reads the first argument and stops the script when it is missing. The :? means error out with this message when the value is empty or unset. Without it, a missing argument would silently become an empty string and the script would back up nothing.
  • mkdir -p "$dest" makes the backup directory and its parents. The -p means make the parents too, and do not complain if it already exists. The quotes around the name matter, because a path with a space in it would split into two arguments without them.
  • diff -r data restored/data compares two directories file by file. The -r means recursive, walk into subdirectories. Without -r, diff can only compare two files. Silence from diff means the two trees match, and that silence is the proof the backup is good.
  • echo "BACKUP OK: $src -> $dest" prints the result. No flags. The script prints it only after the copy, so the message means the copy finished.

Check it: run the backup, then run diff -r on the source and the copy. Diff should print nothing. Then change one file in the copy, run diff again, and watch it catch the change.

Definition: At this level a script is not a shortcut. It is a promise. It guards its arguments, it refuses bad calls, it copies, and it proves the copy with diff. A backup you have never checked is a hope, not a backup.

Types:

  • An argument guard, which refuses a call with nothing to back up.
  • A copy that prints BACKUP OK when it finishes.
  • diff -r, which proves the copy matches the source.
  • A rerun, which proves the script works twice, not once.

Uses: I use this for any script that touches real files: backups, deploys, cleanups. Try it now: run the backup, run diff, change one file, and watch diff catch it.

Dimensions: At a hundred times the folders, the script is the same shape, and the proof is the same. What breaks is trust: a backup nobody has restored is a guess with a log line.

Go deeper

Flags in the next block, letter by letter. I read every flag out loud before I run a command I copied. A flag I cannot explain is a flag I do not run yet.

  • chmod +x first.sh makes the script executable. The +x adds the execute bit. Without it, running the script by name fails with permission denied.
  • bash -n first.sh checks the syntax without running anything. The -n means no execution, parse only. Silence means the syntax is clean.
  • ./first.sh lab runs the script with one argument. The ./ means run the file in this directory. The lab after it is the first argument.
  • name="Rise N"; echo "$name" shows why the quotes matter. With the quotes, echo prints Rise N as one value. Without them, echo $name still prints the same two words here, and the difference shows up only when the value is empty or has spaces that the shell would split. I quote every variable expansion in a script, because the unquoted form breaks on the values I least expect.
  • ls backup1 lists the backup. No flags. The answer should be a.txt, the file that was copied.

Check it: save first.sh, run bash -n first.sh for silence, then run ./first.sh lab and see Hello, lab. Then run the backup and ls backup1 to see the copied file.

The hello script, the whole file. Do this: save it as first.sh, run chmod +x first.sh, then ./first.sh lab. Expected output: Hello, lab, then Shell: ./first.sh, then Args: 1.

#!/bin/bash
set -u
name="${1:-world}"
echo "Hello, $name"
echo "Shell: $0"
echo "Args: $#"

The backup script, the whole file. Do this: save it as nightly-backup.sh, run chmod +x nightly-backup.sh, then bash -n nightly-backup.sh. Expected output: silence, which means it parses. Then run ./nightly-backup.sh srcdata backup1 and expect BACKUP OK: srcdata -> backup1.

#!/bin/bash
set -euo pipefail
src="${1:?usage: nightly-backup.sh SOURCE DEST}"
dest="${2:?usage: nightly-backup.sh SOURCE DEST}"
mkdir -p "$dest"
cp -r "$src"/. "$dest"/
echo "BACKUP OK: $src -> $dest"

What the guard means: ${1:?usage} prints the usage text and exits 1 when $1 is missing or empty. Do this: run ./nightly-backup.sh with no arguments. Expected output: the usage line, exit 1, and nothing copied. That is the script protecting you from a backup with no source.

At 10x the folders, the shape holds and the time grows with the count. At 100x, the design changes: the script gets a log file, a lock so two runs cannot overlap, and a retention rule that deletes old backups. The guard and the check never change, though. Every script still starts with set -euo pipefail.

One time it broke

What broke

A cleanup script I owned deleted the wrong folder. It built a path from a variable that was empty, so the path became the parent folder, and rm -rf removed everything inside it. The script had no set -u, so the empty variable was silent. The folder held a week's worth of test data, and there was no backup, because the backup script was the next item on the list.

Timeline

  • 14:00,I ran the cleanup script with a target folder name, and it printed nothing. Silence looked like success.
  • 14:05,I listed the parent folder and found it empty. The target folder was gone, and so was everything next to it.
  • 14:10,I read the script and found the variable was set from $1, but the call had passed no argument, so $1 was empty.
  • 14:15,I checked for a backup. There was none. The test data was rebuilt by hand over the next two days.
  • 14:30,I rewrote the script with set -u and a guard, ${target:?target folder required}, so an empty call exits 1 and deletes nothing.

What changed after

Two things. First, every script I write starts with set -euo pipefail, and every path built from a variable is guarded. Second, I never run a destructive script before the backup for that folder exists. Try it now: run ./nightly-backup.sh with no arguments and confirm it refuses, then check that nothing was copied.

If you remember nothing else

Build it yourself

Build a backup script for three folders that guards its arguments, copies each folder, and proves each copy with diff. It passes when all three backups exist, all three diffs are silent, and a call with no arguments is refused.

Build it, step by step

  • Write nightly-backup.sh from this lesson, check it with bash -n, and make it executable.
  • Make three source folders with one file each, and back all three up with a for loop.
  • Run diff -r on each pair and confirm all three are silent.
  • Call the script with no arguments and confirm it prints usage and exits 1.
  • Change one file in a source folder, rerun diff, and confirm it catches the change, then rerun the backup and confirm diff is silent again.

What success looks like

Success looks like this: three BACKUP OK lines, three silent diffs, and one refused call with exit 1. If diff prints anything, the copy is stale. If the script runs with no arguments, the guard is missing.

If it fails, check this

If bash -n reports an error, read the line number it names. If the script says permission denied, chmod +x is missing. If the backup copies nothing, the source path is wrong. If diff is noisy after a rerun, the source changed after the backup, and the backup needs to run again.

Next in this series
Lesson 152 — Linux Lab: Processes, Permissions, and Disk