set -euo pipefail: bash scripts that fail safely
Updated · Every command on this page ran in our practice shell
Short answer
Start scripts that change things with set -euo pipefail: -e stops at the first failing command, -u makes unset variables an error, and pipefail makes a pipeline fail if any part fails. Add ${VAR:?} for paths you delete (rm -rf -- "${BUILD_DIR:?}"/*), pass arguments on with "$@", and make temp files with mktemp.
Out of the box, bash keeps going after a command fails, treats a typo'd variable as an empty string, and ignores failures in the middle of a pipeline. That's friendly at the prompt and dangerous in a script that deletes or deploys. A few lines at the top change it. The examples below ran in a practice shell on a bakery's clean-up script.
The bug these settings prevent
This script empties a build folder. If BUILD_DIR was never set, the rm line becomes rm -rf /*.
learner@shellstreak:~/bakery$ cat scripts/clean.sh && echo "rm -rf $BUILD_DIR/*" #!/bin/bash # Empty the build folder before a new build rm -rf $BUILD_DIR/* echo "Cleaned $BUILD_DIR" rm -rf /*
set -u: unset variables are errors
With set -u (nounset), using a variable that was never set stops the script with unbound variable instead of quietly expanding to nothing.
learner@shellstreak:~/bakery$ bash -c 'set -u; echo start; echo "rm -rf $BUILD_DIR/*"; echo done' || echo "stopped early" start stopped early bash: BUILD_DIR: unbound variable
${VAR:?}: stop if empty
set -u doesn't catch a variable that is set but empty. ${BUILD_DIR:?} stops in both cases, and ${BUILD_DIR:?set BUILD_DIR first} prints your message. Together with --, which ends rm's options, the safe line is rm -rf -- "${BUILD_DIR:?}"/*.
learner@shellstreak:~/bakery$ BUILD_DIR=build bash -c 'rm -rf -- "${BUILD_DIR:?}"/* && ls build && echo emptied' emptied
set -e: stop on the first error
With set -e (errexit) the script exits as soon as a command fails, instead of carrying on with half its work done. It has exceptions: failures inside if tests and on the left of && or || don't count, because those are expected to fail sometimes.
set -o pipefail: failures inside pipelines count
A pipeline's exit code is normally the last command's, so false | true succeeds. With pipefail it fails if any part fails, so a broken grep or download in the middle of a pipeline is noticed (and, with set -e, stops the script).
learner@shellstreak:~/bakery$ false | true; echo "without: $?"; set -o pipefail; false | true; echo "with: $?" without: 0 with: 1
Pass arguments on with "$@"
"$@" expands to every argument the script got, each as its own word, spaces intact. Unquoted $@ splits march 01.txt in two, and "$*" glues everything into one string. Put -- before them when they're file names, so a file called -r can't become a flag: cp -- "$@" backups/.
learner@shellstreak:~/bakery$ set -- "march 01.txt" notes.txt; for a in "$@"; do echo "[$a]"; done; for a in $@; do echo "<$a>"; done [march 01.txt] [notes.txt] <march> <01.txt> <notes.txt>
Temp files: mktemp and trap
tmp=$(mktemp) creates a new file with a unique name and private permissions (600). trap 'rm -f "$tmp"' EXIT deletes it when the script ends for any reason, errors included. (The practice shell doesn't support trap.) Never write sort file > file: the shell empties the file before sort reads it. Sort into the temp file, then move it into place.
learner@shellstreak:~/bakery$ tmp=$(mktemp) && sort data/names.txt > "$tmp" && chmod 644 "$tmp" && mv "$tmp" data/names.txt && cat data/names.txt Ana Ben Chloe Dev
Check yourself
One real exercise from the “Shell scripting” unit. No account needed.Use bash -c to run echo hi in a fresh bash.
What's checked: the output of your last command.
Common mistakes
Expecting
set -eto catch a failure inside aniftest or on the left of&&or||. Those are exempt, so check them yourself.Relying on
set -ualone forrmpaths. An empty variable passes; use${VAR:?}too.Forgetting that a file made by
mktempis mode 600. Moving it over a shared file makes that file private;chmodit first.Writing
$@or$*without quotes when passing arguments on.
Questions
Should every bash script start with set -euo pipefail?+
It's a good default for scripts that change things. Know the exceptions of `-e` (commands in `if`, `&&` and `||` don't trigger it), and run shellcheck, which catches most quoting mistakes.
What's the difference between $@ and $*?+
Quoted, `"$@"` gives each argument as a separate word and `"$*"` joins them into one string. Unquoted, both split on spaces. Use `"$@"` to pass arguments on.
Does set -e work in sh as well as bash?+
`set -e` and `set -u` are POSIX and work in `sh`. `set -o pipefail` is newer: bash, zsh and ksh have long had it and it's now in POSIX, but some older `sh` implementations reject it, so use `#!/bin/bash` when you rely on it.
Reading is step one. Now make it stick.
Practise this in a five-minute lesson with instant feedback. No account needed.