Bash Scripting For Automation

This deck introduces the foundations of Bash scripting, a practical skill for anyone who wants to automate repetitive tasks on a Unix-like system. The cards walk you through the essentials, starting with what Bash is and how to prepare and run a script, then moving into variables, parameter expansion, positional parameters, and capturing exit statuses. Together, these topics form the building blocks you need before you can write scripts that do anything meaningful.

The material is well suited to beginners who are new to the command line, as well as developers, sys admins, and curious tinkerers who want a structured way to fill in gaps in their shell knowledge. Because Bash is the default shell on most Linux distributions and is available on macOS, the skills here are widely transferable. Even a basic grasp of these concepts can save you hours of manual work by letting you chain commands together into reliable, repeatable scripts.

To get the most out of the deck, try opening a terminal alongside your study sessions and experimenting with each concept as you review it, since Bash is best learned by doing rather than just reading. Spacing your review over several short sessions will help the syntax sink in more deeply than cramming everything at once. When you finish a batch of cards, challenge yourself to write a tiny script that uses the features you just reviewed, as that small act of recall will reinforce the material far more effectively than passive review alone.

Bash, the Bourne Again SHell, is a Unix shell and command language written as a free software replacement for the original Bourne shell. It serves as the default login shell on most Linux distributions and, before macOS Catalina, on Apple's desktop operating systems as well. Because Bash is installed almost everywhere a Unix-like system runs, it is the natural choice for writing portable automation scripts that glue other tools together.

A Bash script is simply a text file containing commands the shell can interpret. By convention such files carry the .sh extension to signal intent and let editors and tooling recognize them as shell scripts, although the extension is not strictly required for execution. The first line of an executable Bash script must be a shebang — a line such as #!/usr/bin/env bash or #!/bin/bash — which tells the kernel which interpreter to invoke when the file is run directly. #!/usr/bin/env bash is often preferred because it locates Bash through the user's PATH, which keeps the script working even when Bash lives in a non-standard location.

Once the file has the right shebang, it must be made executable before it can be launched as a program. The command chmod +x script.sh (or the equivalent chmod 755 script.sh) sets the execute bit. To run a script in the current directory you invoke it with ./script.sh; the leading ./ is required because the current directory is not normally included in PATH. There is an important distinction between sh script.sh and ./script.sh: the former explicitly invokes the sh interpreter and ignores the shebang, so Bash-only features may break, while the latter honors the shebang and uses the interpreter declared on the first line. To guarantee Bash regardless of how a script is invoked, you can guard at runtime with [[ -n "${BASH_VERSION:-}" ]] || { echo "Need bash"; exit 1; }.

A Bash variable is a named storage location for a value. Variables are assigned with NAME=value — critically, with no spaces around the = sign, because Bash treats VAR = value as a command named VAR with two arguments = and value. Values are referenced as \(NAME or, more explicitly and safely, as \){NAME}. The braced form disambiguates boundaries: \({VAR}suffix correctly means the value of VAR concatenated with a literal suffix, while the unbraced \)VARsuffix is interpreted as a single (and probably non-existent) variable. Without export, a variable is local to the current shell; to make it visible to child processes, use export VAR=value or the two-step VAR=value; export VAR. The declare builtin provides additional type hints: declare -i forces arithmetic on assignment, while declare -r makes a variable read-only and equivalent to readonly.

Bash provides a rich family of parameter expansions that operate on the value of a variable without spawning external commands. \({VAR:-default} substitutes default when VAR is unset or empty, while \){VAR:=default} also assigns the default back to VAR. For required configuration, \({VAR:?message} prints the message to stderr and aborts with status 1 if VAR is unset or empty. Length is queried with \){#var}, substrings with \({var:offset:length} (negative offsets count from the end), and patterns can be removed or replaced: \){var#pattern} strips the shortest prefix, \({var##pattern} the longest, \){var%suffix} the shortest suffix, and \({var%%suffix} the longest. Substitution uses \){var/old/new} (first match), \({var//old/new} (all), \){var/#old/new} (anchored at the start), and \({var/%old/new} (anchored at the end). Bash 4 adds case conversion with \){var^^} (upper) and \({var,,} (lower), with the single-character variants \){var^} and ${var,} capitalizing or decapitalizing the first character.

Quoting is the single most important defensive habit in Bash. Single quotes preserve every character literally, while double quotes preserve everything except \(, backticks, and \. Always double-quote variable expansions ("\)var") to prevent word splitting and globbing, because after unquoted parameter, command, and arithmetic expansions Bash splits the result into words on each character of IFS. The default IFS is space, tab, and newline; many scripts override it to IFS=$'\n\t' to avoid word-splitting surprises on whitespace in filenames. Globbing is the matching of unquoted *, ?, and [...] against filenames; it can be disabled with set -f, and the often-surprising behaviour of empty globs can be changed with shopt -s nullglob (expand to nothing) or shopt -s failglob (treat as an error). Other useful shopt flags include nocaseglob for case-insensitive matching, dotglob for including hidden files, and globstar to enable ** for recursive matching.

For robust scripts, a recommended preamble is set -euo pipefail. set -e causes the shell to exit immediately on any command that returns non-zero, except in contexts where the status is explicitly tested (such as if, while, ||, or &&). set -u treats unset variables as errors when expanded, catching typos and missing configuration. set -o pipefail makes a pipeline's exit status reflect the rightmost command to fail, instead of only the last command's status. The special variable $? holds the exit status of the most recent command (zero is success; non-zero is failure), and Bash reserves several standard codes: 1 for general error, 2 for misuse of a shell builtin, 126 for a command found but not executable, 127 for command not found, and 128 + N for termination by signal N (so 130 is SIGINT, 137 is SIGKILL, and 143 is SIGTERM). For output, prefer printf '%s\n' "$var" over echo, which adds a trailing newline and may interpret backslash escapes inconsistently across shells, and direct error messages to stderr with printf '%s\n' "error" >&2.

Conditional execution in Bash revolves around if, elif, else, and fi. The condition itself is a list of commands whose exit status determines the branch: zero is true and non-zero is false. The three most common test constructs are [ ], [[ ]], and (( )). The single-bracket [ ... ] is the POSIX test builtin: it is portable but performs word splitting on its operands, so variables inside it must be quoted. The double-bracket [[ ... ]] is a Bash keyword that does not word-split its operands and adds Bash-specific features, most notably == glob matching and =~ regular-expression matching. The arithmetic form (( ... )) evaluates an integer expression and returns 0 (true) when the result is non-zero.

File-test operators inside [[ ]] include -e (exists), -f (regular file), -d (directory), -L (symlink), -r, -w, -x (readable, writable, executable), and -s (size greater than zero); they are negated with !. String tests include -z (empty), -n (non-empty), == and != (with the right-hand side optionally a glob), and < and > for lexicographic comparison. For integer comparison, prefer (( a == b )) or the [[ ]] operators -eq, -ne, -lt, -le, -gt, and -ge; do not use the lexicographic < and > operators on integers. A classic pitfall illustrates why quoting matters: if [ \(foo = "bar" ] fails with a syntax error when foo is empty, because word splitting expands it to [ = "bar" ]. The fix is to quote the variable, as in if [ "\)foo" = "bar" ], or to use [[ "$foo" == "bar" ]], which avoids the issue altogether.

Pattern-based multi-way branching uses a case statement, whose right-hand side supports glob patterns and | alternations and which ends with esac. Regex matching lives in [[ ]] as well: [[ "\(s" =~ ^[0-9]+\) ]] tests for an all-digits string, and capture groups after a successful match are available through the \({BASH_REMATCH[1]}, \){BASH_REMATCH[2]}, ... array. To detect which operating system the script is running on, [[ "$(uname)" == "Linux" ]] (or a case over uname -s) gates platform-specific code, and [[ $EUID -eq 0 ]] checks whether the script is running as root.

Bash offers several ways to iterate. The most basic form is for name in list; do body; done, where the list may be a literal set of words, a glob, the output of a command substitution, or an array expansion. Because unquoted globs and command substitutions are subject to word splitting, list iteration over arrays should always be written for item in "${arr[@]}", with both the subscript and the variable quoted, so that elements containing spaces or special characters are preserved correctly. A more verbose variant, the C-style for loop — for (( i=0; i<10; i++ )) — uses arithmetic expression syntax and is convenient for counting loops and parallel iteration.

Conditional loops take the form while condition; do body; done and until condition; do body; done. The condition is re-evaluated after each iteration; a while loop continues as long as the command returns zero, and an until loop continues as long as it returns non-zero — an until is essentially a negated while, handy for retry patterns like until ping -c1 host; do sleep 5; done. Inside any loop, break exits the innermost loop and continue skips to the next iteration; both accept an optional integer level, so break 2 exits two nested loops at once.

Reading a file line by line is a special case of a while loop: while IFS= read -r line; do echo "$line"; done < file.txt. Setting IFS= to an empty string preserves leading and trailing whitespace on each line, and the -r flag prevents read from interpreting backslash escapes. To read fields separated by a delimiter into separate variables, supply a custom IFS: IFS=: read -r user _ uid _ _ home _ < /etc/passwd, where the placeholder _ simply discards fields you do not care about. The same pattern adapted to "\(@" — for arg in "\)@"; do echo "$arg"; done — is the canonical way to iterate over a script's own command-line arguments.

Functions are defined in Bash with name() { commands; } or with the keyword form function name { commands; }. The body of a function executes in the same shell environment as the caller — there is no automatic local scope — so variables assigned inside a function leak into the surrounding scope unless they are declared with local foo=bar, which hides any global of the same name for the duration of the function. Arguments are passed as positional parameters $1, \(2, ..., exactly as in a script, and they are forwarded correctly with "\)@". From Bash 4.3 onward, arrays may be passed by reference using declare -n refname=array. A function exits with return N (where N defaults to the status of the last command), in contrast to exit N, which terminates the entire script — a subtle but important distinction.

Two related mechanisms for modularization are source script.sh (or its POSIX alias . script.sh) and the getopts builtin. source executes a file in the current shell, so any variable assignments or directory changes made by that file persist in the caller — this is why source is the standard way to load a configuration file of Bash variables. There is no separate include mechanism in Bash: modules are sourced, and there is no harm in doing so because source is precisely identical to the dot operator. getopts, by contrast, parses short command-line options such as -a or -b value from a script's own argument list. The typical idiom is while getopts "ab:c" opt; do case \(opt in ...) esac; done, where the option string declares which letters take arguments (those followed by :). The argument to the current option is in \)OPTARG, and $OPTIND points past the consumed options; the usual cleanup is shift $((OPTIND-1)) after the loop, so that "$@" still contains the remaining positional arguments. Before calling any external tool, the robust check for its presence is command -v cmd >/dev/null 2>&1, not the legacy which.

A subtle issue worth knowing is that strict mode does not protect you from a misleading cd. set -e exits when a later command fails for whatever reason, which can hide a failed cd that was supposed to be the cause. The defensive idiom is cd /path || exit 1 or pushd /path || exit to make the directory change itself visible to error handling. The : builtin is a useful no-op that returns 0 and discards its arguments; it is handy as a placeholder in empty function bodies and is the canonical left-hand side of parameter expansion tricks such as : "${VAR:=default}".

Bash supports both indexed arrays and associative arrays (dictionaries). Indexed arrays are declared with arr=(one two three) or by element with arr[0]=one, and individual elements are accessed with \({arr[0]}. All elements are accessed with \){arr[@]} (each as a separate word) or \({arr[*]} (joined into one word), and the element count is \){#arr[@]}. New elements are appended with arr+=(four five). Iteration over indices and values together uses \({!arr[@]} to enumerate the indices: for i in "\){!arr[@]}"; do echo "\(i=\){arr[i]}"; done. Associative arrays require Bash 4 and are declared with declare -A m=([key1]=val1 [key2]=val2); keys come from \({!m[@]} and the length is \){#m[@]}.

The expansions covered in the variables chapter apply naturally to strings held in scalar variables and elements of arrays. Substrings are extracted with ${var:offset:length}, prefixes and suffixes are removed with #, ##, %, and %%, and replacement uses / for the first match, // for all, and the # and % anchors for the start and end. Case conversion uses \({var^^} and \){var,,}. To join an array into a single delimited string, set IFS locally and use "\({arr[*]}", or pipe through printf '%s\n' "\){arr[@]}" | paste -sd , for arbitrary joining.

Arithmetic in Bash is handled by the $(( ... )) expansion, which supports the usual operators: +, -, *, /, %, ** for power, bitwise operators, and the ternary. Inside (( ... )) you can use C-style syntax and assign back to variables, as in ((count++)), and the command form returns 0 when the expression's result is non-zero, making if (( a < b )); then ...; fi a natural integer test. To safely coerce a string to an integer while refusing non-digits, use base-10 notation: n=\(((10#\)s)) errors on trailing garbage instead of silently treating it as octal. All Bash arithmetic is integer-only; for floating-point comparison you must fall back to awk or bc, e.g. awk -v a=1.5 -v b=1.4 'BEGIN{exit !(a>b)}'.

Two substitution forms deserve their own mention. Command substitution — using a command's output as a value — uses $(command), the modern and easily nestable form, rather than the legacy backticks `command` (which nest poorly and behave inconsistently inside double quotes). Process substitution goes further, exposing a command's input or output as a filename referring to a /dev/fd/N pipe: diff <(sort a) <(sort b) is the classic example, replacing temporary files with on-the-fly pipes.

The read builtin is the primary way to ingest input. read -rp "Prompt: " answer reads a single line with a prompt and without backslash interpretation, while while IFS= read -r line; do ...; done < file.txt is the canonical pattern for iterating over a file. The variant read -d DELIM changes the line terminator, which is essential for NUL-delimited input via read -d '' in loops paired with find -print0. To count lines in a file portably, wc -l < file.txt avoids the filename in the output, and grep -c '' file is an alternative that tallies newlines.

Redirection steers streams between commands and files. Stdout is redirected with > (truncate) or >> (append), and stderr with 2> or 2>>. To send stderr to wherever stdout currently points, write 2>&1 — order matters because redirections are processed left to right. To silence both streams, redirect to /dev/null: cmd > /dev/null 2>&1. To capture both into a single variable, out=$(cmd 2>&1) works; to keep them separate, write to temp files: cmd >out 2>err. The exec builtin, when given a redirection but no command, modifies the current shell's file descriptors — exec > file makes all subsequent echo and printf output go to that file until further notice, and is a handy way to redirect a long script's output to a log without sprinkling redirections everywhere.

For multi-line text, Bash provides here-documents and here-strings. A here-document is written cat <<EOF followed by lines and a closing EOF; quoting the delimiter (<<'EOF') disables parameter expansion inside, while an unquoted delimiter allows \(USER and other expansions to be substituted. Here-strings are the single-line equivalent: command <<< "\)variable" feeds the variable's value to the command's standard input without spawning a subshell. To produce filenames safely, use tmp=\((mktemp) for a unique temporary file or tmpd=\)(mktemp -d) for a directory; these avoid the race conditions and predictable-path pitfalls of hand-rolled names in /tmp. To extract the directory and filename parts of a path, basename /a/b/c.sh gives c.sh and dirname /a/b/c.sh gives /a/b. A robust pattern for finding a script's own directory is dir=$(cd "$(dirname "\({BASH_SOURCE[0]}")" && pwd), which prefers \){BASH_SOURCE[0]} over $0 because the latter may be just the program name rather than a real path.

A subshell, written ( commands ) or produced by backgrounding, runs its commands in a child of the current shell. Any changes to variables, the working directory, or shell options do not affect the parent once the subshell exits, which makes subshells ideal for isolating a temporary cd — as in (cd /tmp && cmd) — or for short-lived option changes such as (set -e; cmd). Commands can also be run in the background by appending &; the shell prints a job ID and PID, and synchronization is achieved with wait %1 (a job specifier) or wait $pid (a specific PID). The trap builtin registers code to run when the shell receives a signal or exits, and the most common pattern is trap 'cleanup' EXIT, which fires the cleanup code on any termination path — normal exit, error, or signal, with the lone exception of SIGKILL, which cannot be caught. Pairing trap 'rm -f "\(tmpfile"' EXIT with tmp=\)(mktemp) guarantees reliable cleanup.

Many practical automation tasks involve combining Bash with external commands. find ... -print0 | xargs -0 -n 1 cmd processes filenames safely even when they contain spaces or newlines, and find /path -type f -mtime +7 -delete deletes files older than seven days. The timeout command from coreutils, invoked as timeout 30s cmd, runs a command with a hard deadline and returns non-zero on expiry. To time how long a block takes, the simplest in-script trick is SECONDS=0; cmd; echo "$SECONDS", which gives elapsed seconds in Bash, or use the time builtin for the more detailed real/user/sys breakdown. To schedule periodic execution, crontab -e with an entry like 0 3 * * * /opt/scripts/backup.sh >> /var/log/backup.log 2>&1 runs nightly at 03:00; remember that cron runs scripts with a minimal PATH and environment, so any variables the script depends on must be set explicitly inside the script itself.

To catch real bugs, develop with the linter shellcheck (shellcheck.net) in CI, validate syntax with bash -n script.sh before running, and use set -x (or run as bash -x script.sh) to trace each command just before it executes. Setting PS4='+ \({BASH_SOURCE}:\){LINENO}: ' adds source-file and line-number prefixes to trace output, which is invaluable in long scripts. BASH_ENV is a variable that, when exported, points non-interactive Bash scripts at a startup file to source — useful for ensuring every script on a system runs with the same strict-mode preamble. By default aliases are not expanded in non-interactive scripts; shopt -s expand_aliases turns that on when needed. To log every line of script output with a timestamp, exec > >(while IFS= read -r line; do printf '%s %s\n' "\((date '+%F %T')" "\)line"; done) 2>&1 prepends each line with the current time, or you can pipe everything through ts '[%F %T]' from moreutils.

Finally, a short list of pitfalls that defeat even experienced scripters: never parse ls output, because for f in $(ls) mangles filenames containing spaces or newlines through word splitting and glob expansion — prefer globs (for f in *.txt) or find -print0 with read -d ''; never pass a huge glob directly to a command such as rm *, because Argument list too long errors result — use find ... -exec rm {} + or xargs instead; remember that NUL bytes cannot appear in Bash strings, which is precisely why find -print0 paired with read -d '' exists; and when portability matters, prefer #!/usr/bin/env bash with a runtime guard, avoid Bash-4-only features on older systems, prefer command -v to which, and prefer printf to echo for arbitrary data.