Recently, I refactored an entire deployment setup for a diverse array of apps. That means a big pile of Docker images that all have to be flexible but stable. Some web applications also needed to be restarted in a user-friendly way that displays helpful error messages to developers with a range of skill levels.
It was a lot of work but I sure did get better at Bash scripting. I’m in a good position to write this article because during this project, I wrote down every weird little thing that cost me debugging time.
I could probably write a book that goes into more detail. Instead, I’ve stuffed this post with links.
Your Grandma’s Programming Language
https://en.wikipedia.org/wiki/Bash_(Unix_shell) <– a high quality wikipedia page
Bash was created by Brian Fox (this guy’s an underrated legend) and released in 1989 as an open-source replacement for the Bourne Shell which came out in 1976. Its name is an acronym for Bourne Again SHell.
If you’re used to writing pretty much any other programming language, Bash (and shell scripting in general) can be very un-intuitive. The syntax is unmemorable, variables are weird, scoping is a wild ride, and control flow never seems to do what you think it does.
Much like with CSS, I stopped dreading having to write Bash scripts when I learned a few key things about it: how it works, what it’s really good at, and how to make it behave. I also ran into a lot of stupid little gotchas that I simply had to learn.
Shell scripting is super fun once you master the basics! Nothing makes you feel more like a master hacker than writing a sick one-liner that runs first try.
Note: I’m assuming some prior knowledge of programming and shell scripting. If that’s not you, here is a good resource to start. I assume you at least know how to use a terminal and the following commands: ls, cd, pwd, cat, grep, and have written (or tried to write) a script or two.
By the way, since this goes into the world of Linux and operating systems stuff, I have a note for those of you who live here: it’s OK (even encouraged!) to correct me if I’m wrong, just please be polite. I’m sure I’ll learn some things just from publishing this: that’s the goal.
Versions
The shell scripting language most of us write nowadays is the version of Bash Mac and Linux use for terminal emulators in /bin/bash.
Debian (and by extension, Ubuntu and Linux Mint) now uses a different but mostly compatible shell scripting language (Dash) for the system to use. You may also have Zsh as your primary shell on newer Macs which is similar but different.
Because of all these little variations, it’s a good idea to put #!/bin/bash(or whichever specific shell scripting language you want to use) at the top of files to specify that the shell script should use that and not anything else on the machine.
This will make it a bit more predictable. Stack Overflow answers, for instance, will usually assume you’re using this version.
The Basics
First we’ll go over some fundamentals.
At its core, shell scripting is text streams that pass data between each other. It uses the Unix philosophy of doing one thing well and chaining tiny tools into larger programs — like little machines in a factory line. Each tool outputs to the next in sequence.
Basics of the Unix Philosophy is worth a read.
Syntax
Syntax is relaxed like in a language where syntax isn’t strict: you can use semicolons at the ends of lines if you want and indentation isn’t a big deal.
This is a trap. Syntax matters a lot and is very specific. Additionally, Bash syntax errors are the worst.
It’s especially important to correctly use spaces and semicolons.
These can cause confusing errors like "[grep isn’t a valid command" when you forget the space in one of these [] or “Unexpected end of file” when you forget a semicolon in one of these {}.
When declaring a variable, putting a space between the variable, equal sign, and value will mean something completely different. There is a very important difference between single and double quotes.
Syntax errors always come out sounding like logic errors which makes it even harder to catch typos.
Structure
Shell scripting has control flow: if statements, while loops, for loops, case statements, etc.
What’s different is in the conditions and scoping. However, since it is more geared towards single lines and one-off scripts, these aren’t used as often as they are in other languages.
Here’s an example of a one-liner that uses control flow without any if statements:
tac ~/error.log \
| grep -m1 -E "Error|Running restart" \
| grep -q "Error" \
&& echo "Found error since last restart"
(The \ are line continuations, tac is like cat but outputs the file backward.)
It’s ugly but effective and illustrates the strengths and weaknesses of shell scripts.
Bash has the potential to be very brief and hard to read. You can do a lot in a few lines, but when things break, it can be hard to figure out why. It’s a blessing and a curse. With great power comes great potential to screw everything up.
What is a stream? What is a command?
Each command is a program that does one thing. Grep, for example, searches for things and returns the line. A query goes in, file goes in, lines come out.
You may be thinking: “Yeah, obviously, that’s how all programming works,” but it’s a little bit more complicated than that and especially important to understand here.
These inputs and outputs are bounced from command to command in the form of text streams. There are three places these streams go and come from:
stdin: Standard input.stdout: Standard output.stderr: Standard error output.
It’s a “stream” because lines are output at different points in the command/function execution, instead of all at the end like you might think.
You send text to stdout using commands like printf and echo. An inexperienced shell scripter might think of these as simply logging tools for debugging like with Python or JavaScript. Not so.
Streams enable commands and functions to be strung together into an assembly line of code. A good way to illustrate this is by explaining how functions work.
Functions
I define a function like this:
Now, if you run it from a terminal with $ hello you’ll get:
Hello World!
Something
some more stuff to print Something
echo and printf both send text streams to stdout. If you run our hello function from an interactive shell like your terminal, stdout will print to your console.
We can change this with input redirection and send the output to a file or as input to another command.
It’s a bit like return values in regular scripting language functions except you can have as many of them as you want and they don’t end the function.
If you do want to end the function, there are a couple commands to do that. The exit and return commands take a number code: 0 means success, anything else means failure. return will quit the function while exit will quit the shell itself.
Exit codes are unsigned integers which means you’ll have a fun afternoon of debugging if you’ve had the misfortune of choosing 256 as your failure error code. If for some reason your script needs more than 255 different ways to fail, well, you’re out of luck.
Stream redirection
These are the basics.
Learn Linux, 101: Streams, pipes, and redirects
|is called a pipe and you use it to send output to other commands. For example, we can tryhello | grep 'Hello'. That will send all the output ofhellotogrepwhich will return the lines that contain “Hello”. My favorite everyday usage for pipes ishistory | grep ‘command’when I forgot the exact command I’ve typed before but I know it had a certain word in it.>with a file on the right will redirect the output and print it to a file instead of to the console. The file will be completely overwritten by>. For example,logging_function > tmp_error.log. If you like Python, you may have usedpip freeze > requirements.txt.>>is like>but appends to a file instead of replacing its contents completely. For example,logging_function >> error.log.<is the reverse of>. It sends the contents of a file on the right to a command on the left. Trygrep foo < foo.txt.
Pipes run in parallel. For example, the following will only run for one second:
sleep 1 | sleep 1 | sleep 1 | sleep 1 | sleep 1
Members of a pipe don’t make the next command in line wait for them to be entirely done. They process and send output as they go.
If statements
Nothing illustrates “tiny tools” better than Bash’s if statement which is actually five keywords in a trench coat.
if [ <expression> ]; then<commands>fiNotice how you end the if statement with fi?
It’s like that for case statements as well; case … esac. When I learned this I really hoped that while would be terminated with elihw and until with litnu but these, unfortunately, just end with done.
[ is a command and ] is an argument that tells it to stop accepting other arguments. Then, else, elif, and fi are all keywords.
Take this error for example:
/bin/sh: 1: [: true: unexpected operator
You might think that the script ran into a rogue [ and threw a syntax error. Not so!
What’s actually happening is that the command [ got an unexpected argument: true. The error was actually because I’d used == instead of = which evaluated to true because it was using a weird version of the [ command (this is an example of why you need that #!/bin/bash).
Control flow
I prefer to only use if statements in specific circumstances. Generally I much prefer Bash’s operators: && and ||.
You put them after a command/function and if it returns 0, && will run the thing after it while || will not.
$ will_return_0 && echo "I will print"
$ will_return_0 || echo "I will not print"
$ will_return_1 || echo "I will print"
$ will_return_1 && echo "I will not print"
You can also chain them. Run these two commands:
$ /bin/true && echo "I will print" || echo "I will not print"
$ /bin/false && echo "won't print" || echo "will print"
But watch out! Order matters so you have to do && then || otherwise it won’t work. This won’t work the way you expect:
/bin/false || echo "will print" && echo "won't print"
They’re a bit harder to read if you’re unaccustomed to reading shell scripts but they also don’t do anything else weird. If statements are better for sequences of commands that need to be grouped together where it doesn’t make sense to make them a function.
I also prefer the command: test. It’s the same thing as [ without the misleading syntax. The less familiar looking syntax (I think) is better because it signals to the reader that they may not understand what exactly is going on.
The alternative can lead to some wrong assumptions which wastes time in the long run. This isn’t a common best practice, just my opinion, so take it with a grain of salt.
Variables
Variables in Bash are wild. They work as if you had put their value into the script and run it.
They are not typed (like ints, strings, arrays, etc) and act however is convenient for them at the time: be it strings, commands, numbers, several numbers, etc. They can even expand into multiple keywords if your “string” has spaces in it.
This can lead to some buck wild bugs and it’s why you should never ever accept risky user input to a shell script (like from the internet). If you’re a web developer and know how dangerous eval is: every shell script is a giant eval statement. Remote code executions galore!
For example, try typing the following lines into your terminal:
$ MY_VAR="echo stuff"
$ $MY_VAR
You should see it execute the command and echo “stuff” to the console. This behavior can make longer scripts buggy and unpredictable. For example, try this:
$ HELLO="hello world"
$ test $HELLO = "hello world" && echo yes
It throws an error because Bash reads it like this: test hello world = “hello world”. That’s why an important best practice is to always put variables in double quotes. Like this: test “$HELLO” = “hello world” or [ "$HELLO" = “hello world” ].
It might be helpful to not think of double quotes as string delimiters here. Bash doesn’t treat strings the way other languages do. Quotes are a little bit more like parenthesis (but not Bash parenthesis. Those are sub-shells).
Single vs. double quotes are important to differentiate in Bash. Most of the time, you’ll want to use double quotes. What’s the difference? Double quotes will expand variables, single quotes take them literally. For example:
var=stuff
echo $var
echo "$var"
echo '$var'
Output will be:
stuff
stuff
$var
Another annoying or useful thing about variables (depending on how you look at it) is that they never complain about being undeclared. You can check if a variable has been set like this:
test -z "$empty" && echo "variable is empty"
It’s also possible to add a setting to scripts that exposes unset variables:
set -o nounsetVariables are available to the entire shell by default, but this can be changed.
Scope
Understanding scope is very important to minimizing bugs.
An underutilized feature is using local and readonly variables. local will restrict a variable to a single function and readonly will throw an error if you try to re-set it. You can even chain these together and make local readonly variables, or global readonly variables.
Only global variables should be UPPERCASE. To make a variable available to the whole shell use export VAR="value". The uppercase here is a convention that says the variable is global not that it is constant/immutable like in other languages.
Commands
My least favorite thing about Bash is having to remember arcane little commands. Sed? Cat? What do these mean? The name certainly won’t tell me. Man pages are hard to parse and I certainly won’t remember what order everything is supposed to go in.
It’s like that because back in the day, each character was a lot more expensive than it is now. The idea that code should be readable by humans too was also significantly less mainstream.
Sometimes, looking for accurate information about what exactly a command is supposed to do feels like searching the dusty archives of a massive library. The librarians are mostly smug assholes who give you old manuals with one relevant sentence written in Victorian English.
Luckily, you occasionally run into super wizards on Stack Overflow who don’t mind sharing their hard-earned knowledge. Shout out to super wizards.
Special variables
Sometimes, you’ll run into weird nonsense looking variables like $@ and $!. You can read a list of them:Special Parameters (Bash Reference Manual)
3.4.2 Special Parameters The shell treats several parameters specially. These parameters may only be referenced…www.gnu.org
Some of the useful ones for writing scripts and one-liners are: $- and $*.
These both give you the arguments the command/function is running with. $- gives you the flags (the little dash modifiers) and $1–9 gives you the input. For example:
curl -s example.com
The modifier here is -s and the input is example.com.
When getting the input keywords with $, it’s important to use it like this: ${2}, ${25}, etc. Bash won’t understand something like $42 if there are two digits. You can also do something like this:
iterator=4
echo ${$iterator}
One more nifty one is $!! which, when run in the terminal, expands to the last command. Try sudo $!! when you forget to run a command with sudo.
Sub-shells and brackets
If you see this:
while ( something ); do
Those parenthesis are not what you think they are.
Parenthesis in Bash actually spawn sub-shells. That means they’re sub-processes of the same script. They are essentially child shells that get all of the parent’s context (variables, functions, etc) but can not modify anything about the parent shell.
Sub-shells are (after variables) the second most important thing to understand about shell scripting in order to keep yourself sane. They can pop up where you don’t expect them and make your programs less predictable.
First, what do they do?
Let’s look at some examples:
!#/bin/bash
myVar="foo"
echo $myVar
(
echo "Inside the sub-shell"
echo $myVar
myVar="bar"
echo $myVar
)
echo "Back in the main shell now"
echo $myVarYou’ll get the output:
foo
Inside the sub-shell
foo
bar
Back in the main shell now
foo
Notice how the variable was changed for the context of the sub shell but not for outside of it. Here’s another thing that’s handy about sub-shells:
!#/bin/bash
echo "We start out in home"
pwd
(
echo "In the subshell"
cd /tmp
pwd
)
echo "Back in the main shell now"
pwdThis will yield:
We start out at root
/home/username
In the subshell
/tmp
Back in the main shell now
/home/username
Using exit in a sub-shell will exit only that sub-shell and not the parent script.
Parenthesis aren’t the only way to spawn a sub-shell. If you put a process into the background like with & or nohup, these will also go into sub-shells.
Scripts you run with ./ run in their own shell (not sub-shells of your terminal) whereas scripts you run with source run as if you had typed in the commands directly.
You may expect functions to run in sub-shells but these are actually more like grouped commands.
To make your function run in a sub-shell every time you use it, you have to actually wrap the whole thing in parens. To make your function return a specific exit code without running in a sub-shell or quitting the whole script, use return instead of exit.
Separation of concerns
Sometimes you might need to split your script into files. source is how you do that.
parent.sh:
!#/bin/bash
source /path/to/script.sh
This will basically act as if you had typed the entire content of script.sh into parent.sh — so, kind of like how you would expect if this weren’t Bash. Watch out though! If you set a variable in the parent shell, the inner script will have write access to it.
It’s a much better idea to use this method like you would a library: to import helper functions that can be used in the parent script.
Beware of exit codes and passing values up to the main script! This does not work the way you think it does. If it’s outside of a sub-shell, exit in a sourced file will not exit the inner script: it will exit the whole thing!
Error handling
I like to wrap grouped commands into a function and then handle the case where it failed by doing:
my_function || handle_error
This was a long one. Thanks for reading all the way to the end!