Having used ShellCheck countless times to polish my bash scripts, I am trained to write variable and command expansion like so:
# A variable containing a string does not need quotes
PATH_TO_THING=./thething.txt
# Setting a variable to be the value of a another variable (variable expansion)
# must be escaped in quotes, because of spaces being interpreted as separators.
INPUT="$1.txt"
# Command expansions must be escaped to
OUTPUT_WITH_DATE="$INPUT/file_$(date -I)"
# If we don't escape here, we might accidentally feed ls two commands
ls "$OUTPUT_WITH_DATE"
# If $OUTPUT_WITH_DATE is /Users/A user name/file_2024-01-21, and we don't escape,
# ls will see it as the equivalent of calling separately
ls /Users/A
ls user
ls name/file_2024-01-21
bash will gladly apply a string containing spaces as separate arguments.
The reason for this is that spaces and newlines have a special role in bash related to the Internal Field Separator (IFS).
This can lead to really subtle bugs, and potentially destructive behavior.
Imagine calling rm like so:
# Woops, a space was accidentally entered here:
path=/home/user/.local /etc
# And not knowing any better, we call
rm -rf $path
# Then the above rm call will be equivalent to
rm -rf /home/user/.local
rm -rf /etc
In the above example, instead of deleting etc in our user’s home directory’s
.local directory, we delete the entire .local folder and /etc to go with.
The second half might not run if we are not a superuser or similar, but the
loss of a .local folder might cause destructive loss of important data. Let’s
back up our systems regularly and test our backup strategies.
I have successfully destroyed a Debian installation before, by smuggling a
space into a chown call and accidentally making my root /etc folder
unusable and in turn making Debian unbootable.
At least, when typing out paths in a command prompt, you can easily see potential space-traps and quote your arguments accordingly. But when using variable expansion, and not knowing whether variables contain spaces or not, it’s better to be on the safe side and generously use quotes.
With the fish shell, it’s different. I have recently started writing a lot of scripts for my personal computer and also ported some other system administration scripts that I have been using before to use fish instead of bash.
Being used to bash’s variable expansion gotchas, I am used to writing quotes whenever I can. There are some subtle issues with that.
fish variables are implicit arrays
Everything in fish is an array, and some of these arrays are special PATH arrays.
We define a function that reads out how many arguments are passed to it in fish to examine how quotes are expanded:
function count_args
echo "There are "(count $argv)" arguments:"
echo $argv
end
Which will print:
$ count_args a a a
There are 3 arguments:
a a a
Which makes sense – we are passing three arguments. What happens with quoted arguments is also no big surprise:
$ count_args "a a a"
There are 1 arguments:
a a a
Now, we set a variable to contain a single string, and again no big surprise:
$ set myvar "this is a single argument"
$ count_args "$myvar"
There are 1 arguments:
this is a single argument
fish now, gets rid of the necessity to quote things since it implicitly evaluates variable expansions as single arguments and we can conveniently leave out the quotes:
$ count_args $myvar
There are 1 arguments:
this is a single argument
If we now set a second variable to be an array containing several elements, we can see the different ways variables are expanded:
# Note that no special syntax is needed to define an array
$ set myarray "several" "arguments" "are here"
$ count_args $myarray
There are 3 arguments:
several arguments are here
Since we quoted the arguments, the array is expanded into three different
arguments for our count_args function call. If we want to make sure that
myarray is expanded into exactly one argument, we have to surround it with
quotes:
$ count_args "$myarray"
There are 1 arguments:
several arguments are here
We need to apply bash-like defensive quotes only for variables that are known
to contain several items. Special semantics apply to path-like variables. One
of these variables is, of course, PATH itself. Note the difference between
the two expansions:
$ count $PATH
17
$ count "$PATH"
1
In the first case, counting the items in PATH gives us 17, since the PATH
variable in this shell instance has 17 folders. When quoting the PATH
variable expansion, we only get only one item. Again, in hindsight that totally
makes sense, but being used to reflexively put quotes everywhere, I was still
surprised.
When printing arrays using echo, we won’t notice any differences in argument
counts, of course:
$ echo $myarray
several arguments here
$ echo "$myarray"
several arguments here
And so, thinking that we are safe by just putting quotes around everything can introduce a subtle bug, as we will see now:
fish variables can be path-like
The second gotcha is that some variables are path-like, like the PATH
variable we looked at above. We can declare our own path variable like so:
$ set --path mypath one two
$ echo $mypath
one two
If we try to echo mypath, we will see a difference in behavior between a
quoted expansion, and a simple variable expansion:
$ echo "$mypath"
one:two
$ echo $mypath
one two
As noted in the help section on
path-like variables,
fish introduces special semantics for variables marked as path-like variables.
Any variable can be turned into a path-like variable by using the --path flag
when calling set, as shown above.
In order to maintain backward-compatible behavior with other programs that are invoked from a fish shell and inherit its environment, a colon is implicitly added every time a path-like variable is expanded inside quotes or similar.
If you watch out for this, it’s much more pleasant to work with paths in fish.
For example, when managing work-related documents, paths might contain spaces, or, worse, newlines, and it is quite handy to have well-documented behavior for these. I was about to wag my finger at fish, when I realized that my current habits and workaround for legacy shell behavior are the issue, not fish’s attempt at fixing them.
Nowadays I prefer writing fish scripts over bash scripts, at least for my own
purposes. If programs are instructed to use 0 byte values as separator (\0),
then these can be turned into fish arrays using string split0
as documented here.
Next time you try out fish, take note of these improved semantics, and maybe you will consider making fish your daily driver like I have.