Having used ShellCheck countless times to polish my bash scripts, I write variable and command expansion like so:
# A variable containing a string does not need quotes
PATH_TO_THING=./thething.txt
# Setting a variable to be the value of a another variable (variable expansion)
# must be escaped in quotes, because of spaces being interpreted as separators.
INPUT="$1.txt"
# Command expansions must be escaped to
OUTPUT_WITH_DATE="$INPUT/file_$(date -I)"
# If we don't escape here, we might accidentally feed ls two commands
ls "$OUTPUT_WITH_DATE"
# If $OUTPUT_WITH_DATE is /Users/A user name/file_2024-01-21, and we don't escape,
# ls will see it as the equivalent of calling separately
ls /Users/A
ls user
ls name/file_2024-01-21
bash applies a string containing spaces as separate arguments.
The reason for this is that spaces and newlines have a specific role in bash related to the Internal Field Separator (IFS).
This can lead to subtle bugs, and sometimes destructive behavior. Imagine
calling rm like so:
# Woops, a space was accidentally entered here:
path=/home/user/.local /etc
# And not knowing any better, we call
rm -rf $path
# Then the above rm call will be equivalent to
rm -rf /home/user/.local
rm -rf /etc
In this example, instead of deleting etc in our user’s home directory’s
.local directory, we delete the entire .local folder and /etc to go with.
The second half might not run if we’re not a superuser or similar, but the loss
of a .local folder might cause destructive loss of important data. Let’s back
up our systems and test our backup strategies.
I have successfully destroyed a Debian installation before, by smuggling a
space into a chown call and making my root /etc folder unusable by
accident. This in turn made my Debian installation unbootable.
At least, when typing out paths in a command prompt, you can see potential space-traps and quote your arguments accordingly. But when using variable expansion, and not knowing whether variables contain spaces or not, it’s better to be on the safe side and always use quotes.
With the fish shell, it’s different. I’ve recently started writing a lot of scripts for my personal computer. I’ve also ported some other system administration scripts that I have been using before to use fish instead of bash.
I’m used to bash’s variable expansion gotchas and I use quotes whenever I can. fish has some subtle issues with quoting, as we now see.
fish variables are implicit arrays
Everything in fish is an array. Some of these arrays in turn are PATH arrays. fish interprets PATH arrays slightly differently.
Here, we define a function that reads out how many arguments it receives in fish. You can use it to examine how fish expands quotes. This is the function:
function count_args
echo "There are "(count $argv)" arguments:"
echo $argv
end
This prints:
$ count_args a a a
There are 3 arguments:
a a a
Which makes sense–we’re passing three arguments. What happens with quoted arguments is also no big surprise:
$ count_args "a a a"
There are 1 arguments:
a a a
Now, we set a variable to contain a single string, and again no big surprise:
$ set myvar "this is a single argument"
$ count_args "$myvar"
There are 1 arguments:
this is a single argument
fish now, gets rid of the necessity to quote things since it implicitly evaluates variable expansions as single arguments and we can conveniently leave out the quotes:
$ count_args $myvar
There are 1 arguments:
this is a single argument
Now we set a second variable to be an array containing more than one element.
You can see that fish expands the values myvar to become 3 arguments.
# Note that no special syntax is needed to define an array
$ set myarray "several" "arguments" "are here"
$ count_args $myarray
There are 3 arguments:
several arguments are here
Since we quoted the arguments, fish expands them into three different arguments
for our count_args function call. If we want to make sure that fish expands
myarray into exactly one argument, we need to surround it with quotes:
$ count_args "$myarray"
There are 1 arguments:
several arguments are here
We need to apply bash-like defensive quotes only for variables where we know
that they contain more than one item. Special semantics apply to path-like
variables. One of these variables is PATH itself. Note the difference between
the two expansions:
$ count $PATH
17
$ count "$PATH"
1
In the first case, counting the items in PATH gives us 17, since the PATH
variable in this shell instance has 17 folders. When quoting the PATH
variable expansion, we only get only one item. Again, in hindsight that totally
makes sense, but since I reflexively put quotes everywhere, I was still
surprised.
When printing arrays using echo, we won’t notice any differences in argument
counts:
$ echo $myarray
several arguments here
$ echo "$myarray"
several arguments here
And so, we may thinking that we’re safe by just putting quotes around everything. This can introduce a subtle bug, as we see in the following:
fish variables can be path-like
The second gotcha is that some variables are path-like, like the PATH
variable we looked in the previous section. We can declare our own path
variable like so:
$ set --path mypath one two
$ echo $mypath
one two
If we try to echo mypath, we see a difference in behavior between quoted and
non-quoted variable expansion:
$ echo "$mypath"
one:two
$ echo $mypath
one two
As noted in the help section on
path-like variables,
fish introduces special semantics for variables marked as path-like variables.
We can turn any variable into a path-like variable by using the --path flag
when calling set, just like in the preceding example.
fish maintains backward-compatible behavior with other programs that you invoke from a fish shell and inherit its environment. A colon is implicitly added every time fish expands a path-like variable inside quotes or similar.
If you watch out for this, it’s much more pleasant to work with paths in fish.
For example, when managing work-related documents, paths might contain spaces, or, worse, newlines, and it’s handy to have well-documented behavior for these. I was about to wag my finger at fish. Then, I realized that my current habits and workaround for legacy shell behavior are the issue. fish’s attempt at fixing them isn’t the issue.
Nowadays I prefer writing fish scripts over bash scripts, at least for my own
purposes. If you instruct programs to use 0 byte values as separator (\0), then
fish can turn these into fish arrays using string split0
as documented here.
Next time you try out fish, take note of these improved semantics, and perhaps you may consider making fish your daily driver like I have.