Introduction to Bash command syntax and execution

An overview of how commands are interpreted and executed by the shell.

Introduction to command syntax and execution

Let's look at that mkdir command again, which I'll run from my corn.stanford.edu account (as opposed to my own OS X laptop, which I had done previously only as a demonstration of how the shell and the GUI operate on top of the same system).

In the example below, I'm running the ls command to show that the current directory (which is named ~/stuff, not that it makes a difference) is empty. Then I run mkdir to create new directories. Then I run ls again to show the effects.

Here's the sequence and output:

dun@corn30:~/stuff$ ls
dun@corn30:~/stuff$ mkdir apples oranges pears
dun@corn30:~/stuff$ ls
apples  oranges  pears

And as a GIF:

Arguments

As you can see, the ls command is quite simple, both in execution and purpose; I simply type in ls, hit Enter, and I see the contents of the current directory listed.

However, the structure of mkdir is different: after typing in mkdir, I have to type in the names of directories that I want to create. These text strings – apples, oranges, and pears – are the arguments to the mkdir command.

So unlike ls, the mkdir command requires at least one argument. This makes sense on an intuitive level: what would mkdir actually do if we didn't give it any names from which to create directories?

Filename arguments

Like mkdir, the echo command will just act on the arguments given to it; in echo's case, this involves just outputting the arguments back:

However, some commands, such as cat, wc, and grep, will expect arguments to refer to the names of files that exist.

img

Which commands interpret their arguments which way? It's up to you to read each command's documentation; don't expect to just infer the behavior.

Options

While command-line programs are meant (and forced) to be simple, many of them are designed to accept options, or flags. For example, trying to create a nested directory with mkdir might throw an error:

dun@corn30:~/stuff$ mkdir 1990/September
mkdir: cannot create directory '1990/September': No such file or directory

In this case, mkdir can't create the path 1990/September because the parent directory, 1990 needs to be created before its child directory, September, can be created. With the -p option, mkdir will create all the intermediary parent directories of a desired path:

mkdir -p 1990/September/28

Aliased options and arrangement of options

Oftentimes, the options for programs will have aliases. For example, the --output option for the curl program directs curl to save a downloaded file into the specified filename:

sunet_id@corn30:~$ curl http://www.example.com --output filename.html

The --output option has an alias, -o, so this is equivalent to the above command:

sunet_id@corn30:~$ curl http://www.example.com -o filename.html

The curl program also accepts the --silent option, which prevents the default display of a progress indicator or of any error messages that might come up:

sunet_id@corn30:~$ curl http://www.example.com --silent

That also has an abbreviated alias:

sunet_id@corn30:~$ curl http://www.example.com --silent

And more than one option can be used at a time:

sunet_id@corn30:~$ curl http://www.example.com --silent --output test.html

And both options can be aliased and combined for brevity's sake:

sunet_id@corn30:~$ curl http://www.example.com -s -o test.html
sunet_id@corn30:~$ curl http://www.example.com -so test.html

When in doubt, use the "man" command

If you are a novice to Unix, it's easy to think of it as a cruel operating system full of human-unfriendly tersely-named programs, each with an arbitrary array of options and capricious rules. Well, that's because it is. Unix was created by and for scientists and programmers who needed a framework to build unique tools for unique problems. As Unix became more popular, there was some agreement on a set of standards and best practices for program design, but not always.

And so no, you aren't meant to memorize every single program and the details. That is what the man program is for, and Google, and Stack Overflow. Instead of memorizing trivial details, you need to remember what kinds of problems computers can easily solve, how that applies to your current problem, and then look up the details.

So what is absolutely required of you, if you wish to become a proficient programmer and/or maintain your sanity, is to remember why these Unix tools are designed in such an independently pain-in-the-ass way: so that programmers could solve unique problems by combining tools in a unique way, rather than hope that someone built an all-encompassing-tool that happens to encompass the programmer's unique situation.

Of course, Unix and its derivatives would never have been so ubiquitous and successful if there were absolutely no standards. So there are many similarities in interfaces between the most popular tools. However, you should keep in mind the (likely) possibility that two tools might look alike but have nothing else in common.

For example, these two commands:

curl -vo test.html "www.example.com"
grep -vo test.html "www.example.com"

– have absolutely nothing in common. It's up to you to look up each individual command and its corresponding options.

The many ways to curl

I use the example of curl because it's not only an immensely useful program, but one with a huge array of options, and one that is quite flexible in how it reads arguments and options. These are all equivalent:

curl http://example.com -so test.html
curl -so test.html http://example.com 
curl -o test.html -s http://example.com
curl -s http://example.com -o test.html 
curl -o test.html http://example.com --silent

However, these arrangements of arguments will not have the same effects, or even result in errors:

curl -os test.html http://example.com
curl -o http://example.com -s test.html
curl http://example.com -o -s test.html

To understand why, you have to put yourself in the mindset of the command-line interpreter and think about the ambiguity in such statements. However, that's not particularly easy as a novice Unix user, so it's best to stick with documented examples that you know have worked. And if you see a different variation, before just blindly copying-and-pasting it, make sure you understand the differences.

Keep in mind that options are optional – if you are unsure what each one does, you should be able to run the command without those options. And then add each option, one by one, and notice the effect.

And if it's still unclear, then it's time to consult the man command, or the Internet.