Linux concept: shell

1. The Basics

A shell is a program that reads the commands you type, works out what you mean, and asks the rest of the system to carry them out. It sits between you and the kernel. You never speak to the kernel directly; you speak to the shell, and the shell arranges for programs to run on your behalf. When people say they are "on the command line" or "in a terminal", the program they are really talking to is a shell.

The clearest way to picture it is a simple loop that never stops while you work. The shell prints a prompt, waits for you to type a line, reads that line, breaks it into words, decides what to do, runs it, shows the output, and then prints the prompt again. This is often called the read-eval loop, and it is the whole heartbeat of interactive Unix:

print a prompt   ->   read a line   ->   understand it   ->   run it   ->   show output
      ^                                                                          |
      +--------------------------------------------------------------------------+

Two different jobs hide inside that loop, and it helps to keep them apart. As an interactive command interpreter, the shell is what you type into by hand. As a scripting language, the exact same program reads commands from a file instead, so a list of commands becomes a program you can run again and again. The remarkable thing about Unix shells is that these are not two tools: they are one program doing the same work with a different source of input.

The right mental model: the shell is an ordinary program whose whole job is to run other programs. It is not part of the kernel, it is not "the system" - it is a translator that turns the lines you type into requests the kernel can act on.

This article is about that translator: what it does, where the name comes from, how today's shells grew out of forty years of Unix history, how expansion and quoting really work, the difference between a builtin and a real program, and the surprising truth that the shell, not the command you called, does most of the clever work on your line. The examples are verified against GNU Bash 5.2 on Linux.

1.1 Seeing Which Shell You Run

Your account has a login shell, the shell that starts when you log in. The environment variable SHELL records it, and on most Linux systems it is Bash:

$ echo $SHELL
/bin/bash

That variable tells you your configured shell, but not necessarily the one running this very second. To ask the shell that is actually reading your commands right now, look at $0, the name of the running program:

$ echo $0
bash

The difference matters more than it looks. SHELL is set once at login and never changes; $0 reflects the shell you are in at this moment, which can be different if you started another shell by hand. Keep both in mind: one is your setting, the other is your reality.

1.2 The Shell Is a File Like Any Other

A shell is not magic and not built into the kernel. It is a normal executable program sitting on disk, and you can list it just like any other file:

$ ls -lh /bin/bash
-rwxr-xr-x 1 root root 1.4M ... /bin/bash

The list of shells your system considers valid login shells lives in a plain text file, /etc/shells:

$ cat /etc/shells
# /etc/shells: valid login shells
/bin/sh
/usr/bin/sh
/bin/bash
/usr/bin/bash
/usr/bin/dash

Each line is one program that can serve as somebody's shell. That is the first clue to something we return to at the end: the shell is replaceable. Nothing forces you to use Bash, and swapping it for another program in that list changes the whole feel of your command line without touching the system underneath.

1.3 The Terminal Is Not the Shell

Two words get mixed up constantly, and keeping them apart clears up a lot of confusion. The terminal is the window: an application that draws text on your screen and captures your keystrokes. The shell is the program running inside that window, reading those keystrokes and running your commands. The terminal is the glass; the shell is the voice behind it.

The proof that they are separate is that you can mix and match them freely. The same Bash runs happily inside GNOME Terminal, Konsole, or Alacritty, and any of those terminals will just as happily run Zsh instead. The clearest case is a remote login: when you connect over SSH, there is no terminal emulator on the far machine at all. Your local terminal draws the text, but the shell you are typing into runs on the server, with nothing but a network connection between them. The shell does not own a window; it only reads input and writes output, and something else - a terminal, or SSH - carries those to your screen.

3. A Short History

The shells you use today are the current step in a long line that goes back to the beginning of Unix. Knowing the line explains why there are several shells, why they are almost but not quite compatible, and why the standard one is called what it is.

The first Unix shell was written by Ken Thompson at Bell Labs around 1971. It could run commands and redirect their input and output, but it was a small tool with almost no programming features. The real turning point came in 1979, when Stephen Bourne, also at Bell Labs, released a far more capable shell with Version 7 Unix. It had variables, loops, conditionals, and functions - a genuine programming language - and it kept the name sh. This Bourne shell became the template every later shell measured itself against.

Around the same time, Bill Joy at Berkeley wrote the C shell (csh) for BSD Unix, with a syntax closer to the C language and the first job control and command history. In 1983 David Korn at Bell Labs combined the best of both worlds in the Korn shell (ksh): the Bourne language plus the interactive features of the C shell.

Then came the shell most Linux users meet first. In 1989 Brian Fox wrote Bash for the GNU project, as a free replacement for the Bourne shell. Its name is a pun that tells you exactly what it is:

Bash stands for "Bourne Again SHell" - a free rewrite that is "born again" from the Bourne shell it replaces. The joke is also a promise: it speaks the Bourne language, so old scripts keep working, while adding decades of new interactive comfort on top.

Because Bash was free software and shipped with GNU, it became the default shell on almost every Linux distribution, which is why it is the one this article uses. The line did not stop there. Zsh, written by Paul Falstad in 1990, pushed interactive features furthest and is now the default on macOS. And dash, a small, fast, strictly minimal shell, is used on Debian and Ubuntu as the plain sh, because scripts that only need the standard language run quicker in it.

One standard ties this family together. POSIX defines a common shell language, based on the Bourne and Korn shells, that every serious shell can run. Write a script to that standard and it works in Bash, dash, ksh, and zsh alike. That shared core is why a script written twenty years ago still runs, and why sh is a meaningful name no matter which program is behind it.

5. Moderate Use Cases

Once you see the shell as an interpreter, the next step is to watch it rewrite your line before any command runs. Between the moment you press Enter and the moment a program starts, the shell performs a series of transformations called expansions. Most of the shell's power, and most of its surprises, live here.

5.1 Expansion: the Shell Rewrites Your Line

When you type a line, the shell does not hand it to the command as-is. It first expands several kinds of special text into their real values. The main ones are:

Here is the key point most people miss: the command never sees the thing you typed. It sees the result. When you run ls *.txt, the ls program is never told about the *. The shell expands *.txt into the actual list of matching files first, and ls receives that list as ordinary arguments:

$ ls *.txt
# the shell first turns *.txt into: notes.txt report.txt
# then runs:  ls notes.txt report.txt

This one fact explains a great deal of Unix behaviour, and we come back to it as the biggest surprise in section 7.

5.2 Quoting: Turning Off Expansion

Because the shell treats characters like *, $, and spaces as special, you need a way to tell it "leave this text alone". That is what quoting does. Single quotes switch off all expansion; double quotes switch off most of it but still expand variables:

$ echo $HOME
/home/peter
$ echo "$HOME"        # double quotes: variable still expands
/home/peter
$ echo '$HOME'        # single quotes: taken literally
$HOME

Quoting is not decoration; it is how you control the shell. The classic bug is a filename with a space in it: without quotes the shell splits it into two arguments. Wrap a variable in double quotes - "$file" - and the whole name stays together. This is the single most common source of broken shell scripts.

5.3 Pipes and Redirection: Connecting Programs

The shell's other great power is plumbing. Every program starts with three streams already open, and the shell knows them by number as well as by name:

The shell lets you redirect any of these streams to a file, or connect one program's output to the next program's input with a pipe, written |:

$ ls -l > listing.txt        # send output into a file (> means "write to")
$ ls -l >> listing.txt       # append instead of overwrite
$ sort < names.txt           # take input from a file (< means "read from")
$ ps aux | grep apache       # feed ps output straight into grep

The two output streams are separate on purpose, so normal results and error messages can go to different places. You steer stderr by its number, 2, and you can merge it into stdout with the special form 2>&1, meaning "send stream 2 to wherever stream 1 is going":

$ command 2> errors.txt       # send only error messages to a file
$ command > out.txt 2>&1      # send output AND errors to the same file
$ command 2> /dev/null        # throw error messages away

This is the idea that makes Unix what it is. Each tool does one small job and reads and writes plain text, and the shell snaps them together into a pipeline. The programs do not know they are connected; the shell arranges the streams so that the output of one becomes the input of the next. Almost every serious command line you write is a small pipeline the shell assembles for you.

5.4 Builtins Versus External Programs

When you type a word, the shell may run it in one of two ways. Most commands, like ls or grep, are separate programs the shell finds on disk and launches. But some commands are builtins: code that lives inside the shell itself and runs without starting any new program. Bash has about sixty of them:

$ type -a echo        # echo exists as both a builtin and a program
echo is a shell builtin
echo is /usr/bin/echo
$ type cd
cd is a shell builtin

The distinction is not just trivia. Some commands have to be builtins, because they change the shell itself - and we will see in section 7 exactly why cd is the classic example. For now, remember that a builtin is faster (no new program is started) and can do things an outside program cannot.

7. Something Most Users Do Not Know

7.1 The Shell Expands Your Line, Not the Command

This is the single most important thing to understand about a Unix shell, and most people learn it by accident. When you run rm *.log, the rm program never sees the *. The shell expands *.log into the actual list of matching filenames first, then hands that list to rm. The command has no idea a wildcard was ever involved. This explains behaviour that otherwise looks strange: why * matches nothing in an empty directory and is passed through literally, why quoting a pattern stops it matching, and why you can use wildcards with any command at all, even ones that know nothing about them. The glob is the shell's doing, done before the program starts.

7.2 cd Cannot Be a Real Program

The command that changes your current directory, cd, has to be a shell builtin - it could not possibly be a separate program. Here is why. Each running program has its own current directory, and a program cannot change its parent's directory. If cd were an external program, the shell would start it, that new program would change its own directory, then exit, and your shell would be exactly where it started. The change would vanish with the program that made it. The only way to move the shell's own directory is to run code inside the shell, which is precisely what a builtin is. This is the concrete reason some commands must live inside the shell instead of on disk.

7.3 sh Is Probably Not the Shell You Think

On Debian and Ubuntu, /bin/sh is not Bash. It is a symbolic link to dash, a smaller and stricter shell:

$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 ... /bin/sh -> dash

This matters the moment a script fails only when run as sh script. Bash quietly accepts many convenient extensions that plain sh does not, so a script with #!/bin/sh that uses a Bash-only feature works when you test it in Bash and breaks when the system runs it with dash. The lesson is to match the shebang to the language you actually wrote: use #!/bin/bash if you rely on Bash features, and #!/bin/sh only for strictly standard scripts.

7.4 The Shell Runs Almost Nothing Itself

An interactive shell is mostly a launcher. When you type ls, the shell does not list the directory; it finds the ls program, makes a copy of itself with the fork system call, replaces that copy with ls using exec, and waits for it to finish. The famous fork-and-exec pattern is the heart of how Unix starts programs, and the shell does it for nearly every command you run. The shell's own code handles the prompt, the expansions, the pipes, and the builtins - and then it steps aside and lets a separate program do the actual work.

7.5 You Can Replace Your Shell Entirely

Because a shell is just a program named in /etc/shells, you are free to change the one you get at login. The chsh command ("change shell") does exactly that, and from then on every terminal greets you with the shell you picked:

$ chsh -s /usr/bin/zsh        # make zsh your login shell
$ echo $SHELL
/usr/bin/zsh

This is why the shell is genuinely an outer layer and not a fixed part of the system. People switch from Bash to Zsh for its completion, or use fish for its friendliness, and nothing underneath cares: the kernel, the commands, and the files are all the same. Only the program that reads your commands has changed, which is the whole point of the shell being a shell.

9. Common Mistakes

9.1 Myth versus Reality

9.2 Other Traps to Avoid

• Leaving variables unquoted. An unquoted $file with a space in it becomes two arguments. Quote it as "$file" unless you have a specific reason not to.
• Assuming a script's environment survives. Variables set inside a script vanish when it ends, because the script ran in its own shell. To change your current shell, use source script instead of running it.
• Trusting #!/bin/sh to mean Bash. Test scripts with the shell named in the shebang, not just interactively in Bash, or they may fail only in production.
• Forgetting that expansion happens first. A dangerous rm * is dangerous precisely because the shell expands * into every file before rm runs. Check what a glob matches with echo * first.
• Confusing $SHELL with the running shell. $SHELL is your configured login shell; $0 is the shell running right now. They differ whenever you start another shell by hand.
• Editing the wrong startup file. Put a setting in ~/.bashrc but log in over SSH and see it ignored? That is the login-versus-non-login trap from section 6.1.