Introduction

In recent years, the UNIX operating system has seen a huge boost in its popularity, especially with the emergence of Linux. For programmers and users of UNIX, this comes as no surprise: UNIX was designed to provide an environment that’s powerful yet easy to use.

One of the main strengths of UNIX is that it comes with a large collection of standard programs. These programs perform a wide variety of tasks from listing your files to reading email. Unlike other operating systems, one of the key features of UNIX is that these programs can be combined to perform complicated tasks and solve your problems.

One of the most powerful standard programs available in UNIX is the shell. The shell is a program that provides you with a consistent and easy-to-use environment for executing programs in UNIX. If you have ever used a UNIX system, you have interacted with the shell.

The main responsibility of the shell is to read the commands you type and then ask the UNIX kernel to perform these commands. In addition to this, the shell provides sophisticated programming constructs that enable you to make decisions, repeatedly execute commands, create functions, and store values in variables.

This book concentrates on the standard UNIX shell called the Bourne shell. When Dennis Ritche and Ken Thompson were developing much of UNIX in the early 1970s, they used a very simple shell. The first real shell, written by Stephen Bourne, appeared in the mid 1970s. The original Bourne shell has changed slightly over the years; some features were added and others were removed, but its syntax and its resulting power have remained the same.

The most attractive feature of the shell is that it enables you to create scripts. Scripts are files that contain a list of commands you want to run. Because every script is contained in a file and every file has a name, scripts enable you to combine existing programs to create completely new programs that solve your problems. This book teaches you how to create, execute, modify, and debug shell scripts quickly and easily. After you get used to writing scripts, you will find yourself solving more and more problems with them.

How This Book Is Organized

I assume that you have some familiarity with UNIX and know how to log in, create and edit files, and work with files and directories to a limited extent. If you haven’t used UNIX in a while or you aren’t familiar with one of these topics, don’t worry. The first part of this book reviews this material thoroughly.

This book is divided into three parts:

•  Part I is an introduction to UNIX, the shell, and some common tools.

•  Part II covers programming using the shell.

•  Part III covers advanced topics in shell programming.

Part I consists of Chapters 1 through 6. The following material covered in the individual chapters:

•  Chapter 1, “Shell Basics,” discusses several important concepts related to the shell and describes the different versions of the shell.

•  Chapter 2, “Script Basics,” describes the process of creating and running a shell script. It also covers the login process and the different modes in which the shell executes.

•  Chapters 3, “Working with Files,” and 4, “Working with Directories,” provide an overview of the commands used when working with files and directories. These chapters show you how to list the contents of a directory, view the contents of a file, and manipulate files and directories.

•  Chapter 5, “Manipulating File Attributes,” introduces the concept of file attributes. It covers the different types of files along with modifying a file’s permissions.

•  In UNIX every program runs as a process. Chapter 6, “Processes,” shows you how to start and stop a process. It also explains the term process ID and how you can view them.

By this point, you should have a good foundation in the UNIX basics. This will enable you to start writing shell scripts that solve real problems using the concepts covered in Part II. Part II is the heart of this book, consisting of Chapters 7 through 18. It teaches you about all the tools available when programming in the shell. The following material is covered in these chapters:

•  Chapter 7, “Variables,” explains the use of variables in shell programming, shows you how to create and delete variables, and explains the concept of environment variables.

•  Chapters 8, “Substitution,” and 9, “Quoting,” cover the topics of substitution and quoting. Chapter 8 shows you the four main types of substitution: filename, variables, command, and arithmetic substitution. Chapter 9 shows you the behavior of the different types of quoting and its effect on substitution.

•  Chapters 10, “Flow Control,” and 11, “Loops,” provide complete coverage of flow control and looping. The flow control constructs if and case are covered along with the loop constructs for and while.

•  Chapter 12, “Parameters,” shows you how to write scripts that use command line arguments. The special variables and the getopts command are covered in detail.

•  Chapter 13, “Input/Output,” covers the echo, printf, and read commands along with the < and > input redirection operators. This chapter also covers using file descriptors.

•  Chapter 14, “Functions,” discusses shell functions. Functions provide a mapping between a name and a set of commands. Learning to use functions in a shell script is a powerful technique that helps you solve complicated problems.

•  Chapters 15, “Text Filters,” 16, “Filtering Text Using Regular Expressions,” and 17, “Filtering Text with awk,” cover text filtering. These chapters show you how to use a variety of UNIX commands including grep, tr, sed, and awk.

•  Chapter 18, “Miscellaneous Tools,” provides an introduction to some tools that are used in shell programming. Some of the commands that are discussed include type, find, bc, and remsh.

At this point, you will know enough about the shell and the external tools available in UNIX that you can solve most problems. The last part of the book, Part III, is designed to help you solve the most difficult problems encountered in shell programming. Part III spans Chapters 19 through 24 and covers the following material:

•  Chapter 19, “Dealing with Signals,” explains the concept of signals and shows you how to deliver a signal and how to deal with a signal using the trap command.

•  Chapter 20, “Debugging,” discusses the shell’s built-in debugging tools. It shows you how to use syntax checking and shell tracing to track down bugs and fix them.

•  Chapters 21, “Problem Solving with Functions,” and 22, “Problem Solving with Shell Scripts,” cover problem solving. Chapter 21 covers problems that can be solved using functions. Chapter 22 introduces some real-world problems and shows you how to solve them using a shell script.

•  Chapter 23, “Scripting for Portability,” covers the topic of portability. In this chapter, you rewrite several scripts from previous chapters to be portable to different versions of UNIX.

•  Chapter 24, “Shell Programming FAQs,” is a question-and-answer chapter. Several common programming questions are presented along with detailed answers and examples.

Each chapter in this book includes complete syntax descriptions for the various commands along with several examples to illustrate the use of commands. The examples are designed to show you how to apply the commands to solve real problems. At the end of each chapter are a few questions that you can use to check your progress. Some of the questions are short answer while others require you to write scripts.

After Chapter 24, three appendixes are available for your reference:

•  Appendix A, “Command Quick Reference,” provides you with a complete command reference.

•  Appendix B, “Glossary,” contains the terms used in this book.

•  Appendix C, “Quiz Answers,” contains the answers to all the questions in the book.

About the Examples

As you work through the chapters, try typing in the examples to get a better feeling of how the computer responds and how each command works. After you get an example working, try experimenting with the example by changing commands. Don’t be afraid to experiment. Experiments (both successes and failures) teach you important things about UNIX and the shell.

Many of the examples and the answers to the questions are available for downloading from the following URL:

http://www.csua.berkeley.edu/∼ranga/downloads/tysp1.tar.Z

After you have downloaded this file, change to the directory where the file was saved and execute the following commands:

$ uncompress tysp1.tar.Z
$ tar –xvf tysp1.tar

This creates a directory named tysp1 that contains the examples from this book.

There is no warranty of any kind on the examples in this book. I have tried to make the examples as portable as possible, and to this end I have tested each example on the following versions of UNIX:

•  Sun Solaris versions 2.5.1, 2.6, 2.7

•  Hewlett-Packard HP-UX versions 10.10, 10.20, 11.0

•  Red Hat Linux versions 4.2, 5.1, 5.2

•  FreeBSD version 2.2.6

It is possible that some of the examples might not work on other versions of UNIX. If you encounter a problem or have a suggestion about improvements to the examples or the content of the book, please feel free to contact me at the following email address:

ranga@soda.berkeley.edu

I appreciate any suggestions and feedback you have regarding this book.

Conventions Used in This Book

Features in this book include the following:

---

Notes give you comments and asides about the topic at hand, as well as full explanations of certain concepts.

---

---

Tips provide great shortcuts and hints on how to program in shell more effectively.

---

---

Cautions warn you against making your life miserable and avoiding the pitfalls in programming.

---

Paragraphs containing new terms feature the New Term icon. The new term appears in italic.

At the end of each chapter, you’ll find a handy Summary and a Quiz section (with answers found in Appendix C). Many times, you’ll also find a Terms section.

In addition, you’ll find various typographic conventions throughout this book:

•  Commands, variables, directories, and files appear in text in a special monospaced font.

•  Commands and such that you type appear in boldface type.

•  Placeholders in syntax descriptions appear in a monospaced italic typeface. This indicates that you will replace the placeholder with the actual filename, parameter, or other element that it represents.