Deleting Lines

Say that you run out of mangos and you need to delete them from the list. To accomplish this task, you need to use the sed command d (d as in delete):

/pattern/d

Here pattern is a regular expression.

In this case you can use the following sed command:

$ sed '/^[Mm]ango/d' fruit_prices.txt

Here you request all lines that start with the words mango or Mango to be deleted. The output is as follows:

Fruit           Price/lbs
Banana          0.89
Paech           0.79
Kiwi            1.50
Pineapple       1.29
Apple           0.99

Notice that in this case you did not have to specify the -n option to sed to get the correct output. The p command tells sed to produce additional output, whereas the d command tells sed to modify the regular output.

Now that you have modified the output, you need to update the file. You can do this with the help of the shell:

$ mv fruit_prices.txt fruit_prices.txt.$$
$ sed '/^[Mm]ango/d' fruit_prices.txt.$$ > fruit_prices.txt
$ cat fruit_prices.txt

First, you rename the file fruit_prices.txt to fruit_prices.txt.$$. The value of the variable $$ is the process ID of the current shell. Appending this value to the end of a file is a common practice for creating temporary files.

Next, you use sed to delete the lines starting with Mango or mango from the temporary file. The output of the sed command is redirected into the file fruit_prices.txt.

The final cat command shows us that the update was successful:

Fruit           Price/lbs
Banana          0.89
Paech           0.79
Kiwi            1.50
Pineapple       1.29
Apple           0.99

At this point you can remove the temporary file as follows:

$ rm fruit_prices.txt.$$

Performing Substitutions

By now you might have noticed that Peach is misspelled as Paech in the file fruit_prices.txt. You can fix this misspelling by substituting Paech with the correct spelling Peach. To do this, use the sed command s (s as in substitute):

/pattern/s/pattern1/pattern2/

Here pattern, pattern1, and pattern2 are regular expressions. In the s command pattern1 is replaced with pattern2 on any line that matches pattern.

Frequently pattern is omitted, so you see the s command used as follows:

s/pattern1/pattern2/

Here the s command executes for every input line.

To fix the spelling of Paech, you can use the following sed command:

$ sed 's/Paech/Peach/' fruit_prices.txt

The output looks like the following:

Fruit           Price/lbs
Banana          0.89
Peach           0.79
Kiwi            1.50
Pineapple       1.29
Apple           0.99

Notice that in this case you did not have to specify the -n option to sed to get the correct output. The s command is similar to the d command in that it tells sed to modify the regular output.

Common Errors

A common error with the s command is forgetting one or more of the / characters. For example, say you were to issue the command

$ sed 's/Paech/Peach' fruit_prices.txt

An error message similar to the following is produced:

sed: command garbled: s/Paech/Peach

This is the standard style of sed error messages:

sed: command garbled: command

Here sed could not understand the command. No additional error messages or information are produced. You have to determine what went wrong yourself.

Performing Global Substitutions

Consider the following line:

$ cat nash.txt
things that are eqal to the same thing are eqal to each other

Here the word equal is misspelled as eqal. Try to fix this using the s command as follows:

$ sed 's/eqal/equal/' nash.txt

This produces the output:

things that are equal to the same thing are eqal to each other

As you can see, the first misspelling was fixed, but the second one was not. This is the default behavior of the s command: it only performs one substitution on a line. To perform more than one substitution, you need to use the g (g as in global) operator as follows:

s/pattern1/pattern2/g

Here pattern1 and pattern2 are regular expressions. The g operator tells the s command to substitute every occurrence of pattern1 with pattern2.

In this case, you use it as follows:

$ sed 's/eqal/equal/g' nash.txt

This produces the correct output:

things that are equal to the same thing are equal to each other

Reusing an Expressions Value

Now say that you want to change the list to reflect that the prices are in dollars by appending the $ character in front of each of the prices. You know that by using the following expression, you can match all the lines that end with a price:

/ *[0-9][0-9]*\.[0-9][0-9]$/

The problem, though, is replacing the existing price with a price that is preceded by the $ character. Apparently, you would need to write a separate s command for each fruit in the file.

Fortunately, the s command provides us with the & operator, which enables us to reuse the string that matched pattern1 in pattern2. In this case you need to reuse the price that was matched:

$ sed 's/ *[0-9][0-9]*\.[0-9][0-9]$/\$&/' fruit_prices.txt
Fruit           Price/lbs
Banana          $0.89
Paech           $0.79
Kiwi            $1.50
Pineapple       $1.29
Apple           $0.99