Harley Hahn's Guide to Unix and Linux

A filter is any program that reads and writes textual data, one line at a time, reading from standard input and writing to standard output. As a general rule, most filters are designed as tools, to do one thing well.

Filters are important for two reasons. First, they provide a wide variety of useful services. Second, they can be combined within a pipeline to process data sequentially.

Review Question #2:

You need to solve a difficult problem using filters and a pipeline. What four steps should you follow?

What are the three most important skills you need?

Answer

When you are working on a difficult problem using filters and a pipeline, follow these steps:

1. Frame the problem. Take time to think.

2. Choose your tools. Start with the list in Figure 16-1.

3. Talk to other people. Ask for suggestions.

4. Select options. Read the documentation for the tools you want to use and choose the most suitable options.

When solving such problems, the three most important skills are:

• Being able to think well

• Being able to find and understand the appropriate documentation

• Knowing how to ask other people for help in a way that is effective

Review Question #3:

Why is cat the simplest possible filter?

In spite of its simplicity, cat can be used for a variety of purposes. Name four.

Answer

The cat program is the simplest possible filter because it does nothing more than copy textual data from standard input to standard output. However, because of the power of standard I/O, redirection and pipelines, cat can be used for a variety of purposes:

• Read from the keyboard, create a new file or replace an existing file

• Read from the keyboard, append to an existing file

• Display an existing file

• Copy a file

• Combine multiple files, display output one screenful at a time

• Combine multiple files, save output in a different file

• Combine multiple files, pipe output to another program

Review Question #4:

What is the difference between tac and rev?

Answer

The tac program reverses the lines in a file; rev reverses the characters within each line.

Applying Your Knowledge #1:

A scientist ran an experiment that generated data that accumulated in a sequence of files: data1, data2, data3, data4 and data5. He wants to know how many lines of data he has altogether.

The command wc -l reads from standard input and counts the number of lines. How would you use this command to count the total number of lines in the five files?

Answer

cat data1 data2 data3 data4 data5 | wc -l

Applying Your Knowledge #2:

You have a text file named important. What commands would you use to display the contents of this file in the following four different ways?

(a) As is
(b) Reverse the order of the lines
(c) Reverse the order of the characters within each line
(d) Reverse both the lines and characters

For (b), (c), and (d), which command performs the opposite transformation? How would you test this?

Answer

(a) cat important
(b) tac important
(c) rev important
(d) tac important | rev

Opposite:

(b) tac important
(c) rev important
(d) rev important | tac

To test the opposite commands, use the following pipelines and check that the output matches the original data:

(b) tac important | tac
(c) rev important | rev
(d) tac important | rev | rev | tac

Applying Your Knowledge #3:

In Chapter 6, we discussed the Linux program dmesg, which displays the messages generated when the system was booted. Typically, there are a great many such messages. What command would you use to display the last 25 boot messages?

Answer

dmesg | head -n 25

For Further Thought #1:

Figure 16-1 lists the most important Unix filters. Not counting awk and perl, which are programming languages, there are 19 different filters in the list. For most problems, you will need only a single filter; you will rarely need more than four. Why do you think this is the case?

Can you think of any tools that, in your opinion, are missing from the list?

Answer

The basic Unix filters were designed — by careful thinking and by trial and error — to meet all the basic needs users have when they want to manipulate text.

Because so many people have used (and worked on) these tools for so long, it is difficult to think of basic tools that are missing from the list. However, perhaps some of your more creative students can come up with good suggestions.

For Further Thought #2:

The split program was developed in the early 1970s, when large text files could create problems, because disk space was relatively slow storage and very expensive. Today, disks are fast and cheap. Do we still need a program like split? Why?

Answer

Now that disk are fast and cheap, we don't need split as much as we used to whet the size of text files was limited. However, there are still times when we want to split files for reasons that have nothing to do with storage space. In such cases, split can save you a lot of time.

Return to Previous Page

Jump to top of page

Jump to Exercises & Answers for Chapter 17
Filters: Comparing and Extracting

Exercises: Introduction | Chapter list

Instructor/Student home page