Blocked Pipes

Last updated on August 15, 2003.

When you produce a pipeline in a lab exercise, you may lose points for a "blocked pipe," even if your pipeline works otherwise. Here's how to avoid them.

Pipelines

A pipeline consists of two or more commands in a single command line, connected by pipe operators ( | ), like this:

command1 | command2 

command1 | command2 | command3 

... and so on. Each pipe operator connects the standard output of the preceding command to the standard input of the following command.

• The preceding command must send data to the standard output channel, which would otherwise go to the screen.
• The following command must receive data from the standard input channel, which would otherwise come from the keyboard.
  

Blocked Pipes

A real working pipeline has data flowing through every pipe operator ( | ) in the command line. That means:

(#1) the command before the pipe sends data to the pipe, and

(#2) the command after the pipe receives data from the pipe.

If either of these conditions fails, you get what you could call a "blocked pipe."

Examples and Rules

In an example for #1 above, if the command before the pipe redirects its standard output into a file, it has no data left to send to its standard output channel. Therefore, the pipe gets no data, so the command after the pipe receives no data through its standard input channel from the pipe. Below, command2 gets no data from the pipe, because the pipe gets no data from command1:

command1 > file | command2             {Blocked!} 

Rule #1: Don't redirect data (with > or < ) in the middle of a pipeline.

However, you can always finish a pipeline with output redirection, to store the results in a file, like this:

command1 | command2 > file             {OK} 

In an example for #2 above, if the command after the pipe gets its data from a specified file, it won't receive data from its standard input channel. Therefore, the command ignores the data coming from the pipe. Below, command2 gets no data from the pipe, even though the pipe is trying to transmit data from command1:

command1 | command2 file               {Blocked!} 

Rule #2: Don't specify an input file for a command after a pipe.

And exception to Rule #2 would occur if the command after the pipe takes two or more input files. Then, you can specify the standard input channel as one of the input files by using a dash ( - ). Below, command2 gets data first from the pipe (and thus from command1), and then from a file:

command1 | command2 - file              {OK} 

Sometimes you can produce a pipeline that seems to work, but still contains a blocked pipe. For example, in the pipeline below to count the widgets in a hardware list, the command before the pipe redirects data to a file (violating rule #1), AND the command after the pipe gets its data from that same file, instead of the pipe (violating rule #2).

grep widget hardware.list > widget.temp | wc -l widget.temp   {Blocked!}

The command line works, but:

• no data flows through the pipe after " widget.temp", and
• "wc -l widget.temp" receives no data from the pipe. Therefore, this command line blocks the pipe.

To unblock this pipeline, skip the temporary file. You don't need it:

grep widget hardware.list | wc -l       {OK}

Checking Your Pipeline

To check your pipeline for a blocked pipe, try just entering the first command by itself, and see if it produces output on the screen. If you had a pipeline like this:

command1 | command2 

you'd enter just:

command1 

and see if it produces screen output. If it does, it will send that output to the pipe.

Then, check the second command to see if it gets input from the pipe. You'd enter:

command2 

and see if it produces different output (or none!) compared to the pipeline. If command2's output is the same, it probably isn't getting its data from the pipe, so your pipe is blocked.

"ls" is a good example of a command that gets no data from the standard input channel. "ls" displays the same output whether or not you place it after a pipe. So, if you put "ls" after a pipe, you'll block the pipe.

With these guidelines, you can produce smoothly flowing data throughout your pipelines, and get full credit in your homework!

Eric S. -- 707/573-1460

Back to Eric S. Home Page | Unix Course  | Unix Resources