Piping in Linux : How to use pipes in shell?

One of the most widely used feature in Linux shell is the pipe (|). Piping in a shell helps in redirecting the output of one command as an input to the other. The main advantage of piping is that it helps in combining simple commands to achieve a particular purpose. In fact even if you know some basic commands in Linux, you can make the best use of your knowledge by using piping. One more advantage is that the total number of commands will be reduced because what can be achieved from a single new command can be achieved by combining two already present commands. In addition to this, the number of options for a particular command can be reduced because a new option is not required if we can achieve the same using piping.Before moving on with examples, let’s see the commands which I am going to use to explain piping1. ls :
list directory contents
To list the files in the directory /bin

$ ls /bin
alsacard
alsaunmute
arch
awk
basename
bash
catchgrp
chmod
chown
cp
cpio
csh
.....

2. wc:
To print the number of newlines, words and bytes in files

$ wc test.c
44   93 1179 test.c

where the number of newlines, words and bytes in test.c is 44, 93 and 1179 respectively

To print the number of newlines, -l option is used

$ wc -l test.c
44 test.c

Now suppose to find the count of newlines we write on the terminal

$ wc -l
After writing
press ctrl + d
this will display the
numberof lines
4

Thus wc can take input both from the terminal and from a file

3. grep :
print lines matching a pattern
To print the lines which contain the pattern ‘main’ in test.c

$ grep main test.c
int main (int argc, char *argv[])

Now if we wish to print the line number in addition to the above information, -n option is used

$ grep -n main test.c
25:int main (int argc, char *argv[])

Just like wc, grep can also take input from the terminal

$ grep -n main
we are writing
some words on the terminal
and using grep to find the
line where we wrote main
4:line where we wrote main(^D)

grep as soon as it finds the pattern ‘main’ displays the corresponding line on the terminal.

4. ps :
report a snapshot of the current processes
To see the processes running in the current shell

$ ps PID TTY TIME CMD 593 pts/2 00:00:00 bash 11270 pts/2 00:00:00 ps

To see the details of all the processes run by all the users

$ ps

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.1   2044   512 ?        Ss   Jan29   0:00 init [5]
root         2  0.0  0.0      0     0 ?        S    Jan29   0:00 [migration/0]
root         3  0.0  0.0      0     0 ?        SN   Jan29   0:00 [ksoftirqd/0]
root         4  0.0  0.0      0     0 ?        S    Jan29   0:00 [watchdog/0]
root         5  0.0  0.0      0     0 ?        S<   Jan29   0:00 [events/0]
root         6  0.0  0.0      0     0 ?        S<   Jan29   0:00 [khelper]
...

Let’s see some examples to demonstrate piping

Example 1
To see the number of files or subdirectories starting with the letter ‘a’ in a directory /bin

$ ls /bin/a*
/bin/alsacard
/bin/alsaunmute
/bin/arch/bin
/awk

It is quite clear that the number of files is four. Here we can manually count them, but suppose there are more entries. Let’s use wc for the purpose, wc -l can count the number of newlines. So we can redirect the output of ls to wc. (Note wc can take input from both the file and the terminal)

$ ls /bin/a* | wc -l4

Thus using piping we have achieved our purpose. The output of ls which has to be displayed on the terminal, has been redirected as an input to wc (which was expecting input from the terminal).

Example 2

We can use more than one pipes. Suppose we wish to find the number of commands containg pattern ‘aw’

$ ls /bin/ | grep aw
awk
gawk
igawk
pgawk

The above only displays the commands containing the pattern ‘aw’. But we need the count. Let’s use wc and one more pipe

$ ls /bin/ | grep aw | wc -l4

Thus we can have more than one pipe to achieve a purpose

Example 3
Suppose we wish to know the details of all the processes run by the user ‘root’
We can use ps to search for the process details and grep to search for the pattern ‘root

root         1  0.0  0.1   2044   512 ?        Ss   Jan29   0:00 init [5]
root         2  0.0  0.0      0     0 ?        S    Jan29   0:00 [migration/0]
root         3  0.0  0.0      0     0 ?        SN   Jan29   0:00 [ksoftirqd/0]
root         4  0.0  0.0      0     0 ?        S    Jan29   0:00 [watchdog/0]
root         5  0.0  0.0      0     0 ?        S<   Jan29   0:00 [events/0]
root         6  0.0  0.0      0     0 ?        S<   Jan29   0:00 [khelper]
root         7  0.0  0.0      0     0 ?        S<   Jan29   0:00 [kthread]
...
abcd     12369  0.0  0.1   3892   656 pts/2    R+   13:55   0:00 grep root

You would have noticed in the last line user ‘abcd’ is displayed, because of the pattern root in ‘grep root’.

Example 4
Suppose we wish to know the details of a process like ‘firefox‘. We can use ps to find that. But instead of manually looking for the particular command in the long displayed list from ps, we can make use of the piping. grep can be used to search for a particular pattern. So the output of ps can be redirected to the grep command which can be used for searching the command.

$ ps -x | grep firefox

 5100 ? S 0:00 /bin/sh /usr/lib/firefox-1.5.0.7/firefox -UILocale en-US 5122 ? S 0:00 /bin/sh /usr/lib/firefox-1.5.0.7/run-mozilla.sh /usr/lib/firefox-1.5.0.7/firefox-bin -UILocale en-US 5127 ? Sl 1:03 /usr/lib/firefox-1.5.0.7/firefox-bin -UILocale en-US 5284 pts1 S+ 0:00 grep firefox

Thus instead of designing a new command for searching a particular running process, we have utilized the concept of piping.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s