Using Pipes in Linux Processes

Pipes can be used in threads and processes. The program below demonstrates how pipes can be used in processes. A new process can be created using the system call fork(). It returns two differnt values to the child and parent. The value 0 is returned to the child (new) process and the PID (Process ID) of the child is returned to the parent process. This is used to distinguish between the two processes. In the program given below, the child process waits for the user input and once an input is entered, it writes into the pipe. And the parent process reads from the pipe.

A sample program to demonstrate how pipes are used in Linux Processes

#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>

#define MSGLEN  64

int main(){
    int     fd[2];
    pid_t pid;
    int     result;

    //Creating a pipe
    result = pipe (fd);
    if (result < 0) {
        //failure in creating a pipe
        perror("pipe");
        exit (1);
    }

    //Creating a child process
    pid = fork();
    if (pid < 0) {
         //failure in creating a child
         perror ("fork");
         exit(2);
    }

    if (pid == 0) {
        //Child process
         char message[MSGLEN];

          while(1) {
                    //Clearing the message
                    memset (message, 0, sizeof(message));
                    printf ("Enter a message: ");
                    scanf ("%s",message);

                    //Writing message to the pipe
                    write(fd[1], message, strlen(message));
            }
            exit (0);
    }
    else {
        //Parent Process
         char message[MSGLEN];

         while (1) {
                    //Clearing the message buffer
                    memset (message, 0, sizeof(message));

                    //Reading message from the pipe
                   read (fd[0], message, sizeof(message));
                    printf("Message entered %s\n",message);
            }

            exit(0);
     }
}

Note here that the pipe() system call was called before the system call fork(). The buffer allocated to the pipe is accessible to both the processes.

 

FIFO – Named pipes: mkfifo, mknod

Pipes are commonly used for interprocess communication. But the major disadvantage of pipes is that they can be used only by one process (there are readers and writers within the same process) or the processes which share the same file descriptor table (normally the processes and the child processes or threads created by them). Thus pipes have this big limitation: they cannot pass information between unrelated processes. This is because they do not share the same file descriptor table. But if names are given to the pipes, then one would be able to read or write data to them just like a normal file. The processes need not even share anything with each otherFIFO (First In First Out) are also called named pipes. The main features of FIFO are
1. It implements FIFO feature of the pipes
2. They can be opened just like normal files using their names
3. Data can be read from or written to the fifo

Working with FIFO in a Shell
Creating a FIFO

mkfifo

creates fifo- the named pipes

Syntax

mkfifo [options] fifo_name

Example

$ mkfifo fifo

There is one more way by which we can FIFO using mknod. mknod is used to create block or character special files.

$ mknod [OPTION]... NAME TYPE

To create a FIFO fifo1

$ mknod fifo1 p

where p coressponds to file type : pipe (remember FIFO is a named pipe).

Reading/ Writing data from/to a FIFO
Let’s open two terminals
In the first terminal

$ cat > fifo

we are experimenting with the FIFOThis is second line. After opening the fifo in the second terminal for readingusing cat, you will notice the above two lines displayed there.
Now open the second terminal and go to the directory containing the FIFO ‘fifo’

$ cat fifo

we are experimenting with the FIFOThis is second line. After opening the fifo in the second terminal for reading
Now keep on writing to the first terminal. You will notice that every time you press enter, the coressponding line appears in the second terminal.

Pressing CTRL+D in the first terminal terminates writing to the fifo. This also terminates the second process because reading from the fifo now generates a “BROKEN PIPE” signal. The default action for this is to terminate the process.

Let us now see the details of the file ‘fifo’

$ ls -l fifo
prw-r--r-- 1 user user 0 Feb 14 10:05 fifo

The p in the beginning denotes that it is a pipe.

Let’s see more details about the pipe using stat

$ stat fifo
File: `fifo'Size: 0               Blocks: 0          IO Block: 4096   fifo
Device: fd00h/64768d    Inode: 1145493     Links: 1
Access: (0644/prw-r--r--)  Uid: (    0/    user)   Gid: (    0/    user)
Access: 2008-02-14 10:05:49.000000000 +0530
Modify: 2008-02-14 10:05:49.000000000 +0530
Change: 2008-02-14 10:05:49.000000000 +0530

If you notice carefully, FIFOs just like a normal file possess all the details like inode number, the number of links to it, the access, modification times, size and the access permissions.

As in the case of pipes, there can be multiple readers and writers to a pipe. Try opening multiple terminals to read from and write to a pipe.

 

Operations on Pipe (Advanced Pipe Usage)

we discussed how we can work with pipe() system call, where it is used(Linux shell), how to use pipes in threads(processes).

Here we can discuss on the topic ‘Number of readers and writers accessing a pipe’
A pipe can have more than one reader and more than one writer. Only one thing that must be kept in mind is that writer writes into fd[1] and reader reads from fd[0], where fd is the filedescriptor passed for the pipe() call.

int pipe(int fd[2]);

The program below demonstrates this with two writers and one reader. The writer Writer_ABC writes the capital Alphabet and the Writer_abc writes the small alphabet into the pipe. The reader reads in the order of the letters written by the two writers

#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <pthread.h>

int fd[2];//File descriptor for creating a pipe

//This function continously reads fd[0] for any input data byte
//If available, prints

void *reader()
{
    while(1){
       char    ch;
       int     result;

       result = read (fd[0],&ch,1);
       if (result != 1) {
            perror("read");
            exit(3);
    }    printf ("Reader: %c\n", ch);  }
}

//This function continously writes Capital Alphabet into fd[1]
//Waits if no more space is available

void *writer_ABC()
{
     int     result;
     char    ch='A';

     while(1){
           result = write (fd[1], &ch,1);
           if (result != 1){
               perror ("write");
               exit (2);
           }

           printf ("Writer_ABC: %c\n", ch);
           if(ch == 'Z')
              ch = 'A'-1;

           ch++;
      }
}

//This function continously writes small Alphabet into fd[1]
//Waits if no more space is available

void *writer_abc()
{
  int     result;  char    ch='a';

  while(1){
      result = write (fd[1], &ch,1);
      if (result != 1){
            perror ("write");
            exit (2);
      }

      printf ("Writer_abc: %c\n", ch);
      if(ch == 'z')
            ch = 'a'-1;

     ch++;
  }
}

int main()
{
   pthread_t       tid1,tid2,tid3;
   int             result;

   result = pipe (fd);
   if (result < 0){
       perror("pipe ");
       exit(1);
   }

 pthread_create(&tid1,NULL,reader,NULL);
 pthread_create(&tid2,NULL,writer_ABC,NULL);
 pthread_create(&tid3,NULL,writer_abc,NULL);

 pthread_join(tid1,NULL);
 pthread_join(tid2,NULL);
 pthread_join(tid3,NULL);
}

Output

Writer_abc: o
Writer_abc: p
Writer_abc: q
Writer_abc: r
Writer_abc: s
Writer_abc: t
Writer_abc: u
Reader: A
Reader: B
Reader: C
Reader: D
Reader: E
Reader: F
Reader: G
Reader: H
Reader: I
Reader: J
Reader: K
Reader: L
Reader: M
Reader: N
...

Piping in threads

Pipes are quite useful in threads. Imagine a scenario, where one thread is continously monitoring for new user input and another thread performs some processing on this data input. This can be achieved with the help of shared memory. But in case of shared memory, one has to think about data synchronization. Pipes can be made use of here. One thread waits and reads the user input and writes it into the pipe. The second thread can read from the pipe and perform processing.

Let’s take a simple example. The program below shows the READER-WRITER threads. The WRITER thread continously writes the alphabets into the pipe. and the READER thread reads it. See clearly that fd[2] is declared globally so that both threads can access it. Also note that the pipe() system call was called before threads creation.

To compile a program with threads, we must specify -lpthread to link with the POSIX threads library
$ gcc program -lpthread

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <pthread.h>

int fd[2];//File descriptor for creating a pipe

//This function continously reads fd[0] for any input data byte
//If available, prints

void *reader()
{
   while(1){
      char    ch;
      int     result;

      result = read (fd[0],&ch,1);
      if (result != 1) {
        perror("read");
        exit(3);
      }

      printf ("Reader: %c\n", ch);   }
}

//This function continously writes Alphabet into fd[1]
//Waits if no more space is available

void *writer()
{
   int     result;
   char    ch='A';

   while(1){
       result = write (fd[1], &ch,1);
       if (result != 1){
           perror ("write");
           exit (2);
       }

       printf ("Writer: %c\n", ch);
       if(ch == 'Z')
         ch = 'A'-1;

       ch++;
   }
}

int main()
{
   pthread_t       tid1,tid2;
   int             result;

   result = pipe (fd);
   if (result < 0){
       perror("pipe ");
       exit(1);
   }

   pthread_create(&tid1,NULL,reader,NULL);
   pthread_create(&tid2,NULL,writer,NULL);

   pthread_join(tid1,NULL);
   pthread_join(tid2,NULL);
}

Output

...
Reader: Y
Reader: Z
Reader: A
Reader: B
Reader: C
Reader: D
Reader: E
Reader: F
Reader: G
Reader: H
Reader: I
Reader: J
Reader: K
Reader: L
Reader: M
Reader: N
Reader: O
Reader: P
Reader: Q
Reader: R
Reader: S
Reader: T
Reader: U
Reader: V
Reader: W
Reader: X
Reader: Y
Reader: Z
Reader: A
Reader: B
Reader: C
...

After doing this, let’s check what’s the capacity of a pipe. Anyway pipe also utilizes memory, so there must be an upper limit to the capacity of a pipe. A writer thread keeps on writing to the pipe hoping that the READER thread is reading it. Let’s put some delay in READER thread (sleep() call). so that write will at some point of time stop writing the data to the pipe.

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <pthread.h>

int fd[2];//File descriptor for creating a pipe

//This function continously reads fd[0] for any input data byte
//If available, prints

void *reader()
{

 int     count = 0;
 sleep (25);
  //Delay in starting the reading from the pipe

  while(1){
      char    ch;
      int     result;

      result = read (fd[0],&ch,1);
      if (result != 1) {
          perror("read");
          exit(3);
      } printf ("Reader: %d %c\n",++count,ch);
  }
}

//This function continously writes Alphabet into fd[1]

void *writer()
{
    int     result;
    char    ch='A'; int     count = 0;

    while(1){

       result = write (fd[1], &ch,1);
       if (result != 1){
           perror ("write");
           exit (2);
       }

       printf ("Writer: %d %c\n", ++count, ch);

       if(ch == 'Z')
          ch = 'A'-1;

        ch++;
   }
}

int main()
{
   pthread_t       tid1,tid2;
   int             result;

   result = pipe (fd);

   if (result < 0){
        perror("pipe ");
        exit(1);
   }

   pthread_create(&tid1,NULL,reader,NULL);
   pthread_create(&tid2,NULL,writer,NULL);

   pthread_join(tid1,NULL);
   pthread_join(tid2,NULL);
}

Output
...
Writer: 65524 D
Writer: 65525 E
Writer: 65526 F
Writer: 65527 G
Writer: 65528 H
Writer: 65529 I
Writer: 65530 J
Writer: 65531 K
Writer: 65532 L
Writer: 65533 M
Writer: 65534 N
Writer: 65535 O
Writer: 65536 P

On executing the above program, the WRITER thread stopped at the count = 65536 waiting for READER thread to read the data already written. After the specified delay (here 25 seconds), the READER thread resumes its operation and starts reading. Then both WRITER and READER thread works normally.So pipe() is useful for implementing QUEUE mechanism of sending the data from one thread(process) to another keeping in mind that the pipe also has a capacity limit. So if the reader is blocked or if there is no reader, the writer will block at some point of time if it reached the capacity of the pipe.

 

Using Pipes in Linux Programming

Pipes are one of the most commonly used mechanisms for IPC (inter process communication). IPC is the mechanism by which two or more processes communicate with each other. The commonly used IPC techniques includes shared memory, message queues, pipes and FIFOs (and sometimes even files). Pipes are helpful in the unidirectional form of communication between processes.The pipe() creates a pipe.Syntax

int pipe(fd[2])pipe() creates a pair of file descriptors, pointing to a pipe inode, and places them in the array pointed to by fd. fd[0] is for reading, fd[1] is for writing.Let’s see a simple program how to use piping.

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#define MSG_LEN 64

int main(){

      int     result;
      int     fd[2];
      char    *message="Linux World!!!";
      char    recvd_msg[MSG_LEN];

      result = pipe (fd); //Creating a pipe fd[0] is for reading and fd[1] is for writing

      if (result < 0) {
          perror("pipe ");
          exit(1);
      }

      //writing the message into the pipe

      result=write(fd[1],message,strlen(message));
      if (result < 0) {
          perror("write");
          exit(2);
      }

      //Reading the message from the pipe

      result=read (fd[0],recvd_msg,MSG_LEN);

      if (result < 0) {
          perror("read");
          exit(3);
      }

      printf("%s\n",recvd_msg);
      return 0;
}

Output

Linux World!!!

As seen above , the pipe() system call creates a pipe. Now normal system calls like read() or write() can be used for reading and writing data to the pipe. As you have noticed the fd[0] can only be used for reading from the pipe and does not permit writing. Similarly fd[1] can only be used to perform writing to the pipe.

Seeing the above program, one would feel

what is the neccessity of piping?.

You are writing into the pipe and reading from it. That one can do even from a file.

Now let’s see another scenario. If we add more writes in the above program before reading anything and then start reading, you will find that the first read call will fetch you the first written message and so on. So

pipe() is helpful in Implementing a QUEUE strategy(First in First out)

FIFO: the message which was written first will be available for the first read, then the message which came second, for the second read.

Note: The first read itself can get all of the messages written by different write, if it specifies a large size in the size field (a size which can span the size of one or more writes ) as demonstrated here

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#define MSG_LEN 64

int main(){
   int     result;
   int     fd[2];
   char    message[MSG_LEN];
   char    recvd_msg[MSG_LEN];

   result = pipe (fd); //Creating a pipe//fd[0] is for reading and fd[1] is for writing

   if (result < 0) {
     perror("pipe ");
     exit(1);
   }

   strncpy(message,"Linux World!! ",MSG_LEN);

   result=write(fd[1],message,strlen(message));
   if (result < 0) {
      perror("write");
      exit(2);
   }

   strncpy(message,"Understanding ",MSG_LEN);

   result=write(fd[1],message,strlen(message));
   if (result < 0) {
      perror("write");
      exit(2);
  }

  strncpy(message,"Concepts of ",MSG_LEN);

  result=write(fd[1],message,strlen(message));
  if (result < 0) {
      perror("write");
      exit(2);
 }

 strncpy(message,"Piping ", MSG_LEN);
 result=write(fd[1],message,strlen(message));

 if (result < 0) {
     perror("write");
     exit(2);
 }

 result=read (fd[0],recvd_msg,MSG_LEN);
 if (result < 0) {
      perror("read");
      exit(3);
 }

 printf("%s\n",recvd_msg);
 return 0;

}

Output:

Linux World!! Understanding Concepts of Piping

In the above program, The read() call reads all the messages in a single stretch, because of the MSG_LEN(64) which is greater than sum of the size of the messages written in four different writes. The only thing that we must remember that the read() reads in the order of messages written.