Shell Script to execute a command on every line of a file

There are many times, where you want to execute the same command every line of a file, it can be simple printing on the screen or some other manipulation. This simple shell script reads a file line by line and executes the command specified by the user on the line.

$ cat execute_command_per_line.sh
if [ $# -ne 2 ]
then
   echo "Usage: execute_command_per_line command filename"
   exit 1
fi

command=$1
filename=$2

while read line
do
   $command $line
done < $filename

As you can see that it takes two parameters,

1. Command to be executed
2. Name of the file

Let’s use a simple command like echo, The file contents are as follows

$ cat system_calls.txt
 
aa_change_hat
accept
accept4
access
acct
add_key
adjtime
adjtimex
afs_syscall
alarm

Let’s run the above script

$ ./execute_command_per_line.sh echo system_calls.txt 
aa_change_hat
accept
accept4
access
acct
add_key
adjtime
adjtimex
afs_syscall

Shell Script to extract various sections of a Man page

A Man page has various sections like

1. NAME
2. SYNOPSIS
3. DESCRIPTION
4. EXAMPLE
5. AUTHORS

The above are some of the few sections to name for. You open a man page using the command man. It can also be redirected to a file.

Let’s see the script

$ cat man_section_extract.sh 

if [ $# -ne 3 ]
then
   echo "Usage: man_section_extract man_number section command"
   exit 1
fi
man_number=$1
section=$2
command=$3
   
man $man_number $command> /tmp/man.txt
if [ $? -eq 0 ]
 then
 cat /tmp/man.txt | sed -n -e "/^$section/,/^[A-Z]/ p"| grep -v -e "^[A-Z]" 
fi

As you can see that it takes three arguments,

1. The Man page number (Commmands are usually under man page 1, system calls under man page 2, library calls under man page 3)
2. Section: as described above, It can be NAME, DESCRIPTION
3. Command

Let’s first take a look at a sample man page

NAME
 ls - list directory contents

SYNOPSIS
 ls [OPTION]... [FILE]...

DESCRIPTION
 List information about the FILEs (the current directory by default).  Sort entries alphabetically if none of -cftuvSUX nor --sort.

 Mandatory arguments to long options are mandatory for short options too.

Now let’s use our script to extract the NAME section

$ ./man_section_extract.sh 1 NAME ls
       ls - list directory contents

Let’s extract SYNOPSIS

$ ./man_section_extract.sh 1 SYNOPSIS ls
       ls [OPTION]... [FILE]...

Like this, you can extract various sections of a man page. All you need is to specify the command, section name and man page number

apt-get: Shell Script to remove the packages mentioned in a File

Check this simple shell script which reads all the packages mentioned in a file and removes them one by one.

#! /bin/sh
if [ $# -ne 1 ]
then echo "Usage: read.sh filename";
exit 1;
fi

if [ ! -f $1 ]
then echo "File $1 doesn't exist";
exit 1;
fi

while read line
do
  pkg=`echo $line| awk '{print $1}'`
  echo $pkg
  yes|sudo apt-get remove $pkg
done < $1

To use this script, mention the packages to be removed in a file. Take for example, pkgs.txt contains the files to be removed

$ cat pkgs.txt
vim
emacs

To execute the above script,

$ ./remove.sh pkgs.txt

It will remove all the packages one by one.

Shell script to find the length of a string

awk has a built in function called length with which you can find the length of a string. The following shell script shows how to find the length of the string with awk

#!/bin/sh
#Shell script to find the length of the string

string="Joys of Programming"
length=`echo|awk "{print length("$string")}"`
echo "Length of the string "$string" is $length"

On executing the program

$ ./length.sh 
Length of the string "Joys of Programming" is 19

You can even find the length from the terminal with this simple command

$ awk 'BEGIN {print length("Joys of Programming")}'
19

or

$ echo|awk '{print length("Joys of Programming")}'
19

Thus you can see that the syntax of length function is

length(string)

Arithmetic in Shell Script

The arithmetic operations can be done in shell script using expr command

The following program shows the various arithmetic operations

#!/bin/sh

a=10 b=5

echo `expr $a + $b`

echo `expr $a -  $b`

echo `expr $a * $b`

echo `expr $a / $b`

x=10.5  y=5.5

echo `expr $x+$y|bc`

echo `expr $x-$y|bc`

echo `expr $x*$y|bc`

The output of the program looks like this

15

5

50

2

16.0

5.0

57.7

The bc command is used to perform real arithmetic operations. Here we have piped the result of echo to bc.

String comparison in shell script

The following program shows the comparison of two strings in shell script

#!/bin/sh
str1="happy"

str2="happy"

[ $str1 = $str2 ]

echo $?

The output of the program looks like this

0

Amazed! The metacharacter $? tests the success and failure of the string comparison. The echo statement displays 0 for success and 1 for failure.

Hello World:Shell Script

A simple hello world in shell script will look something like this

#!/bin/sh
echo Hello World!

To execute the program

$ source hello.sh
Hello World!

A Shell script to search for a command pattern

Today I have been looking for a command to compare two files simultaneously (the colour schemes of KDE) and used diff. But that did not suit my requirement of viewing them simultaneously. There are editors like vi which provide simultaneous viewing of the files and I tried

$vimdiff file_1 file_2

To search for some specific command, we can use ‘man’ and the pattern

$man -k diff

and it showed many commands like zdiff,vimdif,..

The advantage of this is that it searches for the pattern and a one line abstract of these commands (available in man pages) for the corresponding pattern

But the major disadvantage is that if you have some commands in your system which does not have a man page, it returns “nothing appropriate”

Here is a simple shell script which searches for command-pattern in the directories in the PATH variable

Just make a change in the PATH_DIR_COUNT depending on the number of directories in it.
(use echo $PATH to find this out)

#!/bin/sh
# A simple Command to search for a pattern in the commands available in the directories mentioned in PATH environment varible
# Usage: cmd_search pattern

PATH_DIR_COUNT=6 #Number of Directories in the PATH environment variable

if test $# -lt 1
then
echo 'Usage: cmd_search pattern'
exit
fi

for i in `seq 1 $PATH_DIR_COUNT`
do
    ls `echo $PATH | cut -d ":" -f $i`|grep $1
done

To use the shell script, save it as cmd_search and

$chmod u+x cmd_search
$ ./cmd_search command_pattern

For example

$./cmd_search diff

Output

bzdiff
combinediff
dehtmldiff
diff
diff3
diff-jars
diffpp
diffstat
editdiff
espdiff
filterdiff
fixcvsdiff
flipdiff
gendiff
grepdiff
hdifftopam
interdiff
lsdiff
pamtohdiff
pdiff
rcsdiff
recountdiff
rediff
sdiff
sgmldiff
splitdiff
tiffdiff
unwrapdiff
vimdiff
zdiff

Shell Script to recursively list files

The shell script is equivalent to “ls -R”
To show what problem may come up in such types of programs, I have taken two versions of the program. In the first version, If file names do not contain spaces or newline character, everything works well. The second version takes care of this.
First Version

#!/bin/sh 
 # Shell script to find out all the files under a directory and 
 #its subdirectories. This takes into consideration only those files
 #or directories which do not have spaces or newlines in their names 

DIR="."

function list_files()
{
 if !(test -d $1) 
 then echo $1; return;
fi

cd $1
 echo; echo `pwd`:; #Display Directory name

 for i in *
 do
 if test -d $i #if dictionary
 then 
 list_files $i #recursively list files
cd ..
 else
 echo $i; #Display File name
fi

 done
}

if [ $# -eq 0 ]
then list_files .
exit 0
fi

for i in $*
do
 DIR=$1 
 list_files $DIR
 shift 1 #To read next directory/file name
done

Second Version

#!/bin/sh 
 # Shell script to find out all the files under a directory and 
 #its subdirectories. This also takes into consideration those files
 #or directories which have spaces or newlines in their names 

DIR="."

function list_files()
{
 if !(test -d "$1") 
 then echo $1; return;
fi

cd "$1"
 echo; echo `pwd`:; #Display Directory name

 for i in *
 do
 if test -d "$i" #if dictionary
 then 
 list_files "$i" #recursively list files
cd ..
 else
 echo "$i"; #Display File name
fi

 done
}

if [ $# -eq 0 ]
then list_files .
exit 0
fi

for i in $*
do
 DIR="$1"
 list_files "$DIR"
 shift 1 #To read next directory/file name
done

As can be clearly seen from the two versions, how quoting the variables has solved the problem of dealing of file name with spaces and newline characters. Quoting is useful to give such argument easily as input to programs which now take it as a single entity instead of taking it as two or more file names because of the whitespace characters.

 

Shell Script To recursively display (list) directories

This is a shell script to recursively display only the directories

#!/bin/sh
# Shell script to find out all the directories under a directory and
#its subdirectories. This also takes into consideration those files
#or directories which have spaces or newlines in their names

DIR="."

function list_files()
{
    if !(test -d "$1")
    then echo $1; return;
    fi

    cd "$1"
    echo; echo `pwd`:; #Display Directory name
for i in *
do
  if test -d "$i" #if dictionary
  then
     echo "$i"; #Display Directory name
  fi
done

for i in *
do
   if test -d "$i" #if dictionary
   then
      list_files "$i" #recursively list files
      cd ..
  fi
done

}

if [ $# -eq 0 ]
then list_files .
exit 0
fi

for i in $*
do
    DIR="$1"
    list_files "$DIR"
    shift 1 #To read next directory/file name
done