CFS264 ‐ Computer and Operating Systems Fundamentals II Page 1 of 6

Handout 10 - Bash Shell Commands and Perl Programming

_____________________________________________________________________________________
Following the steps introduced in Handout1 to log onto the host metrostate.mooo.com, then try the content
discussed below. Please create a directory lab10 and do the following exercises.

Bash Shell Commands and Perl Programming

 How to use “paste” and “join” commands:

 “paste” is used to merge lines of files;
 “join” can also be used to merge lines of files but the merge is based on a common field (key)
 Examples:
 Create files d1 and d2 as follows:
$ cat > d1
Apple $2.15
Pear $2.50
Orange $1.80
^D
$
$ cat > d2
2 $4.30
3 $7.50
2 $3.60
^D
$
 Files d1 and d2 can be merged with “paste” as follow:
$ paste d1 d2
Apple $2.15 2 $4.30
Pear $2.50 3 $7.50
Orange $1.80 2 $3.60
 If we use “join” to merge files d1 and d2 as follow, the files don’t merge because they don’t have a
common field:
$ join d1 d2
join: d2:3: is not sorted: 2 $3.60
join: d1:3: is not sorted: Orange $1.80
$
 Let’s add a common field to files d1 and d2, and re-name the modified files as d3 and d4 as follows:
$ cat > d3
10 Apple $2.15
20 Pear $2.50
30 Orange $1.80
^D
$
$ cat > d4
20 3 $7.50
30 2 $3.60
40 4 $5.40
^D
$
 Let’s use “join” command to merge d3 and d4 as follows, and the two files are merged based on their
common field (the first column of the files). As you can see the lines without the common data are
removed from the merged result:
$ join d3 d4
20 Pear $2.50 3 $7.50
30 Orange $1.80 2 $3.60
 In order to use “join” command, the files must be sorted based on their common field.

_____________________________________________________________________________________________
Please complete the Linux Lab in D2L
CFS264 ‐ Computer and Operating Systems Fundamentals II Page 2 of 6
Handout 10 - Bash Shell Commands and Perl Programming
_____________________________________________________________________________________
 The common field is not necessary to be the first column of a file. If the common field is not the first
column, option “-j” can be used to specify the common field, such as “join –j 2 file1 file2.”

 How to read a file line by line in a bash script using either a while-loop or a for-loop
 A while-loop example:
$ cat d1
Apple $2.15
Pear $2.50
Orange $1.80
$
$ cat > myread
#!/bin/bash
while read -r line
do
echo "$line"
done < $1
^D
$
$ chmod +x myread
$ ./myread d1
Apple $2.15
Pear $2.50
Orange $1.80
$
 A for-loop example:
$ cat d2
2 $4.30
3 $7.50
2 $3.60
$
$ cat > mysum
#!/bin/bash
echo ""
echo "Input numbers are:"
# the following line makes newlines the only separator
IFS=$'\n'
for mynext in `cut -d' ' -f1 $1`
do
echo "$mynext"
let t=t+mynext
done
echo "============="
echo "Total is $t"
echo ""
exit 0
^D
$
$ chmod +x mysum
$ ./mysum d2
Input numbers are:
2
3
2
=============
Total is 7
$

 A for-loop is often used to process a file generated on the fly in a bash shell script.

_____________________________________________________________________________________________
Please complete the Linux Lab in D2L
CFS264 ‐ Computer and Operating Systems Fundamentals II Page 3 of 6
Handout 10 - Bash Shell Commands and Perl Programming
_____________________________________________________________________________________

 Perl script programming

 To compute how much time a user spent on the system, a testing file is created as follow. Our goal here is
to determine how much time a user had spent on the system on Apr 16, 2020.
$ last -wi | grep cfs264sp25 | grep "Jan 13" | head -10 > loginfile4lab10
cfs264sp2561 pts/0 207.153.45.151 Mon Jan 13 20:41 - 20:56 (00:14)
cfs264sp2573 pts/4 66.41.51.45 Mon Jan 13 20:04 - 22:27 (02:23)
cfs264sp2555 pts/5 193.42.0.59 Mon Jan 13 20:00 - 23:12 (03:11)
cfs264sp2573 pts/4 66.41.51.45 Mon Jan 13 19:43 - 20:03 (00:20)
cfs264sp2541 pts/2 98.61.15.234 Mon Jan 13 18:52 - 22:52 (04:00)
cfs264sp2541 pts/2 98.61.15.234 Mon Jan 13 18:46 - 18:51 (00:05)
cfs264sp2555 pts/1 193.42.0.59 Mon Jan 13 18:19 - 20:45 (02:26)
cfs264sp2561 pts/0 207.153.45.151 Mon Jan 13 18:03 - 20:36 (02:33)
cfs264sp2541 pts/0 98.61.15.234 Mon Jan 13 14:24 - 17:36 (03:11)
cfs264sp2541 pts/0 98.61.15.234 Mon Jan 13 13:25 - 14:24 (00:58)
$
 Pick up the records for “cfs264sp2541” (or another user in the file) via a perl script as follow:
$ cat > lab10a.pl
#!/usr/bin/perl -w
open(FILE,$ARGV[1]) or die("Could not open the file $ARGV[1]");
while ( $line = <FILE> ) {
if ( $line =~ /$ARGV[0]/ ) {
print $line;
}
}
close(FILE);
^D
$
$ chmod +x lab10a.pl
$
$ ./lab10a.pl cfs264sp2541 loginfile4lab10
cfs264sp2541 pts/2 98.61.15.234 Mon Jan 13 18:52 - 22:52 (04:00)
cfs264sp2541 pts/2 98.61.15.234 Mon Jan 13 18:46 - 18:51 (00:05)
cfs264sp2541 pts/0 98.61.15.234 Mon Jan 13 14:24 - 17:36 (03:11)
cfs264sp2541 pts/0 98.61.15.234 Mon Jan 13 13:25 - 14:24 (00:58)
$
 Pick up the field for the login time from the above login records:
$ cat > lab10b.pl
#!/usr/bin/perl
open(FILE,$ARGV[1]) or die("Could not open the file $ARGV[1]");
while ( $line = <FILE> ) {
if ( $line =~ /$ARGV[0]/ ) {
print $line;
($record, $time) = split('\(', $line);
print "Time = ", $time;
}
}
close(FILE);
^D
$ chmod +x lab10b.pl
$
$ ./lab10b.pl cfs264sp2541 loginfile4lab10
cfs264sp2541 pts/2 98.61.15.234 Mon Jan 13 18:52 - 22:52 (04:00)
Time = 04:00)
cfs264sp2541 pts/2 98.61.15.234 Mon Jan 13 18:46 - 18:51 (00:05)
Time = 00:05)
cfs264sp2541 pts/0 98.61.15.234 Mon Jan 13 14:24 - 17:36 (03:11)
Time = 03:11)
cfs264sp2541 pts/0 98.61.15.234 Mon Jan 13 13:25 - 14:24 (00:58)

_____________________________________________________________________________________________
Please complete the Linux Lab in D2L
CFS264 ‐ Computer and Operating Systems Fundamentals II Page 4 of 6
Handout 10 - Bash Shell Commands and Perl Programming
_____________________________________________________________________________________
Time = 00:58)

 Try to get hours from variable “$time”:

$ cat > lab10c.pl
#!/usr/bin/perl
open(FILE,$ARGV[1]) or die("Could not open the file $ARGV[1]");
while ( $line = <FILE> ) {
if ( $line =~ /$ARGV[0]/ ) {
print $line;
($record, $time) = split('\(', $line);
print "Time = ", $time;
($h, $m1) = split(':', $time);
print "hour = ", $h, "\n";
print "minute = ", $m1;
}
}
close(FILE);
^D
$
$ chmod +x lab10c.pl
$
$ ./lab10c.pl cfs264sp2541 loginfile4lab10
cfs264sp2541 pts/2 98.61.15.234 Mon Jan 13 18:52 - 22:52 (04:00)
Time = 04:00)
hour = 04
minute = 00)
cfs264sp2541 pts/2 98.61.15.234 Mon Jan 13 18:46 - 18:51 (00:05)
Time = 00:05)
hour = 00
minute = 05)
cfs264sp2541 pts/0 98.61.15.234 Mon Jan 13 14:24 - 17:36 (03:11)
Time = 03:11)
hour = 03
minute = 11)
cfs264sp2541 pts/0 98.61.15.234 Mon Jan 13 13:25 - 14:24 (00:58)
Time = 00:58)
hour = 00
minute = 58)
 Try to get minutes from variable “$m1”:
$ cat > lab10d.pl
#!/usr/bin/perl
open(FILE,$ARGV[1]) or die("Could not open the file $ARGV[1]");
while ( $line = <FILE> ) {
if ( $line =~ /$ARGV[0]/ ) {
print $line;
($record, $time) = split('\(', $line);
print "Time = ", $time;
($h, $m1) = split(':', $time);
print "hour = ", $h, "\n";
($m) = split('\)', $m1);
print "minute = ", $m, "\n";
}
}
close(FILE);
^D
$
$ chmod +x lab10d.pl
$
$ ./lab10d.pl cfs264sp2541 loginfile4lab10
cfs264sp2541 pts/2 98.61.15.234 Mon Jan 13 18:52 - 22:52 (04:00)
Time = 04:00)

_____________________________________________________________________________________________
Please complete the Linux Lab in D2L
CFS264 ‐ Computer and Operating Systems Fundamentals II Page 5 of 6
Handout 10 - Bash Shell Commands and Perl Programming
_____________________________________________________________________________________
hour = 04
minute = 00
cfs264sp2541 pts/2 98.61.15.234 Mon Jan 13 18:46 - 18:51 (00:05)
Time = 00:05)
hour = 00
minute = 05
cfs264sp2541 pts/0 98.61.15.234 Mon Jan 13 14:24 - 17:36 (03:11)
Time = 03:11)
hour = 03
minute = 11
cfs264sp2541 pts/0 98.61.15.234 Mon Jan 13 13:25 - 14:24 (00:58)
Time = 00:58)
hour = 00
minute = 58
$

 How to write a shell script to solve the problem discussed above?

$ cat > lab10.sh
#!/bin/bash
# get hours
for mynext in `grep $1 $2 | cut -d'(' -f2 | cut -c1-2`
do
# for testing purpose
let mynext='10#'$mynext
echo -e "mynext=$mynext"
#
let h=h+mynext
# for testing purpose
echo -e "h=$h"
done
# get minutes
for mynext in `grep $1 $2 | cut -d'(' -f2 | cut -c4-5`
do
# for testing purpose
echo -e "mynext=$mynext"
#
let m=m+mynext
# for testing purpose
echo -e "m=$m"
echo "============="
#
done
# for testing purpose
echo -e "\nhours = $h"
echo -e "\nminutes = $m"
#
let hh=m/60
let mm=m%60
let h=h+hh
echo -e "\n $1 spent $h hours and $mm minutes on sp-cfsics server.\n"
exit 0
^D
$
$ chmod +x lab10.sh
$
$ ./lab10.sh cfs264sp2541 loginfile4lab10
mynext=4
h=4
mynext=0
h=4
mynext=3
h=7

_____________________________________________________________________________________________
Please complete the Linux Lab in D2L
CFS264 ‐ Computer and Operating Systems Fundamentals II Page 6 of 6
Handout 10 - Bash Shell Commands and Perl Programming
_____________________________________________________________________________________
mynext=0
h=7
mynext=00
m=0
=============
mynext=05
m=5
=============
mynext=11
m=16
=============
mynext=58
m=74
=============

hours = 7

minutes = 74

cfs264sp2541 spent 8 hours and 14 minutes on sp-cfsics server.

_____________________________________________________________________________________________
Please complete the Linux Lab in D2L