Improving Command-line Productivity

Writing Simple Bash Scripts

Specifying the Command Interpreter
The first line of a script begins with the notation '#!', commonly referred to as sh-bang or
she-bang, from the names of those two characters, sharp and bang

#!/bin/bash

Executing a Bash Shell Script

A completed shell script must be executable to run as an ordinary command. Use the chmod
command to add execute permission, possibly in conjunction with the chown command to
change the file ownership of the script. Grant execute permission only for intended users of
the script.
If you place the script in one of the directories listed in the shell's PATH environmental
variable, then you can invoke the shell script using the file name alone as with any other
command. The shell uses the first command it finds with that file name; avoid using existing
command names for your shell script file name. Alternatively, you can invoke a shell script
by entering a path name to the script on the command line. The which command, followed by
the file name of the executable script, displays the path name to the command that will be
executed.

[user@host ~]$ which hello

~/bin/hello
[user@host ~]$ echo $PATH
/
home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin
:/usr/local/sbin:/usr/sbin

Quoting Special Characters

A number of characters and words have special meaning to the Bash shell. However,
occasionally you will want to use these characters for their literal values, rather than for their
special meanings. To do this, use one of three tools to remove (or escape) the special
meaning: the backslash (\), single quotes (''), or double quotes ("").

[user@host ~]$ echo # not a comment #

[user@host ~]$ echo \# not a comment #

# not a comment
[user@host ~]$ echo \# not a comment \#

# not a comment #

[user@host ~]$ echo '# not a comment #'

# not a comment #

Providing Output From a Shell Script

The echo command displays arbitrary text by passing the text as an argument to the
command. By default, the text displays on standard output (STDOUT), but it can also be
directed to standard error (STDERR) using output redirection. In the following simple Bash
script, the echo command displays the message "Hello, world" to STDOUT.

[user@host ~]$ cat ~/bin/hello

#!/bin/bash

echo "Hello, world"

[user@host ~]$ hello

Hello, world

The echo command is widely used in shell scripts to display informational or error messages

[user@host ~]$ cat ~/bin/hello

#!/bin/bash

echo "Hello, world"

echo "ERROR: Houston, we have a problem." >&2

[user@host ~]$ hello 2> hello.log

Hello, world

[user@host ~]$ cat hello.log

ERROR: Houston, we have a problem.

vim firstscript.sh

#!/bin/bash

echo "This is my first bash script" > ~/output.txt

echo "" >> ~/output.txt

echo "#####################################################" >> ~/output.txt

sh firstscript.sh

Review the output file generated by the script.

[student@servera ~]$ cat output.txt

This is my first bash script

#####################################################

1. Append the following lines in bold to the firstscript.sh file.

2. #!/bin/bash
3. #
4. echo "This is my first bash script" > ~/output.txt
5. echo "" >> ~/output.txt
6. echo "#####################################################" >>
~/output.txt
7. echo "LIST BLOCK DEVICES" >> ~/output.txt
8. echo "" >> ~/output.txt
9. lsblk >> ~/output.txt
10. echo "" >> ~/output.txt
11. echo "#####################################################" >>
~/output.txt
12. echo "FILESYSTEM FREE SPACE STATUS" >> ~/output.txt
13. echo "" >> ~/output.txt
14. df -h >> ~/output.txt
15. echo "#####################################################" >>
~/output.txt

16. Make the firstscript.sh file executable using the chmod command.

[student@servera ~]$ chmod a+x firstscript.sh

 17. Execute the firstscript.sh script.

[student@servera ~]$ ./firstscript.sh

18. Review the output file generated by the script.

19. [student@servera ~]$ cat output.txt
20. This is my first bash script
21.
22. #####################################################
23. LIST BLOCK DEVICES
24.
25. NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
26. sr0 11:0 1 1024M 0 rom
27. vda 252:0 0 10G 0 disk
28. └─vda1 252:1 0 10G 0 part /
29. vdb 252:16 0 5G 0 disk
30.
31. #####################################################
32. FILESYSTEM FREE SPACE STATUS
33.
34. Filesystem Size Used Avail Use% Mounted on
35. devtmpfs 892M 0 892M 0% /dev
36. tmpfs 915M 0 915M 0% /dev/shm
37. tmpfs 915M 17M 899M 2% /run
38. tmpfs 915M 0 915M 0% /sys/fs/cgroup
39. /dev/vda1 10G 1.5G 8.6G 15% /
40. tmpfs 183M 0 183M 0% /run/user/1000

#####################################################

Using Loops to Iterate Commands

for VARIABLE in LIST; do

COMMAND VARIABLE

Done

[user@host ~]$ for HOST in host1 host2 host3; do echo $HOST; done

[user@host ~]$ for HOST in host{1,2,3}; do echo $HOST; done

[user@host ~]$ for HOST in host{1..3}; do echo $HOST; done

host1 host2 host3

[user@host ~]$ for EVEN in $(seq 2 2 10); do

echo "$EVEN";
done

2 4 6 8 10

Using Exit Codes Within a Script

After a script has processed all of its contents, it exits to the process that called it. However,
there may be times when it is desirable to exit a script before it finishes, such as when an
error condition is encountered

The exit command can be executed with an optional integer argument between 0 and 255,
which represents an exit code. An exit code is a code that is returned after a process has
completed. An exit code value of 0 represents no error. All other nonzero values indicate an
error exit code. You can use different nonzero values to differentiate between different types
of errors encountered. This exit code is passed back to the parent process, which stores it in
the ? variable and can be accessed with $? as demonstrated in the following examples.

This exit code is passed back to the parent process, which stores it in the ? variable and can
be accessed with $? as demonstrated in the following examples.

[user@host bin]$ cat hello

#!/bin/bash

echo "Hello, world"

exit 0

[user@host bin]$ ./hello

Hello, world

[user@host bin]$ echo $?

Testing Script Inputs

Like all commands, the test command produces an exit code upon completion, which is
stored as the value $?. To see the conclusion of a test, display the value of $? immediately
following the execution of the test command. Again, an exit status value of 0 indicates the
test succeeded, and nonzero values indicate the test failed.

The following examples demonstrate the use of the test command using Bash's numeric
comparison operators.
[user@host ~]$ test 1 -gt 0 ; echo $?
0
[user@host ~]$ test 0 -gt 1 ; echo $?
1

Conditional Structures
Using the if/then Construct

if <CONDITION>; then

<STATEMENT>

...

<STATEMENT>

Fi

The following code section demonstrates the use of an if/then construct to start the psacct
service if it is not active.

[user@host ~]$ systemctl is-active psacct > /dev/null 2>&1

[user@host ~]$ if [ $? -ne 0 ]; then
> sudo systemctl start psacct
> fi

Using the if/then/else Construct

if <CONDITION>; then

<STATEMENT>

...

<STATEMENT>

else

<STATEMENT>
...

<STATEMENT>

Fi

Use the ssh and hostname commands to print the host name of servera and serverb to
standard output.

[student@workstation ~]$ ssh student@servera hostname

servera.lab.example.com
[student@workstation ~]$ ssh student@serverb hostname
serverb.lab.example.com

Create a for loop to perform the same task more efficiently.

[student@workstation ~]$ for HOST in servera serverb

do
ssh student@${HOST} hostname
done
servera.lab.example.com
serverb.lab.example.com

Create a shell script to execute the same for loop.

1. Create the /home/student/bin directory to contain the shell script.

[student@workstation ~]$ mkdir ~/bin

2. Verify that the newly created directory is in your PATH environmental variable.
3. [student@workstation ~]$ echo $PATH
4. /
home/student/.local/bin:/home/student/bin::/usr/local/bin:/usr/bin:/usr/loc
al/sbin:/usr/sbin

5. Create a shell script at /home/student/bin/printhostname.sh to perform the for

loop. Use the cat command to verify the content of printhostname.sh.
 6. [student@workstation ~]$ vim ~/bin/printhostname.sh
7. [student@workstation ~]$ cat ~/bin/printhostname.sh
8. #!/bin/bash
9. #Execute for loop to print server hostname.
10. for HOST in servera serverb
11. do
12. ssh student@${HOST} hostname
13. done
14. exit 0

15. Ensure the newly created script is executable.

[student@workstation ~]$ chmod +x ~/bin/printhostname.sh

1. Run the script from your home directory.

2. [student@workstation ~]$ printhostname.sh
3. servera.lab.example.com
4. serverb.lab.example.com

5. Verify that the exit code of your script is 0.

6. [student@workstation ~]$ echo $?

Matching Text in Command Output with Regular

Expressions
Regular Expressions

Option Description
. The period (.) matches any single character.
? The preceding item is optional and will be matched at most once.
* The preceding item will be matched zero or more times.
+ The preceding item will be matched one or more times.
Option Description
{n} The preceding item is matched exactly n times.
{n,} The preceding item is matched n or more times.
{,m} The preceding item is matched at most m times.
{n,m} The preceding item is matched at least n times, but not more than m times.
Alphanumeric characters: '[:alpha:]' and '[:digit:]'; in the 'C' locale and ASCII
[:alnum:]
character encoding, this is the same as '[0-9A-Za-z]'.
Alphabetic characters: '[:lower:]' and '[:upper:]'; in the 'C' locale and ASCII
[:alpha:]
character encoding, this is the same as '[A-Za-z]'.
[:blank:] Blank characters: space and tab.
Control characters. In ASCII, these characters have octal codes 000 through 037,
[:cntrl:] and 177 (DEL). In other character sets, these are the equivalent characters, if
any.
[:digit:] Digits: 0 1 2 3 4 5 6 7 8 9.
[:graph:] Graphical characters: '[:alnum:]' and '[:punct:]'.
Lower-case letters; in the 'C' locale and ASCII character encoding, this is a b c d
[:lower:]
e f g h i j k l m n o p q r s t u v w x y z.
[:print:] Printable characters: '[:alnum:]', '[:punct:]', and space.
Punctuation characters; in the 'C' locale and ASCII character encoding, this is! "
[:punct:] # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ' { | } ~. In other character sets, these
are the equivalent characters, if any.
Space characters: in the 'C' locale, this is tab, newline, vertical tab, form
[:space:]
feed,carriage return, and space.
Upper-case letters: in the 'C' locale and ASCII character encoding, this is A B C
[:upper:]
D E F G H I J K L M N O P Q R S T U V W X Y Z.
[:xdigit:] Hexadecimal digits: 0 1 2 3 4 5 6 7 8 9 A B C D E F a b c d e f.

Matching Regular Expressions with Grep

The grep command provides a regular expression and a file on which the regular expression
should be matched.

[user@host ~]$ grep '^computer' /usr/share/dict/words

The grep command can be used in conjunction with other commands using a pipe operator
(|). For example:

[root@host ~]# ps aux | grep chrony

Table 1.2. Table of Common grep Options

Option Function
Use the regular expression provided but do not enforce case sensitivity (run
-i
case-insensitive).
-v Only display lines that do not contain matches to the regular expression.
Apply the search for data matching the regular expression recursively to a group
-r
of files or directories.
-A
NUMBER
Display NUMBER of lines after the regular expression match.

-B
NUMBER
Display NUMBER of lines before the regular expression match.

With multiple -e options used, multiple regular expressions can be supplied and
-e
will be used with a logical OR.

The following example prints the matched line, along with the 3 lines after it.

$ grep -A 3 -i "example" demo_text

$ grep -B 2 "single WORD" demo_text (matched line along with 3 lines before)

$ grep -C 2 "Example" demo_text Display N lines around match

Options Description
-c : This prints only a count of the lines that match a pattern
-h : Display the matched lines, but do not display the filenames.
-i : Ignores, case for matching
-l : Displays list of a filenames only.
-n : Display the matched lines and their line numbers.
-v : This prints out all the lines that do not matches the pattern
-e exp : Specifies expression with this option. Can use multiple times.

-f file : Takes patterns from file, one per line.

 grep -f pattern.txt tes.txt

-w : Match whole word

-o : display only the matched pattern (instead of displaying entire

line which has matched string by default)

^ : matching the lines that start with a string grep “^unix” tes.txt

$ : matching the lines that end with a string grep “os$” tes.txt

grep -i serverroot /etc/httpd/conf/httpd.conf

In cases where you know what you are not looking for, the -v option is very useful. The -v
option only displays lines that do not match the regular expression. In the following example,
all lines, regardless of case, that do not contain the regular expression server are returned.

grep -v -i server /etc/hosts

To look at a file without being distracted by comment lines use the -v option. In the following
example, the regular expression matches all lines that begin with a # or ; (typical characters
that indicate the line will be interpreted as a comment). Those lines are then omitted from the
output.

[user@host ~]$ grep -v '^[#;]' /etc/ethertypes

The grep command with the -e option allows you to search for more than one regular
expression at a time. The following example, using a combination of less and grep, locates
all occurrences of pam_unix, user root and Accepted publickey in the
/var/log/secure log file.

[root@host ~]# cat /var/log/secure | grep -e 'pam_unix' \

-e 'user root' -e 'Accepted publickey' | less

To search for text in a file opened using vim or less, use the slash character (/) and type the
pattern to find. Press Enter to start the search. Press N to find the next match

[root@host ~]# vim /var/log/boot.log

/Daemons
[root@host ~]# less /var/log/messages

/device

The postfix package was installed today by the start script. Use the grep command to find
the GID and UID for the postfix and postdrop groups and users. To reduce the output of
the grep command, display all logs from a specific Start Time.

1. Use the date command to determine the current time.

[root@servera ~]# date

Fri Mar 22 08:23:56 CET 2019

2. Use the grep command with the date, start time, and GID options to find the postfix
and postdrop user's GID and UID. The lab set-up script ran a few minutes before the
current time. Take this into consideration when searching the /var/log/secure log
file.

[root@servera ~]# grep '^Mar 22 08:2.*GID' /var/log/secure

Modify your regular expression to locate the first two messages in the /var/log/maillog
file. Notice that in this search you are not using the caret character (^) because you are not
searching for the first character in a line.

[root@servera ~]# grep 'postfix' /var/log/maillog | head -n 2

You are required to find the name of the queue directory for the Postfix server. Search the
/etc/postfix/main.cf configuration file for all information about queues. Use the -i option
to ignore case distinctions.

[root@servera ~]# grep -i 'queue' /etc/postfix/main.cf

Confirm that postfix is writing messages to /var/log/messages. Use the less command then
the slash character (/) to search the file. Press n to move to the next entry that matches the
search. Use the q key to quit the less command.

[root@servera ~]# less /var/log/messages

/Postfix

Use the ps aux command to confirm that the postfix server is currently running. Reduce the
output of ps aux by combining it with the grep command.

[root@servera ~]# ps aux | grep postfix

Confirm that the qmgr, cleanup, and pickup queues are correctly configured. Use the grep
command with the -e option to match multiple entries in the same file. The configuration file
is /etc/postfix/master.cf

[root@servera ~]# grep -e qmgr -e pickup -e cleanup /etc/postfix/master.cf

Edit your newly created script file to comply with the following requested information from
the servera and serverb hosts. The systems are configured to use SSH keys for
authentication; a password is not required.

Command or file Content requested

hostname -f Get all the output.
echo "#####" Get all the output.
lscpu Get only the lines that start with the string CPU.
echo "#####" Get all the output.
/
etc/selinux/config
Ignore empty lines. Ignore lines starting with #.

echo "#####" Get all the output.

/var/log/secure Get all "Failed password" entries.
echo "#####" Get all the output.
#!/bin/bash

USR='student'

OUT='/home/student/output'

for SRV in servera serverb

do

ssh ${USR}@${SRV} "hostname -f" > ${OUT}-${SRV}

echo "#####" >> ${OUT}-${SRV}

ssh ${USR}@${SRV} "lscpu | grep '^CPU'" >> ${OUT}-${SRV}

echo "#####" >> ${OUT}-${SRV}

ssh ${USR}@${SRV} "grep -v '^$' /etc/selinux/config|grep -v '^#'" >> ${OUT}-${SRV}

echo "#####" >> ${OUT}-${SRV}

ssh ${USR}@${SRV} "sudo grep 'Failed password' /var/log/secure" >> ${OUT}-${SRV}

echo "#####" >> ${OUT}-${SRV}

done