Working With Text

Pipes
Input/Output Redirection
File Statistics
Working With Text
Text and Text Files
● Text files only contain text.
● A large number of commands can be used to manipulate the content of text
files.
● The shell has features to manage the text output of commands.
○ Pipe - the output of a command can be sent to the input of another command.
○ Redirection – the output of a command can be saved in a file.
cat - Viewing Files in the Terminal
● The cat command can be used to:
⚫ Create text files
⚫ Display text file/s.
● The cat command shows the entire file content:
ivanovn@atlas:~$ cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 ubuntu-1604.sheridanc.on.ca ubuntu-1604
...

● Because the cat command shows the entire file conent, it is good for vewing
small files but not the large files.
Pagers - Viewing Files
● The pager comands can be used to view the content of a text file one
screenfull of data at a time.
● Two commonly used pager commands:
⚫ The more command is a very basic pager command. It only allows to view the content of a text
file one screenful at a time and, once you move to the next screen, you cannot go to the previous
screen.

⚫ The less command allows vewing of files content one screenful at a time and, it also allows
moving forward and backward between the screens as well as searching through the content of
text files.
more - Pager Movement Commands
● To view a file content with the more command, pass the filename as an argument:

ivanovn@atlas:~$ more /etc/passwd

● Use the spacebar to show the next screen of file content.

● Use the enter key to show the next line of the file content.
● Use the letter q (quit) to terminate the more command.
less - Pager Movement Commands
● To view a file with the less command, pass the filename as an argument:
ivanovn@atlas:~$ less /etc/passwd

● Use the letter H or Shift + H to display a help screen while viewing a file with
the less command.
● Below is the list of the most commonly used movement commands:
⚫ Spacebar – next window.

⚫ Letter b – previous window.

⚫ Enter key or letter e – next line.

⚫ Letter y – previous line.

⚫ Letter q – terminate less command

⚫ Letter h (or SHIFT+H) – show help

less - Pager Searching Commands
● We can search forward or backward from the current position while using the less command..

○ Press the foward slash /, enter the search keyword, and press the enter key to start searching
forward from the current position.

○ Press the question mark ?, enter the search keyword, and press the enter key to start
searching backward from the current position.
○ If there are no matches found, then the less command will say “Pattern not found (Press
RETURN)”.
○ If search returns more than one match, then the n (lower case letter n) key can be used to move
the next match and the N (Shift+N, upper case letter N) key can be used to to move to a previous
match.
Head and Tail
● The head and tail commands are used to display the first or last several
lines of a text file.
● By default, both head and tail commands show 10 lines from a text file.
● Use the -n option to specify how many lines to show:
⚫ head -n 3 /etc/passwd – this comand shows the first 3 lines of the
/etc/passwd file.
⚫ tail -n 3 /etc/passwd – this command shows the last 3 lines of the
/etc/passwd file.
Head and Tail (contd.)
● A negative value with the head command, will omit the specified number of
lines from the end of the file and print the rest. As an example: head -n -3
/etc/hosts – this command will not show the last 3 lines of the /etc/hosts file.
Not all implementations of the head will support this.
● If the positive value is given to the tail command, then the tail command will
display the file content starting at the specified line and continuing all the
way to the end: tail -n +10 /etc/hosts
Command Line Pipes
Command Line Pipes
● The pipe | character is used to send the output of one command to the input of another command:
cmd1 | cmd2 | cmd3 – output of cmd1 is sent to cmd2, the output of cmd2 is sent to cmd3, and
output of cmd3 is shown in the terminal.
● Example: viewing the output of a comand one sreenfull at a time:

ivanovn@atlas:~$ ls -l /etc | head

● Multiple pipes can be used.

ivanovn@atlas:~$ cat /etc/passwd | head -n 3 | tail -n 2 | sort

● In the above example, the output of the cat command is sent into the input of the head command.
The output of the head command is sent into the input of the tail command, and the output of the
tail command is sent into the input of the sort command. The sort command will send its output to
the terminal.
Input/Output Redirection
Input/Output Redirection
● Data is sent and recevied in streams.
STDOUT
● The standard output of a command will display to the screen:
sysadmin@localhost:~$ echo "Line 1"
Line 1

● Using the > character, the standard output of a command can be redirected to a file
instead:
sysadmin@localhost:~$ echo "Line 1" > example.txt

● The example.txt file contains the output of the echo command, which can be
viewed with the cat command:
sysadmin@localhost:~$ cat example.txt
Line 1
STDOUT
● Standard output is redirected with the > or >> characters.
● Both, > and >> will create a file if the file does not exist.
● The > will overwrite the content of an existing file.
● The >> will append the new content to the current content of a file.

ivanovn@atlas:~$ echo "Line of text" > file.txt

ivanovn@atlas:~$ cat file.txt
Line of text
ivanovn@atlas:~$ echo "Another line of text" > file.txt
ivanovn@atlas:~$ cat file.txt
Another line of text
ivanovn@atlas:~$ echo "One more line of text" >> file.txt
ivanovn@atlas:~$ cat file.txt
Another line of text
One more line of text
STDERR
● STDERR is redirected with the 2> and 2>> characters.
● Once again, the single > is to overwrite the content of an existing file and the >> is to
append new content to an existing file.

ivanovn@atlas:~$ ls blah
ls: cannot access blah: No such file or directory
ivanovn@atlas:~$ ls blah 2> ls.error.txt
sysadmin@localhost:~$ cat ls.error.txt
ls: cannot access blah: No such file or directory
Redirecting Multiple Streams
● The &> and &>> can be used to redirect both STDOUT and STDERR streams at the same
time.
● Once again, the > is to overwrite the content of an existing file and the >> is to append
the new content to an existing file.

ivanovn@atlas:~$ ls / blah
ls: cannot access 'blah': No such file or directory
/:
bin dev home initrd.img.old lib64 media opt root sbin srv tmp
var vmlinuz.old
ivanovn@atlas:~$ ls / blah &> ls.all.txt
ivanovn@atlas:~$ cat ls.all.txt
ls: cannot access 'blah': No such file or directory
/:
bin dev home initrd.img.old lib64 media opt root sbin srv tmp
var vmlinuz.old
Redirecting Multiple Streams
● The STDERR and STDOUT can be redirected to two different files.

ivanovn@atlas:~$ ls / blah
ls: cannot access 'blah': No such file or directory
/:
bin dev home initrd.img.old lib64 media opt root sbin srv tmp
var vmlinuz.old
ivanovn@atlas:~$ ls / blah > ls.txt 2> ls.err.txt
ivanovn@atlas:~$ cat ls.txt
/:
bin dev home initrd.img.old lib64 media opt root sbin srv tmp
var vmlinuz.old
ivanovn@atlas:~$ cat ls.err.txt
ls: cannot access 'blah': No such file or directory
STDIN
● Most commands will read data from a file when you specify the filename as an argument to the
command: cat /etc/passwd
● Some commands require you to use the input redirection < to specify that the input is to be read
from a file instead of the keyboard.
● For example, the tr command takes the input from the keyboard by default.

ivanovn@atlas:~$ tr 'a-z' 'A-Z'

hello there
HELLO THERE
^C

● Input redirection has to be used to use the tr command to translate the content of a file:
ivanovn@atlas:~$ tr 'a-z' 'A-Z' /etc/hosts
tr: extra operand ‘/etc/hosts’
Try 'tr --help' for more information.
ivanovn@atlas:~$ tr 'a-z' 'A-Z' < /etc/hosts
127.0.0.1 LOCALHOST
127.0.1.1 UBUNTU-1604.SHERIDANC.ON.CA UBUNTU-1604
...
NULL device
● The null device (/dev/null) is like a black hole – any content that is sent to the null
device disappears.

● Example: the find command is used to search the entire file system for a file or a
folder that has the word linux in its name. The errors are being sent to the null device.

ivanovn@atlas:~$ find / -name linux 2> /dev/null

/usr/src/linux-headers-5.4.0-147-generic/include/linux
/usr/src/linux-headers-5.4.0-147-generic/include/generated/uapi/linux
...
Viewing File Statistics
Viewing File Statistics
● The wc command can count the number of lines, words, and bytes (1 byte = 1
character) in a text file or in the output of a command

ivanovn@atlas:~$ wc /etc/passwd
38 60 2018 /etc/passwd

ivanovn@atlas:~$ cat /etc/passwd | wc

38 60 2018

● The wc command has options to view only:

○ Number of lines: the -l option.
○ Number of words: the -w option.
○ Number of bytes: the -c option.
○ Number of characters: the -m option