Redirection, pipeline and filter

● Standard input, output and error

● Redirection
● Pipeline
● Filter utilities

Version 1.0 linuxslides.blogspot.com

Standard input, output, error

standard input standard output

process
(stdin) (stdout)

stdin is an input for process is a running stdout is an output

a process which is linux command line that shows up on
typed from keyboard the screen

standard error
(stderr)

stderr is an error
message that shows
up on the screen

Version 1.0 linuxslides.blogspot.com

Standar output (stdout)

Let's say we executed a command:

$ ls process
Maildir public_html data.txt stdout

$ whoami proses

joni stdout

$ cat /etc/passwd process

root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh stdout
sys:x:3:3:sys:/dev:/bin/sh
...
Version 1.0 linuxslides.blogspot.com
Standar error (stderr)

Let's say we executed another command:

$ hello process
bash: hello: command not found stderr

$ pure-ftpd process

The program 'pure-ftpd' is currently not

installed. You can install it by typing:
sudo apt-get install pure-ftpd stderr
bash: pure-ftpd: command not found

Version 1.0 linuxslides.blogspot.com

Standar input (stdin)

Let's say we executed another command:

$ cat process
Hallo world... stdin (input from keyboard)

Hallo world... stdout (output shows up on the screen)

Ctrl+D EOF (end of file)

Version 1.0 linuxslides.blogspot.com

Redirection

normal redirection symbol

stdin from keyboard from file <

stdout to screen to file > or 1>

stderr to screen to file 2>

Version 1.0 linuxslides.blogspot.com

Redirection stdin
Without redirection:
$ cat process
Hallo world... stdin (input from keyboard)
Hallo world... stdout (output shows up on the screen)

With redirection stdin (input from file, not from keyboard):

process redirection stdin

$ cat < /etc/passwd

root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh stdout
sys:x:3:3:sys:/dev:/bin/sh
...
The command above is the same with: $ cat /etc/passwd
Version 1.0 linuxslides.blogspot.com
Redirection stdout
Redirection stdin (input from file, not from keyboard):
process redirection stdin

$ cat < /etc/passwd

root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh stdout
bin:x:2:2:bin:/bin:/bin/sh

Redirection stdout (output to file, not to screen):

process redirection stdout
QUIZ
$ cat > ~/hallo.txt Try following command:
Hallo world... $ cat >> ~/hallo.txt
stdin (type some lines and press Ctrl+D)
How're you? Bye.. :)
Display the content. What happen?
Ctrl+D EOF (end of file)
To display the content of file, type: $ cat halo.txt
Version 1.0 linuxslides.blogspot.com
Redirection stdout usage
Redirection stdout generally used for capturing and
documenting command's results:

$ ls -l > result-ls.txt
$ cat result-ls.txt

$ ifconfig > ip-address.txt

$ cat ip-address.txt

QUIZ
Is the command below valid? What it uses for?
$ cat < hallo.txt > hallo2.txt

Version 1.0 linuxslides.blogspot.com

Homework

Using redirection technique, assemble commands that will

create a file which is recursively increasing its own size
infinitely. It won't stop until terminated by user, or the hard disk
is full.

Version 1.0 linuxslides.blogspot.com

Redirection stderr

Without redirection stderr:

$ hallo process
bash: hallo: command not found stderr

With redirection stderr:

redirection stderr

$ hallo 2> error.txt

Notice that the error message didn't show up on the screen,
but redirected and saved in file named error.txt. To see:
$ cat error.txt
bash: hallo: command not found

Version 1.0 linuxslides.blogspot.com

Redirection stdout and stderr

$ find / -name xyz > result.txt 2>&1

The meaning of command above is:

● Search results will be redirected in file named result.txt

● If any error messages show up, it will also redirected to the same

file result.txt

Version 1.0 linuxslides.blogspot.com

Pipeline
Pipeline is a technique for “flowing” a command's output as
an input of another command.
process1 output1 = input2 process2 output2 ...

In command line:

$ process1 | process2 | process3 | ...

Command that able to accept an input and

produce an output at the same time called filter
Pipeline example:
$ cat /var/log/messages | more
$ ls /etc | grep passwd
$ dpkg -l | grep apache
Version 1.0 linuxslides.blogspot.com
Filter utilities
● more ● paste
● less ● split
● grep ● sort
● head ● tr
● tail ● wc
● cut ● pr

Version 1.0 linuxslides.blogspot.com

more
more is a filter for paging through text one screenful at a time.
For navigations use enter to scroll per line, or space to scroll
per screen.

$ cat /var/log/messages
$ more /var/log/messages
$ cat /var/log/messages | more

Version 1.0 linuxslides.blogspot.com

less
less is like more, but besides “standard” enter and space
navigations, less is also have additional navigations like up
arrow, down arrow, Page Up, and Page Down.

$ cat /var/log/messages
$ less /var/log/messages
$ cat /var/log/messages | less

Version 1.0 linuxslides.blogspot.com

grep
grep searches lines containing a match to the given pattern. For
example if we want to search all lines containing “root”:
$ grep root /etc/passwd
$ cat /etc/passwd | grep root

To search all lines, NOT containing “root”:

$ grep -v root /etc/passwd

To count how many lines matched:

$ grep -c root /etc/passwd

Search with case sensitive:

$ grep -i ROOT /etc/passwd

Version 1.0 linuxslides.blogspot.com

head and tail
head: Print the first 10 lines of each FILE to standard output:
$ head -5 /etc/passwd
$ cat /etc/passwd | head -2
$ cat -n /etc/passwd | head -3

tail: Print the last 10 lines of each FILE to standard output:

$ tail -5 /etc/passwd
$ cat /etc/passwd | head -2
$ cat -n /etc/passwd | head -3

QUIZ
Using head, tail and pipeline,
assemble commands for
printing only the third row.
Version 1.0 linuxslides.blogspot.com
cut
cut is an utility to print selected column with a specific delimiter.
For example /etc/passwd filled with columns separated by “ : ”.
How many columns are in /etc/passwd?
$ cat /etc/passwd

To print the first column:

$ cut -d: -f1 /etc/passwd

To print the first and the fifth column:

$ cut -d: -f1,5 /etc/passwd

To print the first to the fifth column:

$ cut -d: -f1-5 /etc/passwd

Version 1.0 linuxslides.blogspot.com

paste
paste is an utility for merging columns and print it to standard
output.

Take the first column and save it:

$ cut -d: -f1 /etc/passwd > col1.txt

Take the second column and save it:

$ cut -d: -f2 /etc/passwd > col2.txt

Merge the two files above with paste:

$ paste col1.txt col2.txt

Merge with delimiter “ : “

$ paste -d: col1.txt col2.txt
Version 1.0 linuxslides.blogspot.com
split
split is an utility for splitting a file into pieces with certain size.

Split /etc/passwd into files each contain 10 rows:

$ cat -n /etc/passwd | split -10
$ split -10 /etc/passwd

The results are files with prefix “xa”: xaa, xab, xac:
$ cat xaa

If we want different prefix like “xxx”:

$ split -10 /etc/passwd xxx

To merge again back to original:

$ cat xxx* > passwd.txt
Version 1.0 linuxslides.blogspot.com
sort
sort is a utility for sorting text.

Take field user name from /etc/passwd:

$ cat /etc/passwd | cut -d: -f1

Then sort ascending:

$ cat /etc/passwd | cut -d: -f1 | sort

Then sort descending:

$ cat /etc/passwd | cut -d: -f1 | sort -r

To sort numerically (third field: uid):

$ cat /etc/passwd | cut -d: -f3 | sort -n

Version 1.0 linuxslides.blogspot.com

tr
tr (translate) is an utility for translate or delete characters.

Translate character “a” into “o”:

$ tr “a” “o”
Hallo
Hollo
Ctrl+D

Translate file's contents into capital:

$ cat /etc/passwd | tr “a-z” “A-Z”

Version 1.0 linuxslides.blogspot.com

wc
wc (word count) is an utility for printing newline, word, and byte
counts for each file.

To count lines from /etc/passwd:

$ cat /etc/passwd | wc -l

To count words from /etc/passwd:

$ cat /etc/passwd | wc -w

To count characters from /etc/passwd:

$ cat /etc/passwd | wc -c

To count files in your home directory:

$ ls ~ | wc -l

Version 1.0 linuxslides.blogspot.com

pr
pr is a utility for converting text files for printing.

To format /etc/passwd:
$ cat /etc/passwd | pr

To format /etc/passwd with header:

$ cat /etc/passwd | pr -h “File /etc/passwd”

QUIZ: Try to format /var/log/messages for printing and

redirect it to ~/messages.txt

Version 1.0 linuxslides.blogspot.com