Listing files (ls)

If you want to see the list your UNIX of files on or Linux system, use the ‘ls’ command. It shows the
files /directories in your current directory. Note:

 Directories are denoted in blue color.

 Files are denoted in white.

 You will find similar color schemes in different flavors of Linux.

Suppose, your “Music” folder has following sub-directories and files.

You can use ‘ls -R’ to shows all the files not only in directories but also subdirectories

NOTE: These Linux

basics commands

are case-sensitive.

If you enter,

“ls – r” you will get

an error.

‘ls -al’ gives detailed information of the files. The command provides information in a columnar
format. The columns contain the following information:

1st Column File type and access permissions

2nd Column # of HardLinks to the File

3rd Column Owner and the creator of the file

4th Column Group of the owner

5th Column File size in Bytes

6th Column Date and Time

7th Column Directory or File name

Let’s see an example –

Listing Hidden Files

Hidden items in UNIX/Linux begin with –at the start, of the file or directory. Any Directory/file
starting with a ‘.’ will not be seen unless you request for it. To view
hidden files, use the command.

ls -a

Creating & Viewing Files

The ‘cat’ server command is used to display text files. It can also be used for copying, combining and
creating new text files. Let’s see how it works.

To create a new file, use the command

1. cat > filename

2. Add content

3. Press ‘ctrl + d’ to return to command prompt.

Hhow to create and view files in Linux/Unix

To view a file, use the command –

 cat filename

Let’s see the file we just created –

Let’s see another file sample2

The syntax to combine 2 files is –

cat file1 file2 > newfilename

Let’s combine sample 1 and sample 2.

As soon as you insert this command and hit enter, the files are concatenated, but you do not see a
result. This is because Bash Shell (Terminal) is silent type. Shell Commands will never give you a
confirmation message like “OK” or “Command Successfully Executed”. It will only show a message
when something goes wrong or when an error has occurred.

To view the new combo file “sample” use the command

cat sample

Note: Only text files can be displayed and combined using this command.

Deleting Files

The ‘rm’ command removes files from the system without confirmation. To remove a file use syntax
–

rm filename

How to delete files using Linux/Unix Commands

Moving and Re-naming files

To move a file, use the command.

mv filename new_file_location

Suppose we want to move the file “sample2” to location /home/guru99/Documents.Executing the

command

mv sample2 /home/guru99/Documents

mv command needs super user permission. Currently, we are executing the command as a standard
user. Hence we get the above error. To overcome the error use command.

sudo command_you_want_to_execute

Sudo program allows regular users to run programs with the security privileges of the superuser or
root.

Sudo command will ask for password authentication. Though, you do not need to know the root
password. You can supply your own password. After authentication, the system will invoke the
requested command.

Sudo maintains a log of each command run. System administrators can trackback the person
responsible for undesirable changes in the system.

guru99@VirtualBox:~$ sudo mv sample2 /home/quru99/Documents

[sudo] password for guru99: ****

guru99@VirtualBox:~$

For renaming file:

mv filename newfilename

NOTE: By default, the password you entered for sudo is retained for 15 minutes per terminal. This
eliminates the need of entering the password time and again.

You only need root/sudo privileges, only if the command involves files or directories not owned by
the user or group running the commands

Directory Manipulations

Directory Manipulation in Linux/Unix

Enough with File manipulations! Let’s learn some directory manipulation Linux
commands with examples and syntax.
Creating Directories

Directories can be created on a Linux operating system using the following command

mkdir directoryname

This command will create a subdirectory in your present working directory, which is usually your
“Home Directory”.

For example,

mkdir mydirectory

If you want to create a directory in a different location other than ‘Home directory’, you could use
the following command –

mkdir

For example:

mkdir /tmp/MUSIC

will create a directory ‘Music’ under ‘/tmp’ directory

You can also create more than one directory at a time.

Removing Directories

To remove a directory, use the command –

rmdir directoryname

Example

rmdir mydirectory

will delete the directory mydirectory

Tip: Ensure that there is no file / sub-directory under the directory that you want to delete. Delete
the files/sub-directory first before deleting the parent directory.

Renaming Directory

The ‘mv’ (move) command (covered earlier) can also be used for renaming directories. Use the
below-given format:

mv directoryname newdirectoryname

Let us try it:

How to rename a directory using Linux/Unix Commands

Other Important Commands

The ‘Man’ command

Man stands for manual which is a reference book of a Linux operating system. It is similar to HELP
file found in popular software.

To get help on any command that you do not understand, you can type

man

The terminal would open the manual page for that command.

For an example, if we type man man and hit enter; terminal would give us information on man
command
The History Command

History command shows all the basic commands in Linux that you have used in the past for the
current terminal session. This can help you refer to the old commands you have entered and re-used
them in your operations again.

The clear command

This command clears all the clutter on the terminal and gives you a clean window to work on, just
like when you launch the terminal.
Pasting commands into the terminal

Many times you would have to type in long commands on the Terminal. Well, it can be annoying at
times, and if you want to avoid such a situation then copy, pasting the commands can come to
rescue.

For copying, the text from a source, you would use Ctrl + c, but for pasting it on the Terminal, you
need to use Ctrl + Shift + p. You can also try Shift + Insert or select Edit>Paste on the menu

NOTE: With Linux upgrades, these shortcuts keep changing. You can set your preferred shortcuts via
Terminal> Edit> Keyboard Shortcuts.

Printing in Unix/Linux

How to print a file using Linux/Unix commands

Let’s try out some Linux basic commands with examples that can print files in a format you want.
What more, your original file does not get affected at all by the formatting that you do. Let us learn
about these commands and their use.

pr’ command
This command helps in formatting the file for printing on the
terminal. There are many Linux terminal commands available with
this command which help in making desired format changes on
file. The most used ‘pr’ Unix commands with examples are listed
below.
Option Function
-x Divides the data into ‘x’ columns
-h “header” Assigns “header” value as the report header
-t Does not print the header and top/bottom margins
-d Double spaces the output file
-n Denotes all line with numbers
-l page length Defines the lines (page length) in a page. Default is 56
-o margin Formats the page by the margin number
Let us try some of the options and study their effects.
Dividing data into columns
‘Tools’ is a file (shown below).

We want its content to be arranged in three columns. The syntax

for the same would be:
pr -x Filename
The ‘-x’ option with the ‘pr’ command divides the data into x
columns.

Assigning a header
The syntax is:
pr -h "Header" Filename
The ‘-h’ options assigns “header” value as the report header.

As shown above, we have arranged the file in 3 columns and

assigned a header
Denoting all lines with numbers
The syntax is:
pr -n Filename
This command denotes all the lines in the file with numbers.
These are some of the ‘pr’ command options that you can use to
modify the file format.
Printing a file
Once you are done with the formatting, and it is time for you to
get a hard copy of the file, you need to use the following
command:
lp Filename
or
lpr Filename
In case you want to print multiple copies of the file, you can use
the number modifier.

In case you have multiple printers configured, you can specify a

particular printer using the Printer modifier
Installing Software
In windows, the installation of a program is done by running the
setup.exe file. The installation bundle contains the program as
well various dependent components required to run the program
correctly.

Using Linux/Unix basic commands, installation files in Linux are

distributed as packages. But the package contains only the
program itself. Any dependent components will have to be
installed separately which are usually available as packages
themselves.

You can use the apt commands to install or remove a package.

Let’s update all the installed packages in our system using
command –
sudo apt-get update
The easy and popular way to install programs on Ubuntu is by
using the Software center as most of the software packages are
available on it and it is far more secure than the files downloaded
from the internet.

Linux Command List

Below is a Cheat Sheet of Linux/ Unix basic commands with
examples that we have learned in this Linux commands tutorial
 Command Description
Lists all files and directories in the
ls
present working directory
ls – R Lists files in sub-directories as well
ls – a Lists hidden files as well
Lists files and directories with detailed
ls – al information like permissions, size,
owner, etc.
cat > filename Creates a new file
cat filename Displays the file content
Joins two files (file1, file2) and stores
cat file1 file2 > file3
the output in a new file (file3)
mv file “new file path” Moves the files to the new location
mv filename new_file_name Renames the file to a new filename
Allows regular users to run programs
sudo with the security privileges of the
superuser or root
rm filename Deletes a file
man Gives help information on a command
Gives a list of all past basic Linux
history commands list typed in the current
terminal session
clear Clears the terminal
Creates a new directory in the present
mkdir directoryname working directory or a at the specified
path
rmdir Deletes a directory
 Command Description
mv Renames a directory
pr -x Divides the file into x columns

pr -h Assigns a header to the file

pr -n Denotes the file with Line Numbers

lp -nc
Prints “c” copies of the File
lpr c

lp -d
Specifies name of the printer
lpr -P
Command used to install and update
apt-get
packages

mail -s ‘subject’ -c ‘cc-address’

Command to send email
-b ‘bcc-address’ ‘to-address’

mail -s “Subject” to-address < Command to send email with

Filename attachment
Using Shell Redirection and Piping
The Unix/Linux philosophy revolves around the
concept of programs as building blocks—each one
intended to do one job and do it well. Redirection
lets you connect these commands to files, and
piping enables you to plug commands together like a
childs toy.

How Do I Do That?
Each command has three numbered file
descriptors that are opened automatically:
standard input (stdin, file descriptor 0)
The normal input to the program
standard output (stdout, file descriptor 1)
The normal output from the program
standard error (stderr, file descriptor 2)
Error messages from the program
By default, these file descriptors are connected to
the terminal, if one is available, so standard input
comes from the terminal keyboard, and standard
output and standard error go to the terminal screen.
Programs may open any other connections they need
to read or write files, communicate with other local
programs, or communicate with programs over the
network.
Tip

Connections to the graphical user interface are created by opening a

network connection from the program (client) to the X Window server.
This is distinct from the three standard file descriptors.
Redirection

To redirect the output of a program to a file, use

the greater-than (>) symbol followed by the name of
the file:

$ cal 7 2006
July 2006
Su Mo Tu We Th Fr Sa
1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31
$ cal 7 2006 >

month.txt

$ cat

month.txt

July 2006
Su Mo Tu We Th Fr Sa
1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31

When you redirect output with >, the previous

contents of the file are overwritten. To append (add)
to the file, use >>:

$ cal
 3 2009

>>

month.txt

$ cat

month.txt

July 2006
Su Mo Tu We Th Fr Sa
1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31
March 2009
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31

Error messages are not sent to standard output, so

you can still see the messages even when standard
output is redirected:

$ cal

17 2009

>
 month.txt

cal: illegal month value: use 1-12

To redirect error messages, place the file descriptor

number (2) in front of the redirection symbol (> or >>):

$ cal

17 2009 2

>errors
$ cat

errors

cal: illegal month value: use 1-12

You can redirect both standard output and standard

error:

$ cal

17 2009

>

month.txt 2

>errors

To redirect the input of a command, use the less-

than sign (<) followed by the filename containing the
data you wish to use as the input:

$ echo "
 2^8

" >

problem

$ bc <

problem

256

bc is a calculator program. The first command

places a numeric expression in the file problem; the
second line starts bc, using problem as the input.
The output from bc is the solution of the
expression: 256.
Of course, you can redirect both input and output:

$ bc <

problem

>

result

Piping

A pipe is a mechanism used to connect

the standard output of one program to the standard
input of another program. To create a pipe, insert
the vertical-bar (|) symbol between the two
commands:

$ mount
/dev/mapper/main-root on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/hdc2 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
/dev/mapper/main-home on /home type ext3 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
/dev/sdb on /media/disk type vfat (rw,noexec,nosuid,nodev,shortname=winnt,uid=503)
$ mount | grep

/dev/mapper

/dev/mapper/main-root on / type ext3 (rw)

/dev/mapper/main-home on /home type ext3 (rw)

In this example, the output of the mount command

is used as the input to the grep command, which
outputs only lines that match the specified pattern.
A group of commands connected together with pipe
symbols is known as a pipeline. You can extend a
pipeline by connecting additional commands:

$ mount | grep

/dev/mapper

| sort
/dev/mapper/main-home on /home type ext3 (rw)
/dev/mapper/main-root on / type ext3 (rw)

The input to a pipeline and the output from a pipeline

may be redirected:
$ cut -d: -f1 </etc/passwd|sort|head >

output

$ cat

output

adm
apache
avahi
beaglidx
bin
chip
chris
daemon
dbus
distcache

However, it’s essential that the input redirect take

place at the start of the pipeline (at the command on
the left) and that the output redirection take place at
the end (at the command on the right). Consider
this wrong example:

$ cut -d: -f1 </etc/passwd|sort >

output

|head

In this case, it’s unclear whether the standard

output of sort should be placed in the file output or
used as the standard input to the head command. The
result is undefined (which means don’t do this!).

How Does It Work?

Redirection is set up by the bash shell before the
command is executed. If there is a redirection error
(such as an invalid filename or a file permission
problem), it will be reported by the shell and the
command will not be executed:

$ cal >

foo/bar/baz

bash: foo/bar/baz: No such file or directory

Tip

Note that the error message starts with bash, indicating that it was
produced by the shell and not by the cal command.
A command is not aware of file redirection unless it
has specifically been programed to check the
standard file descriptors or perform special
operations on them (such as changing terminal
characteristics). Redirected file descriptors are
inherited by applications that were started by
commands; in this example, the nice command
starts the cal command, and cal inherits the
redirection set up for nice:

$ nice "cal" >

test.txt

$ cat

test.txt

July 2006
Su Mo Tu We Th Fr Sa
1
 2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31

What About...
...redirecting standard output and standard error to the same destination?

You can use the characters 2>&1 to redirect standard

error to the same destination as standard output:

$ cal 17 2009 >

/tmp/calresult

2>&1

Notice that the order of the redirections matters.

The preceding command will redirect all output
to /tmp/calresult, but this command will not redirect
standard error:

$ cal 17 2009 2>&1 >

/tmp/calresult

The 2>&1 redirection is evaluated first, so standard

error is directed to the same destination as standard
output (which, at that point, is the
terminal); > /tmp/calresult then redirects standard
output by itself.
This construct can also be used with piping:
$ cal 17 2009 2>&1 | head -2

This will feed both the standard output and the

standard error from cal into the standard input
of head.

...redirecting to a device?

Linux treats most devices as files, so you can

redirect data to and from devices easily. This
command copies the first 50 lines of
the /etc/services file directly to a parallel printer
port:

$ head

-50 /etc/services

>

/dev/lp0

...splitting a pipe to send data to two destinations?

The tee command will receive data on standard

input and write one copy to a file and one copy to
standard output. This effectively splits a pipe:

$ cal -y | tee

/tmp/thisyear.txt

| head

-2
Tip

To send a copy of the data to the screen, use tee with the device
file /dev/tty (the current terminal):

$ cal -y | tee /dev/tty | grep Mo | head -1 >/tmp/dow-header.txt

...piping and redirecting data that is not text?

No assumptions are made about the type of data

being piped or redirected; in fact, there are many
programs that are designed to work with piped
graphics, audio, or video data streams. For example,
this pipeline will decode a color JPEG image, scale it
to half-size, convert it to grayscale, normalize it,
convert it back into a JPEG, save a copy
as /tmp/final.jpg, and display the output in a window:

$ djpeg

/usr/share/wallpapers/floating-leaves.jpg

| pnmscale

0.5

|
ppmtopgm | ppmnorm | cjpeg | tee

/tmp/final.jpg

| display -
Piping in Unix or Linux
A pipe is a form of redirection (transfer of standard output to some other destination)
that is used in Linux and other Unix-like operating systems to send the output of one
command/program/process to another command/program/process for further
processing. The Unix/Linux systems allow the stdout of a command to be connected
to the stdin of another command. You can make it do so by using the pipe
character ‘|’.

The pipe is used to combine two or more commands, and in this, the output of one
command acts as input to another command, and this command’s output may act as
input to the next command, and so on. It can also be visualized as a temporary
connection between two or more commands/ programs/ processes. The command line
programs that do the further processing are referred to as filters.

This direct connection between commands/ programs/ processes allows them to

operate simultaneously and permits data to be transferred between them continuously
rather than having to pass it through temporary text files or through the display
screen.
Pipes are unidirectional i.e., data flows from left to right through the pipeline.

Syntax:
command_1 | command_2 | command_3 | .... | command_N

Example of Piping in Unix or Linux

1. List all files and directories and give them as input to `grep` command
using piping in Linux
ls | grep file.txt

ls | grep file.txt

In this first we are using `ls` to list all file and directories in the current directory, then
passing its output to `grep` command and searching for file name `file.txt`. The output
of the ls command is sent to the input of the grep command, and the result is a list of
files that match the search term.
2. List all files and directories and give them as input to `more`
commands using piping in Linux.
$ ls -l | more

$ ls -l | more
The more command takes the output of $ ls -l as its input. The net effect of this
command is that the output of ls -l is displayed one screen at a time. The pipe acts as a
container which takes the output of ls -l and gives it to more as input. This command
does not use a disk to connect standard output of ls -l to the standard input of more
because pipe is implemented in the main memory.
In terms of I/O redirection operators, the above command is equivalent to the
following command sequence.

$ ls -l -> temp
more -> temp (or more temp)
[contents of temp]
rm temp

3. Sort a list of files by size using piping in Linux

ls -l sort -k 5

ls -l sort -k 5

This command lists all the files in the current directory, and then uses the `sort` command to sort the list by the fifth
column, which represents the file size.
4. Use sort and uniq command to sort a file and print unique
values using piping in Linux
$ sort record.txt | uniq

This will sort the given file and print the unique values only.

sort record.txt | uniq

5. Use head and tail to print lines in a particular range in a file.

$ cat sample2.txt | head -7 | tail -5

This command selects the first 7 lines through (head -7) command and that will be input to (tail -5) command which will
finally print last 5 lines from those 7 lines.

cat sample2.txt | head -7 | tail -5

6. Use ls and find to list and print all lines matching a particular
pattern in matching files.
$ ls -l | find ./ -type f -name "*.txt" -exec grep "program" {} \;

This command selects files with .txt extension in the given directory and searches for patterns like “program” in the
above example and prints those which have program in them.
 ls -l | find ./ -type f -name “*.txt” -exec grep “program” {} \;

7. Use cat, grep, tee and wc command to read the particular entry
from user and store in a file and print line count.
$ cat result.txt | grep "Rajat Dua" | tee file2.txt | wc -l

This command selects Rajat Dua and store them in file2.txt and print total number of lines matching Rajat Dua

cat result.txt | grep “Rajat Dua” | tee file2.txt | wc -l

8.How can I redirect the output of a piped command to file in

Unix or Linux?
We can use redirection operator `>` to redirect the output of a piped command.
For Example:
If i have a file name `file.txt` and want to redirect it to a file name `geeks.txt`.
ls | grep 'file' > geeks.txt

Pipes and Redirection in Linux

1. Introduction
Most shells offer the ability to alter the way that application input and output flows. This can
direct output away from the terminal and into files or other applications or otherwise read
input from files instead of the terminal.
In this tutorial we’ll discuss the concepts of pipes and redirection in Linux.

2. Standard Input and Output

Before we understand how redirection works in shells, we first need to understand the
standard input and output process.
All applications have three unique file descriptors that connect them to the outside
world. These are referred to as Standard Input, or stdin; Standard Output, or stdout;
and Standard Error, or stderr.
Standard input is the default mechanism for getting input into an interactive program. This is
typically a direct link to the keyboard when running directly in a terminal and isn’t connected
to anything otherwise.
Standard output is the default mechanism for writing output from a program. This is typically
a link to the output terminal but is often buffered for performance reasons. This can be
especially important when the program is running over a slow network link.
The standard error is an alternative mechanism for writing output from a program. This is typically a
different link to the same output terminal but is unbuffered so that we can write errors immediately
while the normal output can be written one line at a time.

3. Redirecting Into Files

One common need when we run applications is to direct the output into a file instead of
the terminal. Specifically, what we’re doing here is replacing the standard output file
descriptor with one that goes where we want: In this case, a file.
This is typically done with the > operator between the application to run and the file to
write the output into. For example, we can send the output of the ls command into a file
called files as follows:
$ ls > filesCopy
We can do this after any command, including any needed arguments:

$ ls -1 *.txt > text-filesCopy

3.1. Appending To Files

When we use >, then we are writing the output to the file, replacing all of the contents. If
needed, we can also append to the file using >> instead:
$ ls -1 *.txt > files
$ ls -1 *.text >> files
$ ls -1 *.log >> filesCopy
We can use this to build up files using all manner of output if we need to, including just
using echo to exact output lines:
$ echo Text Files > files
$ ls -1 *.txt >> files
$ echo Log Files >> files
$ ls -1 *.log >> filesCopy

3.2. Redirecting Standard Error

On occasion, we need to redirect standard error instead of standard output. This works
in the same way, but we need to specify the exact file descriptor.
All three of the standard file descriptors have ID values, as defined in the POSIX
specification and used in the C language. Standard input is 0, standard output is 1, and the
standard error is 2.
When we use the redirect operators, by default, this applies to standard output. We can
explicitly specify the file descriptor to redirect, though, by prefixing it with the file descriptor
ID.
For example, to redirect standard error from the cat command, we would use 2>:
$ cat does-not-exist 2> logCopy
We can be consistent and use 1> to redirect standard output if we wish, though this is
identical to what we saw earlier.
We can also use &> to redirect both standard output and standard error at the same
time:
$ ls -1 &> logCopy
This sends all output from the command, regardless of which file descriptor it is on, into the
same file. This only works in certain shells, though – for example, we can use this
in bash or zsh, but not in sh.

3.3. Redirecting into File Descriptors

Sometimes we want to redirect into a file descriptor instead of a file. We can achieve this by using
the file descriptor ID, prefixed by &, in place of the filename. For example, we can redirect into
standard error by using >&2:
$ echo Standard Output >&1
$ echo Standard Error >&2Copy
We can combine this with the above to combine file descriptors by redirecting from one
file descriptor into another. A common construct is to combine standard error into
standard output so that both can be used together. We achieve this using 2>&1 – literally
redirecting file descriptor 2 into a file descriptor 1:
# ls -1 2>&1Copy
We can sometimes use this to create new file descriptors simply by using new IDs. This file
descriptor must already have been used elsewhere first; otherwise, it is an error. Most often,
this is used as a file descriptor source first.
For example, we can swap standard output and standard error by going via a third file
descriptor, using 3>&2 2>&1 1>&3:
$ ls -1 3>&2 2>&1 1>&3Copy
This construct doesn’t work correctly in all shells. In bash, the end result is that standard
output and standard error are directly swapped. In zsh the result is that both standard output
and standard error have ended up on standard output instead.

3.4. Redirecting Multiple File Descriptors

We can easily combine the above to redirect standard output and standard error at the
same time. This mostly works exactly as we would expect – we simply combine the two
different redirects on the same command:
$ ls -1 > stdout.log 2> stderr.logCopy
This won’t work as desired if we try to redirect both file descriptors into the same file. What
happens here is that both file descriptors are redirected individually, and whichever comes second
wins, rather than combining both into the same file.

If we want to redirect both into the same file, then we can use &> as we saw above, or else we can
use file descriptor combination operators. If we wish to use the file descriptor combination
operators, then we must do this after we have redirected into the file, or else only the standard
output gets redirected:

$ ls -1 > log 2>&1Copy

4. Reading From Files

Sometimes we also want to achieve the opposite – redirecting a file into an application.
We do this using the < operator, and the contents of the file will replace the standard input
for the application:
$ wc < /usr/share/dict/words
235886 235886 2493109Copy
When we do this, the only input that the application can receive comes from this source,
and it will all happen immediately. It’s effectively the same as when the user types out the
entire contents of the file at the very start of the application.
However, the end of the file is signaled to the application as well, which many applications
can use to stop processing.

5. Piping Between Applications

The final action that we can perform is to direct the output of one application into
another one. This is commonly referred to as piping and uses the | operator instead:
$ ls | wc
11 11 138Copy
This directly connects the standard output of our first application to the standard input of the
second one and then lets the data flow directly between them.

5.1. Handling Standard Error

The standard error isn’t connected by default, so we’ll still get anything written to that
file descriptor appearing in the console. This is by design since the standard error is
designed for error reporting and not for normal application output.
If we want to redirect standard error as well, then we can use the technique from above to
first redirect it into standard output and then pipe into the next application:
$ ls i-dont-exist | wc
ls: i-dont-exist: No such file or directory
0 0 0
$ ls i-dont-exist 2>&1 | wc
1 7 44Copy
If we want only to pipe standard error, then we need to swap the standard output and standard
error file descriptors around, as we saw earlier.
$ ls 3>&2 2>&1 1>&3 | wc -l
some-file
0

$ ls i-dont-exist 3>&2 2>&1 1>&3 | wc -l

1Copy

5.2. Combining Pipes

When piping between applications, we can also build arbitrary chains where we are
piping between many applications to achieve a result:
$ docker images | cut -d' ' -f1 | tail -n +2 | sort -u | wc -l
17Copy
This command looks scary, but we can break this down into individual parts:

 docker images – Get a list of all Docker images

 cut -d’ ‘ -f1 – Cut this output to only return the first column, where columns are space-
separated
 tail -n +2 – Limit this to start from line 2
 sort -u – Sort this list, only returning unique entries
 wc -l – Count the number of lines

So we have a command here to get the number of unique docker images, ignoring the version
of the image.
Many console applications are designed for exactly this use, which is why they can often
consume input from standard input and write to standard output.
Certain applications also have special modes that allow for this – git, for example, has what is
termed porcelain and plumbing commands, where the plumbing commands are specially
designed to be combined in this manner while the porcelain commands are designed for
human consumption.

5.3. Named Pipes

Another significant concept when dealing with IO in Linux is the concept of named
pipes. Named pipes are a form of inter-process communication that allows data to flow
between two processes, but unlike unnamed pipes, they persist in the filesystem beyond the
life of the process and can be accessed as a file. This functionality can be incredibly useful
when setting up consistent communication channels between different parts of your system.
UNIX Communication

Objectives:
First we will discuss the topics left over from Chapter 8. Then, Chapter 9 discusses methods of
communicating in UNIX. The objectives important to this chapter are:

 using write
 using mesg
 using wall
 UNIX mail utilities
 sending files in the mail