DevOps

What is DevOps?

 DevOps is combination of development and operations.

 The main objective of DevOps is to implement collaboration between development and
operations teams.
 DevOps is not a new tool/Technology in the market.
 It is a new culture or process to develop, release and maintain software
products/projects/applications with high quality in very faster way.
 We can achieve this in DevOps by using several automation tools.

For any software development, release and maintenance, there are two groups of engineers will
work in the company.

1) Development Group

2) Non-Development Group or Operations Group or Administrators Group.

Again, this classification can be divided into small sets of groups

1)Development Group:

The people who are involved in below phases of SDLC are considered as Development Group.
1. planning
2. coding
3. build
4. Testing

Below are the roles which take part in above activities.

 Business Analyst (BA)

 System Analyst (SA)
 Design Architect (DA)
 Developers/coders
 Build Engineer
 Test Engineers/QA

2)Operations Group:

The people who are involved in below phases of SDLC are considered as Operations Group.

1) Release
2) Deploy

3) Operate

4) Maintenance

Below are the roles which take part in above activities.

 Release Engineers
 Configuration Engineer
 System Admin
 Database Admin
 Network Admin etc

To understand this new DevOps culture, we have to aware already existing SDLC Models.

SDLC
Software Development Life Cycle

SDLC is a process followed for a software project, within a software organization. It consists of a
detailed plan describing how to develop, maintain, replace and alter or enhance specific software.

In simple term we can say SDLC is a process which contains all the phases from planning to
maintenance of software Development.

Below are some Models of SDLC:

1) Waterfall Model

2) Prototype Model

3) Incremental/Iterative Model

4) Spiral Model
5) RAD Model

6) Big-Bang Model

7) Fish Model

8) V Model

9) Agile Model

10) DevOps Culture

1) Water Fall Model:

 It is the oldest SDLC Model.

 It is also known as Linear sequential Model.

 In this model, each phase must be completed before the next phase can begin and there is no
overlapping of phases.

i.e. all phases will be performed one by one just like flowing of water fall downwards.

Advantages of waterfall Model:

1. It is very simple and easy to implement.

2. Phases won't be overlapped and hence there is no ambiguity.
3. All phases will be executed one by one which gives high visibility to the project managers and
clients about the progress of the project.
4. Best suitable if the requirements are fixed.
5. Best suitable for small projects.

Disadvantages of waterfall Model:

1. It is very rigid model bcz it won't accept requirement changes in the middle.
2. Client satisfaction is very low because most of the times client will add new requirements in
the middle, which won't be supported.
 3. Total project development time is more because testing should be done after complementing
development only.
4. The cost of bug fixing is very high because we cannot identify bugs in the early stages of life
cycle.
5. Not suitable if the requirements keep on changing.
6. Not suitable for large projects.

2) Agile Model:

This is the most frequently used and hot cake model for software development. Agile Model is divided
into several sub models:

1. Rational Unify Process (RUP)

2. Adaptive Software Development (ASD)
3. Feature Driven Development (FDD)
4. Crystal Clear
5. Scrum etc

Among all these models Scrum model is the most popular and frequently used model.

Scrum Model:

1. Scrum is derived from Rugby Game.

2. It is light weight process.
3. It is an iterative /incremental model and it accepts changes very easily.
4. It is people-based model but not plan based model.
5. Team Collaboration and Continuous feedback are strengths of this model.

Water-fall vs Scrum:

1) In water fall model, before starting next phase, the previous phase should be completed. It is very
rigid model and won't accept requirement changes in the middle.

2) But scrum model is not linear sequential model. It is iterative model. Total software will be
developed increment by increment and each increment is called a sprint. Sprint is a
deliverable/shippable product in scrum model.

Points to Remember:
1. Scrum is an agile model that allows us focus on delivering highest quality software in shortest
time.
2. In this model software development follows increment by increment
3. Each increment will take one to 3 weeks duration.
4. 7 to 9 members are responsible in every sprint.

Advantages of Scrum Model:

1. There is maximum chance for quality

2. It ensures effective use of time and money
3. Requirement changes will be accepted so that maximum chance for client satisfaction
4. Project status Tracking is very easy
5. Team gets complete visibility through scrum meetings.
Limitations:

1. The chances of project failure are very high if individuals are not committed or cooperative.
2. Adapting scrum model for large teams is very big challenge
3. Must require experienced and efficient team members
4. If any team member leaves in the middle of project, it can have a huge negative impact on the
project.

DevOps vs Agile Models:

DevOps and Agile, both are not same.

Similarities:

1. Both are software development methodologies. Agile is there in the market for the last 20
years, but DevOps is recent methodology.
2. Both models concentrating on rapid development of software project.

Differences:

1. The differences between these models will starts after development of the project. Agile
methodology always talks about software development, testing and deployment. Once
deployment completed agile methodology has no role. But DevOps model will continue after
deployment also and it is also responsible for operations and monitoring.
2. In Agile Model, separate people are responsible for developing, testing, and deploying the
software. But, in DevOps, the DevOps engineer is responsible for everything; development to
operations, and operations to development.
3. Agile model won't force us to use automation tools. But DevOps model is completely based on
automation.
4. Agile model always giving highest priority for speed, whereas DevOps giving priority for both
speed and automation.
5. In Agile, client is responsible to give the feedback for the sprint. But in DevOps, immediate
feedback is available from the monitoring tools.

Summary:

1) DevOps is not a new Tool/Technology in the market.

2) It is a new culture or process to develop, release and maintain software products.
3) DevOps is combination of Development and Operations.
4) The main objective of DevOps is to implement collaboration between development and operations
teams.
5) The beauty of DevOps is everything is automated and we can use several automation tools for
development and operations.
6) DevOps is not Agile model and it is more than that because it covers both Development and
operations, whereas Agile covers only Development but not operations.
7) DevOps Cycle is an Infinite Loop where everything is continuous.

Implementation Of DevOps:

1. DevOps is built on the foundation of collaboration and shared responsibility. Encourage

collaboration between development, operations, and other stakeholders to break down silos and
promote a culture of teamwork.

2. Establish clear goals and metrics: Define the goals and objectives of your DevOps initiative.
3. Automate processes: Automation is a fundamental aspect of DevOps. Automate repetitive tasks,
such as building, testing, and deploying software, to increase efficiency and reduce errors. Use
tools like continuous integration/continuous delivery (CI/CD) pipelines to automate the software
delivery process.
4. Embrace infrastructure as code (IAC): Infrastructure as code allows you to manage and provision
infrastructure resources programmatically. Use tools like Terraform or CloudFormation to define
and manage your infrastructure, making it more reliable, reproducible, and scalable.
5. Adopt continuous integration and continuous delivery: Implement continuous integration (CI) to
regularly merge code changes into a shared repository and run automated tests. Continuously
deliver (CD) your applications by automating the deployment process, allowing for faster and
more frequent releases.
6. Implement monitoring and feedback loops: Monitor the performance and health of your
applications and infrastructure in real-time. Use monitoring tools and establish feedback loops to
gather insights, identify issues, and make data-driven decisions for continuous improvement.
 Tools And Technologies need to learn:

1. Linux
2. Cloud (AWS)
3. IAC (Terraform)
4. Version Control System (GIT and GITHUB)
5. Configuration Management Tool (Ansible)
6. Containerization (docker)
7. Build Tool (Maven)
8. CI/CD (Jenkin)

Linux/Unix
Prerequisite to for this course:

 Strong Desire to Learn.

 System with minimum 2 GB RAM.

Before we start the Linux Operating System, we will look some basic concept related to Computer
Science.

What is a computer?

A computer is a device that accepts information (in the form of digitalized data) and
manipulates it for some result based on a program, software, or sequence of
instructions on how the data is to be processed.
It has mainly two components:

1] Software

2] Hardware

1] Software:
Software is collection of specialized programs which takes user input, process
algorithm on it and provide desired output to end user.

Types of software:

 Application software.
 Operating systems or system software.
 Firmware.
Application software:

 Application software is nothing but a software which will full fill business
requirement or end users need.
 Ex: Zoom, MS-word, notepad, google chrome, VLC media player.
Operating system:

 An operating system (OS) is the low-level software that manages

 resources, controls peripherals, and provides basic services to other
software.

 An operating system is a software, which will directly interact with Hard

ware.
 An operating system (O.S) is the program that, after being initially loaded into
the computer by a boot program, manages all of the other application
programs in a computer.
 Ex: Ubuntu, fedora, windows, MAC.
Firmware:

 firmware is a computer program that is "embedded" in a hardware device

and is an essential part of the hardware. It is sometimes called embedded
software.

 An example is a microcontroller, a part of the microprocessor that tells the

microprocessor what actions to take.

2] Hardware:
 Hardware refers to the external and internal devices and equipment that
enable you to perform major functions such as input, output, storage,
communication, processing, and more.
 Ex: Keyboard, Mouse, CPU, Printer.

Core Components of computer Hardware:

CPU: Central processing unit:

 It is a is a large chip inside the computer.

 This is the brains of the computer: it controls everything.
 It performs task like reading instructions, data from the random- access memory
(RAM), performing an instruction, and then writing the data back to RAM.

RAM:

 Stand for Random Access Memory.

 RAM temporarily stores data while your computer is running.
 RAM is both readable and writable. You can add, change, and delete data stored in
RAM.
 It is volatile. When the computer is switched off, all the data stored in RAM is lost.
 It is fast.
ROM:

 ROM stands for read-only memory. It is a chip containing data installed by the
manufacturer that is not typically replaced or upgraded. It stores the BIOS.
 ROM is non-volatile memory, which means it does not need power to keep the data
inside it.

Hard Disk:

 The hard drive (sometimes called the hard disk) is the main storage device in
your computer.
 Like RAM, it can be added to and changed, and like ROM it is non-volatile, but it
is slow.
 You can store files and folder here.
 operating system is also stored on the hard drive.
BIOS:

 BIOS stands for Basic Input Output System.

 BIOS, which stands for Basic Input Output System, is software stored on ROM.
 It contains all the basic code for controlling your computer hardware (such as
keyboards, mice, monitors and hard drives).
 The BIOS firmware is non-volatile, meaning that its settings are saved and
recoverable even after power has been removed from the device.
 Size of BIOS Software is up to 32 megabytes

Booting:
Booting is basically the process of starting the computer. When the CPU is first switched on it
has nothing inside the Memory. In order to start the computer, load the Operating System into the
Main Memory and then Computer is ready to take commands from the User.

Why Programmer should Learn Linux?

There are 6 major reasons one should learn Linux operation system.

Lots of IT Infrastructure Is Linux-Based:

 Most of the infrastructure that powers the internet, including routers and
servers, is based on Linux.
 Most of the OS for handheld devices are having Linux based architecture.

Learning How OS Really Work:

 If you want to learn more about how the software that drives your system
works, Linux is a good choice.
 The source code to the kernel and other utilities is available for anyone to
look at.
 https://github.com/torvalds/linux

2. Great for Learning Networking and Security:

 Linux is a great OS for learning networking and security because so many

 servers rely on it.
 Linux distributions also come with many networking utilities that let you test
a network like ping, trace path, or ip, and you can install more networking
tools like tcpdump or Nmap using your package manager to examine
network traffic.

Lots of Programming Tools:

 If you've always wanted to learn to code, the best way to do so is on a Linux

system.

 The biggest reason is that the tools you need to build programs, such as
editors, compilers, debuggers, IDEs, and interpreters either come with Linux
distros for free or can be downloaded through a package manager.

 Software development kits could cost hundreds or even thousands of dollars

on proprietary systems, but Linux is really a software development kit all by
itself.

You Can Make Old Computers Come Alive Again:

 If you see latest version of windows system needs specific set of
configurations to get installed on system, if it is not there those will not get
installed.
 For the old systems Linux is the best OS, it can get installed on any
hardware.

To learn technologies like Cloud Computing, GIT, Containerization Linux knowledge is must.

Overview of UNIX/Linux Operating System

 What is UNIX?
 Flavours of UNIX
 Components of UNIX
 Online Terminal Demo with some basic commands

What is UNIX?
 It is an operating system, by using that, users/applications can communicate with hard ware
components.
 It was developed/created in 1960s.
 With lot of extensions and improvements to base version, several flavours introduced by
organization/companies (flavours like Red hat Linux, ubuntu, CentOS etc)

Features of UNIX:

1. It is FOSS (Freeware and Open-Source Software)

2. UNIX can be used by multiple users simultaneously and hence it is Multi User
operating System.
3. Several tasks can be executed simultaneously and hence it is multi-tasking operating
system.
4. It is user friendly and provides both CUI and GUI Support.
5. When compared with windows, UNIX is more secured.
Flavours of UNIX:

As UNIX is open source, multiple flavours are available with lot of extensions and
Improvements.

 Ubuntu
 RedHat
 Centos
 Fedora
 Slackware
 open salaries
 Sues Linux Enterprise server (SLES)
 Open Sues

All these flavours have lot of similarity. Hence if we are perfect with one flavour, we can work on any
other flavour very easily.

Features of Linux

 Multiuser Capability:
In Multiuser system same computer resources such as Hard disk, memory is accessible to many
user’s

 Multitasking Capability:
It is capable of carrying out more than one job at the same time.

 Communication:
The communication may be within the network of a single main computer, or between two or
more such computer networks. Users can easily exchange mail, data, programs through such
networks.
Distance poses no barrier to passing information.
Data gets share with in fraction of second, doesn’t matter receiver is 2 feet away or thousand
miles.

 Security:

Linux is more secure as compare to other OS because it provides 3 levels security.

i. Credentials (User name and password)
ii. Permission (Read, Write, Execute)
iii. Encryption (Data get encrypted, only user who is having Decryption key can
access it)

 Portability
portability means, Linux OS get installed and work fine on each and every device irrespective
of their built-in hardware.
Components of Linux

Hardware:

Hardware is the physical components of a computer, such as the motherboard, processor, memory,
storage drives, and other devices. It is the hardware that hosts and supports software or programs that
provide instructions for the computer to complete its tasks. Hardware can also include external
input/output devices such as keyboards, mice, monitors, printers, and speakers.

Kernal:
1. It is the core component of UNIX operating system.
2. It is responsible to execute our commands.
3. It is responsible to interact with hardware components.
4. Memory allocation and processor allocation will take care by kernel

Shell:

1. It is the outer layer of UNIX operating System.

2. It reads our command, verify syntax and check whether the corresponding command
related application is available or not.
3. If everything is proper, then shell interprets our command into kernel understandable
form and handover to the kernel.
4. Shell acts as interface between user and kernel.

Application:
Application program, software designed to handle specific tasks for users. Such software directs the
computer to execute commands given by the user and may be said to include any program that
processes data for a user.

Ex: MS-Office, VLC media player

Command Execution flow

Some Basic Commands:

1. pwd - present working directory

2. ls - list out all files and directories
3. mkdir – create directory
4. cd – change directory
5. touch –to create a file
6. rmdir – remove directory
7. rm – to remove file
8. cal – display monthly calendar
9. date – display current date and time
10. help – to display list of commands
11. hello – to display brief system information
12. clear – to clear terminal
13. exit – to logout session
 Linux File System

In Linux everything is treated as File.

All files are divided into 3 types

1) Normal or Ordinary files: These files contain data. It can be either text files (like abc.txt) OR binary
files (like images, videos etc).

2) Directory Files: These files represent directories. In windows, we can use folder terminology where
as in Linux we can use directory terminology. Directory can contain files and sub directories.

3) Device Files: In Linux, every device is represented as a file. By using this file, we can communicate
with that device.

How to check File Type: We have to use 'ls -l' command.

The first character represents the type of file.

1. d --> Directory File

2. - -->Normal File
3. l -->Link File
4. c -->Character Special File
5. b--> Block Special File
6. s --> Socket File

Note: c, b and s are representing system files and mostly used by super user (also known as root user
or admin user)

File System Navigation Commands

1) Every directory implicitly contains 2 directories. and ..

 . Represents Current Directory

 .. Represents Parent Directory

2) $ cd .

Changes to Current Directory (Useless)

3) $ cd ..

Changes to Parent Directory

4) $cd

If we are not passing any argument, then changes to user home directory.

5) $ cd ~

~ Means User Home Directory. It will Changes to User Home Directory.

6) $ cd - -

Means Previous Working Directory. It will Changes to Previous Working Directory.

Linux File System Hierarchy:

Linux file system has Tree Like Structure.

It starts with root(/).

/ is the topmost directory.

This root directory contains the following important sub directories.

Eg. bin, sbin, lib, etc,dev,opt,home,usr,tmp,media etc.

1. bin Directory: bin means binary. This directory contains all binary executables related to our
a. linux commands and those used by both normal user and root user.
2. sbin Directory:
a. sbin means systembin. It contains all binary executables related to super user OR root
b. commands.
c. Eg: Disk partitioning, network management etc.

3. boot:
a. This directory contains the files which are required to boot Linux os.
b. Eg. GRUB, kernal
4. dev Directory:
a. dev means device.
b. In Linux, everything is treated as a file including devices also.
 c. i.e every device is represented as a file. By using these files, we can communicate with
the
d. devices. All device related files will be stored inside dev directory.
e. Eg.
i. tty --> Terminal related File
ii. fd -->Floppy Drive related File
iii. hd -->Hard Disk related File
5. etc Directory: (Edit text configuration)
a. This directory contains all system configuration files. These configurations can be used
to
b. customize behaviour of Linux os.
c. All user’s information available in /etc/passwd file.
d. All groups information available in /etc/group file.
e. Hosts information (ip address and dns names) available in /etc/hosts file.
6. tmp Directory:
a. tmp means temporary. It contains all temporary files created in the current session.
b. If any file is required only for the current session, then create that file inside tmp
directory.
c. These files will be deleted automatically at the time of system shutdown.
d. If any file which is required permanently, then it is not recommended to create inside
tmp directory
7. home:
i. As linux is multi user operating system, for every user a separate directory will
be created to hold his specific data like videos, images, documents etc.
ii. All these user directories will be stored inside home directory.
8. lib Directory:
i. lib means library. It contains Linux os libraries which are required by our
commands and applications.
9. media Directory:
i. The files of automatic mounting will be placed inside media directory.
10. mnt Directory:
i. mnt means mounting.
ii. We have to attach external file system files from Pen drive, CD, external hard
disk etc to the Linux File System. Then only we can use those external files. This
attachment process is called mounting.
iii. In the old operating systems, we have to perform mounting manually. But in
recent operating systems, mounting is performing automatically and we are
not required to perform manually.
iv. The files of manual mounting will be placed inside mnt directory.
11. opt:
i. opt means optional.
ii. This directory contains all 3rd party software installation files.
iii. Eg: If we are installing any software explicitly like google chrome, then the
corresponding installation files will be stored inside opt directory.
12. proc Directory:
i. proc means processes.
ii. In Linux, multiple processes are running simultaneously.
iii. For every process a unique id will be there, which is also known as PID (Process
ID).
iv. The data related to current running processes will be stored inside proc
directory. For every process a separate directory will be created inside proc to
maintain that data.
 v. The name of this directory is same as PID.
13. root Directory: It is the home directory of super user.
14. run: It keeps run time data.
i. It means whenever we are running an application, those application needs
some dependency or libraries or data that data resides here and once that
application get closed its run time data will also get erased.
15. srv :
a. it contains services files.
b. Eg. If I configure a Apache server on my system, then all services file will be resides in
srv .
16. sys: it contains information of Hardware and system information.
17. usr Directory:
i. usr means user. This directory contains all user related software’s and help
data.
18. var Directory:
i. var means variable data. If any data which is keep on changing, such type of
data will be stored inside var directory. log files will be stored inside var

In order to learn Linux, we need to install Linux Operating system on virtual Machine. Before we
install virtual machine, we must have some idea about virtualization process.

Virtualization
Virtualization is a process where a software-based representation of something rather than

a physical one. Virtualization that is managed by a program is called a Hypervisor. A

hypervisor is a software that helps in creating and running virtual machines.

There are two types in this which are as follows:

1. Type-1 native or bare-metal Hypervisors

2. Type-2 or hosted Hypervisors

Type 1, provides direct interaction to the hardware, and it can run directly on the host’s hardware and
control it.

In type 2, an operating system is needed to interact with the hardware. The Hypervisor is

installed along with the operating system

There is separate document provided for installation process of VM Ware and CentOS 7.

Please follow it and install.

Command
In Linux mostly we are going to fire cmd in terminal to fulfil our requirement.

Command: In Linux command is a instruction passed to the shell via terminal

Syntax: [ command] [option] [argument]

 option - option is optional, we do use option in cmd to filter out put as per our requirement.
 Argument – It generally contain the file/folder on which you want to perform the command.

In some command passing argument is mandatory.

If we don’t pass argument, then it will take default argument.

ls, date and cal Commands

1)ls Command:

We can use ls command to list out all files and directories present in the given directory. We can get

manual documentation for any command by using man.

man ls It provides complete information about ls command.

Various options of ls Command:

1) ls It will display all files and directories according to alphabetical order of names.

2) ls -r It will display all files and directories in reverse of alphabetical order.

3) ls | more to display content line by line (To come out we have to use q)

4) ls -l to display long listing of files

5) ls -t : To display all files based on last modified date and time. Most recent is at top and old are at

bottom.

6) ls -rt : to display all files based on reverse of last modified date and time. Old files are at top and

recent files are at bottom.

7) ls -a : means all To display all files including hidden files. Here . and .. also will be displayed.

8) ls -A : means almost all To display all files including hidden files except . and ..

9) ls -F : To display all files by type.

 directory --> /
 executable file --> *
 link file --> @

10) ls -i :

 To display all files including inode number.

 i-node is the address of location, where file attributes are stored.
 The following are various file attributes
1. The size of the file
2. The number of links
3. The owner
4. The group
5. The creation times
6. The last modified time
7. The last accessed time etc

11) ls -R
  R means Recursive.
 It will list all files and directories including sub directory contents also. By default, ls will
display only direct contents but not sub directory contents.

12) ls -s

 The number of blocks used by file will be displayed. 1 Block = 1Kb

 Note: In ubuntu each block is of 1KB but not 4KB.

13) ls -h display in human readable format

2]date Command:

We can use date command to display date and time of system.

Various Options:

1. date +%D To display only date in the form: mm/dd/yy

2. date +%T To display only time in the form: hh:mm:ss
3. date +%d To display only day value
4. date +%m To display only month value
5. date +%y To display only year value in yy form
6. date +%Y To display only year value in yyyy form.
7. date +%H To display only Hour’s value (in 24 hours scale format)
8. date +%M To display only Minutes value
9. date +%S To display only Seconds value
10. date --set “2 MAY 2023 18:00:00” To set system date as 2nd May 2023
11. date -s “2 MAY 2023 18:00:00” To set system date as 2nd May 2023
12. date +%Y%m%d -s “20230526” To set system date as per our specified format.
13. date +%T -s “10:20:04” To set system time as 10:20:04
14. cdate +”%T%p” -s “10:20:04AM” To set system date as per our specified format.

3]cal Command:

 $ cal -->To display current month calendar.

 $ cal 2020 -->To display total year calendar.
 $ cal 1 --> To display 1st year calendar.
 $ cal 9999 -->To display 9999th year calendar.
 $ cal 10000 --> cal: year '10000' not in range 1..9999
 $ cal 08 2019-->To display august 2019th calendar

Note: cal command can provide support only for the years 1 to 9999 (In some flavours).
 Working with Directories
1) Creation of Directories

We can create directories by using mkdir command.

1. mkdir dir1
a. To create a directory
2. mkdir dir1 dir2 dir3
a. To create multiple directories
3. mkdir dir1/dir2/dir3
a. To create dir3. But make sure dir1 and in that dir2 should be available already.
4. mkdir -p dir1/dir2/dir3
• -p means path of directories.
• All directories in the specified path will be created.
• First dir1 will be created and in that dir2 will be created and within
that dir3 will be created.

Wildcard characters

Wildcards (also referred to as meta characters) are symbols or special characters that represent

other characters.

 An asterisk (*) – matches one or more occurrences of any character, including

no character.
 Question mark (?) – represents or matches a single occurrence of any
character.
 Bracketed characters ([ ]) – matches any occurrence of character enclosed in
the square brackets. It is possible to use different types of characters
(alphanumeric characters) numbers, letters, other special characters etc.
 {} – is used to hold or evaluate an expression
 we can create multiple files and folder by using these wildcard characters.

Q1]. Create 5 directories named with dir2, dir3, dir4 and dir5. In these directories create empty files
with a.txt, b.txt, c.txt and d.txt.

Ans:

$ mkdir dir{2..5}

$ touch dir{2..5}/{a..d}.txt

Note: In above example instead of creating each file and folder manually we used wildcard

characters and created multiple files and folders at a time.

2] How to remove Directories

We can remove directories by using rmdir command.

1) $ rmdir dir1

To remove empty directory dir1

2) 2. $ rmdir dir1 dir2 dir3

To remove multiple empty directories

Note: rmdir command will work only for empty directories. If the directory is not empty then we will
get error. We cannot use rmdir for files. Hence the most useless (waste) command in Linux is rmdir.

rnikrad@localhost~$ rmdir Anil/

rmdir failed to remove 'Anil/' Directory not empty

If the directory is not empty then to remove that directory, we should use rm command. All
internal content also will be removed. rm command can work for files also. Hence rm is recommended
to use than rmdir.

rnikrad@localhost~$ rm Anil

rm cannot remove 'Anil' Is a directory

Whenever we are using rm command for directories, we should use -r or -R option. Here case is
not important

$ rm -r Anil

$ rm -R Anil

Note: In Linux operating system, there is no way to perform undo operation. Once we delete a
file or directory, it is impossible to retrieve that. Hence while using rm command, we have to take
special care.

The following command is the most dangerous command in linux, because it removes total file

system.

rm -r /

Various options with rm Command

1. interactive Option(-i)
While removing files and directories, if we want confirmation then we have to use -i option.
rnikrad@localhost~$ rm -ri Anil
rm descend into directory 'Anil'? y
rm remove regular empty file 'Anil/c.txt'? y
rm remove regular empty file 'Anil/d.txt'? y
rm remove regular empty file 'Anil/a.txt'? y
rm remove regular empty file 'Anil/b.txt'? y
rm remove directory 'Anil'? y

2. force removal(-f)
While removing files and directories, if we don't want any error messages, then we should
use -f option. It is opposite to -i option.
rnikrad@localhost~$ rm -r dir99
rm cannot remove 'dir99' No such file or directory
rnikrad@localhost~$ rm -rf dir99
rnikrad@localhost~$
Even dir99 is not available, we won't get any error message, because we used -f option.

3. verbose Option(-v)
If we want to know the sequence of removals on the screen we should go for - v option.
rnikrad@localhost~$ rm -r dir7
 rnikrad@localhost~$ rm -rv abc
removed 'abc/c.txt'
removed 'abc/d.txt'
removed 'abc/a.txt'
removed 'abc/b.txt'
removed directory 'abc'

4. Absolute Path vs Relative Path:

 Absolute Path It is the path from root(/) to destination. ie it is complete path.
 Relative Path It is the path from current working directory to destination
directory. It is always with respect to current location

Consider above diagram. Let’s assume your present working directory is abc .

In this case relative path for a.txt will be ‘Sunil/a.txt’ and absolute path will be /abc/Sunil/a.txt

Working with Files

What is a file in computer?

A file in a computer is a resource for storing information/data, which is available to a computer

program. Information/data can be in the form of text or binary.

Eg: Documents, Images etc.

1] Creation of Files:

In Linux, we can create files in the following ways:

1. By using touch command (to create empty file)

2. By using cat command
 3. By using editors like gedit, vi, nano etc.

touch command:

By using touch command, we can create a empty file, but t if File is already available then

content of the file won't be changed. But last modified date and time (i.e., Timestamp) will be

updated.

Note: We can use touch command for the following two purposes:

 To create an empty file.

 To change timestamp of existing file.

cat Command:

we use cat command to create a file, once we hit command, we need to pass data, then need to

press CTRL+D to save content and exit.

Syntax: cat > [file_name]

Eg: $ cat > file1.txt

Hello there, how are you all?

CTRL+D -->To save and exit

 If file1.txt is not already available, then file1.txt will be created with our provided data. If
 file1.txt is already available with some content, then old data will be over written with our
provided new data.
 We can use cat command to see content of file as well

Eg: cat file1.txt or cat < file1.txt

Note: < symbol is option here and file must be there with that name

Instead of overwriting, if we want append operation then we should use >> with cat command.

cat >> file1.txt extra content ctrl+d

vi editor:

 The default editor that comes with the UNIX operating system is called vi (visual editor).
 Using vi editor, we can edit an existing file or create a new file from scratch.
 we can also use this editor to just read a text file.

Syntax: vi [filename]

 To insert data in a file we need to press ‘i’

 To save and exit we need to press ‘:wq!’

Note: we will discuss vi editors in details in coming session

 Piping
Sometimes we can use output of one command as input to another command. This concept is called
piping. By using piping, multiple commands will work together to fulfil our requirement.

We can implement piping by using vertical bar (|).

$ ls -l /etc | wc

215 1940 11872

First ls got executed and the output of this command will become input to wc command.

tee Command:
tee command is just like T-Junction or T-Pipe. It will take one input but provides two outputs.

Requirement: The output of the ls command should be saved to abc.txt and should be provided as

input to wc command:

ls -l 1>abc.txt | wc

This command won't work because if we are using redirection in the middle of piping, it will break

piping concept.

In piping, if we want to save the output of one command to a file and if we want to pass that output

as input to next command simultaneously, then we should go for tee command.

Eg 1: To save the output of ls command to a file and to display to the terminal simultaneously.
$ ls –l ➔It will display to the terminal

$ ls -l > abc.txt → It will save to the abc.txt but won't display to the terminal.

$ ls -l | tee abc.txt

Use Multiple Commands in a Single Line

We can execute multiple independent commands in a single line by using the following two
ways:
1 st Way: By using semicolon (;)
cmd1; cmd2; cmd3;…. cmdn
First cmd1 will be executed and then cmd2 followed by rest of the commands. If any command
fails in the middle, still rest of the commands will be executed.
Ex : date;whoami
2 nd Way: By using &&

cmd1 && cmd2 && cmd3 &&..... && cmdn First cmd1 will be executed and then cmd2
followed by rest of the commands. If any command fails in the middle, then rest of the
commands won't be executed.

Ex: date && whoami

Command Aliasing
Alias means another alternative name or nickname.

We can give our own more convenient nicknames for unix commands. This concept is called command
aliasing.

Note: we can use type command, to check whether the command is already available or not.

Syntax :

• $ alias nickname='original command'

• $ alias nickname="original command"

List out all available aliases

$ alias

Remove alias

$unalias [alias_name]

Note: we can use alias for lengthy commands where it will make our life easier.

By default, whatever alias we create, those will be persist till our current terminal session is going on,
once we close terminal alias will get vanished. To overcome this issue, we need to persist aliases
permanently.
How to persist aliases permanently?

Whatever aliases we created, are by default available only in the current session. Once we close the
terminal, all aliases will be lost. But we can make our created aliases permanently in our system by
using the following way:

We have to define our aliases in .bashrc file present in our home directory.

Vi .bashrc

Add the following lines in that file.

# myown aliases alias

cls='clear'

alias ddd='date;date;date'

Note: To reflect these aliases, compulsory we have to close and open terminal

Sorting Content of the File

SORT command is used to sort a file, arranging the records in a particular order. By default, the sort
command sortsfile assuming the contents are ASCII. Using options in the sort command can also be
used to sort numerically.
 SORT command sorts the contents of a text file, line by line.
 sort is a standard command-line program that prints the lines of its input or concatenation of all files
listed in its argument list in sorted order.
 The sort command is a command-line utility for sorting lines of text files. It supports sorting
alphabetically, in reverse order, by number etc.
 By default, the entire input is taken as the sort key. Blank space is the default field separator.

The sort command follows these features as stated below:

 Lines starting with a number will appear before lines starting with a letter.
 Lines starting with a letter that appears earlier in the alphabet will appear before lines starting with a
letter that appears later in the alphabet.
 Lines starting with a uppercase letter will appear before lines starting with the same letter in
lowercase.

Syntax: sort [options] [filename]

Note: This command does not actually change the input file.
In above example sorting is based on alphabetical order.

If we want to sort based on reverse of alphabetical order, then we should use -r option.

If the file contains alphanumeric data, then first numbers will be considered and then alphabet
symbols.
If the file contains only numbers, then the sorting is not based on numeric value and it is just based
on digits.

If we want to sort based on numeric value then we have to use -n option. -n means numeric value

By default, sort command will display duplicate lines. If we want only unique lines then we have to
use -u option.

-u meant for unique lines.

Note: We can use -ur and -un options also

 Sorting Tabular Data by using -k Option:

-k means KEYDEF (key definition). Based on which key (column) we have to sort.

Sort based on File Size in ascending Order:

here 5n means 5 column and n means numeric data (5 number column data type) Sort

based on Month:
6 th column provides total date. If we want consider only month then we should use M.
 uniq command in Linux
The linux uniq command is basically used to remove all repeated lines in a file. This command is
used when aline is repeated multiple times and replace these multiple lines with one line. This
command is designed to work on sorted files.
Syntax : $ uniq [option] [input-file] [output-file]
Note : To use uniq command , compulsory the file should be sorted, otherwise it won’t work properly.

With uniq command we can use multiple options:

 -d To display only duplicate lines

 -cTo display number of occurrences of each line
 -i Ignore case while comparing
 -uTo display only unique lines
i.e the lines which are not duplicated

1. To display only duplicate lines:

2. To display number of occurrences of each line:

3. To ignore case while comparing:

 4. To display only unique lines i.e the lines which are not duplicated:

Comparing Files
We can compare data of two files by using the following commands:

1) cmp

2) diff

3) sdiff

4) comm

Note: All above commands will compare data inside the file’s not the filename.

1) cmp Command:

It will compare byte by byte.

Syntax: cmp [file1] [file2]

If content is same then we won't get any output. If the content is different, then it provides
information about only first difference. byte number and line number will be provided.

Ex:
In Above Ex. 36th byte is having miss match.

Note: cmp command won't show all differences and show only first difference.

2) diff Comamnd:

diff stands for difference. This command is used to display the differences in the files by comparing
the files line by line. Unlike its fellow members, cmp and comm, it tells us which lines in one file have
to be changed to make the two files identical.

Syntax: diff [options] File1 File2

If the content is the same then no output.

If the content is different then, it will show all differences.

The important thing to remember is that diff uses certain special symbols and instructions that are
required to make two files identical. It tells you the instructions on how to change the first file to
make it match the second file.

Special symbols are:

a: add

c: change

d: delete

For the diff command we can use the following options.

-q shows message when files are different.

-s shows message when files are same | identical

-y shows comparison line by line (parallel comparison) it is same as sdiff.

3)sdiff Command: We can use sdiff command for side-by-side comparison (parallel comparison)

Note: sdiff and diff -y are same.

4) comm Command:
comm compare two sorted files line by line and write to standard output; the lines that are common
and the lines that are unique.

Suppose you have two lists of people and you are asked to find out the names available in one and
not in the other, or even those common to both. comm is the command that will help you to achieve
this. It requires two sorted files which it compares line by line.

Syntax: $comm [OPTION]... FILE1 FILE2

It displays results in 3 columns

column-1: Data present only in file1.txt but not in file2.txt

column-2: Data present only in file2.txt but not in file1.txt

column-3: Common data of both files.

With comm command we can use the following options

-1-->If we don't want to display column-1

-2 -->If we don't want to display column-2

-3 -->If we don't want to display column-3

-12 -->If we don't want to display columns 1 and 2

Word Count Command (wc Command)

We can use wc command to count number of lines, words and characters present in the given file.

Syntax : wc [filename]

Eg: $ wc a.txt

o/p: 4 26 166 a.txt

• 4 →Number of Lines

• 26 →Number of words
• 166 → Number of characters (File size in bytes)

We can use the following options with wc Command

-l ➔ To print only number of lines

-w ➔ To print only number of words

-c ➔ To print only number of characters

-lw ➔ To print only number of lines and words

-lc ➔ To print only number of lines and characters

-wc ➔ To print only number of words and characters

-L ➔ To print number of characters present in Longest Line.

Ex:

I-node in Operating System

In Unix based operating system each file is indexed by an I-node. I-node are special disk blocks they
are created when the file system is created. The number of I-node limits the total number of
files/directories that can be stored in the file system.

An i-node is a data structure … … that stores all the information about a file except its name and its
actual data.

I-nodes stores metadata about the file it refers to. This metadata contains all the information about
the said file.

• Size
• Permission
• Owner/Group
• Location of the hard drive
• Date/time
• Other information

Every used i-node refers to 1 file. Every file has 1 i-node. Directories, character files, and block
devices are all files. They each have 1 i-node.
For each file in a directory, there is an entry containing the filename and the i-node number
associated with it.

I-nodes are unique at the partition level. You can have two files with the same i-node number given
they are on different partitions. I-nodes information is stored in a table-like structure in the strategic
parts of each partition, often found near the beginning.

Consider above dig. Its show’s a hard disk where there are 4 partitions. partitions A, B, C, D. Each
partition is having Linux file system created, whenever we create partition on disk it will
automatically divided space of that partition into blocks. each block size is 2 kilo bytes. from the total
no of blocks 1 % blocks get reserved for i-node data structure table, rest all blocks will contain actual
data of files and this blocks i-node number get store in i-node data structure table.
Whenever we search for a file kernel first check in file data structure table, where it will find i-node
for that file, then kernel will go to i-node data structure table and will read all meta data of the file.

Meta data: Data about the data is called as meta data.

consider a file “a.txt” which is containing some data, there is some data about the file a.txt like size,
owner, permission, creation date and time these all comes under meta data.

How to check inode in Linux?

• ls -i
• -i stands for i-node number.

In above img. Inode and file name is shown.

To see total number of used and free inode’s

df -ih
 Viewing file’s content
In Linux to see files content we have head, less, more and less command.
head:
We can use head command to view top few lines of content.

Syntax: head [option] [filename]

head file1.txt

 It will display top 10 lines of file1.txt.

 10 is the default value of number of lines.

head -n 30 file1.txt OR head -30 file1.txt

 To display top 30 lines of the file.

 Instead of 30 we can specify any number.

head -n -20 file1.txt

 To display all lines of file1.txt except last 20 lines.

head -c 100 file1.txt

 head -c 100 file1.txt

tail Command:
 We can use tail command to view few lines from bottom of the file.
 It is opposite to head command.

tail file1.txt

Last 10 lines will be displayed.

tail -n 30 file1.txt OR tail -30 file1.txt OR tail -n -30 file1.txt

It will display last 30 lines.

tail -n +4 file1.txt

It will display from 4th line to last line

tail -c 200 file1.txt

It will display 200 bytes of content from bottom of the file.

Consider there is a file “abc.txt” having 20 lines data as below

I’ll tell you how the sun rose, -

A ribbon at a time.

The steeples swam in amethyst,

The news like squirrels ran.

The hills untied their bonnets,

The bobolinks begun.

Then I said softly to myself,

"That must have been the sun!"

But how he set, I know not.

There seemed a purple stile.

Which little yellow boys and girls

Were climbing all the while

Till when they reached the other side,

A dominie in gray

Put gently up the evening bars,

And led the flock away.

Question: we want to see data from exact line number 10 to line number 15 by using head and tail
cmd.

Ans: To see data from line nuber 10 to 15 we have to combine head and tail cmd by using pipe
concept as below.

 head -15 abc.txt|tail -6

here head -15 will give you data line nuber 1 to 15 by using pipe we are sending that data to tail
cmd and tail cmd will give exact last 6 line means line number 10 to 15.

more Command:
We can use more command to view file content page by page.

more file1.txt

It will display first page.

 Enter --> To view next line

 Space Bar --> To view next page
 q -->To quit/exit
 b – go back one page
less Command:

 By using more command, we can view file content page by page only in forward direction.
 If we want to move either in forward direction or in backward direction then we should go
for less command.

less file1.txt

It will display first page

  d -->To go to next page. (d means down)
 b -->To go to previous page. (b means backward)

Useful keys in less:

 Space bar OR Page Down -->One page forward.
 Page Up OR b -->One page backward.
 Right Arrow -->Scroll right.
 Left Arrow -->Scroll left.
 Home OR g -->Jump to the beginning of the file.
 End OR G -->Jump to the end of the file.

Consider there is a file “abc.txt” having 20 lines data as below

I’ll tell you how the sun rose, -

A ribbon at a time.

The steeples swam in amethyst,

The news like squirrels ran.

The hills untied their bonnets,

The bobolinks begun.

Then I said softly to myself,

"That must have been the sun!"

But how he set, I know not.

There seemed a purple stile.

Which little yellow boys and girls

Were climbing all the while

Till when they reached the other side,

A dominie in gray

Put gently up the evening bars,

And led the flock away.

Question: we want to see data from exact line number 10 to line number 15 by using head and tail
cmd.

Ans: To see data from line nuber 10 to 15 we have to combine head and tail cmd by using pipe
concept as below.
  head -15 abc.txt|tail -6

here head -15 will give you data line nuber 1 to 15 by using pipe we are sending that data to tail
cmd and tail cmd will give exact last 6 line means line number 10 to 15.

File Permissions
Linux is a multi-user operating system, so it has security to prevent people from accessing each
other’s confidential files.

Whenever you hit ‘ls -l’ cmd each file will be listed on a separate line in a long format. There is an
example in the window below.

There’s a lot of information in those lines.

1. The first character = ‘-‘, which means it’s a file

‘d’, which means it’s a directory.
2. The next nine characters = (rw-r–r–) show the security
3. The next column shows the owner of the file. (Here it is `root`)
 4. The next column shows the group owner of the file. (Here it is `root` which has
special access to these files)
5. The next column shows the size of the file in bytes.
6. The next column shows the date and time the file was last modified.
7. Last Column = Filename or Directory name. (For example, here are: Rum, Rahul)

File Permissions describe the allowed operations by various users. With respect to file
permissions, all users are categorized into the following 4 types.

Use Case to understand Types of Users:

Project: Beer Bar Software Development.

This project divided into multiple modules. In each module multiple developers are working.

1. CUSTOMER MODULE
A, B, C, D ARE WORKING

2. EMPLOYEES MODULE
X, Y, Z ARE WORKING

3. BRAND’S MODULE
M, N ARE WORKING

4. Delivery MODULE
G, H ARE WORKING
DEVELOPER 'A' CREATED ONE FILE demo.txt

For demo.txt

user/owner: A (The person who created the file)

group: B,C,D (The persons who are working in the same module)

others: X,Y,Z,M,N,G,H (The persons who are working on other modules)

Security permissions in Linux

First, you must think of those nine characters as three sets of three characters (see the box
at the bottom). Each of the three “rwx” characters refers to a different operation you can
perform on the file.

Read, write, and execute.

Note: - means no permission

Above permission use gets diverse in case of normal file and directory file.
Below table show how same permission allow different operation on file and directory.
Operations related to permissions:
We can perform the following 3 operations.

+ --> Add a particular permission to user|group|other|all

- -->Remove a particular permission to user|group|other|all

= -->Assignment a particular permission to user|group|other|all

Change permission of a existing file or directory:

To change permission of file or directory we have chmod command.
chmod means change mode.

We can user chmod cmd by two ways:

1] Operation way (where you use +,-,= operators)

Syntax: $ chmod <user_category> <operation> <permission> file_name/directory_name

Eg: For user add execute permission,for group add write permission,for others remove read
permission

$ chmod u+x,g+w,o-r demo.txt

Note: Only owner and super user (root) can change file permissions.

Let’s check permission of demo.txt file.

File permission are as below

Total 9 permissions. First 3 are user permissions, next 3 are group permissions and next 3 are others
permissions.

user permissions: rw-

user can perform both read and write operations but not execute operation

group permissions: r--

group members can perform only read operation and cannot perform write and execute operations

others permissions: r--

other members can perform only read operation and cannot perform write and execute operations.

User Permissions + Group Permissions + Others Permissions

Note: order is important

Eg 1: $ chmod u+x demo.txt

adding execute permission to the user

Eg 2: $ chmod u+w,g+rw,o+r demo.txt

adding write permission to the user

adding read and write permissions to the group

adding read permission to the others

Eg 3: $chmod u+x,g-w,o+w demo.txt

adding execute permission to the user

removing write permission from the group

adding write permission to the others

Eg 4: $ chmod u=rw,g=rw,o=r demo.txt

Now user permissions: rw-

 group permission: rw-

others permission: r--

Eg 5: $ chmod a=- demo.txt

Now user permissions: ---

group permission: ---

others permission: ---

Eg 6: $ chmod a=rwx demo.txt

Now user permissions: rwx

group permission: rwx

others permission: rwx

Note: If the user not having execute permission on any directory, then he cannot perform read and
write operations also, because to perform these operations he should enter into that directory
which is not possible.

Note: If the user not having read permission on any file, then he cannot execute that file even
though he has executed permission.

2]Numeric Way of changing file:

Numeric Permissions:

We can specify permissions by using octal number.

Octal means base-8 and allowed digits are 0 to 7.

1. 000 -->No Permission

2. 001 -->Execute Permission
3. 010 -->Write Permission
4. 011 -->Write and execute Permissions
5. 100 --> Read Permission
6. 101 -->Read and execute Permissions
7. 110 --> Read and write Permission
8. 111 --> Read, Write and execute Permissions

Note:

 4 -->Read Permission
 2 --> Write Permission
 1 --> Execute Permission

It is easier to remember

 5 -->4+1 -->r-x
 3 -->2+1 --> -wx
 6 --> 4+2 --> rw-
1. Write command for the following permissions

 For user --> Read and write (6)

 For group -->Write and execute (3)
 For others -->Write (2)

$ chmod 632 demo.txt

rnikrad@rnikrad:~/Desktop$ ls -l demo.txt

-rw--wx-w- 1 rnikrad rnikrad 0 Nov 29 20:57 demo.txt

$ chmod 135 demo.txt

---x-wxr-x 1 rnikrad rnikrad 0 Nov 29 20:57 demo.txt

$ chmod 7 demo.txt

rnikrad@rnikrad:~/Desktop$ ls -l demo.txt

-------rwx 1 rnikrad rnikrad 0 Nov 29 20:57 demo.txt

7 means 007

umask Command:
umask means user mask. Hiding permissions. Based on umask value,default permissions will
be there for files and directories.

The default umask value:022

rnikrad@rnikrad: ~/Desktop$ umask

0022

First 0 is sticky bit mostly used in admin related activities. We have to consider only last 3 digits as
umask value.

Default permissions to the file: 777 - umask value

= 777 - 022

= 755 (user --> rwx, group --> read & execute others --> read &
execute)

rnikrad@rnikrad:~/Desktop$ ls -l file1.txt

-rw-r--r-- 1 rnikrad rnikrad 0 Nov 29 21:18 file1.txt

Default permissions to the directory= 777 - umask value

=777-022

=755 (user --> r&w&x, group --> r&x, others --r&x)

Based on our requirement we can set our own umask values.

umask -S 0023
 cut, cp, mv and tr Commands

We can use cut command to extract data from the file.

The file should contain column formatted data, i.e tabular data.

emp.data

eno|ename|esal|eaddr|dept|gender
100|sunny|1000|mumbai|admin|female
200|bunny|2000|chennai|sales|male
300|chinny|3000|delhi|accounting|female
400|vinny|4000|hyderabad|admin|male
500|pinny|5000|mumbai|sales|female

1. Display Character on specific Position in every Record:

$ cut -c 9 emp.dat
-c meant for specific character
e
y
y
n
y
y

2. Display Range of Characters in every Record:

 $cut -c 5-9 emp.dat
It will display 5th to 9th characters in every record
 $cut -c 5- emp.dat
It will display 5th character to last character in every record
 $ cut -c -3 emp.dat
It will display from 1st character to 3rd character in every record.
 $ cut -c 3-5,7-10 emp.dat
It will display 3rd to 5th character and 7th to 10th character in every record.
3. Display Specific Column Data:
$ cut -d '|' -f 3 emp.dat
-d means delimiter (separator). The default delimiter is tab.
-f means field
It will display 3rd Field (OR Column) data.

4. Display Range of Columns:

$ cut -d '|' -f 2-3 emp.dat

It will display 2nd and 3rd Columns data.

 $ cut -d '|' -f 2- emp.dat

It will display from 2nd column to last column data.

$ cut -d '|' -f -3 emp.dat

It will display from 1st column to 3rd column data.

$ cut -d '|' -f 1,3,5 emp.dat

It will display 1st, 3rd and 5th column data.

5. Skip specific Column:

 Display all columns except specific column
$ cut -d '|' --complement -f3 emp.dat
 It will display all columns data except 3rd column.
$ cut -d '|' --complement -f 3,5 emp.dat
 It will display 3rd and 5th column data.
$ cut -d '|' --complement -f 3-5 emp.dat
 It will display 3rd to 5th column data.

Copy Command (cp)

1. To Copy from File1 to File2 (File to File)

 Syntax: $ cp [source_file] [destination_file]
 $ cp file1 file2
Total content of file1 will be copied to file2.
If file2 is not already available, then this command will create that file.
If file2 is already available and contains some data, then this data will be over
write with file1 content.
2. To Copy File to Directory:
 $ cp file1 file2 output/
 file1 and file2 will be copied to output directory.
 Here we can specify any number of files, but last argument should be directory.
Output directory should be available already.
3. To Copy all Files of One Directory to another Directory:
 $ cp dir1/* dir2
 All files of dir1 will be copied to dir2 But dir2 should be available already.
4. To Copy Total Directory to another Directory:
 $ cp dir1 dir2
 cp: -r not specified; omitting directory 'dir1'

Whenever we are copying one directory to another directory, compulsory we should use -r option.

 $ cp -r dir1 dir2
 total dir1 will be copied to dir2
Note:

If the destination directory (dir2) already available then total dir1 will be copied to dir2.If the
destination directory (dir2) not already available, then destination directory will be created and all
files of source directory will be copied to destination directory but source directory won't be copied.

5. To Copy Multiple Directories into a Directories:

 $ cp -r dir1 dir2 dir3 dir4 dir5
 dir1, dir2,dir3 and dir4 will be copied to dir5

Que) Write Command to Copy Data from a.txt, b.txt, c.txt to d.txt?

 $ cp a.txt b.txt c.txt d.txt  It won't work.

We will discuss solution in the next topics.

Moving and Renaming Directories:

Both moving and renaming activities can be performed by using single command: mv
1. Renaming of files:
$ mv oldname newname
a. Eg: $ file1.txt file2.txt
b. file1.txt will be renamed to file2.txt
2. Renaming of Directories:
$ mv dir1 dir2
dir1 will be renamed to dir2
3. Moving files to directory:
$ mv a.txt b.txt c.txt output
a.txt, b.txt and c.txt will be moved to output directory.
4. Moving of all files from one directory to another directory:
$ mv dir1/* dir2
All files of dir1 will be moved to dir2. After executing this command dir1 will become
empty.
5. Moving total directory to another directory:
$ mv dir1 dir2

Note: If dir2 is already available then dir1 will be moved to dir2. If dir1 is not already available then
dir1 will be renamed to dir2.
 tr Command
tr means translate.

This command translates character by character.

rahul@rahul: ~/Desktop$ cat > demo.txt

While learning unix not required to eat.

Consider demo.txt is our file and we are performing below operation on it

1) $ tr 'aeiou' 'AEIOU' < demo.txt

It will replace lower case vowels with upper case vowels in demo.txt

WhIlE lEArnIng UnIx nOt rEqUIrEd tO EAt.

2) $ tr '[a-z]' '[A-Z]' < demo.txt

Every lower-case alphabet symbol will be replaced with upper case alphabet symbol.

WHILE LEARNING UNIX NOT REQUIRED TO EAT.

3) $ tr '[a-zA-Z]' '[A-Za-z]' < demo.txt

Every lower-case character will be replaced with upper case character and every upper

case character will be replaced with lower case character.

tr '[a-zA-Z]' '[A-Za-z]' < demo.txt > temp.txt

4) $ tr 'aeiou' '7' < demo.txt

To replace every lower-case vowel with digit 7.

5) $ tr '|' '\t' < emp.dat

Replace | symbol with tab in emp.dat

6) $ tr -d 'a' < demo.txt

-d means delete

It will delete all occurrence of 'a' in demo.txt

$ tr -d 'aeiou' < demo.txt

It will delete all lower-case vowels in demo.txt

7) $ tr -s 'a' < demo.txt

It replaces sequence of a's with a single a.

-s means squeeze-repeats
 Locate command
locate command in Linux is used to find the files by name. There is two most widely used file
searching utilities accessible to users are called find and locate. The locate utility works better and
faster than find command counterpart because instead of searching the file system when a file
search is initiated, it would look through a database.

This database contains bits and parts of files and their corresponding paths on your system. By
default, locate command does not check whether the files found in the database still exist and it
never reports files created after the most recent update of the relevant database.

Syntax: locate [option] [filename]

In above example locate command provided us absolute path of the file. This path is present in the
mlocate .db

We can see the database by using locate command with -S option:

This database will be updated only once per day. If we are creating or removing files and
directories, to reflect these changes we have to update database explicitly by using updatedb
command. But admin privileges must be required.
Option:

 -i: to ignore case.

 --limit: to limit the no of output lines
 -c, –count: Instead of writing file names on standard output, write the number of matching
entries only.

Locate with -i option:

Locate with --limit option:

Locate with -c option:

Find Command
The find command in UNIX is a command line utility for walking a file hierarchy. It can be used to find
files and directories and perform subsequent operations on them.

By using the ‘-exec’ other UNIX commands can be executed on files or folders found.

Syntax:

$find [where to start searching from] [-options] [what to find]

We can use find command to find files and directories present in our system. It provides more search
options when compared with locate command like:
1. Search only files
 2. Search only directories
3. Search by name
4. Search by size
5. We can use search results automatically for some other commands
6. We can restrict maxdepth etc.

1. $find:

It will find all files and directories in current working directory and below in Linux file system. This is
the default behaviour. We can find all files and directories in the specified directory and below.

 $ find /dev
 $ find /etc

2. maxdepth Option:

usually find command will search in all depth levels. But we can specify the required depth level by
using maxdepth option.

Observe the difference in results by executing the following commands:

Note:

1. For maxdepth option we should use single - but not double --

 -maxdepth Valid
 --maxdepth Invalid

2. find command will find hidden files and directories also.

Find by Type:
  We can find only files or only directories by using type option.
 -type f means only files

Note:

We can use these options simultaneously, but we should use first -maxdepth and then -type.

 $find -type f -maxdepth 2 Generates warning

 $find -maxdepth 2 -type f No warning

Find by Name:

 We can find files and directories by name by using -name option.

 If we want to ignore case then we should use -iname option.

Find Files by Size:

we can find files and folders bases on their size also, for that we need to use -size option

 + symbol means greater than (over)

 - symbol means less than

To list out all file names where size is over 200kb

 $ find / -type f -size +200k This command required root privileges

  $ sudo find / -type f -size +200k | wc -l

To list out all file names where size is over 200kb but less than 4MB.

 $ find / -type f -size +200k -size -4M | wc -l

To list out all file names where file size is less 200kb or more than 4MB.

 $ find / -type f -size -200k -o -size +4M | wc -l

 -o means or

To list out files whose size is exactly 10M

 $find. -size 10M

Note: you can find empty files or folders by using -empty option.

Some Important Option’s with Example:

1. find a file which are accessed with in 1 hour.

$find . -amin -60
2. find a file which are accessed within 1 day.
$find . -atime -1
3. find a file which are accessed after the modification of a given file.
$find -anewer “abc.java”
4. find a file which are modified after the modification of a given file.
$find -newer “abc.java”
5. find the files which are modified 30 minutes back
$find . -not -mmin -30
6. find the files which are modified 1 day back.
$find . -not mtime -1
7. find the files which are changed after the modification of a given file.
$find . -cnewer “abc.java”
8. find the files which are changed within 3 days
$find . -ctime -2
9. find the files bases on the file permission.
$find . -perm 777
10. find the files with file owner’s name.
$find. -user name rnikrad
11. Find a file modified within a range of days
$ find ~ -mtime +1 -mtime -7
Find cmd with -exec option:

The find command is comprised of two main parts, the expression and the action. When we initially
use find, we usually start with the expression part. This is the part that allows us to specify a filter
that defines which files to select.

A classic example would be:

$ find Music/ -name *.mp3

Music/Gustav Mahler/01 - Das Trinklied vom Jammer der Erde.mp3

Music/Gustav Mahler/02 - Der Einsame im Herbst.mp3

Music/Gustav Mahler/03 - Von der Jugend.mp3

This command will result in a list of mp3 files in the Music directory and all its subdirectories. The
action part in this example is the default action, -print. This action prints the resulting paths with
newline characters in between. It’ll run if no other action is specified.

In contrast, the -exec action allows us to execute commands on the resulting paths. Let’s say we want
to run the file command on the list of mp3 files we just found to determine their filetype. We can
achieve this by running the following command:

$ find Music/ -name *.mp3 -exec file {} \;

Music/Gustav Mahler/01 - Das Trinklied vom Jammer der Erde.mp3:

Audio file with ID3 version 2.4.0, contains:MPEG ADTS, layer III, v1, 128 kbps, 44.1 kHz,Stereo

Music/Gustav Mahler/02 - Der Einsame im Herbst.mp3:

Audio file with ID3 version 2.4.0, contains: MPEG ADTS, layer III, v1, 128 kbps, 44.1 kHz

Let’s discuss the arguments passed to the -exec flag, which include:

1) A command: file
2) A placeholder: {}
3) A command delimiter: \;

Now we’ll walk through each of these three parts in-depth. Any command that cannot be executed
by our shell is acceptable here. We should note that this isn’t our shell executing the command,
rather we’re using Linux’s exec directly to execute the command. This means that any shell
expansion won’t work here, as we don’t have a shell. Another effect is the unavailability of shell
functions or aliases.

As a workaround for our missing shell functions, we can export them and call bash -c with our
requested function on our file.

To see this in action, we’ll continue with our directory of Mahler’s mp3 files. Let’s create a shell
function that shows the track name and some details about the quality:

function mp3info () {

TRACK_NAME=$(basename "$1")

FILE_DATA=$(file "$1" | awk -F, '{$1=$2=$3=$4=""; print $0}')

echo "${TRACK_NAME%.mp3}: $FILE_DATA"

If we try to run the mp3info command on all of our files, -exec will complain that it doesn’t know
about mp3info:

 find. -name "*.mp3" -exec mp3info {} \;

 find: ‘mp3info’: No such file or directory

As mentioned earlier, to fix this, we’ll need to export our shell function and run it as part of a
spawned shell:

$ export -f mp3info

$ find . -name "*.mp3" -exec bash -c "mp3info \"{}\"" \;

01 - Das Trinklied vom Jammer der Erde : 128 kbps 44.1 kHz Stereo

02 - Der Einsame im Herbst : 128 kbps 44.1 kHz Stereo

03 - Von der Jugend : 128 kbps 44.1 kHz Stereo

Note that because some of our file names hold spaces, we need to quote the results
placeholder.

The Results Placeholder

 The results placeholder is denoted by two curly braces {}.

 We can use the placeholder multiple times if necessary.

The Delimiter

 We need to provide the find command with a delimiter so it’ll know where our -exec
arguments stop.

Two types of delimiters can be provided to the -exec argument:

the semi-colon (;) or the plus sign (+).

The delimiter determines the way find handles the expression results. If we use the semi colon (;),
the -exec command will be repeated for each result separately. On the other hand, if we use the
plussign (+), all of the expressions’ results will be concatenated and passed as a whole to the -exec
command, which will run only once.

Let’s see the use of the plus sign with another example:

$ find . -name "*.mp3" -exec echo {} +

./Gustav Mahler/01 - Das Trinklied vom Jammer der Erde.mp3 ./Gustav Mahler/02 -

Der Einsame im Herbst.mp3 ./Gustav Mahler/03 - Von der Jugend.mp3 ./Gustav Mahler/04 -

Von der Schönheit.mp3 ./Gustav Mahler/05 - Der Trunkene im Frühling.mp3

./Gustav Mahler/06 - Der Abschied.mp3

From a performance point of view, we usually prefer to use the plus-sign delimiter, as running
separate processes for each file can incur a serious penalty in both RAM and processing time.

However, we may prefer using the semi-colon delimiter in one of the following cases:The tool run by
-exec doesn’t accept multiple files as an argument.

Running the tool on so many files at once might use up too much memory.

We want to start getting some results as soon as possible, even though it’ll take more time to get all
the results.

Difference between find and locate Commands

grep command
Grep is an essential Linux and Unix command. It is used to search text and strings in a given file. grep
filter searches a file for a particular pattern of characters, and displays all lines that contain that
pattern. The pattern that is searched in the file is referred to as the regular expression.

grep stands for:

globally search a regular expression and print it

Syntax:

grep [options] pattern [files]

We can use grep command to search the given pattern in a single or multiple files.

grep <pattern> filename

It prints all matched lines.

Below is some standard grep command explained with examples to get you started with grep on
Linux:

1. Search any line that contains the word in filename on Linux:

$grep 'word' filename

2. Perform a case-insensitive search for the word ‘bar’ in Linux and Unix:

$grep -i 'bar' file1

3. Look for all files in the current directory and in all of its subdirectories in Linux for the word
‘httpd’:
$grep -R 'httpd’.
4. Search and display the total number of times that the string ‘nixcraft’ appears in a file
named frontpage.md:

$grep -c 'nixcraft’ frontpage.md

Consider below Subject.txt file for all below examples:

1. To Search Data in a Single File:

$ grep 'Rahul' subjects.txt

$ grep "Rahul" subjects.txt

$ grep Rahul subjects.txt

It will print same output for all 3 commands:

2. To Search in Multiple Files:

$grep ‘Rahul’ subjects.txt abc.txt

3. To Search in All .txt files:

$grep ‘Rahul’ *.txt

4. To Search Data by ignoring Case:

By default, grep command will consider case. If we want to ignore case then we should use
use -i option.

5. To Display the Number of Occurrences:

We have to use -c option. c means count

6. To Display Line Numbers before Results:

We have to use -n option.

7. To Display only File Names in which Pattern exists:

 We have to use -l option.

8. To Print except matched Lines remaining Lines:

We have to use -v option. It means inverted.

9. To Search for exact word in the File:

For this we have to use -w option.

10. Display before, after and surrounding lines including Search Results:
We have to use -A, -B,-C options
 -A means after
 -B means before
 -C means before and after
Search Multiple Content in a File:

Instead of using -e option, we can use egrep command directly. It is extended grep. It interprets
patterns as an extended reqular expression.

grep with -F Option OR fgrep:

fgrep "Fixed String Global Regular Expression Print"

It will take a group of fixed strings and search for those in the given file. Strings should be separated
by new line.
Instead of using, -F option, we can use directly fgrep.

Note: fgrep can be used only for Strings and cannot be used for regular expressions.
Normal grep command cannot understand some regular expression patterns like | symbol. But
egrep command can understand any regular expression pattern. Hence egrep is the more powerful
than normal grep command.

In this case, (java|unix) is treated as regular expression. It will search for either java or Unix. egrep
and fgrep are deprecated and hence it is recommended to use grep -E and grep -F commands. If
strings are available in the file, then we can use -f option to specify file name

To Print only matched Patterns instead of Total Line:

-o: Print only the matched parts of a matching line, with each such part on a separate output line.

To Search in the Files recursively inside a Directory:

If we want to use grep command for all files present in the specified directory and sub directory
recursively then we should use R option.

Sudoer File
What is a Sudoers File?
A Sudoers file is just like any other file on a Linux Operating System. But it plays a vital role in
managing what a “User” or “Users in a Group” can do on the system.

Why the name Sudoers?

The word Sudo is the abbreviation of the term “Super User Do”. As it clearly suggests — it is just like
telling the Kernel/OS “you have no choice, just do it!”. So the users with Sudo permission are called
as Sudo Users. Management of all these sudo users is done through a file called as Sudoers File. This
file also lives in the /etc directory just like any other configuration file.

Why Sudoers File?

A normal user on Linux will have many restrictions in the system. For example — the user won’t be
able to start a service, change a config file or execute a script on the system by default. From a
security perspective this looks perfect.

But, let us take an example of — you as a system admin have to manage all the users and their
controls. You have a website hosted on Apache in one of your servers. You already have a Web-
Admin to manage it. A user should be created for him on the Web-Server. This User should be
allowed to do two things which a normal user cannot:

 Start/Restart Apache Service in case it stops.

 Edit the config file of Apache.

You have two user options by default:

 Root User which has got all the permissions on the system including Apache.
 A normal user, which by default won’t be able to edit any config Start a Service or files on the
system.

This is when a sudoers file will come handy to you. In Sudoers file we can exactly define:
Web-Admin user can only start Apache Service &He can only edit Apache Config file.Playing with
the Sudoers File

By default, a Sudoers file looks like this:

Primarily we can see five sections in the file — the first three sections in the file are related to ALIAS
of Host, User and Cmd.

We will talk about ALIAS later, first let us talk about the core section:

#User Privilege Specification

root ALL = (ALL:ALL) ALL

Note: ‘#’ indicates comment in the file, OS will ignore this line while executing.

Syntax : User <space> OnHost = (Runas-User:Group) <space> Commands

Example: root ALL = (ALL:ALL) ALL

Read it as — User Root can Run Any Command as Any User from Any Group on Any Host.
  The first ALL is used to define HOSTS. We can define Hostname/Ip-Address instead of ALL.
ALL means any host.
 Second ALL : Third ALL is User:Group. Instead of ALL we can define User or User with the
group like User:Group. ALL:ALL means All users and All groups.

 Last ALL is the Command. Instead of ALL, we can define a command or set of commands. ALL
means all commands.

To put it very simple, it is “who where = (as_whom) what”.

Basically, we can permit a user to do something in only two ways 

 Directly give him the permission to execute the command by modifying the permission of
the file.
 Allow him to execute the command as another user which already has the permission to
execute this command.

In the Sudoers file we will be implementing the latter one.

We will use another example to clearly understand the fields in the syntax:

Example : sysadmin ALL = (root) /usr/bin/cat /etc/shadow

Read this as — User “sysadmin” can Run the command “/usr/bin/cat /etc/shadow” as ROOT user
on all the HOSTS.

#Allow members of group sudo to execute any command

%sudo ALL = (ALL:ALL) ALL

%SystemAdminGroup ALL = (root) /usr/bin/cat /etc/shadow

 Managing Users

 Linux is designed to serve many users at the same time. Users have their own shells through
which they interact with the operating system.
 As a system administrator, you can manage user logins on your system. You can add or
remove users, as well as add and remove groups.
 Any utility to manage a user or group makes use of certain default files, called configuration
files and directories to set up the new account.
 Below is the list of files and purpose:

The Password Files

 A user gains access to an account by providing a correct login and password.

 The system maintains passwords in password files, along with login information like the
username and ID.
 Tools like the passwd command let users change their passwords by modifying these
files; /etc/passwd is the file that traditionally held user passwords, though in encrypted form.
 However, all users are allowed to read the /etc/passwd file, which allows access by users to
the encrypted passwords.
 For better security, password entries are now kept in the /etc/shadow file, which is restricted
to the root user.

/etc/passwd

 When you add a user, an entry for that user is made in the /etc/passwd file, commonly
known as the password file. Each entry takes up one line that has several fields separated by
colons.

The fields are as follows:

 Username: Login name of the user

 Password: Encrypted password for the user’s account
 User ID: Unique number assigned by the system
 Group ID: Number used to identify the group to which the user belongs
 Comment: Any user information, such as the user’s full name
  Home directory: The user’s home directory
 Login shell: Shell to run when the user logs in; this is the default shell, usually /bin/bash
Note: At Password field you will see either X or encrypted password. If shadow file in use it
will be X or else encrypted Password.

/etc/shadow and /etc/gshadow:

 The shadow suite of applications implements a greater level of security.

 Most other user configuration tools support shadow security measures. With shadow
security, passwords are no longer kept in the /etc/password file. Instead, passwords are kept
in a separate file called /etc/shadow.
 Access is restricted to the root user.

The following example shows the /etc/passwd entry for a user.

chris:x: 501:501: Chris:/home/chris:/bin/bash

A corresponding password file, called /etc/gshadow, is also maintained for groups that require
passwords.

Note:

 Each time a user logs in, two profile scripts are executed, a system profile script that is the
same for every user, and a user login profile script that can be customized to each user’s
needs.

/etc/skel:

 When you first add a user to the system, you must provide the user with skeleton versions of
their login, shell, and logout initialization files.
 For the BASH shell, these are the .bash_ profile, .bashrc, and .bash_logout files.
 The useradd command and other user management tools add these files automatically,
copying any files in the directory /etc/skel to the user’s new home directory.
 The /etc/skel directory contains a skeleton initialization file for the .bash_profile, .bashrc, and
.bash_logout files.
 As a superuser, you can configure the .bash_profile or .bashrc file in the /etc/skel directory
any way you want. Usually, basic system variable assignments are included that define
pathnames for commands and command aliases.
 The PATH and BASH_ENV variables are defined in .bash_profile. Once users have their own
.bash_profile or. bashrc file, they can redefine variables or add new commands as they
choose.

/etc/login.defs:

 System wide values used by user and group creation utilities such as useradd and usergroup
are kept in the /etc/login.defs file.
 Here you will find the range of possible user and group IDs listed.
 UID_MIN holds the minimum number for user IDs and UID_MAX the maximum number.
  Various password options control password controls—such as PASS_MIN_LEN, which
determines the minimum number of characters allowable in a password.

/etc/group and /etc/gshadow:

 The system file that holds group entries is called /etc/group.

 The file consists of group records, with one record per line and its fields separated by colons.
 A group record has four fields: a group name, a password, its ID, and the users who are part
of this group.
 The Password field can be left blank.

The fields for a group record are as follows:

 Group name The name of the group, which must be unique

 Password With shadow security implemented, this field is an x, with the password indicated
in the /etc/gshadow file
 Group ID The number assigned by the system to identify this group
 Users The list of users that belong to the group, separated by commas

EX:

engines:x:100: chris,robert,valerie,aleina

Adding and Removing Users with useradd, usermod, and userdel :

 Linux provides the useradd, usermod, and userdel commands to manage user accounts.
 All these commands take in their information as options on the command line.
 If an option is not specified, they use predetermined default values. These are command line
operations.

useradd:

With the useradd command, you enter values as options on the command line, such as the name of
a user, to create a user account. It then creates a new login and directory for that name using all the
default features for a new account.

Syntax: useradd [option] [username]

Ex: useradd Rahul

Some Imp. Options:

Option Description
-d dir Sets the home directory of the new user.
-e mm/dd/yy Sets an expiration date for the account (none, by default). Specified as month/day/year.
-f days Sets the number of days an account remains active after its password expires
-g group Sets a group.
-m Creates user’s home directory, if it does not exist
-M Does not create user’s home directory
-s shell Sets the login shell of the new user. This is /bin/bash by default, the BASH shell.
 -u userid Sets the user ID of the new user. The default is the increment of the highest number used
so far

usermod:

 The usermod command enables you to change the values for user.
 You can change the home directory or the user ID. You can even change the username for
the account.
 The usermod command takes the same options as useradd, listed previously in above table.

userdel:

 When you want to remove a user from the system, you can use the userdel command to
delete the user’s login.
 With the -r option, the user’s home directory will also be removed.
 In the next example, the user chris is removed from the system:
 # userdel -r chris

Managing Groups Using groupadd, groupmod, and groupdel :

You can also manage groups with the groupadd, groupmod, and groupdel commands.

groupadd and groupdel

 With the groupadd command, you can create new groups.

 When you add a group to the system, the system places the group’s name in the /etc/group
file and gives it a group ID number.
 If shadow security is in place, changes are made to the /etc/gshadow file.
 The groupadd command only creates the group category. You need to add users to the group
individually.

In the following example, the groupadd command creates the engines group:

# groupadd engines

You can delete a group with the groupdel command. In the next example, the engines group is
deleted:

# groupdel engines

groupmod

You can change the name of a group or its ID using the groupmod command.
Enter groupmod -g with the new ID number and the group name. To change the name of a group,
you use the -n option.

Enter groupmod -n with the new name of the group, followed by the current name. In the next
example, the engines group has its name changed to trains:

# groupmod -n trains engines

Password Tools

passwd command in Linux is used to change the user account passwords. The root user reserves the
privilege to change the password for any user on the system, while a normal user can only change
the account password for his or her own account.

Syntax: passwd [options] [username]

Some Important options:

 -d, –delete: This option deletes the user password and makes the account password-less.
 -e, –expire: This option immediately expires the account password and forces the user to
change password on their next login.
 -i, –inactive INACTIVE_DAYS: This option is followed by an integer, INACTIVE_DAYS, which
is the number of days after the password expires that the account will be deactivated.

 -l, –lock:
1. Lock the password of user.
2. This appends the encrypted password of the user with a character ‘!’, and thus
making it unable to match with any of input password combinations.
3. This does not disable the account but prevents the user from logging in using a
password.
4. Though other authentication methods like ssh keys can be used to login to the
account.
 -u, –unlock: Unlock the password of an account.
 Chage tool

The chage command is used to view and change the user password properties.

The chage command let you specify an expiration limit for a user’s password.

A user can be required to change his or her password every month, every week, or at a given date.
Once the password expires, the user is prompted to enter a new one. You can issue a warning
beforehand, telling the user how much time is left before the password expires.

If there is an account that you want to close, you can permanently expire a password. You can even
shut down accounts that are inactive too long.

Syntax: chage [options] username

Some Important Options:

The -M option with the number of days sets the maximum time that a password can be valid.

chage -M 7 chris

To set a particular date for the account to expire, use the -E option with the date specified
mm/dd/yyyy.

chage -E 07/30/2003 chris

To find out what the current expiration settings are for a given account, use the -l option.

chage -l chris

You can also combine your options into one command.

chage -M 7 -E 07/30/2003 chris