DevOps Linux Lecture-3
DevOps Linux Lecture-3
www.corvit.com
Topics
• Users, Groups, Permissions
• Crontab
• Text Editors, Other Essential tools
• Backup Techniques
• I/O Redirection
• Partitions
www.corvit.com
Linux User Accounts
• Each user on Linux has a separate account with a password.
• User accounts can be members of user groups.
• The root account is known as a super-user and is all powerful (like
Administrator on Windows)
• Typically we log into a Linux system and escalate our privileges to
become a super-user using the sudo command (super-user do)
– Like ”Run as Administrator” on Windows.
www.corvit.com
Linux User Accounts
• Users have a defined home directory, typically something like
/home/username
– The tilda (~) character is used as an alias for a user’s home
directory.
You can add new users using the useradd command:
– $ useradd bob
$ sudo adduser bob
And delete users using the userdel command:
– $ userdel –r bob
$ sudo deluser --remove-home bob
www.corvit.com
Linux User Groups
• The following example shows how to create a new user named
“bob” and set the login group to users
$ sudo useradd –g users bob
• To verify the user’s GID, use the id command
$ id –gn bob
• To create a new group
$ sudo addgroup groupname
୦ To to add user to the group
$ sudo usermod -aG groupname username
To list groups against a user
groups username
www.corvit.com
Linux File Permissions
www.corvit.com
Linux File Permissions
• We use indicators to represent these values, and form short “permission statements”
such as u+x where “u” means “user”(who), “+” means add (what), and “x” means the
execute permission (which.)
• The “who” values we can use are :
- u: User, meaning the owner of the file
- g: Group, meaning members of the group the file belongs to.
- o: Others, meaning people not governed by the u and g permissions.
- a: All, meaning all of the above.
If none of these are used, chmod behaves as if “a” had been used.
• The “what” values we can use are :
- “-”: Minus sign. Removes the permission
- “+“: Plus sign. Grants the permission. The permission is added to the existing
permissions. If you want to have this permission and only this permission set, use the =
option.
- “=”: Equal sign. Set a permission and remove others.
• The “which” values we can use are:
- r: read permission
- w: write permission
- x: execute permission
www.corvit.com
Linux File Permissions
The digits you can use and what they represent are listed here:
• 0: (000) No Permission
• 1: (001) Execute Permission
• 2: (010) Write Permission
• 3: (011) Write and Execute Permission
• 4: (100) Read Permission
• 5: (101) Read and Execute Permissions.
• 6: (110) Read and Write Permissions.
• 7: (111) Read, Write and Execute Permissions
www.corvit.com
Linux File Permissions
• Change the owner of a file using the chown command.
– $ chown file1
• Change the group associated with a file using the chgrp command.
– $ chgrp webusers file1
• Change the file permissions using the chmod command.
– $ chmod a+r file1
Set the permissions so that all users can read the file
– $ chmod a+x file1
Set the permissions to that everyone can execute the file
• We can also use a numeric argument (octal) to set the file permissions:
– $ chmod 600 file1
Set read and write permissions for the file owner
– $ chmod 775 file1
Give everyone read+execute, owner and group get all access
• Current user would have read and write permissions and the group and other users to
have read permissions only. We can do using the following command:
- chmod u=rw,og=r new_file
Using the “=” operator means we wipe out any existing permissions and then set the
ones specified.
www.corvit.com
Text Editors
• Easy way to create an empty text file is to use the touch command.
– $ touch myfile.txt
• You can also use one of the basic text editors to create a file.
– $ nano myfile.txt (recommended for beginners)
– $ vi myfile.txt
• For nano:
- ctrl+o (to save the changes made to the file)
- ctrl+x (to exit)
• For vim:
- Use escape(Esc) to change the mode.
- after pressing Esc press “i” to change mode to insert and make changes
to file.
- After making changes to file press Esc to change mode again.
- write “:x” to save and exit OR “:wq”
www.corvit.com
Crontab
• The Crontab is a list of commands that you want to run on a regular schedule.
• Crontab stands for “Cron Table”
• Crontab format
MIN HOUR DOM MON DOW CMD
• Crontab fields and Allowed Ranges
www.corvit.com
Crontab
• Crontab uses 24 hour format.
• Crontab would use the same time zone as of the hosted machine.
So be careful while creating cron jobs on virtual machines which
are in other time zone.
• Use * for all values
• Use any editor like nano or vim to edit the job and save the file.
• Example:
- 30 08 10 06 * /home/maverick/full-backup
- 0 3 15 * * sudo docker image prune –a –f
- 00 11, 16 * * * /home/maverick/bin/incremental-backup
- 00 09-18 * * * /home/maverick/bin/check-db-status
- 00 09-18 * * 1-5 /home/maverick/bin/check-db-status
- */10 * * * * /home/maverick/check-disk-space
- @reboot CMD
www.corvit.com
Crontab Keywords
Keyword Equivalent
@yearly 0011*
@daily 00***
@hourly 0****
www.corvit.com
Crontab Commands
• To create a new job
crontab –e
• To remove cron
crontab –r –i
• To list jobs
crontab –l
• To create a job to run with a specific user
crontab –u user –e
www.corvit.com
File Globbing
• Globbing is the use of pathname expansion to refer to one or more
files.
• Uses special characters to expand pathname:
– * matches all characters (wildcard)
– ? matches a single character
– $ ls *.jpg (list all jpeg files)
– $ ls ?.jpg (list jpeg files with 1 character names)
– $ rm [A-Z]*.jpg (remove jpeg files that start with capital letter)
www.corvit.com
Package Management
• Every Linux system comes with applications pre-installed on the
system in packages.
• Trying to figure out how to install and remove applications can be
tricky because of package dependencies.
• A package management system is used to:
– Track installed packages
– Install new packages and dependencies
– Remove packages
• A couple different package management systems exist and each
distribution may use a different one.
www.corvit.com
Installing Package
• Amazon Linux uses the common yum packaging system.
• Yum provides an easy way to update the currently installed
packages on a system.
– $ sudo yum update –y
– Kind of like running a Windows Update
• Install new applications on the system using the command:
– $ sudo yum install httpd
• Remove a package using the command:
– $ sudo yum remove httpd
www.corvit.com
Installing Package
www.corvit.com
Processes
• Each program on Linux runs as a processes or set of processes.
– Some are interactive and terminate quickly after execution.
– Some are long running and run in the background (called
services or daemons).
• View current running processes on the system using the ps
comand (use - ax arguments to show all processes).
– $ ps –ax
• A handy way to look at the current resource utilization on the
system is by using the top command.
– $ top
• Forcibly stop a process using the kill command.
– $ kill
www.corvit.com
Services
• Linux uses a service management program called
service to manage the state of services.
– $ sudo service start httpd
Starts the Apache webserver on the system.
– $ sudo service stop httpd
Stops the Apache webserver on the system.
• A service can be configured to automatically start when
a system starts up using the chkconfig command.
– $ sudo chkconfig httpd on
www.corvit.com
Archiving Files
• The tar command is used to compress a group of files into an
archive. The command is also used to extract, maintain, or
modify tar archives.
• Syntax : tar [options] [archive-file] [file or directory to be
archived]
• Options:
-c: Creates Archive
-x: Extracts the archive
-f: Creates archive with given filename
-t: displays or lists files in archived file
-v: Shows details about the results of running tar
-A: Concatenates the archive files
-z: Compresses the tar file using gzip
-j: Compresses the tar file using bzip
-W: Verifies an archive file
-r: Updates or adds file or directory in already existing .tar file
www.corvit.com
Archiving Files Usage Examples
• Extract an archive: • Extract files from gzip tar Archive archive.tar.gz:
$ tar xfv archive.tar $ tar xvzf archive.tar.gz
(Options: x=extract, f=file, • Create a compressed tar archive file using
v=verbose) bzip2:
$ tar cvfj archive.tar.tz example.txt
• Create and archive with files or folder: (Options: j=compress with bzip2, smaller file
size but takes longer than -z)
$ tar cfv archive.tar file1 file2 file3 • Update existing tar file by adding todo.txt file to
(Options: c=create) archive:
• Create compressed archives: $ tar rvf archive.tar todo.txt
(Options: r=add file)
$ tar cfzv archive.tar file1 file2 file3
(Options: z=compress with gzip) • List contents of tar file:
$ tar tf file.tar
• Show all files of an archive: (options: t=display, f=file)
$ tar tvf archive.tar
• Create a compressed archive of current
• Create an uncompressed archive of all directory but exclude certain directories:
.txt files in current directory: $ tar --exclude=‘./folder’
$ tar cfv archive.tar *.txt --exclude=‘./upload/folder2’ cfzv archive.tar
(folder and folder2 are excluded)
www.corvit.com
Compressing and Archiving Tools
• zip, unzip, bzip2 and tar file • zip compressing file and packaging it
compression and archiving tool in unix into a zip file keeping the input file
like systems. untouched.
• gzip and bzip2 are for compressing $ zip data.zip data1.txt data2.txt
single file. bzip2 is more efficient that • unzip extract compressed files from a
gzip to compress the file but takes zip package
more time to do the compression. $ unzip data.zip
• zip and tar are for compressing and • tar compress and archive files. It
archiving file. tar can use gzip and supports archive compressing through
bzip2 to do the compress and then gzip and bzip2. If you are compressing
archive. more the 2 files, tar is recommended
• gzip compress and gunzip instead of gzip or bzip2.
decompresses the gz files. $ tar –zcvf data.tgz data1.txt data2.txt
www.corvit.com
Backup Techniques
Full Mirror Incremental Differential
Backup 2 All Data All data selected Changes from Backup 1 Changes from Backup 1
Backup 3 All Data All data selected Changes from Backup 2 Changes from Backup 1
Backup 4 All Data All data selected Changes from Backup 3 Changes from Backup 1
https://linuxconfig.org/how-to-create-incremental-and-differential-
backups-with-tar
www.corvit.com
I/O Redirection
• Linux programs accept input data
from a keyboard and output to a
terminal by default.
- Data input path is called stdin
(standard input)
- Data output path is called stdout
(standard output)
• Redirection is a feature in Linux such
that when executing a command, you
can change the standard input/output
devices.
• The basic workflow of any Linux
command is that it takes an input and
give an output.
• The standard input is the keyboard
(value:0). The standard output device
is the terminal screen (value:1) and
standard error (value:2).
www.corvit.com
I/O Redirection
• Redirecting input : The < or 0< symbol is used to redirect inputs.
$ cat < file
cat command would take input from file and list content of file
$wc –l < file
input file into the word count program to count the number of
lines.
• Redirecting output: The > or 1> symbol is used to redirect
output.
$ ls > file
write the output of the directory listing to file.
$ ls /test >> file
append data to file instead of overwriting.
• Redirecting Error: The 2> symbol is used to redirect error
messages.
$ cat 2> errorfile
if there is no file named errorfile in the current directory then the
error messages are sent to the standard error device.
www.corvit.com
Partitions
There are two kinds of major partitions on a Linux
system:
• Data Partition: Normal Linux system data,
including the root partition containing all the data
to start up and run the system.
• Swap Partition: Expansion of the computer’s
physical memory, extra memory on hard disk.
www.corvit.com
Swap
• Swap is a space on a
disk that is used when
the amount of
physical RAM memory
is full.
• When a Linux system
runs out of RAM,
inactive pages are
moved from the RAM
to the swap space.
• Swap space can take
the form of either a
dedicated swap
partition or a swap
file.
https://www.devopsschool.com/blog/what-is-swap-in-linux-and-useful-commands-to-work-with-swap-space-in-linux/
www.corvit.com
Does Linux use MBR or GPT
• For old Linux systems with legacy BIOS, use MBR.
Otherwise, stick with GPT.
MBR GPT
4 Primary partitions
or 3 primary partitions + an
Maximum Partition Number 128 primary partitions
infinite number of logical
partitions.
www.corvit.com
Partitions
Primary Extended
• Can hold operating system • Can be further subdivided into
boot files an unlimited number of logical
• Cannot be further subdivided partitions
into logical drives. • Cannot be directly formatted
• Can be formatted with a file with a file system.
system. • However logical partitions
within an extended partition
can be formatted with a file
system.
www.corvit.com