Os Unit 6 Notes
Os Unit 6 Notes
Linux
Q. Write a note on History of Unix and Linux?
History of Unix
The history of Unix dates back to the mid-1960s, when the Massachusetts Institute of
Technology, AT&T Bell Labs, and General Electric were jointly developing an
experimental time-sharing operating system called Multics for the GE-645 mainframe.
Multics introduced many innovations, but also had many problems.
Evolution of Unix
In 1969, a team of developers of Bell Labs started a project to make a common software
for all the computers and named it as 'Unix'.
It was simple and elegant, used 'C' language instead of assembly language and its code was
recyclable.
As it was recyclable, a part of its code now commonly called 'kernel' was used to develop
the operating system and other functions and could be used on different systems. Also its
source code was open source.
Initially, Unix was only found in large organizations like government, university, or larger
financial corporations with mainframes and minicomputers (PC is a microcomputer).
Unix Expansion
In eighties, many organizations like IBM, HP and dozen other companies started creating
their own Unix.
It results in a mess of Unix dialects.
Then in 1983, Richard Stallman developed GNU project with the goal to make it freely
available Unix like operating system and to be used by everyone.
But his project failed in gaining popularity.
Many other Unix like operating system came into existence but none of them was able to
gain popularity.
1
History Of Linux
Evolution of Linux
In 1991, Linus Torvalds a student at the university of Helsinki, Finland, thought to have a
freely available academic version of Unix started writing its own code.
Later this project became the Linux kernel. He wrote this program specially for his own
PC as he wanted to use Unix 386 Intel computer but couldn't afford it.
He did it on MINIX using GNU C compiler.
GNU C compiler is still the main choice to compile Linux code, but other compilers are
also used like Intel C compiler.
He started it just for fun but ended up with such a large project.
Firstly, he wanted to name it as 'Freax' but later it became 'Linux'.
He published the Linux kernel under his own license and was restricted to use as
commercially. Linux uses most of its tools from GNU software and are under GNU
copyright.
In 1992, he released the kernel under GNU General Public License.
Linux Today
Today, supercomputers, smart phones, desktop, web servers, tablet, laptops and home
appliances like washing machines, DVD players, routers, modems, cars, refrigerators, etc
use Linux OS.
Overview Of Linux - Linux Goals
Linux® is an open-source operating system (OS).
An operating system is the software that directly manages a system’s hardware and
resources, like CPU, memory, and storage.
The OS sits between applications and hardware and makes the connections between all of
your software and the physical resources that do the work.
Linux is a community of open-source Unix like operating systems that are based on the
Linux Kernel.
It was initially released by Linus Torvalds on September 17, 1991.
2
It is a free and open-source operating system and the source code can be modified and
distributed to anyone commercially or noncommercially under the GNU General Public
License.
Initially, Linux was created for personal computers and gradually it was used in other
machines like servers, mainframe computers, supercomputers, etc.
Nowadays, Linux is also used in embedded systems like routers, automation controls,
televisions, digital video recorders, video game consoles, smartwatches, etc.
The biggest success of Linux is Android(operating system) it is based on the Linux kernel
that is running on smartphones and tablets.
Due to android Linux has the largest installed base of all general-purpose operating
systems.
Linux is generally packaged in a Linux distribution.
Linux Distribution
Linux distribution is an operating system that is made up of a collection of software based
on Linux kernel or you can say distribution contains the Linux kernel and supporting
libraries and software.
And you can get Linux based operating system by downloading one of the Linux
distributions and these distributions are available for different types of devices like
embedded devices, personal computers, etc. Around 600 + Linux Distributions are
available and some of the popular Linux distributions are:
1. MX Linux 2. Manjaro 3. Linux Mint 4. elementary 5. Ubuntu 6. Debian 7.Solus
8. FedoraopenSUSE 9. Deepin
Linux Goals
Main design goals are speed, efficiency, and standardization.
1. Kernel: Kernel is the core of the Linux based operating system. It virtualizes the common
hardware resources of the computer to provide each process with its virtual resources. This
3
makes the process seem as if it is the sole process running on the machine. The kernel is also
responsible for preventing and mitigating conflicts between different processes. Different types
of the kernel are:
Monolithic Kernel, Hybrid kernels,Exo kernels,Micro kernels
2. System Library: Isthe special types of functions that are used to implement the functionality
of the operating system.
3. Shell: It is an interface to the kernel which hides the complexity of the kernel’s functions from
the users. It takes commands from the user and executes the kernel’s functions.
4. Hardware Layer: This layer consists of all peripheral devices like RAM/ HDD/ CPU etc.
5. System Utility: It provides the functionalities of an operating system to the user.
Advantages of Linux
4
Bridge
Bonded interface
The Linux bonding driver provides a method for aggregating multiple network interfaces into a
single logical "bonded" interface.
The behaviour of the bonded interface depends on the mode; generally speaking, modes provide
either hot standby or load balancing services.
Team device
Similar a bonded interface, the purpose of a team device is to provide a mechanism to group
multiple NICs (ports) into one logical one (teamdev) at the L2 layer.
The main thing to realize is that a team device is not trying to replicate or mimic a bonded interface.
What it does is to solve the same problem using a different approach, using, for example, a lockless
(RCU) TX/RX path and modular design.
5
ifconfig
ifconfig is a command line tool for UNIX-like systems that allows for diagnosing and configuring
network interfaces. At boot time, it sets up network interfaces such as Loopback and Ethernet. Most
of the time, however, ifconfig is used for network diagnostics.
Before diving into details of its output, let’s first make clear what is an interface.
network interface
A network interface is a software interface to networking hardware. Linux kernel distinguishes
between two types of network interfaces: physical and virtual.
Physical network interface represents an actual network hardware device such as network interface
controller (NIC). In practice, you’ll often find eth0 interface, which represents Ethernet network
card.
interfaces
There are 3 network interfaces on the box:
eth0 is a physical interface representing Ethernet network card. It’s used for communication with
other computers on the network and on the Internet.
lo is a special virtual network interface called loopback device. Loopback is used mainly for
diagnostics and troubleshooting, and to connect to services running on local host.
docker0 is a virtual bridge interface created by Docker. This bridge creates a separate network for
docker containers and allows them to communicate with each other.
interface details
Link encap shows how packets are encapsulated for transmission. Most interfaces wrap packets
in Ethernet frames.
HWaddr is hardware address of the ethernet interface (also known as MAC address).
inet addr is IPv4 address assigned to the interface.
Bcast is broadcast address for the interface.
Mask is network mask for the interface.
inet6 addr is IPv6 address assigned to the interface.
Scope is scope of IPv6 address. It can be link-local or global. Link-local address is used in local
area network and is not routable. Global address is routable.
UP indicates that kernel modules related to the interface have been loaded and interface is activated.
6
BROADCAST indicates that interface is configured to handle broadcast packets, which is required
for obtaining IP address via DHCP.
RUNNING indicates that interface is ready to accept data.
MULTICAST indicates that interface supports multicasting.
MTU is maximum transmission unit. IP datagrams larger than MTU bytes will be fragmented into
multiple Ethernet frames.
Metric determines the cost of using the interface. Interfaces with lower cost have higher priority.
interface stats
RX packets is a total number of packets received.
RX errors shows a total number of packets received with error. This includes too-long-frames
errors, ring-buffer overflow errors, CRC errors, frame alignment errors, fifo overruns, and missed
packets.
RX dropped is a number of dropped packets due to unintended VLAN tags or receiving IPv6 frames
when interface is not configured for IPv6.
RX overruns is a number of received packets that experienced FIFO overruns, caused by rate at
which a buffer gets full and kernel isn’t able to empty it.
RX frame is a number of misaligned frames, i.e. frames with length not divisible by 8.
TX packets is total number of packets transmitted.
7
Shell is broadly classified into two categories –
1. Command Line Shell
2. Graphical shell
Command Line Shell
Shell can be accessed by user using a command line interface.
A special program called Terminal in linux/macOS or Command Prompt in Windows OS
is provided to type in the human readable commands such as “cat”, “ls” etc.
and then it is being execute.
The result is then displayed on the terminal to the user.
Graphical shell
Graphical shells provide means for manipulating programs based on graphical user
interface (GUI), by allowing for operations such as opening, closing, moving and resizing
windows, as well as switching focus between windows.
Window OS or Ubuntu OS can be considered as good example which provide GUI to
user for interacting with program. User do not need to type in command for every
actions.
There are several shells are available for Linux systems like –
BASH (Bourne Again SHell) – It is most widely used shell in Linux systems.
It is used as default login shell in Linux systems and in macOS.
It can also be installed on Windows OS.
CSH (C SHell) – The C shell’s syntax and usage are very similar to the C programming
language.
KSH (Korn SHell) – The Korn Shell also was the base for the POSIX Shell standard
specifications etc.
9
ss or socket statistics is a good alternative to netstat it directory gather information from
kernel space and play fast in comparision to the netstat utility.
Kernel is a small and special code which is the core component of Linux OS and directly
interacts with hardware.
It is the intermediate level between software and hardware which provides low level service
to user mode’s components.
It is fully developed in C language and file system architecture Moreover, it has different
blocks which manage various operations.
In this tutorial we will learn about kernel architecture of Linux.
Kernel runs a number of processes concurrently and manages the various resources. It is
viewed as a resource manager when several programs run concurrently on a system.
In this case, the kernel is an instance that shares available resources like CPU time, disk
space, network connections etc.
Types of kernel
Kernel architecture is classified into two types:
1.Monolithic
2. Micro Kernel
Monolithic Kernels
In the traditional monolithic kernel architecture, all the basic system services like process
and memory management, interrupt handling etc. were packaged into a single module in
kernel space.
Module is an object file in which the whole code can be linked to kernel at run time.
Huge space is consumed by traditional monolithic kernel and maintenance level is very
low.
It takes few hours for recompilation if some new feature is added because all the services
are attached into one single module.
Modern monolithic kernel architecture consists of different modules which can be
dynamically loaded and un-loaded. In this way maintainability is very easy because kernel
needs to take care of only loaded module.
Recompilation is not required upon added feature or some changes. Monolithic kernel is
faster than micro kernel.
Micro kernels
Monolithic kernel architecture supports the modular approach. All service modules are not
run in kernel space as compared to monolithic kernel. Device driver management, protocol
stack, file system etc are run in user space. This reduces the kernel code size and also
increases the security.
10
Fundamental Architecture of Linux
User Space
All user programs and applications are executed in user space. User Space cannot directly
access the memory and hardware.
It accesses the hardware through kernel space. Processes or programs which are running in
user space only access some part of memory by system call.
Due to full protection, crashes in user mode are recoverable.
GNU C library provides the mechanism switching user space application to kernel space.
Kernel Space
All kernel programs are executed in kernel space.
Kernel space accesses full part of memory and directly interacts with hardware like RAM,
Hard disk etc
It is divided in different blocks and modules which manage all operations (like file
management, memory management, process management etc.) in kernel space and
applications running in user space.
Kernel space consists of system call interface, Kernel (core component of Linux) and
device module.
Kernel is independent from hardware.
It is common for all Hardware processors which are supported by Linux. You can run
kernel on any processor like Intel, ARM, Atmel etc.
It acts as a resource manager in Kernel space and performs process management, file
management, memory management, Interrupt handler, scheduling of process, etc.
It is a powerful structure which handles all kinds of operations.
Architecture of Kernel
11
Kernel architecture follows modular approach.
Each block in kernel (i.e. file management) is nothing but piece of code written in C
language.
Each block consists of powerful structure for handling various operations.
Process management
Process management is handling the management of various processes which are run at the
same time. Process is an instance of ‘program in execution’ like open the file, access drive,
access external resource (i.e. printer), etc. and can be created and destroyed.
Process management gives the information of what’s happening with a process and manages
its priorities, such as which address should be assigned to process, file allocated to process,
state of process (like running, waiting, stop) etc.
Memory management
Memory management is the most important part of kernel which handles assignment of
address space to process and application.
Basically, memory management assigns virtual memory instead of physical memory,
whereby latter is the actual address space in RAM.
Don’t be confused between physical and virtual memory.
Assignment of virtual address overcomes the limitation of assignment of physical memory.
Conversion from physical address to virtual address is performed by MMU (Memory
Management Unit which provides the protection of memory interference, sharing of
memory and allocation of virtual memory.
Physical address space is divided into some block of memory called frame that contains a
number of pages.
12
File is simple data structure (in form of some byte of memory) which can store any kind of
data like text, image, video, music and whatever you want in computer.
Linux file system is organized as a directory in tree form. Directory is a collection or group
of files.
Two types of directory are available in Linux system:
1) root
2) Sub directory.
Root is main directory and cannot be accessed without permission of super user.
It is the parent and main directory in file system denoted as forward slash ( / ).
Sub directory is under the root directory which can be created or renamed by user.
Linux supports number of file types like ext2, ext3, device file, block file, networking file
system etc.
Each file system is stored in different a disk partition.
13
Process Control :
This system calls perform the task of process creation, process termination, etc.
The Linux System calls under this are fork() , exit() , exec().
fork()
A new process is created by the fork() system call.
A new process may be created with fork() without a new program being
run-the new sub-process simply continues to execute exactly the same
program that the first (parent) process was running.
It is one of the most widely used system calls under process management.
exit()
The exit() system call is used by a program to terminate its
execution.
The operating system reclaims resources that were used by the
process after the exit() system call.
exec()
A new program will start executing after a call to exec()
Running a new program does not require that a new process be
created first: any process may call exec() at any time.
The currently running program is immediately terminated, and the
new program starts executing in the context of the existing process.
File Management:
File management system calls handle file manipulation jobs like creating a file,
reading, and writing, etc.
The Linux System calls under this are open(), read(), write(), close().
open():
It is the system call to open a file.
This system call just opens the file, to perform operations such as
read and write, we need to execute different system call to perform
the operations.
read():
This system call opens the file in reading mode
We can not edit the files with this system call.
Multiple processes can execute the read() system call on the same
file simultaneously.
write():
This system call opens the file in writing mode
14
We can edit the files with this system call.
Multiple processes can not execute the write() system call on the
same file simultaneously.
close():
This system call closes the opened file.
Device Management :
Device management does the job of device manipulation like reading from
device buffers, writing into device buffers, etc.
The Linux System calls under this is ioctl().
ioctl():
ioctl() is referred to as Input and Output Control.
ioctl is a system call for device-specific input/output operations and
other operations which cannot be expressed by regular system calls.
Information Maintenance:
It handles information and its transfer between the OS and the user program.
In addition, OS keeps the information about all its processes and system calls are
used to access this information.
The System calls under this are getpid(), alarm(), sleep().
getpid():
getpid stands for Get the Process ID.
The getpid() function shall return the process ID of the calling
process.
The getpid() function shall always be successful and no return
value is reserved to indicate an error.
alarm():
This system call sets an alarm clock for the delivery of a signal that
when it has to be reached.
It arranges for a signal to be delivered to the calling process.
sleep():
This System call suspends the execution of the currently running
process for some interval of time
Meanwhile, during this interval, another process is given chance to
execute
Communication:
These types of system calls are specially used for inter-process communications.
Two models are used for inter-process communication
1. Message Passing(processes exchange messages with one another)
2. Shared memory(processes share memory region to communicate)
The system calls under this are pipe() , shmget() ,mmap().
pipe():
15
The pipe() system call is used to communicate between different
Linux processes.
It is mainly used for inter-process communication.
The pipe() system function is used to open file descriptors.
shmget():
shmget stands for shared memory segment.
It is mainly used for Shared memory communication.
This system call is used to access the shared memory and access the
messages in order to communicate with the process.
mmap():
This function call is used to map or unmap files or devices into
memory.
The mmap() system call is responsible for mapping the content of
the file to the virtual memory space of the process.
These are the various system calls involved in LINUX operating system.
Process
A process is a computer program under execution. Linux is running many processes
at any given time.
We can monitor them on the terminal using the ps command or on the System
Monitor UI.
For instance, let’s see an example of using the ps command to view all the processes
running on the machine:
As we run new commands/applications or the old commands complete, we can see
the number of processes grow and shrink dynamically.
Linux processes are isolated and do not interrupt each other’s execution.
With a PID, we can identify any process in Linux. Internally, the kernel uniquely
allocates this number and releases it for reuse after the process exits.
We can see PID as the second column in the output of the above ps command.
Since many processes are running at any given time in Linux, they have to share the
CPU.
The process of switching between two executing processes on the CPU is called
process context switching.
Process context switching is expensive because the kernel has to save old registers
and load current registers, memory maps, and other resources.
Single-Threaded Process
Any thread created within the process shares the same memory and resources of
the process.
In a single-threaded process, the process and thread are the same, as there’s only
one thing happening.
We can also validate ps -eLf output from our previous discussion that PID and
LWP are the same for the single-threaded process.
Multi-Threaded Process
In a multi-threaded process, the process has more than one thread. Such a process
accomplishes multiple tasks simultaneously or almost at the same time.
Threads also can switch faster compared to the processes in the CPU.
Internally, the thread has only a stack in the memory, and they share the heap
(process memory) with the parent process.
Due to this nature of thread, we also call it a Light-Weight Process (LWP).
There are both benefits and drawbacks of sharing the same memory with other
threads.
17
The most important benefit is that we can create threads faster than processes
since we don’t have to allocate memory and resources. The other benefit is the low
cost of inter-thread communication.
Thread
A thread is a lightweight process.
A process can do more than one unit of work concurrently by creating one or more
threads.
These threads, being lightweight, can be spawned quickly.
Let’s see an example and identify the process and its thread in Linux using the ps -
eLf command. We’re interested in PID, LWP, and NLWP attributes:
1.Real-time Processes
2.Conventional Processes
Real-time Processes
Real-time processes are required to ‘obey’ response time constraints without any
regard to the system’s load.
In different words, real-time processes are urgent and cannot be delayed no
matter the circumstances.
An example of a real-time process in Linux is the migration process which is
responsible for distributing processes across CPU cores.
Conventional Processes
Conventional processes don’t have strict response time constraints and they can
suffer from delays in case the system is ‘busy’.
An example of a conventional process can be the browser process you’re using to
read this post.
Each process type has a different scheduling algorithm, and as long as there are
ready-to-run real-time processes they will run and make the conventional
processes wait.
2. MBR
3. GRUB
20
Grub configuration file is /boot/grub/grub.conf (/etc/grub.conf is a link to this).
The following is sample grub.conf of CentOS.
As you notice from the above info, it contains kernel and initrd image.
So, in simple terms GRUB just loads and executes Kernel and initrd images.
4. Kernel
5. Init
6. Runlevel programs
When the Linux system is booting up, you might see various services getting
started. For example, it might say “starting sendmail …. OK”.
Those are the runlevel programs, executed from the run level directory as defined
by your run level.
21
Depending on your default init level setting, the system will execute the programs
from one of the following directories.
Run level 0 – /etc/rc.d/rc0.d/
Run level 1 – /etc/rc.d/rc1.d/
Run level 2 – /etc/rc.d/rc2.d/
Run level 3 – /etc/rc.d/rc3.d/
Run level 4 – /etc/rc.d/rc4.d/
Run level 5 – /etc/rc.d/rc5.d/
Run level 6 – /etc/rc.d/rc6.d/
22