Linux Kernel
Linux Kernel
Kernel Page 1
Kernel (Intro)
Friday, August 30, 2019 6:09 PM
A kernel is the critical component of an operating system. It works as a bridge between the applications and data processing at the hardware level with the help of its inter-process
communication and system calls.
Whenever an operating system is loaded into memory, firstly, the kernel is loaded, and stays there till the operating system gets shut down. The kernel is responsible for taking care of low-level
tasks such as task management, memory management, risk management, etc.
The kernel is responsible for:
A Kernel is the core component of any Linux based operating system. It represents the core aspect of the Linux distributions for Desktop computers and servers. It has a monolithic architecture,
and the operating system operates entirely in the kernel space. The monolithic kernel not only encircles the Central Processing Unit, IPC, and memory, but also has system server calls, device
drivers, and file system management. Linux kernel works as a layer between the software and hardware of a device.
Operations of a Kernel
The kernel is termed as a heart of any operating system as it controls all other programmes in a system. When a device starts, the kernel goes through a process called initialization function,
such as checking memory. It takes care of the memory allocation part and creates an environment for running the applications without any disturbances.
Kernel works as a service provider, so the programs can request the kernel for accomplishing multiple tasks such as requesting the use of the disk, network card, or other pieces of hardware,
and also, kernel sets interrupt for the CPU to enable multitasking. It protects the computational environment by not letting the faulty programmes enter into the operational functions of others. It
stops the unauthorised programs at the entrance by not allowing the memory space and limits the CPU time that they consume.
Types of kernels
Monolithic kernel
It is a widely used kernel by the operating systems. In a Monolithic architecture, the kernel consists of various modules which can dynamically be loaded and unloaded. This kind of architecture
would extend the capabilities of the OS and allows easy extensions to the kernel.
Maintenance of a kernel becomes easy with monolithic architecture because it allows a concerned module to load and unload when there is a need to fix a bug in a particular module. So, it
eliminates the tedious task of bringing down and recompiling the whole kernel for little changes. It is easier in the monolithic kernel to unload the module that is no more in usage.
MicroKernel
Microkernel has evolved as an alternative to the monolithic kernel to address the issue of the ever-growing size of kernel code which monolithic kernel failed to do. This architecture allows some
basic services like protocol stack, device driver management, file system, etc., to run in userspace. This could enhance the capability of OS with minimum code, improved security, and ensures
the stability.
It restricts the damages to the impacted areas by leaving the rest of the system to function correctly without any interruptions. In Microkernel architecture, all the basic OS services are available
to programmes via interprocess communication (IPC). Microkernel allows direct interaction between the device drivers and hardware.
Hybrid kernel
The hybrid kernel can decide what it wants to run in user mode, and in supervisor mode. In hybrid kernel environment, usually, things like device drivers, file system I/O would run in user mode,
whereas server calls and IPC are kept in supervisor mode. It gives the best experience of both the worlds.
Kernel space
Kernel space is found in an elevated state which provides full access to the hardware devices and protects the memory space. This memory space and user space together called as Kernel-
space. In a kernel space environment, core access to the system services and hardware are maintained and provided as a service to the rest of the system.
User Space
Kernel Page 2
The userspace or userland is a code which runs outside operating system kernel environment.
A userspace is defined as various applications or programs or libraries that an operating system uses to connect with the kernel. Because of the complicated process to access the memory,
malicious functions can be restricted only to the user system.
If you want to add code to the Linux kernel, the first thing you need to do is to add some source files to the kernel source tree. There may be situations where you are required to add code to the
kernels while it is running, this process is called as loadable kernel module.
• LKMs save the memory because they are loaded into memory only when required.
The Linux Kernel provides different interfaces to the user space applications that execute different tasks and have different properties. It consists of two separate Application Programming
Interfaces (APIs): one is kernel userspace, and the other is kernel internal. Kernel user space is the Linux API userspace, and allows the programs in the user space into system services and
resources of the kernel.
A system call is a programmatic process in which a program requests a service from the kernel of an operating system. It includes various hardware services such as connecting with hardware
devices, and creating a communication interface among the integral parts of the Kernel. System call creates an efficient interface between an operating system and a process.
Process management:
The Kernel takes care of creating and destroying the different processes, and monitors their connection to the outside world such as input and output. It handles the communication between
different methods via signals, interprocess communication primitive, or pipes. In addition to all these, it also has a scheduler which controls the processes in sharing the CPU.
Memory management:
Memory is a vital component of an Operating system and kernel takes care of it. Linux manages the available memory and hardware mechanisms for virtual and physical mappings.
Memory management isn't just managing 4KB buffers, and it is much more than that. Linux also provides abstractions other than 4kb buffers, known as a slab allocator. Slab allocator uses the
4kb buffer as its base but then allocates structures from inside by monitoring things like, which pages are full, empty, and partially used. This allows the scheme to grow dynamically and in
supporting the more significant needs of the system.
Virtual file system (VFS) is an important integral part of the kernel and facilitates common interface abstraction for the file system. The VFS creates a switching layer between the file system
supported by the kernel and SCI (System Call Interface).
In addition to the above things, Linux supports various types of file systems that require different ways of organizing data to store in physical format. For instance, a disk can be formatted with
the commonly used FAT file system, or Linux standard ext3 file system, or several others.
Device Drivers:
A vast part of the source code of kernel is stored in the device drivers, and that makes a specific hardware device usable. The Linux provides a driver subdirectory that is further divided into
various devices that are supported, such as I2C, Bluetooth, serial, etc.
Architecture-dependent code:
Even though much of the Linux runs on its independent architecture, some elements should be considered for the architecture efficiency and normal operation. Linux has many subdirectories,
and each architecture subdirectory has many numbers of other subsidiaries. And, these subdirectories focus on the specific tasks of the kernel such as memory management, boot, kernel, etc.
As we are aware of the concept called update, we do have that option in the kernel to update it from the older version to a newer one. The retention of old configurations is significant, and to
achieve this, one has to back up the configuration file in the kernel source directory. If anything goes wrong while updating the kernel, follow the below steps.
• Download the latest source code from the kernel.org main page.
Kernel Page 3
• Apply variations to the old version tree to make it as a new one.
• Reconfigure the kernel on the basis of the older kernel configuration file that you had backed up.
The files and systems in the Linux kernel system are where most of the users find difficulties, majorly because it is hard to tell which files are in which directories if you don’t have the
knowledge. For this reason, we shall try to look at the organization of the file systems. We shall also learn how to create, delete, move, and rename directories. Additionally, we shall learn how
to edit files and change permissions.
The file system layout
The UNIX file system can aptly fit into a one-line description; “Everything on a UNIX system that is not a process is a file”. This statement holds true for files that are a little bit more than just
files. Therefore, a Linux system does not differentiate between a file and directory mainly because the directory, in essence, is a file containing names of other files, services, texts, images, and
programs. Additionally, a Linux system also treats input and output devices as files. The general understanding is that the files are a sort of in a tree structure on the main hard drive; this is for
easy management and order. Most of the files on a Linux system are regular files, regardless of the data they hold, be it programs, executable files or normal data.
While we have already said that everything in a Linux system is a file, there is a general understanding that there are some exceptions. For instance:
Directories: A file list of other files
Special files: These are the mechanisms used for input and output. Special files are in /dev.
Links: This is a system to make a file, including directory visible in various parts of the “system tree”.
Domain (sockets): These are special types of files similar to the IP/TCP sockets. These files are protected by the file system access control and they provide inter-process networking.
Named pipes: These types of files are the bridge between processes. They are more or less the same as sockets and enhance communication between processes without the use of networks
or sockets semantics.
Remember that I had indicated that most computer users generalize that the file system is more or less like a tree, here is a good example of a Linux file system tree.
It is important to note that depending on the UNIX system in use, the file system tree may change; some files and directories may change.
The file system tree starts at the slash or the trunk, which, if you look at our table is the (/) forward slash. This is what we call the root directory; it is the underlying directory for all files.
Directories are one level below the slash or root directory often have the slash in their proceeding names to indicate their position and to prevent confusion with other files or directories with
similar names.
A question that plagues most Linux users is where programs and program files are stored when they are installed on the system. Let us examine this for a minute. Linux uses two partitions: The
data partition where the system data, including the root directories and all system resources required to start the system are located, and the swap partition, which is an expansion of the
physical memory on the computer. All files (including programs) are stored in this root directory in accordance to the Linux tree file system we have already looked at.
Manipulating files
To show file names, properties, date of creation, permission, type, size, link files and owners, the Is command is the easiest way.
Creating and deleting files and directories on your system is very important when you want to create new files or delete redundant directories to free up space. Because the graphical interface is
much or less that of MS DOS, creating files is not that difficult. Deleting files on the other hand is a moderately difficult. There are some popular file managers for the GNU/Linux, with most of
them being executable files that are accessible from the desktop manager, home directory icon or the command line using the following commands.
Managing files
Nautilus: This is the default file manager in the Gnome GNU desktop. There are very useful resources on how to use this tool online.
Kernel Page 4
For easier file management, the above applications are worth the time of reading through the documentation and the effort. It is also important to note that there are many more file
management applications, but these are the most popular and have a moderate difficulty level. Additionally, these tools optimize the UNIX commands in a specific manner.
To keep files and things in one place, you must allocate specific file default locations by creating directories and subdirectories for them. You can do this by using the mkdir command. For
instance:
1 john:~> cd archive
2 john:~/archive> mkdir 1999 2000 2001
3 john:~/archive>ls
4 1999/ 2000/ 2001/
5 john:~/archive>mkdir 2001/reports/Suppliers-Industrial/
6 mkdir: cannot create directory `2001/reports/Suppliers-Industrial/':
Additionally, you can create subdirectories easily in one-step by using the – p option. For instance:
1 john:~> cd archive
2 john:~/archive> mkdir 1999 2000 2001
3 john:~/archive>ls
4 1999/ 2000/ 2001/
5 john:~/archive>mkdir 2001/reports/Suppliers-Industrial/
6 mkdir: cannot create directory `2001/reports/Suppliers-Industrial/':
Conclusion
The Linux Kernel plays an important role in resource allocation to different applications. The kernel acts as a centralized place to connect the hardware and software and runs the applications
in a system. Linux kernel has got more popularity due to its open source nature. Users can customize this OS according to their requirements. Therefore, it’s been used by a wide variety of
devices.
The modular characteristic of the Linux kernel allows a wide range of modifications without rebooting the system. The Flexibility of the kernel enables its users to perform their level best.
Moreover, the monolithic nature of this kernel has greater computational power than the microkernel.
From <https://mindmajix.com/linux-kernel-tutorial>
Architechture
Components of the Linux Kernel
A kernel is simply a resource manager; the resource being managed may be a process, memory or hardware device. It manages and arbitrates access to
the resource between multiple competing users. The Linux kernel exists in the kernel space, below the userspace, which is where the user’s applications
are executed. For the user space to communicate with the kernel space, a GNU C Library is incorporated which provides a forum for the system call
interface to connect to the kernel space and allow transition back to the userspace.
1. The system call interface; this is the topmost and undertakes the basic actions such as read and write.
2. The kernel code; is located below the system call interface, it is common to all of the processor architectures supported by Linux, it is sometimes
defined as architecture-independent kernel code.
3. The architecture-dependent code; it is under the architecture-independent code, forms what is usually referred to as a Board Support Package
(BSP) – this contains a small program called the bootloader that places the Operating System and device drivers into memory.
The architectural perspective of the Linux kernel consists of: System call interface, Process Management, the Virtual File system, Memory Management,
Network Stack, Architecture and the Device Drivers.
1. System call interface; is a thin layer that is used to undertake function calls from user space into the kernel. This interface may be architecture
dependent
2. Process management; is mainly there to execute the processes. These are referred to as the thread in a kernel and are representing an individual
virtualization of the particular processor
3. Memory management; memory is managed in what are known as pages for efficiency. Linux includes the methods in which to manage the available
memory as well as the hardware mechanisms for physical and virtual mappings. Swap space is also provided
4. Virtual file system; it provides a standard interface abstraction for the file systems. It provides a switching layer between the system call interface and
the file systems supported by the kernel.
5. Network stack; is designed as a layered architecture modeled after the particular protocols.
6. Device drivers; a significant part of the source code in the Linux kernel is found in the device drivers that make a particular hardware device
usable. Device driver tutorial
7. Architecture-dependent code; those elements that depend on the architecture on which they run, hence must consider the architectural design for
normal operation and efficiency.
Interfaces
System calls and Interrupts
Kernel Page 5
System calls and Interrupts
Applications pass information to the kernel through system calls. A library contains functions that the applications work with. The libraries then, through the
system call interface, instruct the kernel to perform a task that the application wants. What is a Linux System Call?
Interrupts offer a way through which the Linux kernel manages the systems’ hardware. If hardware has to communicate with a system, an interrupt on the
processor does the trick, and this is passed on to the Linux kernel.
Linux ABI
This refers to the kernel-user space ABI (Application Binary Interface). This is explained as the interface that exists between program modules. When
comparing API and ABI, the difference is that ABI’s are used to access external codes that are already compiled while API are structures for managing
software. Defining an important ABI is majorly the work of Linux distributions than it is for the Linux kernel. A specific ABI should be defined for each
instruction set, for example, x86-64. End-users of Linux products are interested in the ABIs rather than the API.
1. Cgroups subsystem.
2. The Direct Rendering Manager’s system calls.
3. A readahead feature.
4. Getrandom call that is present in V 3.17.
5. System calls such as futex, epoll, splice, dnotify, fanotify and inotify.
More information about POSIX Standard is here.
The loadable kernel module can be compared to the kernel extensions in other operating systems. You can put a module into the kernel by either loading it
as an LKM or binding it into the base kernel.
• Rebuilding your kernel often is not necessary, saving time and avoids errors.
• They assist in figuring out system problems such as bugs.
• LKMs save you space as you only have them loaded when you need to use them.
• Give much faster maintenance and debugging time.
Uses of LKMs
1. Device drivers; the kernel exchanges information with hardware through this. A kernel must have a device’s driver before using it.
2. Filesystem drivers; this translates the contents of a filesystem
3. System calls; programs in the user space utilize system calls to acquire services from the kernel.
4. Network drivers; interprets a network protocol
5. Executable interpreters; loads and manages an executable.
Compiling the Linux Kernel
Unlike what most people say, compiling the Linux kernel is a simple task. The following is a step-by-step illustration of the process using one of the Linux
distributions: Fedora 13 KDE. (It is advisable to backup your data and grub.conf just in case something goes wrong)
Kernel Page 6
distributions: Fedora 13 KDE. (It is advisable to backup your data and grub.conf just in case something goes wrong)
1. Get the latest source code from the main kernel.org website
2. Apply the variations to the old source tree to bring it up to the latest version.
3. Reconfigure the kernel based on the preceding kernel configuration file you had backed up.
4. Build the new kernel.
5. Now you can install the new build the kernel.
Downloading the new source; the Linux kernel developers understand that some users may not want to download the full source code for the kernel
updates, as this would waste time and bandwidth. Therefore, a patch is made available which can upgrade an older kernel release. Users only need to
know which patch applies to a particular version, since a kernel patch file will only update the source code from one specific release. The different patch
files can be applied through the following ways;
Conclusion
The Linux kernel mainly acts as a resource manager acting as an abstract layer for the applications. The applications have a connection with the kernel
which in turn interacts with the hardware and services the applications. Linux is a multitasking system allowing multiple processes to execute concurrently.
The Linux kernel is popular due to its open source nature that allows users to alter the kernel to what is suitable for them and their hardware. Therefore it
can be used in a variety of devices, unlike other operating systems.
The modular characteristic of the Linux kernel adds more thrill to its users. This is because of the wide variety of modifications that can be made here
without rebooting the system. The flexibility gives its users a big room to actualize their imaginations.
Moreover, the monolithic nature of the kernel is a great advantage as it has a high processing ability than the microkernel. The main setback with the Linux
type of kernel is that if any of its services fail, then the whole system goes down with it. Latest versions have been designed in a way that if a new service
is added, there is no need of modifying the whole operating system. This is an improvement when compared to previous versions.
From <https://linuxhint.com/linux-kernel-tutorial-beginners/>
Kernel Page 7
How to work with kernel tunables
There are three ways to modify kernel tunables.
1. Using the sysctl command
2. By manually modifying configuration files in the /etc/sysctl.d/ directory
3. Through a shell, interacting with the virtual file system mounted at /proc/sys
From <https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/kernel_administration_guide/listing_of_kernel_parameters_and_values#kernel_command_line_parameters>
Control groups
What is a control group?
NOTE
Control Group Namespaces are a Technology Preview in Red Hat Enterprise Linux 7.5
Linux Control Groups (cgroups) enable limits on the use of system hardware, ensuring that an individual process running inside a cgroup only utilizes as much as has been allowed in the cgroups configuration.
Control Groups restrict the volume of usage on a resource that has been enabled by a namespace. For example, the network namespace allows a process to access a particular network card, the cgroup ensures
that the process does not exceed 50% usage of that card, ensuring bandwidth is available for other processes.
Control Group Namespaces provide a virtualized view of individual cgroups through the /proc/self/ns/cgroup interface.
The purpose is to prevent leakage of privileged data from the global namespaces to the cgroup and to enable other features, such as container migration.
Because it is now much easier to associate a container with a single cgroup, containers have a much more coherent cgroup view, it also enables tasks inside the container to have a virtualized view of the cgroup
it belongs to.
4.1.2. What is a namespace?
Namespaces are a kernel feature that allow a virtual view of isolated system resources. By isolating a process from system resources, you can specify and control what a process is able to interact with.
Namespaces are an essential part of Control Groups.
4.1.3. Supported namespaces
The following namespaces are supported from Red Hat Enterprise Linux 7.5 and later
• Mount
○ The mount namespace isolates file system mount points, enabling each process to have a distinct filesystem space within wich to operate.
• UTS
○ Hostname and NIS domain name
• IPC
Kernel Page 8
• IPC
○ System V IPC, POSIX message queues
• PID
○ Process IDs
• Network
○ Network devices, stacks, ports, etc.
• User
○ User and group IDs
• Control Groups
○ Isolates cgroups
NOTE
Kernel Page 9