CONTENTS

 Introduction to security in operating system

Analysis and definition of security in o.s

Security-Focused o.s

Security-Evaluated o.s

Trends in security in o.s

Unix security

Security in ordinary operating system

Windows security o.s

Memory protection

Isolated system
Introduction to security in operating system :
The operating system (OS) is the backbone of every modern computer
system, managing the system’s resources and executing applications.
Its security is critical as a vulnerability in the OS or any applications
running on it can expose the entire system to risk
secure OSes is, among others, the use in container-focused OSes and
intelligent vehicles where digital features are increasing, as well as in
mobile phones.
Hardening measures will vary as different use cases have different
requirements for a secure OS. In addition, different technologies are
used to complement the security provided by the OS.
Analysis and definition of security in o.s :

Every modern computer system runs a core piece of software

executed on top of the hardware. This software is the operating
system (OS). It is responsible for allocating the primary resources
of the system .
The crucial role of the OS, its security (or the lack of security)
might have a significant impact on the whole system: a
vulnerability in the OS, or any applications running in it, exposes a
danger to all the other applications running in the system as well
as to all the data stored in it. This situation becomes highly
problematic when the system stores important (confidential) data
or runs critical applications in high-risk facilities (e.g., satellite
communications, power plants, banking systems, aircraft systems,
and SCADA systems). Therefore, it is essential to improve OS
security to ensure data integrity, confidentiality and availability.
Security-Focused os :

A security-focused operating system should guarantee the secure

or trusted execution of components that might not be secure
(programs). That is, the OS should protect the rest of the system
from modules that an attacker might exploit to get control of the
system, for instance, using sandboxing, compartmentalization or
by isolating cryptography functions and key management. Qube
OS is one such OS, which is especially valuable in industries
where sensitive data has to be securely segregated. Other
examples include Tails OS and React OS .
The operating system can reduce its
exposure to security threats, as the keys and sensitive data are not
accessible to the operating system or other software. Similarly to
an HSM, a Trusted Execution Environment, or TEE (Chap. 18) is
a secure area of a computing device, typically implemented on the
chip itself, that provides a secure environment for executing
sensitive operations. The TEE is also used to provide secure
storage for cryptographic keys and other sensitive data, such as
passwords and certificates, which would protect such assets in a
scenario where the OS is compromised. For example, iOS uses a
dedicated, isolated and hardware backed subsystem called secure
enclave to isolate important cryptographic tasks. And on Android
smart phones, it depends on the manufacturer of the smart phone.

Security-
Evaluated o.s :

A security-evaluated OS is an OS that has achieved certification

from an external security-auditing organization. However, they
still need to implement more security mechanisms to make certain
system areas more secure (e.g., cryptographic modules, fine-
grained access control) according to the criteria. Some of the most
popular evaluation criteria are Common Criteria [2], FIPS 140-2
[3], and ITSEC [4]. Examples of such OSs are SUSE Linux or
some Red Hat Linux Enterprise versions, Windows 10 Enterprise,
etc.
Even though a baseline exists for achieving a minimum level
of security, the ultimate set of requirements to make a secure OS
depends on the specific use case. For instance, a mobile OS has
different requirements than a container-focused OS. Therefore,
different measures can be taken to harden the underneath
operating system for each specific use case.
Trends in security in o.s :

the rest of the system could be affected. For instance, they disrupt
the applications running on the top or steal critical business
information. On the other hand, if an attacker compromises an
application running inside a container, he/she could try to escape
the container and gain access to the host OS and/or pivot to other
containers, achieving the same results as in the previous example.
With that in mind, it seems reasonable that a container-focused OS
might also be security-focused, including features such as those
mentioned in Sect. . That is why recently, the first standards on
container security are emerging . Examples of well-known
container-focused OSes are Flat Car Container Linux or Bottle
rocket . However, those are not considered secure OSes since
some of the features mentioned in previous sections are not
implemented. Another example of such an OS that focuses on
security is ARCA OS, from Cy Sec a Swiss startup launched in
2018 in Lausanne, EPFL.
Unix security :

UNIX operating system, we will try to place our observations in a

wider context than just the UNIX system or one particular version
of the UNIX system. UNIX system security is neither better nor
worse than that of other systems. Any system that provides the
same facilities as the UNIX system will necessarily have similar
hazards. From its inception, the UNIX system was designed to be
user friendly, and most decisions that pitted security against ease
of use were heavily weighted in favor of ease of use.
The result has been that the UNIX system
has become a fertile test bed for the development of reasonable
security procedures that interfere to the minimum possible extent
with ease of use. The major weakness of any information system
such as the UNIX system resides in the habits and attitudes of the
user community. Naivete and carelessness will produce awful
security under almost any conditions.
Security in ordinary operating system :

The reference monitor and protection system are stored in the

kernel, but this does not guarantee tamper-protection. First, the
protection system is discretionary, so it may be tampered by any
running process. Untrusted user processes can modify permissions
to their user’s data arbitrarily, so enforcing security goals on user
data is not possible. Second, the UNIX kernel is not as protected
from untrusted user processes as theMultics kernel is. Both use
protection rings for isolation, but the Multics system also
explicitly specifies gates for verifying the legality of the ring
transition arguments. While UNIX kernels often provide
procedures to verify system call arguments, such procedures are
may be misplaced. Finally, user-level processes have a variety of
interfaces to access and modify the kernel itself above and beyond
system calls, ranging from the ability to install kernel modules to
special file systems to interfaces through netlink sockets to direct
access to kernel memory . Ensuring that these interfaces can only
be accessed by trusted code has become impractical.
Windows security o.s :

The Microsoft Windows operating system has

changed a lot from the simple personal computer operating
system, DOS. It has grown from a single product offering to a full
family of products to meet different needs. The current versions of
Windows address a wide variety of computing needs, from
portable devices and workstations to enterprise-class, high-
performance platforms. The Windows operating system is
designed to be a modular system to provide the widest variety of
services for most platform requirements .

The current versions of client and server Windows

operating systems in use today are all based on the legacy
Windows NT code base. Windows NT was Microsoft’s first
operating system designed with security in mind. The first
commercial version of Windows NT was version 3.1, released in
1993. At the time, Windows NT was a ground-breaking product
from Microsoft. Several versions of Windows share a common
ancestry back to Windows NT .
Memory protection :

• Relocation process takes a program considering address 0 for the

first instruction and replaces rest of the addresses with the actual
addresses in memory that the program is residing at.
• Every time each address adds constant relocation factor to get
the address. The first address of the program becomes the value of
relocation factor.
• The fence register acts as a relocation device defined in
hardware. To generate address of the program, the fence register
contents are read and added to the address for each instruction of
the program.
Isolated system :

Isolation is and will remain an important building block for secure

systems, the focus on isolation mechanisms obscures two difficult,
recent lessons. First, modern hardware is highly optimized for
performance which makes isolation difficult. Second, even with
fully effective isolation mechanisms, secure systems will likely
consist of multiple isolated but communicating environments,
where the communication creates new side channels. This paper
advocates redundant protections for isolated computation using a
combination of software and hardware techniques. It also
advocates securing and optimizing the communication among
multiple, distinct isolated computations, often within the same
machine.