Operating

System Security
Chapter-12
Content
12.1- Introduction To OS Security
12.2- System Security Planning
12.3- OS Hardening
12.4- Application Security
12.5- Security Maintenance
12.6- Linux/Unix Security
12.7- Windows Security
12.8- Virtualization Security
 Introduction To Operating System Security

Computer client and server systems are central components of the IT infrastructure
for most organizations, may hold critical data and applications, and are a necessary
tool for the function of an organization.

So we need to be aware of the possible vulnerabilities in operating systems and

applications, and also the presence of worms scanning for such vulnerabilities at
high rates.

Thus, A system may be compromised during the installation, To ensure this doesn't
happen we need to check for latest patches or implement other hardening methods.
Hence, building and deploying system should be a planned process designed
to counter such a threat, and maintain security during operational lifetime.

[SCAR08] states that this process should contain:

● Assess risks and plan the system deployment.

● Secure the underlying operating system and then the key applications.
● Ensure any critical content is secured.
● Ensure appropriate network protection mechanisms are used.
● Ensure appropriate processes are used to maintain security.
 System Security Planning
The first step in deploying a system is planning. Careful planning can ensure that the new system is
as secure as possible and follows any necessary policies.

The aim of specific system installation planning is to maximize security and minimizing costs.

This planning process needs to determine the security requirements for the system, its applications
and data, and of its users.

This then guides the selection of appropriate software for the operating system and applications,
and provides guidance on appropriate user configuration and access control settings. It also guides
the selection of other hardening measures required. The plan also needs to identify appropriate
personnel to install and manage the system, noting the skills required and any training needed.
[SCAR08] provides a list of items that should be considered during the system security
planning process:

1) The purpose of the system, the type of information stored, the applications and
services provided, and their security requirements.
2) User roles and privileges of each role.
3) User authentication methods.
4) Information access management.
5) Access to information stored on the host like DB server is managed.
6) System administrator management (remote or local).
7) Any additional security measures required on the system, including the use of host
firewalls, anti-virus or other malware protection mechanisms, and logging.
 OS Hardening
The First and foremost thing in securing a system is to secure its base OS upon which all other
applications run.

A good security foundation needs a properly installed, patched, and configured operating
system.

Unfortunately, the default configuration for many operating systems often maximizes ease of
use and functionality, rather than security.

Further, every organization has its own security need based on the requirements the
Operating Systems security profile and configuration differ.

Appropriate security configuration guides and checklists exist for most common operating
systems, and these should be consulted, though always informed by the specific needs of each
organization and their systems. In some cases, automated tools may be available to further
assist in securing the system configuration.
[SCAR08] suggests the following basic steps that should be used to secure an operating
system:

● Install and patch the operating system.

● Harden and configure the operating system to adequately address the identified
security needs of the system by:
○ Removing unnecessary services, applications, and protocols.
○ Configuring users, groups, and permissions.
○ Configuring resource controls.
● Install and configure additional security controls, such as anti-virus, host based
firewalls, and intrusion detection systems (IDS), if needed.
● Test the security of the basic operating system to ensure that the steps taken
adequately address its security needs.
 Application Security
Application security describes security measures at the application level that aim to
prevent data or code within the app from being stolen or hijacked. It encompasses the
security considerations that happen during application development and design, but it
also involves systems and approaches to protect apps after they get deployed.

Application security may include hardware, software, and procedures that identify or
minimize security vulnerabilities. A router that prevents anyone from viewing a
computer’s IP address from the Internet is a form of hardware application security. But
security measures at the application level are also typically built into the software, such
as an application firewall that strictly defines what activities are allowed and prohibited.
Procedures can entail things like an application security routine that includes protocols
such as regular testing.
Application Security can be done in 2 ways:

Application Configuration:

This may include creating and specifying appropriate data storage areas for
the application, and making appropriate changes to the application or service
default configuration details.

Encryption Technology:

Encryption is a key enabling technology that may be used to secure data both
in transit and when stored. If such technologies are required for the system,
then they must be configured, and appropriate cryptographic keys created,
signed, and secured.
 Security Maintenance
Once the system is built securely, and deployed, the process of maintaining security is
continuous. This to tackle the constant change in environments and discovery of new
vulnerabilities, and hence exposure to new threats.

[SCAR08] suggests the following steps should be included in the maintenance process:

● Monitoring and analyzing logging information.

● Performing regular backups.
● Recovering from security compromises.
● Regularly testing system security.
● Using appropriate software maintenance processes to patch and update all
critical software, and to monitor and revise configuration as needed.
Logging:

[SCAR08] notes that logging can is a basic process of identifying a threat that has already
happened. It generates a lot of information the developer checks the information for any
threats in the system.

Data Backup and Archives :

Regular backup of data is another critical measure for maintaining integrity of the system,
data loss can occur due to many reasons like hardware or software failure, accidental or
deliberate corruption.

Backup is the process of making copies of data at regular intervals, which can be used to
recover lost data when needed.

Archive is process of retaining data for long intervals of time in order to meet legal and
operational requirements to access past data
 Linux/Unix Security.
Often Linux/Unix OS vendors make do not provide a step by step documentation for
securing the system. However, we can generalize the basic concept of securing the
environment that you can customize based on your institutes requirements.

● Choose a good, Operating System

● Stay current with patches
● Use a firewall
● Use file integrity monitoring and change auditing
● Keeping clocks in sync
● Application and Service Configuration
● Remote Access Controls
There are many objectives and goals to be considered when securing a operating
system. When configuring Unix operating system security, consider the critical
principles of security known as the confidentiality, integrity, and availability (CIA) triad.
In addition to incorporating security controls that relate to the CIA triad, three other
security features directly affect CIA and aid the overall site security program: access
control, auditing, and backups.
 Windows Security
Compared to other Systems Microsoft Windows Operating System is made user friendly
and the configuration and maintenance is made easy to every user, so MS windows is
used as a “General Purpose” Operating System.

Apart from the positives being so widely used OS windows devices are more prone to
Attacks by hackers, and consequently security countermeasures are needed to deal
with these challenges.

Again, there are a large range of resources available to assist administrators of these
systems, including reports such as [SYMA07], online resources such as the “Microsoft
Security Tools & Checklists,” and specific system hardening guides such as those
provided by the “NSA—Security Configuration Guides.”
Users Administration and Access Controls

Application and Service Configuration

Other Security Controls

Security Testing
 Virtualization Security
Virtualization refers to a technology that provides an abstraction of the computing
resources used by some software, which thus runs in a simulated environment called a
virtual machine (VM).
Virtualized security, or security virtualization, refers to security solutions that are
software-based and designed to work within a virtualized IT environment. This differs
from traditional, hardware-based network security, which is static and runs on devices
such as traditional firewalls, routers, and switches.
In contrast to hardware-based security, virtualized security is flexible and dynamic.
Instead of being tied to a device, it can be deployed anywhere in the network and is
often cloud-based. This is key for virtualized networks, in which operators spin up
workloads and applications dynamically; virtualized security allows security services and
functions to move around with those dynamically created workloads.
Advantages:

Cost Effective

Flexibility

Operational Efficiency