CPSC 506

Computer Security I
<MEETING TIME AND INSTRUCTOR>
WEEK 7

1
Operating System
Security
CH. 12
What is “hardening”?
Hardening is the process of strengthening the security of an operating system and the services
that run on it.
◦ One of the primary duties of many security professionals.
◦ A multi-step process, requiring knowledge of many areas of system and network operation
◦ Will require us to integrate all the knowledge we have learned so far.

Why is hardening necessary?

◦ Currently deployed operating systems, including Windows, Linux, and MacOS, do not have sufficient
security in their default installation state.
◦ Need to provide a reasonable level of security despite the expected presence of vulnerabilities.

3
Big Picture
The security professional needs a good overall feel for the characteristic weak spots of each
operating system….
…in order to deploy the most
needful and effective defense
measures first.
Hardening activities are done at all
three layers.
Only the top layer may seem to be
exposed to direct attack; however,
a compromised lower layer can
make the upper layers an easy
target.

4
Planning
As emphasized throughout this course, security does not happen when we just deploy different
defensive measures at random.
In OS hardening, the watchword of planning is:
“What is the system used for?”
The set of factors for planning system hardening includes:
◦ Type of information to be stored on the system and its sensitivity level
◦ The applications and services provided and their security requirements
◦ Categories of users of the system and what types of access and privileges each will have.
◦ How users are authenticated
◦ What access the system has to information and services on other hosts
◦ Who will administer the system, and whether it will be managed remotely or locally.

5
The hardening process
We will discuss 6 types of activity, in the order they should be approached.

6
Hardening I: Initial Setup and Patching
A networked system is vulnerable to exploit during its
installation.
◦ A newly-installed system is in an unpatched state
◦ Many operations run with elevated privileges during installation.

1) Therefore, the first direction for security during installation

is to do it on a protected network.
◦ Of course, the most secure network is no network at all.

2) This stage also includes securing the system boot

procedure.
◦ Set BIOS passwords
◦ Disable booting from other media (USB sticks, network, CD-ROMs,
second hard drive)
◦ Enable secure boot, to prevent VM rootkits

7
Initial Setup and Patching (cont’d)
3) Install only the minimum number of services and software packages required for the system
to fulfill its role.
◦ An instance of minimizing the attack surface.
◦ Of course, this must be balanced against usability of the system.

4) As soon as the system can be brought online, update the OS and all installed applications to
the latest patches, and record and implement the system’s patching policy.
◦ Automatic install of every update may not always be the best policy, especially if the system has high
availability requirements.

8
Hardening II: Remove unnecessary
services, applications, and protocols
Oftentimes you will not have access to a system to harden it at the time of installation. That’s
where this step comes in.
Because any software package running on the machine may contain vulnerabilities, the fewer
there are, the more risk is reduced.
This is less secure than choosing not to install unnecessary software in the first place.
◦ The system may be compromised before a vulnerable application has been removed.

9
Hardening III: Configure Users, Groups,
and Authentication
This phase concerns the implementation of access control and authentication, the two pillars of
system security.
1) In the planning phase, should consider categories of users and the privileges they should and
should not have.
2) Decide whether users and groups are specified and authenticated locally or from a centralized
authentication server.
3) Restrict elevated privileges to users that require them.
4) Default accounts should be secured.
◦ Passwords changed from the defaults!
◦ Unnecessary default accounts should be disabled/deleted.

Possibly implement role-based or mandatory access controls. (RBAC, MAC)

10
Hardening IV: Configure resource
controls
This is the resource-focused, rather than user-focused, aspect of access control.
◦ Limit which users can execute certain programs, especially system services.
◦ Limit which users can access, add, or remove system devices.
◦ Limit which users can read or write certain directory trees. (Example: web server
directories, shared data directories)
◦ If the system has databases, set which users can access these.

11
Hardening V: Install defensive tools
Host-based firewall
IDS
IPS
Anti-virus
The planning phase should determine which of these are appropriate for the system.

12
Hardening VI: Test the system’s security
Systematic records should be kept of each activity of the previous phases.
Then, a checklist should be assembled for testing each of the aspects of security.
While executing the list, make additional records of any unexpected behavior.
Use of scanning or vulnerability testing tools may be appropriate.
Penetration tests are sometimes called for, but can leave a production system in a less secure
state. They are usually done against whole networks.
A report of the process and test results should be produced and stored in a secure external
location.

13
Maintenance
The process of host-based security does not end after the initial hardening
procedure.
Security maintenance includes:
◦ Monitoring and analyzing logging information
◦ Performing regular backups
◦ Recovering from security compromises
◦ Regularly testing system security
◦ Using appropriate software maintenance processes to patch and update all critical software,
and to monitor and revise configurations as needed

14
Hardening through Encryption
Encryption is a key enabling technology that may be used to secure data both in transit and
when stored.
◦ However, encryption is not a “magic bullet” for security.

Messaging encryption Must be configured and appropriate cryptographic keys created, signed,
and secured
◦ If secure network services are provided using TLS or IPsec, suitable public and private keys must be
generated for each of them
◦ If secure network services are provided using SSH, appropriate server and client keys must be created

Disk encryption is used to prevent physical access to the data on a locked or powered-
down machine.

15
Linux Security Overview

16
Aspects of securing Linux/Unix systems
Patching
◦ “Linux”, properly speaking, is just a kernel. A complete Linux-based operating system is an amalgam of
open-source software packages from a variety of sources, assembled into a distribution.
◦ Distribution maintainers have the responsibility to take security patches from “upstream” source and
push them as OS updates
◦ Important to choose a distribution with a good security record (and possibly paying for support).

Application and Service Configuration

• Most commonly implemented using separate text files for each application and service
• Generally located either in the /etc directory or in the installation tree for a specific application
• Individual user configurations that can override the system defaults are located in hidden “dot” files in
each user’s home directory

17
Aspects of securing Linux/Unix systems (2)
Logging
Logging has been an intrinsic part of Unix operating systems from the beginning.
They are a great help to the security administrator, if you know how to use them.

18
Common Linux security weak points
Remote exploits through server programs written in C or C++.
Lax permission settings due to the coarse-grained nature of traditional Unix permissions may
leave the system vulnerable to attack, including insider attacks.
◦ Newer Unix systems use ACLs for more fine-grained permissions; but they are still not so commonly
used.

19
Unix Security Technology: chroot jails
Started from BSD operating system
• Restricts the server’s view of the file system to just a specified portion
• Uses chroot system call to confine a process by mapping the root of the filesystem to some
other directory
• File directories outside the chroot jail aren’t visible or reachable
• Main disadvantage is added complexity

20
SELinux
The SELinux (Security Enhanced Linux) project allows for increasing the default
security and implementing more sophisticated access control policies on Linux
installations.
It provides elements of Mandatory Access Control and RBAC.
https://selinuxproject.org/page/Main_Page

21
Windows
Security
Overview

22
Aspects of securing Windows systems
Application and Service Configuration
Windows stores application as well as OS configuration parameters in a centralized database
known as the registry.
◦ Changing application settings through its GUI will modify the registry
◦ Sets of configuration settings can be stored in .reg “hive files” and merged into the registry.

Defensive tools
Built-in firewall enabled by default, freely available anti-virus
◦ May need to augment these with more robust commercial solutions
◦ One possible concern is unpredictable/undesirable interactions between products from different
vendors.

23
Aspects of securing Windows systems (2)
Administration and Access Control
ACL-based permissions allow more fine-grained control and also have generally sensible defaults
regarding, e.g., permissions inheritance
User Account Control (UAC)
◦ Assists with ensuring users with administrative rights only use them when required, otherwise accesses
the system as a normal user
◦ Example of the principle of minimum privilege

Low Privilege Service Accounts

◦ Used for long-lived service processes such as file, print, and DNS services

Can also implement Mandatory Access Controls (MAC) with security labels.

24
Common Windows security weak points
Due to its prevalence as a desktop operating system, the majority of malware in existence
targets Windows.
◦ Recall that the Stuxnet malware targeted Windows machines controlling nuclear facilities.

Dependence on a single company (Microsoft) to report and patch vulnerabilities

◦ On the plus side, their income stream depends on the perception that the OS is secure.
◦ But since the OS is closed-source, the perception does not necessarily match reality.

Increasing data collection (telemetry) by the Windows OS creates security/privacy concerns

25
Mobile Device Security
24.2
What makes mobile device security
difficult?
The threats involved are not essentially different (we have talked about many of them)
◦ Malware, data theft

But the nature mobile devices makes good security policy more difficult to enforce.
From the perspective of wireless networks, mobile devices are a particularly vulnerable
endpoint.
Once upon a time,
Prior to the widespread use of smartphones and tablets,…
Corporate IT security was more controllable.
◦ User devices were Windows desktop PCs
◦ A well-defined perimeter between the internal
network and outside networks
But Now….
More and more organizations implementing “BYOD” policies
Rapid introduction of new devices and new device types
Rapidly changing software stack
Notion of a static network perimeter is dissolving (Cloud-based applications)
Mobile Device Security Issues
Lack of Physical Security Controls
◦ Security policy must be based on the assumption that any
mobile device could be stolen at any time

Use of untrusted networks

◦ Potentially susceptible to eavesdropping and man-in-the-
middle attacks

Use of apps created by unknown parties

◦ App permissions may allow spying on sensitive data
Mobile Device Security Issues (2)
Use of Cloud-based apps
◦ Especially with automatic synchronization and
sharing of data that comes into the device

Location Services
◦ GPS capability may allow attackers to track
devices containing sensitive data

Use of devices with insufficient security

◦ Devices may be using unpatched and vulnerable
OS software or have insecure default settings for
encryption, etc.
Mobile Device Security Strategy
Organizations may require user-owned devices With devices owned by the company,
to undergo a security audit before allowing organizations can implement more proactive
them to connect to the corporate network: policies :
◦ Require devices to auto-lock with password or
PIN May maintain remote access to the device,
◦ Keep OS and anti-virus up to date
including the ability to wipe in case of theft or
loss
Of course, organizations may also prohibit the May disable installation of third-party
use of owned devices or not allow them to applications
connect to internal networks.
May disable connection to unknown Wi-Fi
networks
Plus utilizing all other means available to strengthen network
security (Authentication, Access Control, IDS, Firewalls, VPN)
Assignments
1. Read through Chapter 12 of the textbook
2. Homework 7, posted on Blackboard
3. Final Project: OS Hardening
◦ You’ll be given a Linux virtual machine, and it’s your job to “harden” it, changing settings and setting
up security services.
◦ Due at the end of the term (not before next week’s lecture)

33