Module 5

Chapter 2: OS Protection
 Goals of Protection
• To ensure that each shared resource is used only
in accordance with system policies, which may be
set either by system designers or by system
administrators.
• To ensure that errant programs cause the
minimal amount of damage possible.
• Note that protection systems only provide
the mechanisms for enforcing policies and
ensuring reliable systems. It is up to
administrators and users to implement those
mechanisms effectively.
 Principles of Protection
• Operating system provides system calls
and services that allow applications to be
written with fine-grained access controls.
• It provides mechanisms to enable
privileges when they are needed and to
disable them when they are not needed.
• Also beneficial is the creation of audit
trails for all privileged function access.
The audit trail allows the programmer,
systems administrator, or law-
enforcement officer to trace all protection
and security activities on the system
 Principles of Protection
• Managing users with the principle of least
privilege entails creating a separate account
for each user, with just the privileges that the
user needs.
• An operator who needs to mount tapes and
backup files on the system has access to just
those commands and files needed to
accomplish the job.
• Some systems implement role-based access
control (RBAC) to provide this functionality.
• Computers implemented in a computing
facility under the principle of least privilege
can be limited to running specific services,
accessing specific remote hosts via specific
services, and doing so during specific times.
 The Security Problem
• One of the most common types of violations include:
– Breach of Confidentiality - Theft of private or confidential
information, such as credit-card numbers, trade secrets, patents,
secret formulas, manufacturing procedures, medical information,
financial information, etc.
– Breach of Integrity - Unauthorized modification of data, which
may have serious indirect consequences. For example a popular
game or other program's source code could be modified to open
up security holes on users systems before being released to the
public.
– Breach of Availability - Unauthorized destruction of data, often
just for the "fun" of causing havoc and for bragging rites.
Vandalism of web sites is a common form of this violation.
– Theft of Service - Unauthorized use of resources, such as theft of
CPU cycles, installation of daemons running an unauthorized file
server, or tapping into the target's telephone or networking
services.
– Denial of Service, DOS - Preventing legitimate users from using
the system, often by overloading and overwhelming the system
with an excess of requests for service.
 The Security Problem

• One common attack is masquerading, in

which the attacker pretends to be a trusted
third party.
• A variation of this is the man-in-the-middle,
in which the attacker masquerades as both
ends of the conversation to two targets.
• A replay attack involves repeating a valid
transmission. Sometimes this can be the
entire attack, ( such as repeating a request
for a money transfer ), or other times the
content of the original message is replaced
with malicious content.
 Protection of System
• There are four levels at which a system must be protected:

1.Physical -
The easiest way to steal data is to pocket the backup tapes.
Also, access to the root console will often give the user special
privileges, such as rebooting the system as root from removable
media.
Even general access to terminals in a computer room offers some
opportunities for an attacker, although today's modern high-
speed networking environment provides more and more
opportunities for remote attacks.
 Protection of System
2.Human - More and more attacks today are made via social
engineering, which basically means fooling trustworthy people
into accidentally breaching security.
– Phishing involves sending an innocent-looking e-mail or
web site designed to fool people into revealing confidential
information. E.g. spam e-mails pretending to be from e-Bay,
PayPal, or any of a number of banks or credit-card
companies.
– Dumpster Diving involves searching the trash or other
locations for passwords that are written down.
– Password Cracking involves divining users passwords,
either by watching them type in their passwords, knowing
something about them like their pet's names, or simply
trying all words in common dictionaries.
 Protection of System
3.Operating System - The OS must protect itself
from security breaches, such as runaway
processes ( denial of service ), memory-access
violations, stack overflow violations, the
launching of programs with excessive privileges,
and many others.
4.Network - As network communications become
ever more important and pervasive in modern
computing environments, it becomes ever more
important to protect this area of the system
This is a growing area of concern as wireless
communications and portable devices become
more and more prevalent.
 Program Threats

1. Trojan Horse
2. Trap Door
3. Logic Bomb
4. Stack and Buffer Overflow
5. Viruses
 Program Threats
1.Trojan Horse:
• A program that secretly performs some maliciousness
in addition to its visible actions.
• One dangerous opening for Trojan horses is long
search paths, and in particular paths which include the
current directory ( "." ) as part of the path.
• Another classic Trojan Horse is a login emulator,
which records a users account name and password,
issues a "password incorrect" message, and then logs off
the system.
• The user then tries again ( with a proper login prompt ),
logs in successfully, and doesn't realize that their
information has been stolen.
• Spyware is a version of a Trojan Horse that is often
included in "free" software downloaded off the Internet.
• Spyware programs generate pop-up browser windows,
and may also accumulate information about the user
and deliver it to some central site.
 Program Threats
2. Trap Door is when a designer or a
programmer ( or hacker ) deliberately inserts
a security hole that they can use later to access
the system.
• Because of the possibility of trap doors, once
a system has been in an untrustworthy state,
that system can never be trusted again. Even
the backup tapes may contain a copy of
some cleverly hidden back door.
• A clever trap door could be inserted into a
compiler, so that any programs compiled
with that compiler would contain a security
hole. This is especially dangerous, because
 Program Threats

3. Logic Bomb is code that is not designed

to cause havoc all the time, but only when a
certain set of circumstances occurs, such as
when a particular date or time is reached or
some other noticeable event.
• A classic example is the Dead-Man
Switch, which is designed to check
whether a certain person ( e.g. the author
) is logging in every day, and if they don't
log in for a long time ( presumably
because they've been fired ), then the logic
bomb goes off and either opens up
security holes or causes other problems.
 Program Threats
4.Stack and Buffer Overflow
This is a classic method of attack, which exploits bugs in system code that
allows buffers to overflow. Consider what happens in the following code,
for example, if argv[ 1 ] exceeds 256 characters:
The strcpy command will overflow the buffer, overwriting adjacent
areas of memory .
 Program Threats
• Stack and Buffer Overflow
How does overflowing the buffer cause a security breach?
– The "bottom" of the stack is actually at a high memory address,
and the stack grows towards lower addresses.
– However the address of an array is the lowest address of the array,
and higher array elements extend to higher addresses. ( I.e. an
array "grows" towards the bottom of the stack.
– In particular, writing past the top of an array, as occurs when a
buffer overflows with too much input data, can eventually
overwrite the return address, effectively changing where the
program jumps to when it returns.
•
 Program Threats
Stack and Buffer Overflow
• Now that we know how to change where the program returns to
by overflowing the buffer,
• the second step is to insert some nefarious code, and then get the
program to jump to our inserted code.
• Only opportunity to enter code is via the input into the buffer,
which means there isn't room for very much.
• One of the simplest and most obvious approaches is to insert the
code for "exec( /bin/sh )". To do this requires compiling a program
that contains this instruction, and then using an assembler or
debugging tool to extract the minimum extent that includes the
necessary instructions.
• The bad code is then padded with as many extra bytes as are
needed to overflow the buffer to the correct extent, and the
address of the buffer inserted into the return address location.
• The resulting block of information is provided as "input", copied
into the buffer by the original program, and then the return
statement causes control to jump to the location of the buffer and
start executing the code to launch a shell.
 Program Threats
Stack and Buffer
Overflow
 Program Threats
5. Virus is a fragment of code embedded in an otherwise legitimate
program, designed to replicate itself ( by infecting other programs ),
and ( eventually ) wreaking havoc.

• Forms of viruses include:

1.File - A file virus attaches itself to an executable file, causing it to
run the virus code first and then jump to the start of the original
program. These viruses are termed parasitic, because they do not
leave any new files on the system, and the original program is still
fully functional.
2.Boot - A boot virus occupies the boot sector, and runs before the
OS is loaded. These are also known as memory viruses, because in
operation they reside in memory, and do not appear in the file
system.
3.Macro - These viruses exist as a macro ( script ) that are run
automatically by certain macro-capable programs such as MS
Word or Excel. These viruses can exist in word processing
documents or spread sheet files.
4.Source code viruses look for source code and infect it in order to
spread.
 Program Threats

5.Polymorphic viruses change every time they spread -

Not their underlying functionality, but just
their signature, by which virus checkers recognize them.
6.Encrypted viruses travel in encrypted form to escape
detection. In practice they are self-decrypting, which then
allows them to infect other files.
7.Stealth viruses try to avoid detection by modifying parts
of the system that could be used to detect it.
8.Tunneling viruses attempt to avoid detection by
inserting themselves into the interrupt handler chain, or
into device drivers.
9.Multipartite viruses attack multiple parts of the system,
such as files, boot sector, and memory.
10.Armored viruses are coded to make them hard for anti-
virus researchers to decode and understand.
Boot Sector Computer virus
 System and Network Threats
1. Worm:
• A worm is a process that uses the fork / spawn
process to make copies of itself in order to wreak
havoc on a system. Worms consume system resources,
often blocking out other, legitimate processes.
• Worms that propagate over networks can be
especially problematic, as they can tie up vast
amounts of network resources and bring down large-
scale systems.
• One of the most well-known worms was launched by
Robert Morris, a graduate student at Cornell, in
November 1988. This worm consisted of two parts:
– A small program called a grappling hook, which was
deposited on the target system through one of three
vulnerabilities, and
– The main worm program, which was transferred onto the
target system and launched by the grappling hook
 System and Network Threats

•
 System and Network Threats
• The three vulnerabilities exploited by the Morris Internet worm
were as follows:
1.rsh ( remote shell ) is a utility that was in common use at early time
for accessing remote systems without having to provide a password.
If a user had an account on two different computers ( with the same
account name on both systems ), then the system could be configured to
allow that user to remotely connect from one system to the other without
having to provide a password.
Many systems were configured so that any user ( except root ) on system
A could access the same account on system B without providing a
password.

2.finger is a utility that allows one to remotely query a user database, to

find the true name and other information for a given account name on a
given system.
Unfortunately the finger daemon ( which ran with system privileges )
had the buffer overflow problem, so by sending a special 536-character
user name the worm was able to fork a shell on the remote system
running with root privileges.
 System and Network Threats
2.Port Scanning is technically not an attack, but rather a search for
vulnerabilities to attack.

• The basic idea is to systematically attempt to connect to every

known ( or common or possible ) network port on some remote
machine, and to attempt to make contact.

• Once it is determined that a particular computer is listening to a

particular port, then the next step is to determine what daemon
is listening, and whether or not it is a version containing a known
security flaw that can be exploited.

• Because port scanning is easily detected and traced, it is usually

launched from zombie systems, i.e. previously hacked systems
that are being used without the knowledge or permission of their
rightful owner.
 System and Network Threats
3. Denial of Service
• attacks do not attempt to actually access or
damage systems, but merely to clog them up so
badly that they cannot be used for any useful
work.
• Tight loops that repeatedly request system
services are an obvious form of this attack.
• DOS attacks can also involve social engineering,
such as the Internet chain letters that say "send
this immediately to 10 of your friends, and then
go to a certain URL", which clogs up not only the
Internet mail system but also the web server to
which everyone is directed