UNIT-5 NOTES

UNIT – V

PROTECTION: System protection-Goals of protection, principles of protection, domain of

protection access matrix, implementation of access matrix, access control, revocation of
access rights. (T1: Ch-13)

SECURITY: System security-The security problem, program threats, system and network
threats, implementing security defenses, firewalling to protect systems(T1: Ch -18).

Goals of Protection

• Obviously to prevent malicious misuse of the system by users or programs.

• To ensure that each shared resource is used only in accordance with
system policies, which may be set either by system designers or by system
administrators.
• To ensure that errant programs cause the minimal amount of damage possible.
• Note that protection systems only provide the mechanisms for enforcing policies and
ensuring reliable systems. It is up to administrators and users to implement those
mechanisms effectively.

Principles of Protection

• The principle of least privilege dictates that programs, users, and systems be given
just enough privileges to perform their tasks.
• This ensures that failures do the least amount of harm and allow the least of harm to
be done.
• For example, if a program needs special privileges to perform a task, it is better to
make it a SGID program with group ownership of "network" or "backup" or some
other pseudo group, rather than SUID with root ownership. This limits the amount of
damage that can occur if something goes wrong.
• Typically each user is given their own account, and has only enough privilege to
modify their own files.
• The root account should not be used for normal day to day activities - The System
Administrator should also have an ordinary account, and reserve use of the root
account for only those tasks which need the root privileges

Operating Systems Page 1

 UNIT-5 NOTES

Note that mechanisms are distinct from policies. Mechanisms determine how something will
be done; policies decide what will be done.
Domain of Protection

• A computer can be viewed as a collection of processes and objects ( both HW & SW ).

• The need to know principle states that a process should only have access to those
objects it needs to accomplish its task, and furthermore only in the modes for which it
needs access and only during the time frame when it needs access.
• The modes available for a particular object may depend upon its type.

Domain Structure

• A protection domain specifies the resources that a process may access.

• Each domain defines a set of objects and the types of operations that may be invoked
on each object.
• An access right is the ability to execute an operation on an object.
• A domain is defined as a set of < object, { access right set } > pairs, as shown below.
Note that some domains may be disjoint while others overlap.

System with three protection domains.

• The association between a process and a domain may be static or dynamic.

o If the association is static, then the need-to-know principle requires a way of
changing the contents of the domain dynamically.
o If the association is dynamic, then there needs to be a mechanism for domain
switching.
• Domains may be realized in different fashions - as users, or as processes, or as
procedures. E.g. if each user corresponds to a domain, then that domain defines the
access of that user, and changing domains involves changing user ID.

Operating Systems Page 2

 UNIT-5 NOTES

A domain can be realized in a variety of ways:

• Each user may be a domain. In this case, the set of objects that can be accessed depends on
the identity of the user. Domain switching occurs when the user is changed—generally when
one user logs out and another user logs in.
• Each process may be a domain. In this case, the set of objects that can be accessed depends
on the identity of the process. Domain switching occurs when one process sends a message to
another process and then waits for a response.
• Each procedure may be a domain. In this case, the set of objects that can be accessed
corresponds to the local variables defined within the procedure. Domain switching occurs
when a procedure call is made.
Example: UNIX

• UNIX associates domains with users.

• Certain programs operate with the SUID bit set, which effectively changes the user ID,
and therefore the access domain, while the program is running. ( and similarly for the
SGID bit. ) Unfortunately this has some potential for abuse.
• An alternative used on some systems is to place privileged programs in special
directories, so that they attain the identity of the directory owner when they run. This
prevents crackers from placing SUID programs in random directories around the
system.
• Yet another alternative is to not allow the changing of ID at all. Instead, special
privileged daemons are launched at boot time, and user processes send messages to
these daemons when they need special tasks performed.

Example: MULTICS

• The MULTICS system uses a complex system of rings, each corresponding to a different
protection domain, as shown below:

MULTICS ring structure

Operating Systems Page 3

 UNIT-5 NOTES

• Rings are numbered from 0 to 7, with outer rings having a subset of the privileges of
the inner rings.
• Each file is a memory segment, and each segment description includes an entry that
indicates the ring number associated with that segment, as well as read, write, and
execute privileges.
• Each process runs in a ring, according to the current-ring-number, a counter associated
with each process.
• A process operating in one ring can only access segments associated with higher
( farther out ) rings, and then only according to the access bits. Processes cannot
access segments associated with lower rings.
• Domain switching is achieved by a process in one ring calling upon a process operating
in a lower ring, which is controlled by several factors stored with each segment
descriptor:
o An access bracket, defined by integers b1 <= b2.
o A limit b3 > b2
o A list of gates, identifying the entry points at which the segments may be
called.
• If a process operating in ring i calls a segment whose bracket is such that b1 <= i <= b2,
then the call succeeds and the process remains in ring i.
• Otherwise a trap to the OS occurs, and is handled as follows:
o If i < b1, then the call is allowed, because we are transferring to a procedure
with fewer privileges. However if any of the parameters being passed are of
segments below b1, then they must be copied to an area accessible by the called
procedure.
o If i > b2, then the call is allowed only if i <= b3 and the call is directed to one of
the entries on the list of gates.
• Overall this approach is more complex and less efficient than other protection
schemes.

Access Matrix

• The model of protection that we have been discussing can be viewed as an access
matrix, in which columns represent different system resources and rows represent
different protection domains. Entries within the matrix indicate what access that
domain has to that resource.

Operating Systems Page 4

 UNIT-5 NOTES

Access matrix

• Domain switching can be easily supported under this model, simply by providing
"switch" access to other domains:

Access matrix of Figure 14.3 with domains as objects

• The ability to copy rights is denoted by an asterisk, indicating that processes in that
domain have the right to copy that access within the same column, i.e. for the same
object. There are two important variations:
o If the asterisk is removed from the original access right, then the right
is transferred, rather than being copied. This may be termed a transfer right as opposed
to a copy right.
o If only the right and not the asterisk is copied, then the access right is added to the new
domain, but it may not be propagated further. That is the new domain does not also
receive the right to copy the access. This may be termed a limited copy right, as shown in
Figure 14.5 below:

Operating Systems Page 5

 UNIT-5 NOTES

Access matrix with copy rights

• The owner right adds the privilege of adding new rights or removing existing ones:

Access matrix with owner rights

Operating Systems Page 6

 UNIT-5 NOTES

• Copy and owner rights only allow the modification of rights within a column. The
addition of control rights, which only apply to domain objects, allow a process
operating in one domain to affect the rights available in other domains. For example in
the table below, a process operating in domain D2 has the right to control any of the
rights in domain D4.

Modified access matrix of above Figure

Implementation of Access Matrix

Global Table

• The simplest approach is one big global table with < domain, object, rights > entries.
• Unfortunately this table is very large ( even if sparse ) and so cannot be kept in
memory ( without invoking virtual memory techniques. )
• There is also no good way to specify groupings - If everyone has access to some
resource, then it still needs a separate entry for every domain.

Access Lists for Objects

• Each column of the table can be kept as a list of the access rights for that particular
object, discarding blank entries.
• For efficiency a separate list of default access rights can also be kept, and checked first.

Capability Lists for Domains

• In a similar fashion, each row of the table can be kept as a list of the capabilities of that
domain.
• Capability lists are associated with each domain, but not directly accessible by the
domain or any user process.

Operating Systems Page 7

 UNIT-5 NOTES

• Capability lists are themselves protected resources, distinguished from other data in
one of two ways:
o A tag, possibly hardware implemented, distinguishing this special type of data.
( other types may be floats, pointers, booleans, etc. )
o The address space for a program may be split into multiple segments, at least
one of which is inaccessible by the program itself, and used by the operating
system for maintaining the process's access right capability list.

A Lock-Key Mechanism

• Each resource has a list of unique bit patterns, termed locks.

• Each domain has its own list of unique bit patterns, termed keys.
• Access is granted if one of the domain's keys fits one of the resource's locks.
• Again, a process is not allowed to modify its own keys.

Comparison

• Each of the methods here has certain advantages or disadvantages, depending on the
particular situation and task at hand.
• Many systems employ some combination of the listed methods.

Access Control

• Role-Based Access Control, RBAC, assigns privileges to users, programs, or roles as

appropriate, where "privileges" refer to the right to call certain system calls, or to use
certain parameters with those calls.
• RBAC supports the principle of least privilege, and reduces the susceptibility to abuse
as opposed to SUID or SGID programs.

Operating Systems Page 8

 UNIT-5 NOTES

Role-based access control in Solaris 10

Revocation of Access Rights

• The need to revoke access rights dynamically raises several questions:

o Immediate versus delayed - If delayed, can we determine when the revocation
will take place?
o Selective versus general - Does revocation of an access right to an object
affect all users who have that right, or only some users?
o Partial versus total - Can a subset of rights for an object be revoked, or are all
rights revoked at once?
o Temporary versus permanent - If rights are revoked, is there a mechanism for
processes to re-acquire some or all of the revoked rights?
• With an access list scheme revocation is easy, immediate, and can be selective, general,
partial, total, temporary, or permanent, as desired.
• With capabilities lists the problem is more complicated, because access rights are
distributed throughout the system. A few schemes that have been developed include:
o Reacquisition - Capabilities are periodically revoked from each domain, which
must then re-acquire them.
o Back-pointers - A list of pointers is maintained from each object to each
capability which is held for that object.
o Indirection - Capabilities point to an entry in a global table rather than to the
object. Access rights can be revoked by changing or invalidating the table entry,

Operating Systems Page 9

 UNIT-5 NOTES

which may affect multiple processes, which must then re-acquire access rights
to continue.
o Keys - A unique bit pattern is associated with each capability when created,
which can be neither inspected nor modified by the process.
▪ A master key is associated with each object.
▪ When a capability is created, its key is set to the object's master key.
▪ As long as the capability's key matches the object's key, then the
capabilities remain valid.
▪ The object master key can be changed with the set-key command,
thereby invalidating all current capabilities.
▪ More flexibility can be added to this scheme by implementing a list of
keys for each object, possibly in a global table.

Capability-Based Systems (Optional)

Example: Hydra

• Hydra is a capability-based system that includes both system-defined rights and user-
defined rights. The interpretation of user-defined rights is up to the specific user
programs, but the OS provides support for protecting access to those rights, whatever
they may be
• Operations on objects are defined procedurally, and those procedures are themselves
protected objects, accessed indirectly through capabilities.
• The names of user-defined procedures must be identified to the protection system if it
is to deal with user-defined rights.
• When an object is created, the names of operations defined on that object
become auxiliary rights, described in a capability for an instance of the type. For a
process to act on an object, the capabilities it holds for that object must contain the
name of the operation being invoked. This allows access to be controlled on an
instance-by-instance and process-by-process basis.
• Hydra also allows rights amplification, in which a process is deemed to
be trustworthy, and thereby allowed to act on any object corresponding to its
parameters.
• Programmers can make direct use of the Hydra protection system, using suitable
libraries which are documented in appropriate reference manuals.

Operating Systems Page 10

 UNIT-5 NOTES

Example: Cambridge CAP System

• The CAP system has two kinds of capabilities:

o Data capability, used to provide read, write, and execute access to objects. These
capabilities are interpreted by microcode in the CAP machine.
o Software capability, is protected but not interpreted by the CAP microcode.
▪ Software capabilities are interpreted by protected (privileged) procedures, possibly
written by application programmers.
▪ When a process executes a protected procedure, it temporarily gains the ability to read or
write the contents of a software capability.
▪ This leaves the interpretation of the software capabilities up to the individual subsystems,
and limits the potential damage that could be caused by a faulty privileged procedure.
▪ Note, however, that protected procedures only get access to software capabilities for the
subsystem of which they are a part. Checks are made when passing software capabilities
to protected procedures that they are of the correct type.
▪ Unfortunately the CAP system does not provide libraries, making it harder for an
individual programmer to use than the Hydra system.

Language-Based Protection (Optional)

o As systems have developed, protection systems have become more powerful, and also
more specific and specialized.
o To refine protection even further requires putting protection capabilities into
the hands of individual programmers, so that protection policies can be
implemented on the application level, i.e. to protect resources in ways that are
known to the specific applications but not to the more general operating
system.

Compiler-Based Enforcement

• In a compiler-based approach to protection enforcement, programmers directly

specify the protection needed for different resources at the time the resources are
declared.
• This approach has several advantages:
1. Protection needs are simply declared, as opposed to a complex series of
procedure calls.

Operating Systems Page 11

 UNIT-5 NOTES

2. Protection requirements can be stated independently of the support provided

by a particular OS.
3. The means of enforcement need not be provided directly by the developer.
4. Declarative notation is natural, because access privileges are closely related to
the concept of data types.
• Regardless of the means of implementation, compiler-based protection relies upon the
underlying protection mechanisms provided by the underlying OS, such as the
Cambridge CAP or Hydra systems.
• Even if the underlying OS does not provide advanced protection mechanisms, the
compiler can still offer some protection, such as treating memory accesses differently
in code versus data segments. ( E.g. code segments cannot be modified, data segments
can't be executed. )
• There are several areas in which compiler-based protection can be compared to
kernel-enforced protection:
o Security. Security provided by the kernel offers better protection than that
provided by a compiler. The security of the compiler-based enforcement is
dependent upon the integrity of the compiler itself, as well as requiring that
files not be modified after they are compiled. The kernel is in a better position
to protect itself from modification, as well as protecting access to specific files.
Where hardware support of individual memory accesses is available, the
protection is stronger still.
o Flexibility. A kernel-based protection system is not as flexible to provide the
specific protection needed by an individual programmer, though it may provide
support which the programmer may make use of. Compilers are more easily
changed and updated when necessary to change the protection services offered
or their implementation.
o Efficiency. The most efficient protection mechanism is one supported by
hardware and microcode. Insofar as software based protection is concerned,
compiler-based systems have the advantage that many checks can be made off-
line, at compile time, rather that during execution.
The concept of incorporating protection mechanisms into programming languages is in its
infancy, and still remains to be fully developed. However the general goal is to provide
mechanisms for three functions:

Operating Systems Page 12

 UNIT-5 NOTES

0. Distributing capabilities safely and efficiently among customer processes. In

particular a user process should only be able to access resources for which it
was issued capabilities.
1. Specifying the type of operations a process may execute on a resource, such as
reading or writing.
2. Specifying the order in which operations are performed on the resource, such
as opening before reading.

Security

The Security Problem

We say that a system is secure if its resources are used and accessed as intended under all
circumstances. Unfortunately, total security cannot be achieved.

Security violations (or misuse) of the system can be categorized as intentional

(malicious) or accidental. It is easier to protect against accidental misuse than against
malicious misuse. For the most part, protection mechanisms are the core of protection from
accidents. The following list includes several forms of accidental and malicious security
violations. We should note that in our discussion of security, we use the terms intruder and
cracker for those attempting to breach security. In addition, a threat is the potential for a
security violation, such as the discovery of a vulnerability, whereas an attack is the attempt
to break security.
• Breach of confidentiality. This type of violation involves unauthorized reading of data (or
theft of information). Typically, a breach of confidentiality is the goal of an intruder. Capturing
secret data from a system or a data stream, such as credit-card information or identity
information for identity theft, can result directly in money for the intruder.
• Breach of integrity. This violation involves unauthorized modification of data. Such attacks
can, for example, result in passing of liability to an innocent party or modification of the
source code of an important commercial application.
• Breach of availability. This violation involves unauthorized destruction of data. Some
crackers would rather wreak havoc and gain status or bragging rights than gain financially.
Website defacement is a common example of this type of security breach.
• Theft of service. This violation involves unauthorized use of resources. For example, an
intruder (or intrusion program) may install a daemon on a system that acts as a file server.

Operating Systems Page 13

 UNIT-5 NOTES

• Denial of service. This violation involves preventing legitimate use of the system. Denial-
of-service (DOS) attacks are sometimes accidental.

Methods to breach security

• One common attack is masquerading, in which the attacker pretends to be a trusted

third party. A variation of this is the man-in-the-middle, in which the attacker
masquerades as both ends of the conversation to two targets.
• A replay attack involves repeating a valid transmission. Sometimes this can be the
entire attack, ( such as repeating a request for a money transfer ), or other times the
content of the original message is replaced with malicious content.

Standard security attacks

There are four levels at which a system must be protected:

1. Physical - The easiest way to steal data is to pocket the backup tapes. Also, access to
the root console will often give the user special privileges, such as rebooting the
system as root from removable media. Even general access to terminals in a computer

Operating Systems Page 14

 UNIT-5 NOTES

room offers some opportunities for an attacker, although today's modern high-speed
networking environment provides more and more opportunities for remote attacks.
2. Human - There is some concern that the humans who are allowed access to a system
be trustworthy, and that they cannot be coerced into breaching security. However
more and more attacks today are made via social engineering, which basically means
fooling trustworthy people into accidentally breaching security.
o Phishing involves sending an innocent-looking e-mail or web site designed to
fool people into revealing confidential information. E.g. spam e-mails
pretending to be from e-Bay, PayPal, or any of a number of banks or credit-card
companies.
o Dumpster Diving involves searching the trash or other locations for
passwords that are written down. ( Note: Passwords that are too hard to
remember, or which must be changed frequently are more likely to be written
down somewhere close to the user's station. )
o Password Cracking involves divining user’s passwords, either by watching
them type in their passwords, knowing something about them like their pet's
names, or simply trying all words in common dictionaries. (Note: "Good"
passwords should involve a minimum number of characters, include non-
alphabetical characters, and not appear in any dictionary (in any language), and
should be changed frequently. Note also that it is proper etiquette to look away
from the keyboard while someone else is entering their password. )
3. Operating System - The OS must protect itself from security breaches, such as
runaway processes (denial of service), memory-access violations, stack overflow
violations, the launching of programs with excessive privileges, and many others.
4. Network - As network communications become ever more important and pervasive in
modern computing environments, it becomes ever more important to protect this area
of the system. (Both protecting the network itself from attack, and protecting the local
system from attacks coming in through the network.) This is a growing area of concern
as wireless communications and portable devices become more and more prevalent.

Operating Systems Page 15

 UNIT-5 NOTES

Program Threats

• There are many common threats to modern systems. Only a few are discussed here.

Trojan Horse

• A Trojan Horse is a program that secretly performs some maliciousness in addition to

its visible actions.
• Some Trojan horses are deliberately written as such, and others are the result of
legitimate programs that have become infected with viruses, ( see below. )
• One dangerous opening for Trojan horses is long search paths, and in particular paths
which include the current directory ( "." ) as part of the path. If a dangerous program
having the same name as a legitimate program ( or a common mis-spelling, such as "sl"
instead of "ls" ) is placed anywhere on the path, then an unsuspecting user may be
fooled into running the wrong program by mistake.
• Another classic Trojan Horse is a login emulator, which records a users account name
and password, issues a "password incorrect" message, and then logs off the system.
The user then tries again ( with a proper login prompt ), logs in successfully, and
doesn't realize that their information has been stolen.
• Two solutions to Trojan Horses are to have the system print usage statistics on
logouts, and to require the typing of non-trappable key sequences such as Control-Alt-
Delete in order to log in. ( This is why modern Windows systems require the Control-
Alt-Delete sequence to commence logging in, which cannot be emulated or caught by
ordinary programs. I.e. that key sequence always transfers control over to the
operating system. )
• Spyware is a version of a Trojan Horse that is often included in "free" software
downloaded off the Internet. Spyware programs generate pop-up browser windows,
and may also accumulate information about the user and deliver it to some central
site. ( This is an example of covert channels, in which surreptitious communications
occur. ) Another common task of spyware is to send out spam e-mail messages, which
then purportedly come from the infected user.

Trap Door

• A Trap Door is when a designer or a programmer ( or hacker ) deliberately inserts a

security hole that they can use later to access the system.

Operating Systems Page 16

 UNIT-5 NOTES

• Because of the possibility of trap doors, once a system has been in an untrustworthy
state, that system can never be trusted again. Even the backup tapes may contain a
copy of some cleverly hidden back door.
• A clever trap door could be inserted into a compiler, so that any programs compiled
with that compiler would contain a security hole. This is especially dangerous, because
inspection of the code being compiled would not reveal any problems.

Logic Bomb

• A Logic Bomb is code that is not designed to cause havoc all the time, but only when a
certain set of circumstances occurs, such as when a particular date or time is reached
or some other noticeable event.
• A classic example is the Dead-Man Switch, which is designed to check whether a
certain person ( e.g. the author ) is logging in every day, and if they don't log in for a
long time ( presumably because they've been fired ), then the logic bomb goes off and
either opens up security holes or causes other problems.

Stack and Buffer Overflow

• This is a classic method of attack, which exploits bugs in system code that allows
buffers to overflow. Consider what happens in the following code, for example, if argv[
1 ] exceeds 256 characters:
o The strcpy command will overflow the buffer, overwriting adjacent areas of
memory.
o ( The problem could be avoided using strncpy, with a limit of 255 characters
copied plus room for the null byte. )

Viruses

• A virus is a fragment of code embedded in an otherwise legitimate program, designed

to replicate itself ( by infecting other programs ), and ( eventually ) wreaking havoc.

Operating Systems Page 17

 UNIT-5 NOTES

• Viruses are more likely to infect PCs than UNIX or other multi-user systems, because
programs in the latter systems have limited authority to modify other programs or to
access critical system structures ( such as the boot block. )
• Viruses are delivered to systems in a virus dropper, usually some form of a Trojan
Horse, and usually via e-mail or unsafe downloads.
• Viruses take many forms ( see below. ) Figure 15.5 shows typical operation of a boot
sector virus:

A boot-sector computer virus

• Some of the forms of viruses include:

o File - A file virus attaches itself to an executable file, causing it to run the virus
code first and then jump to the start of the original program. These viruses are
termed parasitic, because they do not leave any new files on the system, and
the original program is still fully functional.
o Boot - A boot virus occupies the boot sector, and runs before the OS is loaded.
These are also known as memory viruses, because in operation they reside in
memory, and do not appear in the file system.
o Macro - These viruses exist as a macro ( script ) that are run automatically by
certain macro-capable programs such as MS Word or Excel. These viruses can
exist in word processing documents or spreadsheet files.
Operating Systems Page 18
 UNIT-5 NOTES

o Source code viruses look for source code and infect it in order to spread.
o Polymorphic viruses change every time they spread - Not their underlying
functionality, but just their signature, by which virus checkers recognize them.
o Encrypted viruses travel in encrypted form to escape detection. In practice
they are self-decrypting, which then allows them to infect other files.
o Stealth viruses try to avoid detection by modifying parts of the system that
could be used to detect it. For example the read( ) system call could be modified
so that if an infected file is read the infected part gets skipped and the reader
would see the original unadulterated file.
o Tunneling viruses attempt to avoid detection by inserting themselves into the
interrupt handler chain, or into device drivers.
o Multipartite viruses attack multiple parts of the system, such as files, boot
sector, and memory.
o Armored viruses are coded to make them hard for anti-virus researchers to
decode and understand.

System and Network Threats

• Most of the threats described above are termed program threats, because they attack
specific programs or are carried and distributed in programs. The threats in this
section attack the operating system or the network itself, or leverage those systems to
launch their attacks.

Worms

• A worm is a process that uses the fork / spawn process to make copies of itself in
order to wreak havoc on a system. Worms consume system resources, often blocking
out other, legitimate processes. Worms that propagate over networks can be especially
problematic, as they can tie up vast amounts of network resources and bring down
large-scale systems.
• One of the most well-known worms was launched by Robert Morris, a graduate
student at Cornell, in November 1988. Targeting Sun and VAX computers running BSD
UNIX version 4, the worm spanned the Internet in a matter of a few hours, and
consumed enough resources to bring down many systems.
• This worm consisted of two parts:

Operating Systems Page 19

 UNIT-5 NOTES

1. A small program called a grappling hook, which was deposited on the target
system through one of three vulnerabilities, and
2. The main worm program, which was transferred onto the target system and
launched by the grappling hook program.

The Morris Internet worm.

• The three vulnerabilities exploited by the Morris Internet worm were as follows:
1. rsh ( remote shell ) is a utility that was in common use at that time for
accessing remote systems without having to provide a password. If a user had
an account on two different computers ( with the same account name on both
systems ), then the system could be configured to allow that user to remotely
connect from one system to the other without having to provide a password.
Many systems were configured so that any user ( except root ) on system A
could access the same account on system B without providing a password.
2. finger is a utility that allows one to remotely query a user database, to find the
true name and other information for a given account name on a given system.
For example "finger [email protected]" would access the finger
daemon at somemachine.edu and return information regarding joeUser.
Unfortunately the finger daemon ( which ran with system privileges ) had the
buffer overflow problem, so by sending a special 536-character user name the
worm was able to fork a shell on the remote system running with root
privileges.
3. sendmail is a routine for sending and forwarding mail that also included a
debugging option for verifying and testing the system. The debug feature was
convenient for administrators, and was often left turned on. The Morris worm

Operating Systems Page 20

 UNIT-5 NOTES

exploited the debugger to mail and execute a copy of the grappling hook
program on the remote system.
• Once in place, the worm undertook systematic attacks to discover user passwords:
1. First it would check for accounts for which the account name and the password
were the same, such as "guest", "guest".
2. Then it would try an internal dictionary of 432 favorite password choices. ( I'm
sure "password", "pass", and blank passwords were all on the list. )
3. Finally it would try every word in the standard UNIX on-line dictionary to try
and break into user accounts.
• Once it had gotten access to one or more user accounts, then it would attempt to use
those accounts to rsh to other systems, and continue the process.
• With each new access the worm would check for already running copies of itself, and 6
out of 7 times if it found one it would stop. ( The seventh was to prevent the worm
from being stopped by fake copies. )
• Fortunately the same rapid network connectivity that allowed the worm to propagate
so quickly also quickly led to its demise - Within 24 hours remedies for stopping the
worm propagated through the Internet from administrator to administrator, and the
worm was quickly shut down.
• There is some debate about whether Mr. Morris's actions were a harmless prank or
research project that got out of hand or a deliberate and malicious attack on the
Internet. However the court system convicted him, and penalized him heavy fines and
court costs.
• There have since been many other worm attacks, including the W32.Sobig.F@mm
attack which infected hundreds of thousands of computers and an estimated 1 in 17 e-
mails in August 2003. This worm made detection difficult by varying the subject line of
the infection-carrying mail message, including "Thank You!", "Your details", and "Re:
Approved".

Port Scanning

• Port Scanning is technically not an attack, but rather a search for vulnerabilities to
attack. The basic idea is to systematically attempt to connect to every known ( or
common or possible ) network port on some remote machine, and to attempt to make
contact. Once it is determined that a particular computer is listening to a particular
Operating Systems Page 21
 UNIT-5 NOTES

port, then the next step is to determine what daemon is listening, and whether or not it
is a version containing a known security flaw that can be exploited.
• Because port scanning is easily detected and traced, it is usually launched from zombie
systems, i.e. previously hacked systems that are being used without the knowledge or
permission of their rightful owner. For this reason it is important to protect
"innocuous" systems and accounts as well as those that contain sensitive information
or special privileges.
• There are also port scanners available that administrators can use to check their own
systems, which report any weaknesses found but which do not exploit the weaknesses
or cause any problems. Two such systems are nmap
( http://www.insecure.org/nmap ) and nessus ( http://www.nessus.org ). The former
identifies what OS is found, what firewalls are in place, and what services are listening
to what ports. The latter also contains a database of known security holes, and
identifies any that it finds.

Denial of Service

• Denial of Service ( DOS ) attacks do not attempt to actually access or damage systems,
but merely to clog them up so badly that they cannot be used for any useful work.
Tight loops that repeatedly request system services are an obvious form of this attack.
• DOS attacks can also involve social engineering, such as the Internet chain letters that
say "send this immediately to 10 of your friends, and then go to a certain URL", which
clogs up not only the Internet mail system but also the web server to which everyone
is directed. ( Note: Sending a "reply all" to such a message notifying everyone that it
was just a hoax also clogs up the Internet mail service, just as effectively as if you had
forwarded the thing. )
• Security systems that lock accounts after a certain number of failed login attempts are
subject to DOS attacks which repeatedly attempt logins to all accounts with invalid
passwords strictly in order to lock up all accounts.
• Sometimes DOS is not the result of deliberate maliciousness. Consider for example:
o A web site that sees a huge volume of hits as a result of a successful advertising
campaign.
o CNN.com occasionally gets overwhelmed on big news days, such as Sept 11,
2001.

Operating Systems Page 22

 UNIT-5 NOTES

o CS students given their first programming assignment involving fork( ) often

quickly fill up process tables or otherwise completely consume system
resources. :-)

Implementing Security Defenses

Security Policy

The first step toward improving the security of any aspect of computing is to have a security
policy. Policies vary widely but generally include a statement of what is being secured. For
example, a policy might state that all outside accessible applications must have a code review
before being deployed, or that users should not share their passwords, or that all connection
points between a company and the outside must have port scans run every six months.
Without a policy in place, it is impossible for users and administrators to know what is
permissible, what is required, and what is not allowed. The policy is a road map to security,
and if a site is trying to move from less secure to more secure, it needs a map to know how to
get there. Once the security policy is in place, the people it affects should know it well. It
should be their guide. The policy should also be a living document that is reviewed and
updated periodically to ensure that it is still pertinent and still followed.

Vulnerability Assessment

How can we determine whether a security policy has been correctly implemented? The best
way is to execute a vulnerability assessment. Such assessments can cover broad ground, from
social engineering through risk assessment to port scans. Risk assessment, for example,
attempts to value the assets of the entity in question (a program, a management team, a
system, or a facility) and determine the odds that a security incident will affect the entity and
decrease its value. When the odds of suffering a loss and the amount of the potential loss are
known, a value can be placed on trying to secure the entity. The core activity of most
vulnerability assessments is a penetration test, in which the entity is scanned for known
vulnerabilities. A scan within an individual system can check a variety of aspects of the
system:
• Short or easy-to-guess passwords
• Unauthorized privileged programs, such as setuid programs
• Unauthorized programs in system directories
• Unexpectedly long-running processes

Operating Systems Page 23

 UNIT-5 NOTES

• Improper directory protections on user and system directories

• Improper protections on system data files, such as the password file, device drivers, or the
operating-system kernel itself
• Dangerous entries in the program search path (for example, the Trojan horse discussed in
Section 15.2.1)
• Changes to system programs detected with checksum values
• Unexpected or hidden network daemons
Any problems found by a security scan can be either fixed automatically or reported to the
managers of the system. Networked computers are much more susceptible to security attacks
than are standalone systems.

Intrusion(Obstruction) Detection

Securing systems and facilities is intimately linked to intrusion detection. Intrusion

detection, as its name suggests, strives to detect attempted or successful intrusions into
computer systems and to initiate appropriate responses to the intrusions. Intrusion detection
encompasses a wide array of techniques that vary on a number of axes, including the
following:
• The time at which detection occurs. Detection can occur in real time (while the intrusion is
occurring) or after the fact.
• The types of inputs examined to detect intrusive activity. These may include user-shell
commands, process system calls, and network packet headers or contents. Some forms of
intrusion might be detected only by correlating information from several such sources.
• The range of response capabilities. Simple forms of response include alerting an
administrator to the potential intrusion or somehow halting the potentially intrusive activity.
These degrees of freedom in the design space for detecting intrusions have yielded a wide
range of solutions, known as intrusion-detection systems (IDSs) and intrusion-
prevention systems (IDPs). IDS systems raise an alarm when an intrusion is detected, while
IDP systems act as routers, passing traffic unless an intrusion is detected.

Operating Systems Page 24

 UNIT-5 NOTES

Anomaly detection can find previously unknown methods of intrusion (so-called zero-day
attacks). Signature-based detection, in contrast, will identify only known attacks that can be
codified in a recognizable pattern.

Digital signature

A digital signature is a mathematical technique used to validate the authenticity and integrity
of a message, software or digital document. The digital equivalent of a handwritten signature
or stamped seal, a digital signature offers far more inherent security, and it is intended to
solve the problem of tampering and impersonation in digital communications.

Digital signatures can provide the added assurances of evidence of origin, identity and
status of an electronic document, transaction or message and can acknowledge informed
consent by the signer.

Operating Systems Page 25

 UNIT-5 NOTES

How digital signatures work

Digital signatures are based on public key cryptography, also known as asymmetric
cryptography. Using a public key algorithm, such as RSA, one can generate two keys that are
mathematically linked: one private and one public.

Digital signatures work because public key cryptography depends on two mutually
authenticating cryptographic keys. The individual who is creating the digital signature uses
their own private key to encrypt signature-related data; the only way to decrypt that data is
with the signer's public key. This is how digital signatures are authenticated.

How to create a digital signature

To create a digital signature, signing software -- such as an email program -- creates a one-
way hash of the electronic data to be signed. The private key is then used to encrypt the hash.
The encrypted hash -- along with other information, such as the hashing algorithm -- is the
digital signature.

Virus Protection

As we have seen, viruses can and do wreak havoc on systems. Protection from viruses thus is
an important security concern. Antivirus programs are often used to provide this protection.
Some of these programs are effective against only particular known viruses. They work by
searching all the programs on a system for the specific pattern of instructions known to make
up the virus. When they find a known pattern, they remove the instructions, disinfecting the
program. Antivirus programs may have catalogs of thousands of viruses for which they
search.
Both viruses and antivirus software continue to become more sophisticated. Some
viruses modify themselves as they infect other software to avoid the basic pattern-match
approach of antivirus programs. Antivirus programs in turn now look for families of patterns
rather than a single pattern to identify a virus. In fact, some antivirus programs implement a
variety of detection algorithms. They can decompress compressed viruses before checking for
a signature. Some also look for process anomalies. A process opening an executable file for
writing is suspicious, for example, unless it is a compiler. Another popular technique is to run
a program in a sandbox, which is a controlled or emulated section of the system. The
antivirus software analyzes the behavior of the code in the sandbox before letting it run

Operating Systems Page 26

 UNIT-5 NOTES

unmonitored. Some antivirus programs also put up a complete shield rather than just
scanning files within a file system. They search boot sectors, memory, inbound and outbound
e-mail, files as they are downloaded, files on removable devices or media, and so on. The best
protection against computer viruses is prevention, or the practice of safe computing.

Purchasing unopened software from vendors and avoiding free or pirated copies from
public sources or disk exchange offer the safest route to preventing infection. For macro
viruses, one defense is to exchange Microsoft Word documents in an alternative file format
called rich text format (RTF). Unlike the native Word format, RTF does not include the
capability to attach macros.
Another defense is to avoid opening any e-mail attachments from unknown users.
Another safeguard, although it does not prevent infection, does permit early detection. A user
must begin by completely reformatting the hard disk, especially the boot sector, which is
often targeted for viral attack.

Firewalling to Protect Systems and Networks

A firewall is a computer, appliance, or router that sits between the trusted and the untrusted.
A network firewall limits network access between the two security domains and monitors
and logs all connections. It can also limit connections based on source or destination address,
source or destination port, or direction of the connection. For instance, web servers use HTTP
to communicate with web browsers. A firewall therefore may allow only HTTP to pass from
all hosts outside the firewall to the web server within the firewall.

In fact, a network firewall can separate a network into multiple domains. A common
implementation has the Internet as the untrusted domain; a semi trusted and semi secure
network, called the demilitarized zone (DMZ), as another domain; and a company’s
computers as a third domain (Figure 15.10). Connections are allowed from the Internet to the
DMZ computers and from the company computers to the Internet but are not allowed from
the Internet or DMZ computers to the company computers. Optionally, controlled
communications maybe allowed between the DMZ and one company computer or more. For
instance, a web server on the DMZ may need to query a database server on the corporate
network. With a firewall, however, access is contained, and any DMZ systems that are broken
into still are unable to access the company computers.

Operating Systems Page 27

 UNIT-5 NOTES

Firewalls do not prevent attacks that tunnel, or travel within protocols or connections that
the firewall allows. A buffer-overflow attack to a web server will not be stopped by the
firewall, for example, because the HTTP connection is allowed; Likewise, denial of- service
attacks can affect firewalls as much as any other machines. Another vulnerability of firewalls
is spoofing, in which an unauthorized host pretends to be an authorized host by meeting
some authorization criterion. In addition to the most common network firewalls, there are
other, newer kinds of firewalls, each with its pros and cons. A personal firewall is a software
layer either included with the operating system or added as an application. Rather than
limiting communication between security domains, it limits communication to (and possibly
from) a given host.
An application proxy firewall understands the protocols that applications speak
across the network. For example, SMTP is used for mail transfer. An application proxy accepts
a connection just as an SMTP server would and then initiates a connection to the original
destination SMTP server. It can monitor the traffic as it forwards the message, watching for
and disabling illegal commands, attempts to exploit bugs, and so on. Some firewalls are
designed for one specific protocol.

An XML firewall, for example, has the specific purpose of analyzing XML traffic and blocking
disallowed or malformed XML.

System-call firewalls sit between applications and the kernel, monitoring system-call
execution. For example, in Solaris 10, the “least privilege” feature implements a list of more
than fifty system calls that processes may or may not be allowed to make.

Operating Systems Page 28