System Security

Presented by:
Sara Latif
 Contents
• Security and Protection
• The security problem
• Program threats
• System and network threats
• Cryptography as a security tool
• User authentication
• Implementin security defenses
• Firewalls to protect systems and networks
• Computer –Security Classification
• An Example:Windows XP
 The Security Problem
• Security must consider external environment of the
system, and protect the system
• Intruders (crackers) attempt to breach security
• Threat is potential security violation
• Attack is attempt to breach security
• Attack can be accidental or malicious
• Easier to protect against accidental than malicious misuse
 Security Violation Categories
Breach of confidentiality Unauthorized reading of data. Goal of
the intruder: Capture secret data
Breach of integrity Unauthorized modification of data. E.g.
modification of source code
Breach of availability Unauthorized destruction of data
Theft of service Unauthorized use of resources. E.g. intruder may install a
daemon that acts as a file server
Denial of service Preventing legitimate use of the system
 Methods to Breach the Security

• Masquerading (breach authentication)

– in Masquerading the attacker pretends to be a trusted
third party. A variation of this is the man-in-the-middle, in
which the attacker masquerades as both ends of the
conversation to two targets.
 Methods to Breach the Security

• Replay attack
– A replay attack involves repeating a valid transmission.
Sometimes this can be the entire attack, ( such as
repeating a request for a money transfer ), or other times
the content of the original message is replaced with
malicious content.
 Methods to Breach the Security

• Man-in-the-middle attack
– Intruder sits in data flow, masquerading as sender
to receiver and vice versa
 Methods to Breach the Security

• Session hijacking
– Intercept an already-established session to bypass
authentication
 Security Measure Level
• Impossible to have absolute security, but make cost to perpetrator sufficiently high
to deter most intruders
• Security must occur at four levels to be effective:
– Physical
• Data centers, servers, connected terminals
– Human
• Avoid social engineering, phishing, dumpster diving
– Operating System
• Protection mechanisms, debugging
– Network
• Intercepted communications, interruption, DOS
• Security is as weak as the weakest link in the chain
 Program Threats
• Many variations, many names
• Trojan Horse
– Code segment that misuses its environment
– Exploits mechanisms for allowing programs written by users to be executed by
other users
– Spyware, pop-up browser windows, covert channels
– Up to 80% of spam delivered by spyware-infected systems
• Trap Door
– Specific user identifier or password that circumvents normal security
procedures
– Could be included in a compiler
– How to detect them?
 Program Threats…
• Logic Bomb
– Program that initiates a security incident under certain circumstances
• Stack and Buffer Overflow
– Exploits a bug in a program (overflow either the stack or memory
buffers)
– Failure to check bounds on inputs, arguments
– Write past arguments on the stack into the return address on stack
– When routine returns from call, returns to hacked address
• Pointed to code loaded onto stack that executes malicious code
– Unauthorized user or privilege escalation
C Program with Buffer-overflow Condition
Layout of Typical Stack Frame
 Modified Shell Code
#include <stdio.h>
int main(int argc, char *argv[])
{
execvp(‘‘\bin\sh’’,‘‘\bin \sh’’,
NULL);
return 0;
}
Hypothetical Stack Frame
 Program Threats (Cont.)
• Viruses
– Code fragment embedded in legitimate program
– Self-replicating, designed to infect other computers
– Very specific to CPU architecture, operating system, applications
– Usually borne via email or as a macro
• Visual Basic Macro to reformat hard drive
Sub AutoOpen()
Dim oFS
Set oFS =
CreateObject(’’Scripting.FileSystemObject’’)
vs = Shell(’’c:command.com /k format c:’’,vbHide)
End Sub
 Program Threats (Cont.)
• Virus dropper inserts virus onto the system

• Many categories of viruses, literally many thousands of viruses

– File / parasitic
– Boot / memory
– Macro
– Source code
– Polymorphic to avoid having a virus signature
– Encrypted
– Stealth
– Tunneling
– Multipartite
– Armored
A Boot-sector Computer Virus
 System and Network Threats
• Worms – use spawn mechanism; standalone program
• Internet worm
_Exploited UNIX networking features (remote access) and bugs in finger
and sendmail programs
_Grappling hook program uploaded main worm program
• Port scanning
_Automated attempt to connect to a range of ports on one or a range of
IP addresses
• Denial of Service
_Overload the targeted computer preventing it from doing any useful
work
_Distributed denial-of-service (DDOS) come from multiple sites at once
The Morris Internet Worm
 Cryptography as a Security Tool

 Broadest security tool available

• Source and destination of message cannot be trusted without
cryptography
• Means to constrain potential senders(sources) and/or
recievers(destinations) of messages
• Based on Secrets (keys)
Secure Communication over
Insecure Medium
 Encryption
• Encryption algorithm consists of
– Set K of keys
– Set M of Messages
– Set C of cipher texts (encrypted messages)
– A function E : K → (M→C). That is, for each k  K, E(k) is a function for generating cipher
texts from messages
• Both E and E(k) for any k should be efficiently computable functions
– A function D : K → (C → M). That is, for each k  K, D(k) is a function for generating
messages from cipher texts
• Both D and D(k) for any k should be efficiently computable functions
• An encryption algorithm must provide this essential property: Given a cipher text c  C, a
computer can compute m such that E(k)(m) = c only if it possesses D(k)
– Thus, a computer holding D(k) can decrypt cipher texts to the plaintexts used to produce
them, but a computer not holding D(k) cannot decrypt cipher texts
– Since cipher texts are generally exposed (for example, sent on the network), it is
important that it be infeasible to derive D(k) from the cipher texts
 Symmetric Encryption
• Same key used to encrypt and decrypt
– E(k) can be derived from D(k), and vice versa
• DES is most commonly used symmetric block-encryption algorithm (created by US Govt)
– Encrypts a block of data at a time
• Triple-DES considered more secure
• Advanced Encryption Standard (AES), two fish up and coming
• RC4 is most common symmetric stream cipher, but known to have vulnerabilities
– Encrypts/decrypts a stream of bytes (i.e., wireless transmission)
– Key is a input to pseudo-random-bit generator
• Generates an infinite keystream
 Asymmetric Encryption
• Public-key encryption based on each user having two keys:
– public key – published key used to encrypt data
– private key – key known only to individual user used to decrypt data

• Must be an encryption scheme that can be made public without making it

easy to figure out the decryption scheme
– Most common is RSA block cipher
– Efficient algorithm for testing whether or not a number is prime
– No efficient algorithm is know for finding the prime factors of a
number
 Asymmetric Encryption (Cont.)
• Formally, it is computationally infeasible to derive D(kd , N) from E(ke
, N), and so E(ke , N) need not be kept secret and can be widely
disseminated
– E(ke , N) (or just ke) is the public key
– D(kd , N) (or just kd) is the private key
– N is the product of two large, randomly chosen prime numbers
p and q (for example, p and q are 512 bits each)
– Encryption algorithm is E(ke , N)(m) = mke mod N, where ke
satisfies kekd mod (p−1)(q −1) = 1
– The decryption algorithm is then D(kd , N)(c) = ckd mod N
 Asymmetric Encryption Example
• For example. make p = 7and q = 13
• We then calculate N = 7∗13 = 91 and (p−1)(q−1) = 72
• We next select ke relatively prime to 72 and< 72, yielding 5
• Finally,we calculate kd such that kekd mod 72 = 1, yielding 29
• We how have our keys
– Public key, ke, N = 5, 91
– Private key, kd , N = 29, 91
• Encrypting the message 69 with the public key results in the cyphertext 62
• Cyphertext can be decoded with the private key
– Public key can be distributed in cleartext to anyone who wants to communicate with holder of
public key
Encryption and Decryption using RSA
Asymmetric Cryptography
 Cryptography (Cont.)

• Note symmetric cryptography based on

transformations, asymmetric based on mathematical
functions
– Asymmetric much more compute intensive
– Typically not used for bulk data encryption
 Authentication
• Constraining set of potential senders of a message
– Complementary and sometimes redundant to encryption
– Also can prove message unmodified

• Algorithm components
– A set K of keys
– A set M of messages
– A set A of authenticators
– A function S : K → (M→ A)
• That is, for each k  K, S(k) is a function for generating authenticators from messages
• Both S and S(k) for any k should be efficiently computable functions
– A function V : K → (M× A→ {true, false}). That is, for each k  K, V(k) is a
function for verifying authenticators on messages
• Both V and V(k) for any k should be efficiently computable functions
 Authentication (Cont.)
• For a message m, a computer can generate an authenticator a  A such that
V(k)(m, a) = true only if it possesses S(k)

• Thus, computer holding S(k) can generate authenticators on messages so that any
other computer possessing V(k) can verify them

• Computer not holding S(k) cannot generate authenticators on messages that can
be verified using V(k)

• Since authenticators are generally exposed (for example, they are sent on the
network with the messages themselves), it must not be feasible to derive S(k) from
the authenticators
 Authentication – Hash Functions
• Basis of authentication

• Creates small, fixed-size block of data (message digest, hash value) from m

• Hash Function H must be collision resistant on m

– Must be infeasible to find an m’ ≠ m such that H(m) = H(m’)

• If H(m) = H(m’), then m = m’

– The message has not been modified

• Common message-digest functions include MD5, which produces a 128-bit hash,

and SHA-1, which outputs a 160-bit hash
 Authentication - MAC
• Symmetric encryption used in message-authentication code (MAC)
authentication algorithm

• Simple example:
– MAC defines S(k)(m) = f (k, H(m))
• Where f is a function that is one-way on its first argument
– k cannot be derived from f (k, H(m))
• Because of the collision resistance in the hash function, reasonably assured no
other message could create the same MAC
• A suitable verification algorithm is V(k)(m, a) ≡ ( f (k,m) = a)
• Note that k is needed to compute both S(k) and V(k), so anyone able to
compute one can compute the other
 Authentication – Digital Signature
• Based on asymmetric keys and digital signature algorithm

• Authenticators produced are digital signatures

• In a digital-signature algorithm, computationally infeasible to derive S(ks ) from V(kv)

– V is a one-way function
– Thus, kv is the public key and ks is the private key

• Consider the RSA digital-signature algorithm

– Similar to the RSA encryption algorithm, but the key use is reversed
– Digital signature of message S(ks )(m) = H(m)ks mod N
– The key ks again is a pair d, N, where N is the product of two large, randomly chosen prime numbers p and q
– Verification algorithm is V(kv)(m, a) ≡ (akv mod N = H(m))
• Where kv satisfies kvks mod (p − 1)(q − 1) = 1
 Authentication (Cont.)
• Why authentication if a subset of encryption?
– Fewer computations (except for RSA digital signatures)
– Authenticator usually shorter than message
– Sometimes want authentication but not
confidentiality
• Signed patches et al
– Can be basis for non-repudiation
 Key Distribution

• Delivery of symmetric key is huge challenge

– Sometimes done out-of-band

• Asymmetric keys can proliferate – stored on key ring

– Even asymmetric key distribution needs care –
man-in-the-middle attack
Man-in-the-middle Attack on
Asymmetric Cryptography
 Digital Certificates
• Proof of who or what owns a public key

• Public key digitally signed a trusted party

• Trusted party receives proof of identification from entity and certifies that
public key belongs to entity

• Certificate authority are trusted party – their public keys included with
web browser distributions
– They vouch for other authorities via digitally signing their keys, and so
on
 User Authentication
• Crucial to identify user correctly, as protection systems depend on user ID

• User identity most often established through passwords, can be considered a special case of either keys or
capabilities

• Passwords must be kept secret

– Frequent change of passwords
– History to avoid repeats
– Use of “non-guessable” passwords
– Log all invalid access attempts (but not the passwords themselves)
– Unauthorized transfer

• Passwords may also either be encrypted or allowed to be used only once

– Does encrypting passwords solve the exposure problem?
• Might solve sniffing
• Consider shoulder surfing
• Consider Trojan horse keystroke logger
• How are passwords stored at authenticating site?
 Passwords
• Encrypt to avoid having to keep secret
– But keep secret anyway (i.e. Unix uses superuser-only readably file /etc/shadow)
– Use algorithm easy to compute but difficult to invert
– Only encrypted password stored, never decrypted
– Add “salt” to avoid the same password being encrypted to the same value
• One-time passwords
– Use a function based on a seed to compute a password, both user and computer
– Hardware device / calculator / key fob to generate the password
• Changes very frequently
• Biometrics
– Some physical attribute (fingerprint, hand scan)

• Multi-factor authentication
– Need two or more factors for authentication
• i.e. USB “dongle”, biometric measure, and password
 Firewalling to Protect Systems
and Networks
• A network firewall is placed between trusted and untrusted hosts
– The firewall limits network access between these two security domains
• Can be tunneled or spoofed
– Tunneling allows disallowed protocol to travel within allowed protocol (i.e.,
telnet inside of HTTP)
– Firewall rules typically based on host name or IP address which can be spoofed
• Personal firewall is software layer on given host
– Can monitor / limit traffic to and from the host
• Application proxy firewall understands application protocol and can control them
(i.e., SMTP)
• System-call firewall monitors all important system calls and apply rules to them
(i.e., this program can execute that system call)
 Computer Security Classifications
• U.S. Department of Defense outlines four divisions of computer security: A, B, C,
and D
• D – Minimal security
• C – Provides discretionary protection through auditing
– Divided into C1 and C2
• C1 identifies cooperating users with the same level of protection
• C2 allows user-level access control
• B – All the properties of C, however each object may have unique sensitivity labels
– Divided into B1, B2, and B3
• A – Uses formal design and verification techniques to ensure security
 Example: Windows
• Security is based on user accounts
– Each user has unique security ID
– Login to ID creates security access token
• Includes security ID for user, for user’s groups, and special privileges
• Every process gets copy of token
• System checks token to determine if access allowed or denied
• Uses a subject model to ensure access security
– A subject tracks and manages permissions for each program that a user runs
• Each object in Windows has a security attribute defined by a security descriptor
– For example, a file has a security descriptor that indicates the access
permissions for all users
Questions?
Thanks