Chapter 8 - Email Security
Chapter 8 - Email Security
Ola Flygt
Linnaeus University, Sweden
http://homepage.lnu.se/staff/oflmsi/
[email protected]
+46 470 70 86 49
1
Outline
ª Email delivery on Internet
ª Pretty Good Privacy (PGP)
ª S/MIME
ª DNS-based authentication of named entities
(DANE)
ª SMTP Strict Transport Security (SMTP STS)
ª DomainKeys Identified Mail (DKIM)
ª Sender policy framework (SPF)
ª Domain-based Messaging Authentication,
Reporting and Conformance (DMARC)
2
Internet Mail Architecture
Message transfer Message transfer Message transfer
agent (MTA) agent (MTA) agent (MTA)
SMTP SMTP
SMTP (SMTP,
local)
(SMTP,
SMTP
local)
Message user Message Message store
agent (MUA) author (MS)
(IMAP, POP,
local)
3
E-Mail components
ª Administrative management domain
(ADMD)
ª Internet e-mail provider
ª Examples include a department that
operates a local mail relay, an IT
department that operates an enterprise
mail relay, and an ISP that operates a public
shared e-mail service
ª Each ADMD can have different operating
policies and trust-based decision making
ª Domain name system (DNS)
ª A directory lookup service that provides a
mapping between the name of a host on
the Internet and its numerical address
4
© 2017 Pearson Education, Ltd., All rights reserved.
Simple Mail Transfer Protocol
(SMTP, RFC 822)
ª SMTP Limitations - Can not transmit, or has a problem
with:
ª executable files, or other binary files (jpeg image)
ª “national language” characters (non-ASCII)
ª messages over a certain size
ª ASCII to EBCDIC translation problems
ª lines longer than a certain length (72 to 254
characters)
ª Has undergone several revisions, the most current being
RFC5321 (October 2008)
5
Mail access protocols
Post Office Protocol (POP3)
7
Multipurpose Internet Mail
Extensions (MIME) MIME specification includes the following
ª An extension to the RFC 5322 elements:
framework that is intended Five new message
header fields are
to address some of the defined, which may be
included in an RFC
problems and limitations of 5322 header; these
the use of Simple Mail fields provide
information about the
Transfer Protocol (SMTP) body of the message A number of content
ª Is intended to resolve these formats are defined,
problems in a manner that is thus standardizing
representations that
compatible with existing support multimedia
RFC 5322 implementations electronic mail
ª The specification is provided Transfer encodings are
defined that enable the
in RFCs 2045 through 2049 conversion of any
content format into a
form that is protected
from alteration by the
mail system 8
Header fields in MIME
ª MIME-Version: Must be “1.0” -> RFC 2045, RFC
2046
ª Content-Type: More types being added by
developers (e.g. application/word)
ª Content-Transfer-Encoding: How message has
been encoded (e.g. radix-64)
Optional fields:
ª Content-ID: Unique identifying character string.
ª Content Description: Needed when content is not
readable text (e.g. mpeg)
9
Header fields in MIME,
Content-Type
10
Header fields in MIME,
Content-Transfer-Encoding
11
Canonical Form
ª An important concept in MIME is that of
canonical form. Messages, especially those
with attachments, should be in canonical
form. The alternative is Native Form.
ª Conversion into canonical form may include
character set conversion, transformation of
audio data, compression etc.
12
MIME Example
Return-Path: <[email protected]>
Received: from webt.msi.vxu.se (localhost [127.0.0.1])
by babbage (Cyrus v2.1.16) with LMTP; Wed, 13 Feb 2008 09:22:44 +0100
X-Sieve: CMU Sieve 2.2
Received: from mailinone.vxu.se (mailinone.vxu.se [194.47.65.80])
by webt.msi.vxu.se (8.12.10/8.12.10) with ESMTP id m1D8MgvG008161
for <[email protected]>; Wed, 13 Feb 2008 09:22:43 +0100 (MET)
Received: from [194.47.95.160] by mailinone.vxu.se
(Sun Java System Messaging Server 6.2-8.01 (built Nov 27 2006))
with ESMTPSA id <[email protected]> for
Ola.Flygt@tcp_msi-daemon (ORCPT [email protected]); Wed,
13 Feb 2008 09:22:41 +0100 (MET)
Date: Wed, 13 Feb 2008 09:22:39 +0100
From: Ola Flygt <[email protected]>
Subject: A simple MIME example
To: Ola Flygt <[email protected]>
Message-id: <[email protected]>
MIME-version: 1.0 (Apple Message framework v753)
X-Mailer: Apple Mail (2.753)
Content-type: text/plain; charset=US-ASCII; format=flowed
Content-transfer-encoding: 7bit
X-Spam-Debug: msi.vxu.se 0
X-Spam-Report: msi.vxu.se 0 ()
X-Spam-Score: 0
X-Spam-Flag: NO
X-Scanned-By: MIMEDefang 2.49 on 194.47.94.71
</x-flowed>
13
MIME Example 2
Return-Path: <[email protected]>
Received: from webt.msi.vxu.se (localhost [127.0.0.1])
by babbage (Cyrus v2.1.16) with LMTP; Wed, 13 Feb 2008 09:24:08 +0100
X-Sieve: CMU Sieve 2.2
Received: from mailinone.vxu.se (mailinone.vxu.se [194.47.65.80])
by webt.msi.vxu.se (8.12.10/8.12.10) with ESMTP id m1D8O7vG009405
for <[email protected]>; Wed, 13 Feb 2008 09:24:07 +0100 (MET)
Received: from [194.47.95.160] by mailinone.vxu.se
(Sun Java System Messaging Server 6.2-8.01 (built Nov 27 2006))
with ESMTPSA id <[email protected]> for
Ola.Flygt@tcp_msi-daemon (ORCPT [email protected]); Wed,
13 Feb 2008 09:24:07 +0100 (MET)
Date: Wed, 13 Feb 2008 09:24:05 +0100
From: Ola Flygt <[email protected]>
Subject: A HTML example
To: Ola Flygt <[email protected]>
Message-id: <[email protected]>
MIME-version: 1.0
X-Mailer: Apple Mail (2.753)
Content-type: multipart/alternative; boundary=Apple-Mail-46--901936981
X-Spam-Debug: msi.vxu.se 0
X-Spam-Report: msi.vxu.se 0 ()
X-Spam-Score: 0
X-Spam-Flag: NO
X-Scanned-By: MIMEDefang 2.49 on 194.47.94.71
--Apple-Mail-5--263420395
Content-Type: text/plain;
charset=ISO-8859-1;
format=flowed
Content-Transfer-Encoding: quoted-printable
--Apple-Mail-5--263420395
Content-Disposition: inline;
filename=new01.gif
Content-Type: image/gif;
x-unix-mode=0644;
name="new01.gif"
Content-Transfer-Encoding: base64
R0lGODlhIwAUALMIAAAAAIAAAACAAICAAAAAgIAAgACAgICAgMDAwP8AAAD/AP//AAAA//8A/wD/
/////yH5BAEAAAgALAAAAAAjABQAQARZEMlJq724LrA7+IvHcUtmnmgWhshatu77mnFtx2muWyEp
gr4OC9VbjWIk304ynMyYz6V0Sj09o7DlbVvLbZLCorFpFQJdnzTHK/6Bw1gedOWku6r4vH6PiAAA
Ow==
--Apple-Mail-5--263420395
Content-Type: text/plain;
charset=ISO-8859-1;
format=flowed
Content-Transfer-Encoding: quoted-printable
16
Why Is PGP Popular?
ª It is available free on a variety of
platforms
ª Based on well known algorithms
ª Wide range of applicability
ª Not developed or controlled by
governmental or standards organizations
ª Now however also standardized
(RFC 3156)
17
Operational Description
18
Authentication
ª Combination of SHA-1 and RSA provides an
effective digital signature scheme
ª Because of the strength of RSA the recipient is
assured that only the possessor of the matching
private key can generate the signature
ª Because of the strength of SHA-1 the recipient is
assured that no one else could generate a new
message that matches the hash code
ª As an alternative, signatures can be generated using
DSS/SHA-1
ª Detached signatures are supported
ª Each person’s signature is independent
and therefore applied only to the document
19
Confidentiality
ª Provided by encrypting messages to be transmitted or to be stored locally
as files
ª In both cases the symmetric encryption algorithm CAST-128 may be used
ª Alternatively IDEA or 3DES may be used
ª The 64-bit cipher feedback (CFB) mode is used
20
Compression
ª PGP compresses the message after
applying the signature but before
encryption
ª The placement of the compression
algorithm is critical
ª The compression algorithm used is ZIP
(described in appendix G)
21
E-mail Compatibility
ª The scheme used is radix-64 conversion (see
appendix K)
ª The use of radix-64 expands the message by 33%
22
Segmentation and
Reassembly
ª Internet Email delivery often restricted
to a maximum message length of
50,000 octets
ª Longer messages must be broken up
into segments
ª PGP automatically subdivides a
message that is to large
ª The receiver strip of all e-mail headers
and reassemble the block
23
PGP Cryptographic Functions
Source A Destination B
E[PRa, H(M)]
PUa
PRa
H DP
M || Z Z-1
EP M
Compare
PUb PRb
Ks EP DP
M M
Z EC || DC Z-1
25
PGP Operation – Summary
X file convert from radix 64
X R64–1[X]
decrypt key, X
Yes Yes
Signature generate signature Confidentiality Ks D(PRb, E(PUb, Ks))
required? X signature || X required?
X D(Ks, E(Ks, X))
No No
Compress
X Z(X) Decompress
X Z–1(X)
convert to radix 64
X R64[X]
(a) Generic Transmission Diagram (from A) (b) Generic Reception Diagram (to B)
26
Figure 8.2 Transmission and Reception of PGP Messages
Format of PGP Message
27
PGP Key Rings
ª each PGP user has a pair of keyrings:
ª public-key ring contains all the public-keys of other
PGP users known to this user, indexed by key ID
ª private-key ring contains the public/private key
pair(s) for this user, indexed by key ID & encrypted
keyed from a hashed passphrase
ª security of private keys thus depends on the
pass-phrase security
28
PGP Key Rings
29
PGP Message Generation
30
PGP Message Reception
31
Obtaining trust
ª How can we obtain trust in the the
other parties public key?
ªCertificate
ªGetting the key personally
ª PGP tries to give trust in a new way
ªWeb of trust
32
The Use of Trust
ª The user maintains a data structure with
certified keys together with the following
fields
ª Owner trust field, assigned by user
ª Signature trust field, cached copies of respective
owner trust fields
ª Key legitimacy field, calculated by PGP
ª Trust for each key is calculated as weighted
sum of trust in signatures for this key
33
34
Revoking Public Keys
ª The owner issue a key revocation
certificate.
ªNormal signature certificate with a revoke
indicator.
ªCorresponding private key is used to sign
the certificate
35
PGP Key servers
ª Public key servers act as a phonebook for PGP keys,
allowing a person to use an email address, name, or key
fingerprint to search for a full key and download it
ª There are many PGP public key servers, but they usually
share their key collections with each other
ª Key servers can't verify whether the keys they publish
are genuine or forgeries. Anyone can upload a key to a
public key server—in anyone's name
ª In order to check the authenticity of a key, you need to
check its signatures, or confirm its fingerprint with the
original user in a trustworthy way
ª PGP allows you to sign other people's keys, which is a
way of using your own key to assert that a certain key is
the right one to use to contact another person. This
creates the web of trust
36
Key Servers poisoning
ª In June 2019 an attack was made on the
Key Server infrastructure
ª Several keys were spammed by being
signed thousands of times
ª The information added to a Key Server
can not be removed (for security reasons)
but in this case it means that people
syncing with these poisoned servers can
crash due to the size of the signed keys
ª There is still no remedy for this problem
Source: https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f 37
S/MIME
ª Secure/Multipurpose Internet Mail
Extension (currently version 4.0, 2019)
ª Defined in RFC 8551
ª S/MIME was intended to become the
industry standard for companies and
other large organisations
ª PGP for personal e-mail security
38
S/MIME Functions
ª Enveloped Data: Encrypted content and
encrypted session keys for recipients.
ª Signed Data: Message Digest encrypted with
private key of the signer and then content +
signature is encoded using base64 encoding.
ª Clear-Signed Data: Signed but not encoded
except the signature.
ª Signed and Enveloped Data: Various
orderings for encrypting and signing.
39
Algorithms Used
ª Message Digesting: SHA-1 and SHA-256
ª Digital Signatures: RSA and DSA (DSS)
ª Secret-Key Encryption: AES, Triple-DES,
RC2/40
ª Public-Private Key Encryption: RSA with
key sizes of 512 and 1024 bits, and Diffie-
Hellman (for session keys).
40
S/MIME Content Types
Type Subtype smime Parameter Description
Multipart Signed A clear-signed message in two
parts: one is the message and the
other is the signature.
Application pkcs7-mime signedData A signed S/MIME entity.
pkcs7-mime envelopedData An encrypted S/MIME entity.
pkcs7-mime degenerate An entity containing only public-
signedData key certificates.
pkcs7-mime Compr A compressed S/MIME entity.
essed
Data
pkcs7- signe The content type of the signature
signature dData subpart of a multipart/signed
message.
41
Securing a MIME Entity
ª S/MIME secures a MIME entity with a signature, encryption, or
both
ª The MIME entity is prepared according to the normal rules for
MIME message preparation
ª The MIME entity plus some security-related data, such as
algorithm identifiers and certificates, are processed by
S/MIME to produce what is known as a PKCS object
ª A PKCS object is then treated as message content and
wrapped in MIME
ª In all cases the message to be sent is converted to canonical
form
Certificate storage
Key generation Registration
and retrieval
47
Email delivery on internet
48
SMTP Strict Transport
Security (SMTP STS)
ª Top email providers, namely Google,
Microsoft, Yahoo!, Comcast, LinkedIn, and 1&1
Mail & Media Development, have joined forces
to develop a new email standard that makes
sure the emails you send are going through an
encrypted channel and cannot be sniffed.
ª The primary goal of SMTP STS is to prevent
Man-in-the-Middle (MitM) attacks that have
compromised past efforts like STARTTLS at
making SMTP a more secure protocol.
Reference: http://thehackernews.com/2016/03/smtp-sts-email-security.html 49
DNS-based authentication of
named entities (DANE)
ª DANE is a protocol to allow X.509 certificates,
commonly used for Transport Layer Security (TLS),
to be bound to DNS names using DNSSEC
ª It is proposed in RFC 6698 as a way to
authenticate TLS client and server entities
without a certificate authority (CA)
ª The purpose of DANE is to replace reliance on the
security of the CA system with reliance on the
security provided by DNSSEC
ª DANE defines a new DNS record type, TLSA, that
can be used for a secure method of authenticating
SSL/TLS certificates
50
© 2017 Pearson Education, Ltd., All rights reserved.
SMTP STS vs. DANE
ª SMTP STS does not require DNSSEC (but it is
recommended)
ª SMTP STS defines a policy cache in the mail
server
ª SMTP STS requires x509 certificates that
validate against a root-CA-certificate
(no "self-signed" certs)
ª SMTP STS requires a HTTPS server to serve a
policy JSON document
ª SMTP STS requires validation of the HTTPS
connection to fetch the policy document
51
SMTP STS vs. DANE
ª DANE does require DNSSEC
ª DANE has no policy cache (but the TTL on
TLSA records can work as such)
ª DANE allows "self-signed" certificates
ª DANE policy can be changed by switching the
TLSA record in DNS
ª DANE TLS-cert rollover need to be in sync with
TLSA record(s)
ª DANE relies on the trust on the DNSSEC chain
52
DomainKeys Identified Mail
ª DomainKeys Identified Mail (DKIM) lets an
organization take responsibility for a message while
it is in transit.
ª The organization is a handler of the message, either
as its originator or as an intermediary. Their
reputation is the basis for evaluating whether to
trust the message for delivery.
ª Technically DKIM provides a method for validating a
domain name identity that is associated with a
message through cryptographic authentication.
53
DomainKeys Identified Mail
ª Protocol for signing and verifying the
originating domain for a message in transit
ª Prevents domain-level forgery
ª Helps deal with spam and phishing
ª Increase effectiveness of blacklists
ª Ensure the identity of an sender domain
ª Backwards compatible
ª Works with all existing MTAs and MUAs
54
DKIM Goals
ª Based on message content, itself
ª Not related to path
ª Transparent to end users
ª No client User Agent upgrades required
ª But extensible to per-user signing
ª Allow signature delegation
ª Outsourcing
ª Low development, deployment, use costs
ª Avoid large PKI, new Internet services
ª No trusted third parties (except DNS)
55
Technical High-points
ª Signs body and selected parts of header
ª Signature transmitted in DKIM-Signature:
header
ª Public key stored in DNS
ª In _domainkey subdomain
ª Uses TXT RR (see http://en.wikipedia.org/wiki/List_of_DNS_record_types)
56
How it works – Sending Servers
ª Set up: The domain owner (typically the
team running the email systems within a
company or service provider) generates a
public/private key pair to use for signing all
outgoing messages (multiple key pairs are
allowed). The public key is published in DNS,
and the private key is made available to their
DomainKey-enabled outbound email servers.
This is step "A" in the diagram to the right.
ª Signing: When each email is sent by an
authorized end-user within the domain, the
DomainKey-enabled email system
automatically uses the stored private key to
generate a digital signature of the message.
This signature is then pre-pended as a header
to the email, and the email is sent on to the
target recipient's mail server. This is step "B"
in the diagram to the right.
57
How it works – Receiving Servers
ª Preparing: The DomainKeys-enabled receiving email
system extracts the signature and claimed From: domain
from the email headers and fetches the public key from
DNS for the claimed From: domain. This is step "C" in
the diagram to the right.
ª Verifying: The public key from DNS is then used by the
receiving mail system to verify that the signature was
generated by the matching private key. This proves that
the email was truly sent by, and with the permission of,
the claimed sending From: domain and that its headers
and content weren't altered during transfer.
ª Delivering: The receiving email system applies local
policies based on the results of the signature test. If the
domain is verified and other anti-spam tests don't catch
it, the email can be delivered to the user's inbox. If the
signature fails to verify, or there isn't one, the email can
be dropped, flagged, or quarantined. This is step "D" in
the diagram on the right.
58
DomainKeys Identified Mail
ª Example Signature Header
DKIM-Signature: v=1; a=rsa-sha256; d=example.net; s=brisbane;
c=relaxed/simple; q=dns/txt; t=1117574938; x=1118006938;
h=from:to:subject:date:keywords:keywords;
bh=MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI=;
b=dzdVyOfAKCdLXdJOc9G2q8LoXSlEniSbav+yuU4zGeeruD00lszZVoG4ZHRNiYzR
ª Where the tags used are:
v, version t, signature timestamp
a, signing algorithm x, expire time
d, domain h, header fields - list of those that have
s, selector been signed
c, canonicalization algorithm(s) for bh, body hash
header and body b, signature of headers and body
q, default query method
60
Sender policy framework (SPF)
ª SPF is the standardized way for a sending
domain to identify and assert the mail
senders for a given domain
ª Addresses the problem of any host being able
to use any domain name for each of the
various identifiers in the mail header, not
just the domain name where the host is
located
ª Defined in RFC 7208
ª SPF works by checking a sender’s IP address
against the policy encoded in any SPF record
found at the sending domain
62
© 2017 Pearson Education, Ltd., All rights reserved.
Sender policy framework
Operation
Inbox
Inbound
Sender Mail Server Junk Email
Internet
+
Quarantine
Authorization
SPF Record
Pass/Fail Further
Policy
?
Lookup Checks
Block/Delete
DNS
63
Figure
© 2017 Pearson Education, Ltd., All 8.9 Sender Policy Framework Operation
rights reserved.
DMARC
ª Two mechanisms have traditionally been used to
accomplish email authentication - DomainKeys
identified mail (DKIM) and sender policy framework
(SPF). Recently, these standards have been
integrated into DMARC (Domain-based Messaging
Authentication, Reporting and Conformance)
ª Google, Yahoo, Microsoft and many others use it to
enforce email security by aligning sender and
recipient information
ª Is defined in RFC 7489, March 2015
64
DMARK overview
ª A DMARC policy allows a sender to indicate that their
emails are protected by SPF and/or DKIM, and tells a
receiver what to do if neither of those authentication
methods passes - such as junk or reject the message
ª It removes the guesswork from the receiver's handling of
these failed messages, limiting or eliminating the user's
exposure to potentially fraudulent & harmful messages
ª It also provides a way for the email receiver to report
back to the sender about messages that pass and/or fail
DMARC evaluation
65
Sender Receiver
Pass
DMARK
Retrieve verified
DKIM DKIM domains
DKIM
Functional
Apply
SPF SPF DMARC
policy
Retrieve
Sending mail server “envelope from”
Flow
attaches DKIM signature via SPF
Fail
Update periodic
Standard validation
Aggregate Report
tests at receiver
to be sent to sender
(including IP
blocklists, Quaran-
Failure
reputation, rate Block tine
limits, etc)
report
66
© 2017 Pearson Education, Ltd., All rights reserved.
DMARC debate
ª In April 2014, Yahoo changed its DMARC policy
to p=reject, thereby causing misbehaviour in
several mailing lists. It was not yet a standard
protocol, and missed a provision for such sudden
changes. This change has caused significant
debate in the IETF mailing list about whether
DMARC was ready for deployment, about
whether it is good for the Internet at all, and
about whether its making process was correct.
67
Use of SPF and DMARC
ª An OTech study (March 2017) found that 86 percent
of major online businesses are using Sender Policy
Framework (SPF) to determine whether messages
that claim to be from the businesses’ email
addresses actually come from the businesses.
ª Fewer than 10 percent of the businesses, however,
have implemented the supplemental technology
Domain Message Authentication Reporting &
Conformance (DMARC) in a manner which would
allow the businesses to receive intelligence on
potential spoofing attempts and to instruct ISPs to
automatically reject any unauthenticated messages
that claimed to be from the businesses’ email
addresses
Source: https://www.ftc.gov/news-events/press-releases/2017/03/online-businesses-could-do-more-protect-their-reputations-prevent 68
Sender
DNS
The Inter-
relationship of
DNSSEC, SPF,
msg
msg Sending Receiving
sig MTA
MTA
DKIM, DMARC,
sig
DANE and
msg
Sender
MUA sig
S/MIME for
Receiver
DNS
Receiver
MUA
Assuring msg
Message
Authenticity
and Integrity
69
© 2017 Pearson Education, Ltd., All rights reserved.