SHA-1 and MD5 Cryptographic Hash Functions Securit
SHA-1 and MD5 Cryptographic Hash Functions Securit
73-80
REVIEW
Roman Jasek *
Despite their obsolescence and recommendations they are phased out from production environment, MD5 and SHA-1 cryptographic hash
functions remain defaults frequently offered in many applications, e.g., database managers. In the article, we present a security overview of
both algorithms and demonstrate the necessity to abandon them in favor of more resilient alternatives due to low computational requirements
necessary to reverse engineer the message digests, or to future proof security due to advances in hardware performance and scalability. Suita-
bility procedures and their methods of use are part of this article.
Keywords: Algorithm, bcypt, function, hashing, MD5, PBKDF2, security, SHA-1, scrypt.
* Roman Jasek
Department of Applied Informatics, Tomas Bata University of Zlin, Czech Republic
E-mail: [email protected]
COMMUNICATIONS 1/2015 ●
73
REVIEW
which imposes computational penalty when reverse engineering in such a way. Therefore, encryption modules included by default
hashes. Section 4 continues by describing key strengthening in many instances of DBMSs will be considered: MD5 and
and mentions suitable alternatives to MD5 and SHA-1 designed SHA-1.
specifically with key stretching and strengthening in mind, and
brief concluding remarks.
Security in various forms, i.e., message authentication [4] and
protecting data in transit [5] is imperative for data confidentiality,
integrity, and availability. We believe the article will contribute
to safer organizational environments by incentivizing system
administrators and appropriate parties to migrate from insecure or
weakened cryptographic hash functions to alternatives scrutinized
by academia and security community.
74 ● COMMUNICATIONS 1/2015
REVIEW
COMMUNICATIONS 1/2015 ●
75
REVIEW
B. SHA-1
The Secure Hash Algorithm 1 was designed by the United
States National Security Agency (NSA) in 1995 as a successor to
the 1993’s SHA-0. With a 160-bit digest iterated for 80 rounds, it
was used for protecting sensitive unclassified information as well
as in Internet protocols such as Secure Sockets Layer (SSL) and
Secure Shell (SSH) [21]. SHA-1 is represented as a 40-character
sequence. Fig. 5 An SHA-1 encryption round [23]
SHA-1 encryption round is depicted in Fig. 5. A-E are 32b
words identical to MD5, F a non-linear function, <<< bitwise shift Because computational complexity of attacks on SHA-1 has
for arbitrary number of positions, Kt a round constant, and Wt an been steadily decreasing, SHA-2 (SHA-224, SHA-256, SHA-384,
input data block. The output of F enters an adder modulo 232. The SHA-512) class was devised as a direct successor. However, both
function is based on Merkle–Damgard construction. systems are based on identical algorithmic operations and it is
Touted an MD5’s replacement, SHA-1 saw enormous rise expected optimized SHA-1 attacks will be applicable to SHA-2,
in applications which led to its thorough examination by the as well.
cryptographic community. Previously, research focused on its In 2012, a successor to SHA-1 and SHA-2 was selected by
predecessor, SHA-0 for which a collision was found using the NIST (National Institute of Standards and Technology) after
disturbance vectors with “complexity [of] 239 hash operations. an open competition, aiming to choose a function dissimilar to its
Compared with existing attacks on SHA-0, our method is much predecessors. Currently published reverse engineering attempts
more efficient and real collisions can be found quickly on a typical break 46 out of 64 rounds for SHA-256 [26] and equivalent
PC. The techniques… are also applicable to SHA-1. As SHA-0 amount of rounds for the 80-round SHA-512, it is expected the full
may be viewed as a simple variant of SHA-1, the analysis… serves system will be targeted eventually despite it being computationally
to verify effectiveness of these new techniques for other SHA infeasible at present time. SHA-3 utilizes functions with sponge
variants” [22]. construction [27], making harder for the attacker to differentiate
The result showed it is possible to find a collision in SHA-1 it from a random oracle, a theoretical scenario in which any
while requiring fewer computations than it would take to brute- input is encrypted randomly in a black-box setting. Any outside
force the hash, the most time- and resource-intensive cryptanalytic agent cannot discern whether the output was produced based on
process. a random function or a genuine encryption algorithm if no other
Further attempts were made to reduce the number of information (timing measurements, heat emissions, cycle counts)
operations after which the collision is found. A significant is known. Independent on SHA-2, known attack vectors are not
breakthrough was made in 2006 when “…for the first time an applicable to SHA-3.
actual collision for 64-step SHA-1 is produced, with an expected Two theoretical vectors against SHA-3 were proposed: a zero-
work factor of 235 compression function computations” [24]. sum attack applicable to the 9-round reduced version with no
Since then, several attempts have been made to extend the attack effect on its security [28]; and an improved zero-sum distinguisher
on full SHA-1 with mixed results. The latest discovery is dated which applies to all 24 rounds and lowers the number of
to 2013 when a researcher estimated theoretical number of operations from 21579 to 21570 [29]. Both were published before
computations for full SHA-1 to 261 operations [25]. the final version of SHA-3 was selected; no practical cryptanalytic
breakthroughs on the final implementation has been published
as of yet.
76 ● COMMUNICATIONS 1/2015
REVIEW
In 2011, National Institute of Standards and Technology bits. Therefore, using a salt makes it difficult for the attacker to
asserted that “…the known research results indicate that SHA-1 generate a table of resulting [hashes] for even a small subset of
is not as collision resistant as expected. The collision security the most-likely passwords” [34].
strength is significantly less than an ideal has function (i.e., [269] Even when generating (pseudo)random salts, they may be
compared to [280])…. [C]ollision resistance has been shown to rendered ineffective if the attacker can exploit vulnerabilities in the
affect some (but not all) applications that use digital signatures” way they are concatenated and added to the strings. Two frequent
[30]. omissions are salt reuse and short salts. The former is “ineffective
because if two users have the same password, they’ll still have the
same hash. An attacker can still use a reverse lookup table attack
3. Best practices to run a dictionary attack on every hash at the same time. They
just have to apply the salt to each password guess before they hash
Regardless of the hash function, the security best practice it,” the latter does not prevent the attacker to “build a lookup table
for storing sensitive data such as user credentials (logins, for every possible salt.… To make it impossible for an attacker
passwords) is to utilize randomized hashing. As the hash to create a lookup table for every possible salt, the salt must be
itself is deterministic (two identical strings produce identical long. A good rule of thumb is to use a salt that is the same size
outputs), additional probabilistically-generated data need to be as the output of the hash function” [35]. A one-time (pseudo)
supplanted and processed along with the input data stream. Titled random data string is titled nonce; encryption schemes have been
cryptographic salt, its purpose is to increase time factor involved proposed which makes it impossible to decrypt (reverse engineer)
when adversary employs rainbow tables, a list of pre-computed the product without the nonce [36].
values which speeds the process of iterating through the whole A break-through occurs when an attack vector enabling pre-
search space. Adding salt is depicted in Fig. 6. image extraction after lower number of operations (and thus time
factor involved) is discovered than during exhaustive search. It
is defined as “[a]n attack that uses a brute-force technique of
successively trying all the words in some large, exhaustive list”
[37].
Cryptographic salts also make time-memory tradeoff
difficult to implement. First described in 1980 [38], the
technique trades time dedicated to calculating candidate
solutions for a pre-computed lookup data array where a simple
search algorithm can be applied to find the correct value.
A threshold exists, though, above which table lookups become
costly and ineffective. After several improvements, a new version
was introduced in 2003 making use of non-merging rainbow
chains, addressing the issue in the original proposal [39]. The
technique achieved 99.9% success rate when reverse engineering
Microsoft Windows LM hashes with a lookup table the size of
Fig. 6 Adding cryptographic salt to a password before hashing as well 1.4GB. As the prices of storage media decreases per Moore’s law,
as concatenating another salt with the message digest [31] rainbow tables in tens of terabytes will proliferate which utilize
high-speed storage media such as SSD (Solid-State Drive).
NIST recommends “[t]he random value… [to] be a message- If the input to the hash function concatenated with a salt
independent bit string of at least 80 bits, but no more than prior to being reduced to a fixed-size output, the attacker is
1024 bits… [which] shall have sufficient randomness to meet forced to pre-compute the lookup array for every possible salt
the desired security strength…” [32]. Cryptographic salt should, value. Therefore, security depends on uniqueness and length
therefore, be generated using a random number generator whose of the random value being appended or prepended to the
output meets randomness criteria, e.g., Linear Complexity, (presumably) non-random input string. Salts, key strengthening
Approximate Entropy, Binary Matrix Rank, and Serial Tests and key stretching make time-memory tradeoff difficult to balance
[33]. A single value should not be used globally, instead a per- compared to a brute-force attack. Key stretching is discussed
user or per-application salt stored in a database separate from in Section 4. Key strengthening was devised in 1994 and splits
the hashes is recommended. It is introduced to force threat agent the salt in two parts: public and secret [40]. While the public
to generate large sets of candidate hashes for to the string being part is stored, the secret is securely deleted after first use and
reverse engineered. “[T]he number of possible resulting [hashes] becomes unknown. When the user enters a password, the server
is approximately 2sLen where sLen is the length of the salt in must perform a brute-force attack using the public part of the
COMMUNICATIONS 1/2015 ●
77
REVIEW
salt to determine the secret portion, increasing both per-user through data arrays stored in memory. As demonstrated in Fig. 7,
computational requirements and security. The attacker, though, PBKDF2 utilizes HMAC.
must exhaustively search the whole hash space, i.e., both parts of
the salt. The salt or its part and the algorithm used to generate the
output must be known server-side to allow comparison of the data
to the stored value. No plaintext-formatted data should be stored
at any point, only the fingerprints.
Compared to alternatives discussed below, implementing
cryptographic hashes does not guarantee the adversary will not
be able to extract the pre-image. Should system administrators
be forced to select one of the countermeasures, i.e., transitioning
to a new cryptographic hash function or adding salt to the
existing infrastructure, the former should be strongly preferred
as it results in significantly higher computational demands during
reverse engineering. Moore’s law dictates effectiveness of salting
will decrease as hardware performance increase. By deploying
strong cryptographic hash function, the work factor can be
easily tweaked by means of parameters used during hashing, e.g.,
iteration count and parallelizability.
78 ● COMMUNICATIONS 1/2015
REVIEW
per Moore’s law and future proofing should be taken into account Acknowledgement
when selecting suitable cryptographic hash function to deploy. The work was performed with financial support of research
Hash functions have seen increased use in areas such as project NPU I No. MSMT-7778/2014 by the Ministry of
concurrent algorithm design [45] and continue to be active Education of the Czech Republic and also by the European
research field. Regional Development Fund under the Project CEBIA-Tech No.
CZ.1.05/2.1.00/03.0089.
References
[1] PCI Security Standards Council. Payment Card Industry Data Security Standard 2.0 [Online]. Available: https://www.
pcisecuritystandards.org/security_standards/documents.php, 2010.
[2] EU: Directive 95/46/EC of the European Parliament and of the Council of 24 October 1995 on the Protection of Individuals with
Regard to the Processing of Personal Data and on the Free Movement of such Data [Online]. Available: http://eur-lex.europa.eu/
LexUriServ/LexUriServ. do?uri= CELEX:31995L0046:en:HTML, 1995.
[3] MOORE, G. E.: Cramming More Components onto Integrated Circuits, Electronics, vol. 38, No. 8, pp. 4-8, April 1965.
[4] LEE, T.-Y., LEE, H.-M.: Encryption and Decryption Algorithm of Data Transmission in Network Security, WSEAS Trans. Inf. Sc.
Appl., vol. 3, No. 12, pp. 2557-2562, 2006.
[5] QAWASMEH, E., MASADEH, E.: Developing and Investigation of a New Technique Combining Message Authentication and
Encryption, WSEAS Trans. Inf. Sc. Appl., vol. 3, no. 7, pp. 1417-1422, 2006.
[6] SCHNEIER, B.: Applied Cryptography, Second Edition: Protocols, Algorithms, and Source Code in C. New Jersey : Wiley, 1996.
[7] FEISTEL, H.: Cryptography and Computer Privacy, Sci. Am., vol. 228, no. 5, pp. 15-23, May 1973.
[8] GOTHBERG, D.: Avalanche effect.svg, 2006 [Online]. Available: https://commons.wikimedia.org/ wiki/File:Avalanche_effect.
svg
[9] SUNACHIT: MD5.svg, 2005 [Online] Available: https://commons.wikimedia.org/wiki/File: MD5.svg
[10] RIVEST, R.: The MD5 Message Digest Algorithm, 1992 [Online]. Available: http://tools.ietf.org/ html/rfc1321
[11] WANG, X., YU, H.: How to Break MD5 and Other Hash Functions, Lect. Notes Comput. Sc., No. 3494, pp. 561-577, 2005.
[12] DAMGARD, I. B.: A Design Principle for Hash Functions, Lect. Notes Comput. Sc., No. 435, pp. 416-427, 1990, doi: 10.1007/0-
387-34805-0_39
[13] MERKLE, R. C.: A Certified Digital Signature, Lect. Notes Comput. Sc., No. 435, pp. 218-238, 1990, doi: 10.1007/0-387-34805-
0_21
[14] SPRENGERS, M.: GPU-based Password Cracking: On the Security of Password Hacking Schemes regarding Advances in Graphics
Processing Units, M. S. thesis [Online]. Fac. Sc., Radboud Univ. Nijmegen, Nijmegen, The Netherlands, 2012. Available: http://
enricopagliarini. com/wp-content/uploads/2012/02/thesis.pdf
[15] WANG, X., FENG, D., LAI, X, YU, H.: Collision for Hash Functions MD4, MD5, HAVAL-128 and RIPEMD, 2004 [Online].
Available: http://eprint.iacr.org/2004/199
[16] LENSTRA, A., WANG, X., De WEGER, B.: Colliding X.509 Certificates, 2005 [Online]. Available: http://eprint.iacr.
org/2005/067
[17] SOTIROV, A., STEVENS, M., APPELBAUM, J., LENSTRA, A. et al.: MD5 Considered Harmful Today, 2008 [Online]. Available:
http://www.win.tue.nl/hashclash/rogue-ca/
[18] KLIMA, V.: Finding MD5 Collisions - a Toy for a Notebook, 2006 [Online]. Available: http://eprint.iacr.org/2005/075
[19] US-CERT: MD5 Vulnerable to Collision Attacks, 2008 [Online]. Available: http://www.kb.cert.org/vuls/id/836068
[20] STEVENS, M.: Single-block Collision for MD5, 2012 [Online]. Available: http://marc-stevens.nl/research/md5-1block-collision/
[21] EATLAKE, D. 3rd, JONES, P.: US Secure Hash Algorithm 1 (SHA1), 2001 [Online]. Available: tools.ietf.org/html/rfc3174
[22] WANG, X., YU. H. IN, Y. L.: Efficient Collision Search Attacks on SHA-0, Lect. Notes Comput. Sc., vol. 3621, pp. 1-16, 2005, doi:
10.1007/11535218_1
[23] PIETRYGA: SHA-1.svg, 2007 [Online]. Available: https://commons.wikimedia.org/wiki/File:SHA-1.svg
[24] CANNIERE, C. RECHBERGER, C.: Finding SHA-1 Characteristics: General Results and Applications, Lect. Notes Comput. Sc.,
No. 4284, pp. 1-20, 2006.
[25] STEVENS, M.: New Collision Attacks on SHA-1 Based on Optimal Joint Local-collision Analysis, Lect. Notes Comput. Sc., No.
7881, pp. 245-261, 2013, doi: 10.1007/978-3-642-38348-9_15
COMMUNICATIONS 1/2015 ●
79
REVIEW
[26] LAMBERGER, M, MENDEL, F.: Higher-Order Differential Attack on Reduced SHA-256, 2011 [Online]. Available: http://eprint.
iacr.org/2011/037
[27] BERTONI, G., DAEMEN, J., PEETERS, M. ASSCHE, G.: Sponge Functions, Proc. ECRYPT Hash Workshop 2007, Barcelona,
1997.
[28] AUMASSON, J. P., MEIER, W.: Zero-sum Distinguishers for Reduced Keccak-f and for the Core Functions of Luffa and Hamsi, 2009
[Online]. Available: https://131002.net/data/papers/AM09.pdf
[29] MING, D. XUAJIA, L.: Improved Zero-sum Distinguisher for Full Round Keccak-f Permutation, 2011 [Online]. Available: http://
eprint.iacr.org/2011/023
[30] POLK, T., CHEN, L., TURNR, S., HOFFMAN, P.: Security Considerations for the SHA-0 and SHA-1 Message-Digest Algorithms,
2011 [Online]. Available: http://tools.ietf.org/html/rfc6194
[31] FERNANDEZ, D.: How to Encrypt User Passwords, 2013 [Online]. Available: http://www.jasypt.org/howtoencryptuserpasswords.
html
[32] DANG, O.: NIST Special Publication 800-106: Randomized Hashing for Digital Signatures, 2009 [Online]. Available: http://csrc.
nist.gov/publications/nistpubs/800-106/NIST-SP-800-106.pdf
[33] RUKHIN, A., SOTO, J., NECHVATAL, J., SMID, M.: NIST Special Publication 800-22, Revision 1a: A Statistical Test Suite for
Random and Pseudorandom Number Generators for Cryptographic Applications, 2010 [Online]. Available: http://csrc.nist.gov/
publications/nistpubs/ 800-22-rev1a/SP800-22rev1a.pdf
[34] TURAN, M. S., BARKER, E., BURR, CHEN, L.: NIST Special Publication 800-132: Recommendation for Password-Based Key
Derivation, Part 1: Storage Applications, 2010 [Online]. Available: http://csrc.nist.gov/publications/nistpubs/800-132/nist-sp800-
132.pdf
[35] HORNBY, T.: Salted Password Hashing - Doing it Right, 2013 [Online]. Available: https://crackstation.net/hashing-security.htm
[36] WU, M.-L.: Nonce-aware Encryption Scheme, WSEAS Trans. Inf. Sc. Appl., vol. 6, No. 9, pp. 1513-1522, 2009.
[37] SHIREY, R.: Internet Security Glossary, Version 2, 2007 [Online]. Available: https://tools.ietf. org/html/rfc4949
[38] HELLMAN, M.: A Cryptanalytic Time-Memory Trade-Off, IEEE Trans. Inf. Th., vol. 26, No. 4, pp. 401-406, 1980.
[39] OECHSLIN, P.: Making a Faster Time-Memory Trade-Off, Proc. of 23rd Annu. Int. Cryptology Conf. (CRYPTO 2003), Santa
Barbara, pp. 617-630, 2003.
[40] MANBER, U.: A Simple Scheme to Make Passwords Based on One-Way Functions Much Harder to Crack, 1994 [Online]. Available:
http://webglimpse.net/pubs/TR94-34.pdf
[41] BELLARE, M., CANETTI, R., KRAWCZYK, H.: Keying Hash Functions for Message Authentication, 1996 [Online]. Available:
http://cseweb.ucsd.edu/~mihir/papers/kmd5.pdf
[42] MICHAILH, E., KAKAROUNTAS, A.P., E. FOTOPOULOU, E., GOUTIS, C. E.: Novel Hardware Implementation for
Generating Message Authentication Codes, WSEAS Trans. Commun., vol. 4, No. 11, pp. 1276-1283, 2005.
[43] SHINER, J.: Defending Against Crackers: Peanut Butter Keeps Dogs Friendly, Too, 2011 [Online]. Available: http://blog.agilebits.
com/2011/05/05/defending-against-crackers-peanut-butter-keeps-dogs-friendly-too/
[44] PERCIVAL: Stronger Key Derivation via Sequential Memory-Hard Functions, 2009 [Online]. Proc. BSDCan’09, Ottawa, 2009.
Available: http://www.bsdcan.org/2009/schedule/attachments/ 87_scrypt.pdf
[45] DUDAS, A., JUHASZ, S.: Blocking and Non-blocking Concurrent Hash Tables in Multi-core Systems, WSEAS Trans. Comput.,
vol. 12, No. 2, pp. 74-84, 2013.
80 ● COMMUNICATIONS 1/2015