Explore the full ebook collection and download it now at ebookball.

com

Reverse Engineering for Beginners 1st Edition by

Dennis Yurichev B0DLZR55FH

https://ebookball.com/product/reverse-engineering-for-
beginners-1st-edition-by-dennis-yurichev-b0dlzr55fh-16234/

OR CLICK HERE

DOWLOAD EBOOK

Browse and Get More Ebook Downloads Instantly at https://ebookball.com

Click here to visit ebookball.com and download ebookball now
 Your digital treasures (PDF, ePub, MOBI) await
Download instantly and pick your perfect format...

Read anywhere, anytime, on any device!

Ghidra Software Reverse Engineering for Beginners Analyze

Identify and Avoid Malicious Code and Potential Threats in
Your Networks and Systems 1st Edition by David
18002079729781800207974
https://ebookball.com/product/ghidra-software-reverse-engineering-for-
beginners-analyze-identify-and-avoid-malicious-code-and-potential-
threats-in-your-networks-and-systems-1st-edition-by-
david-18002079729781800207974-16064/
ebookball.com

Ghidra Software Reverse Engineering for Beginners Analyze

identify and avoid malicious code and potential threats in
your networks and systems 1st Edition by AP David ISBN
1800201842 9781800201842
https://ebookball.com/product/ghidra-software-reverse-engineering-for-
beginners-analyze-identify-and-avoid-malicious-code-and-potential-
threats-in-your-networks-and-systems-1st-edition-by-ap-david-
isbn-1800201842-9781800201842-160/
ebookball.com

Reversing Secrets of Reverse Engineering 1st Edition by

Eldad Eilam 0764574817 9780764574818

https://ebookball.com/product/reversing-secrets-of-reverse-
engineering-1st-edition-by-eldad-eilam-0764574817-9780764574818-16232/

ebookball.com

Sockets Shellcode Porting and Coding Reverse Engineering

Exploits and Tool Coding for Security Professionals 1st
Edition by James C Foster ISBN 1597490059 9781597490054
https://ebookball.com/product/sockets-shellcode-porting-and-coding-
reverse-engineering-exploits-and-tool-coding-for-security-
professionals-1st-edition-by-james-c-foster-
isbn-1597490059-9781597490054-16216/
ebookball.com
Practical Reverse Engineering X86 X64 ARM Windows Kernel
Reversing Tools and Obfuscation 1st Edition by Bruce Dang,
Alexandre Gazet, Elias Bachaalany 1118787315 9781118787311
https://ebookball.com/product/practical-reverse-
engineering-x86-x64-arm-windows-kernel-reversing-tools-and-
obfuscation-1st-edition-by-bruce-dang-alexandre-gazet-elias-
bachaalany-1118787315-9781118787311-16242/
ebookball.com

Hacking Hacking Practical Guide for Beginners 1st Edition

by Jeff Simon ISBN B01LXVJRNV

https://ebookball.com/product/hacking-hacking-practical-guide-for-
beginners-1st-edition-by-jeff-simon-isbn-b01lxvjrnv-15730/

ebookball.com

Hacking Hacking Practical Guide for Beginners 1st Edition

by Jeff Simon ISBN B01LXVJRNV

https://ebookball.com/product/hacking-hacking-practical-guide-for-
beginners-1st-edition-by-jeff-simon-isbn-b01lxvjrnv-15704/

ebookball.com

Thoracic Radiology A Guide for Beginners 1st edition by

Iacopo Carbone,Michele Anzidei 9783030357658 3030357651

https://ebookball.com/product/thoracic-radiology-a-guide-for-
beginners-1st-edition-by-iacopo-carbone-michele-
anzidei-9783030357658-3030357651-6066/

ebookball.com

Basic Electronics for Scientists and Engineers 1st edition

by Dennis Eggleston ISBN 0521154308 978-0521154307

https://ebookball.com/product/basic-electronics-for-scientists-and-
engineers-1st-edition-by-dennis-eggleston-
isbn-0521154308-978-0521154307-17674/

ebookball.com
Reverse Engineering for Beginners

Dennis Yurichev
 Reverse Engineering for Beginners

Dennis Yurichev
<dennis(a)yurichev.com>

c ba
©2013-2016, Dennis Yurichev.
This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
license. To view a copy of this license, visit https://creativecommons.org/licenses/by-sa/4.0/.
Text version (May 11, 2017).
The latest version (and Russian edition) of this text is accessible at beginners.re.
The cover was made by Andy Nechaevsky: facebook.

i
Call for translators!

You may want to help me with translating this work into languages other than English and Russian. Just
send me any piece of translated text (no matter how short) and I’ll put it into my LaTeX source code.
Read here.
Speed isn’t important, because this is an open-source project, after all. Your name will be mentioned as
a project contributor. Korean, Chinese, and Persian languages are reserved by publishers. English and
Russian versions I do by myself, but my English is still that horrible, so I’m very grateful for any notes
about grammar, etc. Even my Russian is flawed, so I’m grateful for notes about Russian text as well!
So do not hesitate to contact me: dennis(a)yurichev.com .

ii
Abridged contents

1 Code patterns 1

2 Important fundamentals 448

3 Slightly more advanced examples 467

4 Java 661

5 Finding important/interesting stuff in the code 700

6 OS-specific 734

7 Tools 789

8 Case studies 793

9 Examples of reversing proprietary file formats 928

10 Other things 988

11 Books/blogs worth reading 1002

12 Communities 1005

Afterword 1007

Appendix 1009

Acronyms used 1038

Glossary 1043

Index 1045

iii
Contents

1 Code patterns 1
1.1 The method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Some basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 A short introduction to the CPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.2 Numeral systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Empty function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.1 x86 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.2 ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.3 MIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.4 Empty functions in practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Returning value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4.1 x86 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4.2 ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4.3 MIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4.4 In practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5 Hello, world! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5.1 x86 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5.2 x86-64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.5.3 GCC—one more thing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.5.4 ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.5.5 MIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
1.5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1.5.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1.6 Function prologue and epilogue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1.6.1 Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1.7 Stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1.7.1 Why does the stack grow backwards? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
1.7.2 What is the stack used for? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
1.7.3 A typical stack layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
1.7.4 Noise in stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
1.7.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
1.8 printf() with several arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
1.8.1 x86 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
1.8.2 ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
1.8.3 MIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
1.8.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
1.8.5 By the way . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
1.9 scanf() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
1.9.1 Simple example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
1.9.2 Popular mistake . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
1.9.3 Global variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
1.9.4 scanf() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
1.9.5 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
1.10Accessing passed arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
1.10.1x86 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
1.10.2x64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
1.10.3ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
1.10.4MIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
1.11More about results returning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
1.11.1Attempt to use the result of a function returning void . . . . . . . . . . . . . . . . . . . . . 107
1.11.2What if we do not use the function result? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
1.11.3Returning a structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

iv
CONTENTS
1.12Pointers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
1.12.1Swap input values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
1.12.2Returning values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
1.13GOTO operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
1.13.1Dead code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
1.13.2Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
1.14Conditional jumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
1.14.1Simple example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
1.14.2Calculating absolute value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
1.14.3Ternary conditional operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
1.14.4Getting minimal and maximal values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
1.14.5Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
1.14.6Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
1.15switch()/case/default . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
1.15.1Small number of cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
1.15.2A lot of cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
1.15.3When there are several case statements in one block . . . . . . . . . . . . . . . . . . . . . 179
1.15.4Fall-through . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
1.15.5Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
1.16Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
1.16.1Simple example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
1.16.2Memory blocks copying routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
1.16.3Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
1.16.4Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
1.17More about strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
1.17.1strlen() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
1.17.2Boundaries of strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
1.18Replacing arithmetic instructions to other ones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
1.18.1Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
1.18.2Division . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
1.18.3Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
1.19Floating-point unit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
1.19.1IEEE 754 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
1.19.2x86 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
1.19.3ARM, MIPS, x86/x64 SIMD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
1.19.4C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
1.19.5Simple example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
1.19.6Passing floating point numbers via arguments . . . . . . . . . . . . . . . . . . . . . . . . . . 230
1.19.7Comparison example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
1.19.8Some constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
1.19.9Copying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
1.19.10Stack, calculators and reverse Polish notation . . . . . . . . . . . . . . . . . . . . . . . . . . 267
1.19.11x64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
1.19.12Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
1.20Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
1.20.1Simple example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
1.20.2Buffer overflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
1.20.3Buffer overflow protection methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
1.20.4One more word about arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
1.20.5Array of pointers to strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
1.20.6Multidimensional arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
1.20.7Pack of strings as a two-dimensional array . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
1.20.8Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
1.21By the way . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
1.21.1Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
1.22Manipulating specific bit(s) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
1.22.1Specific bit checking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
1.22.2Setting and clearing specific bits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
1.22.3Shifts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
1.22.4Setting and clearing specific bits: FPU1 example . . . . . . . . . . . . . . . . . . . . . . . . . 317
1.22.5Counting bits set to 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
1.22.6Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
1.22.7Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
1 Floating-point unit

v
CONTENTS
1.23Linear congruential generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
1.23.1x86 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
1.23.2x64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
1.23.332-bit ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
1.23.4MIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
1.23.5Thread-safe version of the example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
1.24Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
1.24.1MSVC: SYSTEMTIME example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
1.24.2Let’s allocate space for a structure using malloc() . . . . . . . . . . . . . . . . . . . . . . . . 349
1.24.3UNIX: struct tm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
1.24.4Fields packing in structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
1.24.5Nested structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
1.24.6Bit fields in a structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
1.24.7Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
1.25Unions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
1.25.1Pseudo-random number generator example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
1.25.2Calculating machine epsilon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
1.26FSCALE replacement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
1.26.1Fast square root calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
1.27Pointers to functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
1.27.1MSVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
1.27.2GCC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
1.27.3Danger of pointers to functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
1.2864-bit values in 32-bit environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
1.28.1Returning of 64-bit value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
1.28.2Arguments passing, addition, subtraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
1.28.3Multiplication, division . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
1.28.4Shifting right . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
1.28.5Converting 32-bit value into 64-bit one . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406
1.29SIMD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
1.29.1Vectorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
1.29.2SIMD strlen() implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
1.3064 bits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
1.30.1x86-64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
1.30.2ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
1.30.3Float point numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
1.30.464-bit architecture criticism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
1.31Working with floating point numbers using SIMD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
1.31.1Simple example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
1.31.2Passing floating point number via arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . 436
1.31.3Comparison example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
1.31.4Calculating machine epsilon: x64 and SIMD . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
1.31.5Pseudo-random number generator example revisited . . . . . . . . . . . . . . . . . . . . . 440
1.31.6Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
1.32ARM-specific details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
1.32.1Number sign (#) before number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
1.32.2Addressing modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
1.32.3Loading a constant into a register . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
1.32.4Relocs in ARM64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
1.33MIPS-specific details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
1.33.1Loading a 32-bit constant into register . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
1.33.2Further reading about MIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447

2 Important fundamentals 448

2.1 Integral datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449
2.1.1 Bit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449
2.1.2 Nibble AKA nybble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449
2.1.3 Byte . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
2.1.4 Wide char . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
2.1.5 Signed integer vs unsigned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
2.1.6 Word . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
2.1.7 Address register . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452
2.1.8 Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452
2.2 Signed number representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

vi
CONTENTS
2.2.1 Using IMUL over MUL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456
2.2.2 Couple of additions about two’s complement form . . . . . . . . . . . . . . . . . . . . . . . 456
2.3 AND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
2.3.1 Checking if a value is on 2n boundary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
2.3.2 KOI-8R Cyrillic encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458
2.4 AND and OR as subtraction and addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459
2.4.1 ZX Spectrum ROM text strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459
2.5 XOR (exclusive OR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
2.5.1 Everyday speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
2.5.2 Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
2.5.3 RAID2 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
2.5.4 XOR swap algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
2.5.5 XOR linked list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462
2.5.6 By the way . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462
2.5.7 AND/OR/XOR as MOV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
2.6 Population count . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
2.7 Endianness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
2.7.1 Big-endian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
2.7.2 Little-endian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
2.7.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
2.7.4 Bi-endian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
2.7.5 Converting data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
2.8 Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465
2.9 CPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465
2.9.1 Branch predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465
2.9.2 Data dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465
2.10Hash functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465
2.10.1How do one-way functions work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466

3 Slightly more advanced examples 467

3.1 Double negation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467
3.2 strstr() example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468
3.3 Temperature converting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468
3.3.1 Integer values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468
3.3.2 Floating-point values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470
3.4 Fibonacci numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472
3.4.1 Example #1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473
3.4.2 Example #2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476
3.4.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480
3.5 CRC32 calculation example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
3.6 Network address calculation example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
3.6.1 calc_network_address() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
3.6.2 form_IP() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
3.6.3 print_as_IP() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487
3.6.4 form_netmask() and set_bit() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488
3.6.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
3.7 Loops: several iterators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
3.7.1 Three iterators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
3.7.2 Two iterators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490
3.7.3 Intel C++ 2011 case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492
3.8 Duff’s device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493
3.8.1 Should one use unrolled loops? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
3.9 Division using multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
3.9.1 x86 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
3.9.2 How it works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497
3.9.3 ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497
3.9.4 MIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499
3.9.5 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499
3.10String to number conversion (atoi()) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499
3.10.1Simple example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499
3.10.2A slightly advanced example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503
3.10.3Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
3.11Inline functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506
2 Redundant Array of Independent Disks

vii
CONTENTS
3.11.1Strings and memory functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506
3.12C99 restrict . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
3.13Branchless abs() function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
3.13.1Optimizing GCC 4.9.1 x64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
3.13.2Optimizing GCC 4.9 ARM64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518
3.14Variadic functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518
3.14.1Computing arithmetic mean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518
3.14.2vprintf() function case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522
3.14.3Pin case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
3.14.4Format string exploit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
3.15Strings trimming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525
3.15.1x64: Optimizing MSVC 2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525
3.15.2x64: Non-optimizing GCC 4.9.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527
3.15.3x64: Optimizing GCC 4.9.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
3.15.4ARM64: Non-optimizing GCC (Linaro) 4.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529
3.15.5ARM64: Optimizing GCC (Linaro) 4.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 530
3.15.6ARM: Optimizing Keil 6/2013 (ARM mode) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531
3.15.7ARM: Optimizing Keil 6/2013 (Thumb mode) . . . . . . . . . . . . . . . . . . . . . . . . . . . 531
3.15.8MIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532
3.16toupper() function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533
3.16.1x64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534
3.16.2ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535
3.16.3Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
3.17Obfuscation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
3.17.1Text strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
3.17.2Executable code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
3.17.3Virtual machine / pseudo-code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
3.17.4Other things to mention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
3.17.5Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
3.18C++ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 540
3.18.1Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 540
3.18.2ostream . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556
3.18.3References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557
3.18.4STL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558
3.18.5Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591
3.19Negative array indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 592
3.19.1Addressing string from the end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 592
3.19.2Addressing some kind of block from the end . . . . . . . . . . . . . . . . . . . . . . . . . . . 592
3.19.3Arrays started at 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593
3.20Packing 12-bit values into array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595
3.20.1Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595
3.20.2Data structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 595
3.20.3The algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596
3.20.4The C/C++ code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596
3.20.5How it works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 598
3.20.6Optimizing GCC 4.8.2 for x86-64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 599
3.20.7Optimizing Keil 5.05 (Thumb mode) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601
3.20.8Optimizing Keil 5.05 (ARM mode) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604
3.20.9(32-bit ARM) Comparison of code density in Thumb and ARM modes . . . . . . . . . . . . 605
3.20.10
Optimizing GCC 4.9.3 for ARM64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605
3.20.11
Optimizing GCC 4.4.5 for MIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607
3.20.12
Difference from the real FAT12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 609
3.20.13
Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 610
3.20.14
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 610
3.20.15
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 610
3.21More about pointers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 610
3.21.1Working with addresses instead of pointers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 611
3.21.2Passing values as pointers; tagged unions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613
3.21.3Pointers abuse in Windows kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614
3.21.4Null pointers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618
3.21.5Array as function argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 622
3.21.6Pointer to function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623
3.21.7Pointer as object identificator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623
3.22Loop optimizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624

viii
CONTENTS
3.22.1Weird loop optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624
3.22.2Another loop optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626
3.23More about structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 628
3.23.1Sometimes a C structure can be used instead of array . . . . . . . . . . . . . . . . . . . . . 628
3.23.2Unsized array in C structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 629
3.23.3Version of C structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 630
3.23.4High-score file in “Block out” game and primitive serialization . . . . . . . . . . . . . . . . 632
3.24memmove() and memcpy() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636
3.24.1Anti-debugging trick . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 637
3.25setjmp/longjmp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 638
3.26Other weird stack hacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640
3.26.1Accessing arguments/local variables of caller . . . . . . . . . . . . . . . . . . . . . . . . . . . 640
3.26.2Returning string . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 642
3.27OpenMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643
3.27.1MSVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645
3.27.2GCC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 647
3.28Another heisenbug . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 648
3.29Windows 16-bit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 649
3.29.1Example#1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 650
3.29.2Example #2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 650
3.29.3Example #3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 651
3.29.4Example #4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 652
3.29.5Example #5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654
3.29.6Example #6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 658

4 Java 661
4.1 Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 661
4.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 661
4.1.2 Returning a value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 661
4.1.3 Simple calculating functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 666
4.1.4 JVM3 memory model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 668
4.1.5 Simple function calling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669
4.1.6 Calling beep() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 670
4.1.7 Linear congruential PRNG4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 671
4.1.8 Conditional jumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 672
4.1.9 Passing arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674
4.1.10Bitfields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675
4.1.11Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 676
4.1.12switch() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 678
4.1.13Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 679
4.1.14Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 687
4.1.15Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 689
4.1.16Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 692
4.1.17Simple patching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694
4.1.18Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 699

5 Finding important/interesting stuff in the code 700

5.1 Identification of executable files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 700
5.1.1 Microsoft Visual C++ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 700
5.1.2 GCC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 701
5.1.3 Intel Fortran . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 701
5.1.4 Watcom, OpenWatcom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 701
5.1.5 Borland . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 702
5.1.6 Other known DLLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703
5.2 Communication with outer world (function level) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703
5.3 Communication with the outer world (win32) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703
5.3.1 Often used functions in the Windows API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704
5.3.2 Extending trial period . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704
5.3.3 Removing nag dialog box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704
5.3.4 tracer: Intercepting all functions in specific module . . . . . . . . . . . . . . . . . . . . . . . 704
5.4 Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705
5.4.1 Text strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705
3 Java virtual machine
4 Pseudorandom number generator

ix
CONTENTS
5.4.2 Finding strings in binary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 710
5.4.3 Error/debug messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 711
5.4.4 Suspicious magic strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 711
5.5 Calls to assert() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 712
5.6 Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 712
5.6.1 Magic numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 713
5.6.2 Specific constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715
5.6.3 Searching for constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715
5.7 Finding the right instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715
5.8 Suspicious code patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716
5.8.1 XOR instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716
5.8.2 Hand-written assembly code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717
5.9 Using magic numbers while tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 718
5.10Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 718
5.10.1Some binary file patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 719
5.10.2Memory “snapshots” comparing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 726
5.11ISA5 detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 727
5.11.1Incorrectly disassembled code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 727
5.11.2Correctly disassembled code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732
5.12Other things . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732
5.12.1General idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 732
5.12.2Order of functions in binary code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 733
5.12.3Tiny functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 733
5.12.4C++ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 733

6 OS-specific 734
6.1 Arguments passing methods (calling conventions) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734
6.1.1 cdecl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734
6.1.2 stdcall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734
6.1.3 fastcall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 735
6.1.4 thiscall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 737
6.1.5 x86-64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 737
6.1.6 Return values of float and double type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 740
6.1.7 Modifying arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 740
6.1.8 Taking a pointer to function argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 741
6.2 Thread Local Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742
6.2.1 Linear congruential generator revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 743
6.3 System calls (syscall-s) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 747
6.3.1 Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 748
6.3.2 Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 748
6.4 Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 748
6.4.1 Position-independent code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 748
6.4.2 LD_PRELOAD hack in Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 751
6.5 Windows NT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 753
6.5.1 CRT (win32) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 753
6.5.2 Win32 PE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 757
6.5.3 Windows SEH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764
6.5.4 Windows NT: Critical section . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787

7 Tools 789
7.1 Binary analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 789
7.1.1 Disassemblers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 789
7.1.2 Decompilers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 790
7.1.3 Patch comparison/diffing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 790
7.2 Live analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 790
7.2.1 Debuggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 790
7.2.2 Library calls tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 790
7.2.3 System calls tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 791
7.2.4 Network sniffing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 791
7.2.5 Sysinternals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 791
7.2.6 Valgrind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 791
7.2.7 Emulators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 791
7.3 Other tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 792
5 Instruction Set Architecture

x
CONTENTS
7.4 Something missing here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 792

8 Case studies 793

8.1 Task manager practical joke (Windows Vista) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 794
8.1.1 Using LEA to load values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 797
8.2 Color Lines game practical joke . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 799
8.3 Minesweeper (Windows XP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 802
8.3.1 Finding grid automatically . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 807
8.3.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 808
8.4 Hacking Windows clock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 808
8.5 Dongles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815
8.5.1 Example #1: MacOS Classic and PowerPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815
8.5.2 Example #2: SCO OpenServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 822
8.5.3 Example #3: MS-DOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 832
8.6 “QR9”: Rubik’s cube inspired amateur crypto-algorithm . . . . . . . . . . . . . . . . . . . . . . . . 837
8.7 Encrypted database case #1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 864
8.7.1 Base64 and entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 864
8.7.2 Is it compressed? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 867
8.7.3 Is it encrypted? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 868
8.7.4 CryptoPP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 868
8.7.5 Cipher Feedback mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 870
8.7.6 Initializing Vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 872
8.7.7 Structure of the buffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 873
8.7.8 Noise at the end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 875
8.7.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 875
8.7.10Post Scriptum: brute-forcing IV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 876
8.8 Overclocking Cointerra Bitcoin miner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 876
8.9 Breaking simple executable cryptor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 880
8.9.1 Other ideas to consider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 885
8.10SAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 885
8.10.1About SAP client network traffic compression . . . . . . . . . . . . . . . . . . . . . . . . . . . 885
8.10.2SAP 6.0 password checking functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896
8.11Oracle RDBMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 900
8.11.1 V$VERSION table in the Oracle RDBMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 900
8.11.2 X$KSMLRU table in Oracle RDBMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 908
8.11.3 V$TIMER table in Oracle RDBMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 909
8.12Handwritten assembly code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 913
8.12.1 EICAR test file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 913
8.13Demos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 914
8.13.110 PRINT CHR$(205.5+RND(1)); : GOTO 10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 914
8.13.2Mandelbrot set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 917
8.14Other examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 927

9 Examples of reversing proprietary file formats 928

9.1 Primitive XOR-encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 928
9.1.1 Norton Guide: simplest possible 1-byte XOR encryption . . . . . . . . . . . . . . . . . . . . 929
9.1.2 Simplest possible 4-byte XOR encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 932
9.1.3 Simple encryption using XOR mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 936
9.1.4 Simple encryption using XOR mask, case II . . . . . . . . . . . . . . . . . . . . . . . . . . . . 942
9.2 Analyzing using information entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 948
9.2.1 Analyzing entropy in Mathematica . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 948
9.2.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 955
9.2.3 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 955
9.2.4 A word about primitive encryption like XORing . . . . . . . . . . . . . . . . . . . . . . . . . . 955
9.2.5 More about entropy of executable code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 955
9.2.6 Random number generators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 956
9.2.7 Entropy of various files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 956
9.2.8 Making lower level of entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 957
9.3 Millenium game save file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 958
9.4 fortune program indexing file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 965
9.4.1 Hacking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 969
9.4.2 The files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 970
9.5 Oracle RDBMS: .SYM-files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 970
9.6 Oracle RDBMS: .MSB-files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 980

xi
CONTENTS
9.6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 987
9.7 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 987
9.8 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 987

10 Other things 988

10.1Executable files patching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 988
10.1.1Text strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 988
10.1.2x86 code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 988
10.2Function arguments number statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 989
10.3Compiler intrinsic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 989
10.4Compiler’s anomalies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 990
10.4.1Oracle RDBMS 11.2 and Intel C++ 10.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 990
10.4.2MSVC 6.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 990
10.4.3Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 991
10.5Itanium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 991
10.68086 memory model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 993
10.7Basic blocks reordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994
10.7.1Profile-guided optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994
10.8My experience with Hex-Rays 2.2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 996
10.8.1Bugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 996
10.8.2Odd peculiarities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 997
10.8.3Silence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 999
10.8.4Comma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1000
10.8.5Data types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1001
10.8.6My plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1001
10.8.7Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1001

11 Books/blogs worth reading 1002

11.1Books and other materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1002
11.1.1Reverse Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1002
11.1.2Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1002
11.1.3C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1002
11.1.4x86 / x86-64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1003
11.1.5ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1003
11.1.6Assembly language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1003
11.1.7Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1003
11.1.8UNIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1003
11.1.9Programming in general . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1003
11.1.10
Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1004

12 Communities 1005

Afterword 1007
12.1Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1007

Appendix 1009
.1 x86 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1009
.1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1009
.1.2 General purpose registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1009
.1.3 FPU registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1013
.1.4 SIMD registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1015
.1.5 Debugging registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1015
.1.6 Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1016
.1.7 npad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1028
.2 ARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1029
.2.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1029
.2.2 Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1030
.2.3 32-bit ARM (AArch32) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1030
.2.4 64-bit ARM (AArch64) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1031
.2.5 Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1031
.3 MIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1032
.3.1 Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1032
.3.2 Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1033

xii
CONTENTS
.4 Some GCC library functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1033
.5 Some MSVC library functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1033
.6 Cheatsheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1034
.6.1 IDA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1034
.6.2 OllyDbg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1034
.6.3 MSVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1034
.6.4 GCC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1035
.6.5 GDB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1035

Acronyms used 1038

Glossary 1043

Index 1045

xiii
CONTENTS
Preface

There are several popular meanings of the term “reverse engineering”: 1) The reverse engineering of
software: researching compiled programs; 2) The scanning of 3D structures and the subsequent digital
manipulation required in order to duplicate them; 3) Recreating DBMS6 structure. This book is about the
first meaning.

Topics discussed in-depth

x86/x64, ARM/ARM64, MIPS, Java/JVM.

Topics touched upon

Oracle RDBMS ( 8.11 on page 900), Itanium ( 10.5 on page 991), copy-protection dongles ( 8.5 on
page 815), LD_PRELOAD ( 6.4.2 on page 751), stack overflow, ELF7 , win32 PE file format ( 6.5.2 on
page 757), x86-64 ( 1.30.1 on page 421), critical sections ( 6.5.4 on page 787), syscalls ( 6.3 on page 747),
TLS8 , position-independent code (PIC9 ) ( 6.4.1 on page 748), profile-guided optimization ( 10.7.1 on
page 994), C++ STL ( 3.18.4 on page 558), OpenMP ( 3.27 on page 643), SEH ( 6.5.3 on page 764).

Prerequisites

Basic C PL10 knowledge. Recommended reading: 11.1.3 on page 1002.

Exercises and tasks

…are all moved to the separate website: http://challenges.re.

About the author

Dennis Yurichev is an experienced reverse engineer and programmer.
He can be contacted by email: dennis(a)yurichev.com.

Praise for Reverse Engineering for Beginners

• “Now that Dennis Yurichev has made this book free (libre), it is a contribution to the world of free
knowledge and free education.” Richard M. Stallman, GNU founder, software freedom activist.
• “It’s very well done .. and for free .. amazing.”11 Daniel Bilar, Siege Technologies, LLC.
6 Database management systems
7 Executable file format widely used in *NIX systems including Linux
8 Thread Local Storage
9 Position Independent Code: 6.4.1 on page 748
10 Programming language
11 twitter.com/daniel_bilar/status/436578617221742593

xiv
CONTENTS
• “... excellent and free”12 Pete Finnigan, Oracle RDBMS security guru.
• “... book is interesting, great job!” Michael Sikorski, author of Practical Malware Analysis: The
Hands-On Guide to Dissecting Malicious Software.
• “... my compliments for the very nice tutorial!” Herbert Bos, full professor at the Vrije Universiteit
Amsterdam, co-author of Modern Operating Systems (4th Edition).
• “... It is amazing and unbelievable.” Luis Rocha, CISSP / ISSAP, Technical Manager, Network & Infor-
mation Security at Verizon Business.
• “Thanks for the great work and your book.” Joris van de Vis, SAP Netweaver & Security specialist.
• “... reasonable intro to some of the techniques.”13 Mike Stay, teacher at the Federal Law Enforce-
ment Training Center, Georgia, US.
• “I love this book! I have several students reading it at the moment, plan to use it in graduate
course.”14 Sergey Bratus , Research Assistant Professor at the Computer Science Department at
Dartmouth College
• “Dennis @Yurichev has published an impressive (and free!) book on reverse engineering”15 Tanel
Poder, Oracle RDBMS performance tuning expert .
• “This book is some kind of Wikipedia to beginners...” Archer, Chinese Translator, IT Security Re-
searcher.
• “First class reference for people wanting to learn reverse engineering. And it’s free for all.” Mikko
Hyppönen, F-Secure.

Thanks

For patiently answering all my questions: Andrey “herm1t” Baranovich, Slava “Avid” Kazakov.
For sending me notes about mistakes and inaccuracies: Stanislav “Beaver” Bobrytskyy, Alexander Ly-
senko, Alexander “Solar Designer” Peslyak, Federico Ramondino, Mark Wilson, Shell Rocket, Zhu Ruijin,
Changmin Heo, Vitor Vidal, Stijn Crevits, Jean-Gregoire Foulon16 , Ben L., Etienne Khan, Norbert Szetei17 ,
Marc Remy, Michael Hansen..
For helping me in other ways: Andrew Zubinski, Arnaud Patard (rtp on #debian-arm IRC), noshadow on
#gcc IRC, Aliaksandr Autayeu, Mohsen Mostafa Jokar.
For translating the book into Simplified Chinese: Antiy Labs (antiy.cn), Archer.
For translating the book into Korean: Byungho Min.
For translating the book into Dutch: Cedric Sambre (AKA Midas).
For translating the book into Spanish: Diego Boy, Luis Alberto Espinosa Calvo, Fernando Guida.
For translating the book into Portuguese: Thales Stevan de A. Gois.
For translating the book into Italian: Federico Ramondino18 , Paolo Stivanin19 , twyK.
For translating the book into French: Florent Besnard20 , Marc Remy21 , Baudouin Landais, Téo Dacquet22 .
For translating the book into German: Dennis Siekmeier23 , Julius Angres24 , Dirk Loser25 .
For proofreading: Alexander “Lstar” Chernenkiy, Vladimir Botov, Andrei Brazhuk, Mark “Logxen” Cooper,
Yuan Jochen Kang, Mal Malakov, Lewis Porter, Jarle Thorsen, Hong Xie.
12 twitter.com/petefinnigan/status/400551705797869568
13 reddit
14 twitter.com/sergeybratus/status/505590326560833536
15 twitter.com/TanelPoder/status/524668104065159169
16 https://github.com/pixjuan
17 https://github.com/73696e65
18 https://github.com/pinkrab
19 https://github.com/paolostivanin
20 https://github.com/besnardf
21 https://github.com/mremy
22 https://github.com/T30rix
23 https://github.com/DSiekmeier
24 https://github.com/JAngres
25 https://github.com/PolymathMonkey

xv
CONTENTS
Vasil Kolev26 did a great amount of work in proofreading and correcting many mistakes.
For illustrations and cover art: Andy Nechaevsky.
Thanks also to all the folks on github.com who have contributed notes and corrections27 .
Many LATEX packages were used: I would like to thank the authors as well.

Donors

Those who supported me during the time when I wrote significant part of the book:
2 * Oleg Vygovsky (50+100 UAH), Daniel Bilar ($50), James Truscott ($4.5), Luis Rocha ($63), Joris van
de Vis ($127), Richard S Shultz ($20), Jang Minchang ($20), Shade Atlas (5 AUD), Yao Xiao ($10), Pawel
Szczur (40 CHF), Justin Simms ($20), Shawn the R0ck ($27), Ki Chan Ahn ($50), Triop AB (100 SEK), Ange
Albertini (e10+50), Sergey Lukianov (300 RUR), Ludvig Gislason (200 SEK), Gérard Labadie (e40), Sergey
Volchkov (10 AUD), Vankayala Vigneswararao ($50), Philippe Teuwen ($4), Martin Haeberli ($10), Victor
Cazacov (e5), Tobias Sturzenegger (10 CHF), Sonny Thai ($15), Bayna AlZaabi ($75), Redfive B.V. (e25),
Joona Oskari Heikkilä (e5), Marshall Bishop ($50), Nicolas Werner (e12), Jeremy Brown ($100), Alexandre
Borges ($25), Vladimir Dikovski (e50), Jiarui Hong (100.00 SEK), Jim Di (500 RUR), Tan Vincent ($30),
Sri Harsha Kandrakota (10 AUD), Pillay Harish (10 SGD), Timur Valiev (230 RUR), Carlos Garcia Prado
(e10), Salikov Alexander (500 RUR), Oliver Whitehouse (30 GBP), Katy Moe ($14), Maxim Dyakonov ($3),
Sebastian Aguilera (e20), Hans-Martin Münch (e15), Jarle Thorsen (100 NOK), Vitaly Osipov ($100), Yuri
Romanov (1000 RUR), Aliaksandr Autayeu (e10), Tudor Azoitei ($40), Z0vsky (e10), Yu Dai ($10).
Thanks a lot to every donor!

mini-FAQ

Q: What are prerequisites for reading this book?

A: Basic understanding of C/C++ is desirable.
Q: Can I buy Russian/English hardcopy/paper book?
A: Unfortunately no, no publisher got interested in publishing Russian or English version so far. Meanwhile,
you can ask your favorite copy shop to print/bind it.
Q: Is there epub/mobi version?
A: The book is highly dependent on TeX/LaTeX-specific hacks, so converting to HTML (epub/mobi is a set
of HTMLs) will not be easy.
Q: Why should one learn assembly language these days?
A: Unless you are an OS28 developer, you probably don’t need to code in assembly—latest compilers
(2010s) are much better at performing optimizations than humans 29 .
Also, latest CPU30 s are very complex devices and assembly knowledge doesn’t really help one to under-
stand their internals.
That being said, there are at least two areas where a good understanding of assembly can be helpful:
First and foremost, security/malware research. It is also a good way to gain a better understanding of
your compiled code whilst debugging. This book is therefore intended for those who want to understand
assembly language rather than to code in it, which is why there are many examples of compiler output
contained within.
Q: I clicked on a hyperlink inside a PDF-document, how do I go back?
A: In Adobe Acrobat Reader click Alt+LeftArrow. In Evince click “<” button.
Q: May I print this book / use it for teaching?
A: Of course! That’s why the book is licensed under the Creative Commons license (CC BY-SA 4.0).
Q: Why is this book free? You’ve done great job. This is suspicious, as many other free things.
26 https://vasil.ludost.net/
27 https://github.com/dennis714/RE-for-beginners/graphs/contributors
28 Operating System
29 Avery good text about this topic: [Agner Fog, The microarchitecture of Intel, AMD and VIA CPUs, (2016)]
30 Central processing unit

xvi
CONTENTS
A: In my own experience, authors of technical literature do this mostly for self-advertisement purposes.
It’s not possible to get any decent money from such work.
Q: How does one get a job in reverse engineering?
A: There are hiring threads that appear from time to time on reddit, devoted to RE31 (2016). Try looking
there.
A somewhat related hiring thread can be found in the “netsec” subreddit: 2016.
Q: I have a question...
A: Send it to me by email (dennis(a)yurichev.com).

About the Korean translation

In January 2015, the Acorn publishing company (www.acornpub.co.kr) in South Korea did a huge amount
of work in translating and publishing my book (as it was in August 2014) into Korean.
It’s now available at their website.
The translator is Byungho Min (twitter/tais9). The cover art was done by my artistic friend, Andy Nechaevsky:
facebook/andydinka. They also hold the copyright to the Korean translation.
So, if you want to have a real book on your shelf in Korean and want to support my work, it is now available
for purchase.

About the Persian/Farsi translation

In 2016 the book has been translated by Mohsen Mostafa Jokar (who is also known to Iranian community
by his translation of Radare manual32 ). It is available on the publisher’s website33 (Pendare Pars).
40 page excerpt: https://beginners.re/farsi.pdf.
Registration of the book in National Library of Iran: http://opac.nlai.ir/opac-prod/bibliographic/
4473995.

About the Chinese translation

In April 2017, translation to Chinese has been finished by Chinese PTPress publisher. They are also the
Chinese translation copyright holder.
It’s available for order here: http://www.epubit.com.cn/book/details/4174. Some kind of review and
history behind the translation: http://www.cptoday.cn/news/detail/3155.
Principal translator is Archer, to whom I owe so much. He was extremely meticulous (in good sense)
and reported most of known mistakes and bugs, which is very important to literature like this book. I’ll
recommend his services to any other author!
Guys from Antiy Labs has also helped with translation. Here is preface written by them.

31 reddit.com/r/ReverseEngineering/
32 http://rada.re/get/radare2book-persian.pdf
33 http://goo.gl/2Tzx0H

xvii
Chapter 1

Code patterns

Everything is comprehended through

comparison

Author unknown

1.1 The method

When the author of this book first started learning C and, later, C++, he used to write small pieces of
code, compile them, and then look at the assembly language output. This made it very easy for him
to understand what was going on in the code that he had written. 1 . He did it so many times that the
relationship between the C/C++ code and what the compiler produced was imprinted deeply in his mind.
It’s easy to imagine instantly a rough outline of C code’s appearance and function. Perhaps this technique
could be helpful for others.
Sometimes ancient compilers are used here, in order to get the shortest (or simplest) possible code snip-
pet.
By the way, there is a great website where you can do the same, with various compilers, instead of
installing them on your box. You can use it as well: https://gcc.godbolt.org/.

Exercises

When the author of this book studied assembly language, he also often compiled small C-functions and
then rewrote them gradually to assembly, trying to make their code as short as possible. This probably
is not worth doing in real-world scenarios today, because it’s hard to compete with latest compilers in
terms of efficiency. It is, however, a very good way to gain a better understanding of assembly. Feel free,
therefore, to take any assembly code from this book and try to make it shorter. However, don’t forget to
test what you have written.

Optimization levels and debug information

Source code can be compiled by different compilers with various optimization levels. A typical compiler
has about three such levels, where level zero means disable optimization. Optimization can also be tar-
geted towards code size or code speed. A non-optimizing compiler is faster and produces more under-
standable (albeit verbose) code, whereas an optimizing compiler is slower and tries to produce code that
runs faster (but is not necessarily more compact). In addition to optimization levels, a compiler can in-
clude in the resulting file some debug information, thus producing code for easy debugging. One of the
important features of the ´debug’ code is that it might contain links between each line of the source code
and the respective machine code addresses. Optimizing compilers, on the other hand, tend to produce
output where entire lines of source code can be optimized away and thus not even be present in the re-
sulting machine code. Reverse engineers can encounter either version, simply because some developers
1 In fact, he still does it when he can’t understand what a particular bit of code does.

1
1.2. SOME BASICS
turn on the compiler’s optimization flags and others do not. Because of this, we’ll try to work on examples
of both debug and release versions of the code featured in this book, where possible.

1.2 Some basics

1.2.1 A short introduction to the CPU

The CPU is the device that executes the machine code a program consists of.
A short glossary:
Instruction : A primitive CPU command. The simplest examples include: moving data between registers,
working with memory, primitive arithmetic operations. As a rule, each CPU has its own instruction
set architecture (ISA).
Machine code : Code that the CPU directly processes. Each instruction is usually encoded by several
bytes.
Assembly language : Mnemonic code and some extensions like macros that are intended to make a
programmer’s life easier.
CPU register : Each CPU has a fixed set of general purpose registers (GPR2 ). ≈ 8 in x86, ≈ 16 in x86-
64, ≈ 16 in ARM. The easiest way to understand a register is to think of it as an untyped temporary
variable. Imagine if you were working with a high-level PL and could only use eight 32-bit (or 64-bit)
variables. Yet a lot can be done using just these!
One might wonder why there needs to be a difference between machine code and a PL. The answer lies
in the fact that humans and CPUs are not alike—it is much easier for humans to use a high-level PL like
C/C++, Java, Python, etc., but it is easier for a CPU to use a much lower level of abstraction. Perhaps
it would be possible to invent a CPU that can execute high-level PL code, but it would be many times
more complex than the CPUs we know of today. In a similar fashion, it is very inconvenient for humans
to write in assembly language, due to it being so low-level and difficult to write in without making a huge
number of annoying mistakes. The program that converts the high-level PL code into assembly is called
a compiler. 3 .

A couple of words about different ISAs

The x86 ISA has always been one with variable-length instructions, so when the 64-bit era came, the x64
extensions did not impact the ISA very significantly. In fact, the x86 ISA still contains a lot of instructions
that first appeared in 16-bit 8086 CPU, yet are still found in the CPUs of today. ARM is a RISC4 CPU
designed with constant-length instructions in mind, which had some advantages in the past. In the very
beginning, all ARM instructions were encoded in 4 bytes5 . This is now referred to as “ARM mode”. Then
they thought it wasn’t as frugal as they first imagined. In fact, most used CPU instructions6 in real world
applications can be encoded using less information. They therefore added another ISA, called Thumb,
where each instruction was encoded in just 2 bytes. This is now referred as “Thumb mode”. However, not
all ARM instructions can be encoded in just 2 bytes, so the Thumb instruction set is somewhat limited. It is
worth noting that code compiled for ARM mode and Thumb mode may of course coexist within one single
program. The ARM creators thought Thumb could be extended, giving rise to Thumb-2, which appeared
in ARMv7. Thumb-2 still uses 2-byte instructions, but has some new instructions which have the size of
4 bytes. There is a common misconception that Thumb-2 is a mix of ARM and Thumb. This is incorrect.
Rather, Thumb-2 was extended to fully support all processor features so it could compete with ARM mode—
a goal that was clearly achieved, as the majority of applications for iPod/iPhone/iPad are compiled for the
Thumb-2 instruction set (admittedly, largely due to the fact that Xcode does this by default). Later the
64-bit ARM came out. This ISA has 4-byte instructions, and lacked the need of any additional Thumb mode.
However, the 64-bit requirements affected the ISA, resulting in us now having three ARM instruction sets:
ARM mode, Thumb mode (including Thumb-2) and ARM64. These ISAs intersect partially, but it can be
said that they are different ISAs, rather than variations of the same one. Therefore, we would try to add
2 General Purpose Registers
3 Old-school Russian literature also use term “translator”.
4 Reduced instruction set computing
5 By the way, fixed-length instructions are handy because one can calculate the next (or previous) instruction address without

effort. This feature will be discussed in the switch() operator ( 1.15.2 on page 174) section.
6 These are MOV/PUSH/CALL/Jcc

2
1.2. SOME BASICS
fragments of code in all three ARM ISAs in this book. There are, by the way, many other RISC ISAs with
fixed length 32-bit instructions, such as MIPS, PowerPC and Alpha AXP.

1.2.2 Numeral systems

Humans accustomed to decimal numeral system probably because almost all ones has 10 fingers. Nev-
ertheless, number 10 has no significant meaning in science and mathematics. Natural numeral system
in digital electronics is binary: 0 is for absence of current in wire and 1 for presence. 10 in binary is 2 in
decimal; 100 in binary is 4 in decimal and so on.
If the numeral system has 10 digits, it has radix (or base) of 10. Binary numeral system has radix of 2.
Important things to recall: 1) number is a number, while digit is a term of writing system and is usually
one character; 2) number is not changed when converted to another radix: writing notation is (and way
of representing it in RAM7 ).
How to convert a number from one radix to another?
Positional notation is used almost everywhere, this means, a digit has some weight depending on where
it is placed inside of number. If 2 is placed at the rightmost place, it’s 2. If it is placed at the place one
digit before rightmost, it’s 20.
What does 1234 stand for?
103 ⋅ 1 + 102 ⋅ 2 + 101 ⋅ 3 + 1 ⋅ 4 = 1234 or 1000 ⋅ 1 + 100 ⋅ 2 + 10 ⋅ 3 + 4 = 1234
Same story for binary numbers, but base is 2 instead of 10. What does 0b101011 stand for?
25 ⋅ 1 + 24 ⋅ 0 + 23 ⋅ 1 + 22 ⋅ 0 + 21 ⋅ 1 + 20 ⋅ 1 = 43 or 32 ⋅ 1 + 16 ⋅ 0 + 8 ⋅ 1 + 4 ⋅ 0 + 2 ⋅ 1 + 1 = 43
Positional notation can be opposed to non-positional notation such as Roman numeric system 8 . Perhaps,
humankind switched to positional notation because it’s easier to do basic operations (addition, multipli-
cation, etc.) on paper by hand.
Indeed, binary numbers can be added, subtracted and so on in the very same as taught in schools, but
only 2 digits are available.
Binary numbers are bulky when represented in source code and dumps, so that is where hexadecimal
numeral system can be used. Hexadecimal radix uses 0..9 digits and also 6 Latin characters: A..F. Each
hexadecimal digit takes 4 bits or 4 binary digits, so it’s very easy to convert from binary number to
hexadecimal and back, even manually, in one’s mind.

hexadecimal binary decimal

0 0000 0
1 0001 1
2 0010 2
3 0011 3
4 0100 4
5 0101 5
6 0110 6
7 0111 7
8 1000 8
9 1001 9
A 1010 10
B 1011 11
C 1100 12
D 1101 13
E 1110 14
F 1111 15

How to understand, which radix is used in a specific place?

Decimal numbers are usually written as is, i.e., 1234. But some assemblers allows to make emphasis on
decimal radix and this number can be written with ”d” suffix: 1234d.
7 Random-access memory
8 About numeric system evolution, see [Donald E. Knuth, The Art of Computer Programming, Volume 2, 3rd ed., (1997), 195–213.]

3
1.2. SOME BASICS
Binary numbers sometimes prepended with ”0b” prefix: 0b100110111 (GCC9 has non-standard language
extension for this10 ). There is also another way: ”b” suffix, for example: 100110111b. I’ll try to stick to
”0b” prefix throughout the book for binary numbers.
Hexadecimal numbers are prepended with ”0x” prefix in C/C++ and other PLs: 0x1234ABCD. Or they are
has ”h” suffix: 1234ABCDh—this is common way of representing them in assemblers and debuggers. If
the number is started with A..F digit, 0 is to be added before: 0ABCDEFh. There was also convention
that was popular in 8-bit home computers era, using $ prefix, like $ABCD. I’ll try to stick to ”0x” prefix
throughout the book for hexadecimal numbers.
Should one learn to convert numbers in mind? A table of 1-digit hexadecimal numbers can easily be
memorized. As of larger numbers, probably, it’s not worth to torment yourself.
Perhaps, the most visible to all people hexadecimal numbers are in URL11 s. This is the way how non-Latin
characters are encoded. For example: https://en.wiktionary.org/wiki/na%C3%AFvet%C3%A9 is the
URL of Wiktionary article about “naïveté” word.

Octal radix

Another numeral system heavily used in past of computer programming is octal: there are 8 digits (0..7)
and each is mapped to 3 bits, so it’s easy to convert numbers back and forth. It has been superseded by
hexadecimal system almost everywhere, but surprisingly, there is *NIX utility used by many people often
which takes octal number as argument: chmod .

As many *NIX users know, chmod argument can be a number of 3 digits. The first digit is rights for owner
of file, second is rights for group (to which file belongs), third is for everyone else. And each digit can be
represented in binary form:

decimal binary meaning

7 111 rwx
6 110 rw-
5 101 r-x
4 100 r--
3 011 -wx
2 010 -w-
1 001 --x
0 000 ---

So each bit is mapped to a flag: read/write/execute.

Now the reason why I’m talking about chmod here is that the whole number in argument can be repre-
sented as octal number. Let’s take for example, 644. When you run chmod 644 file , you set read/write
permissions for owner, read permissions for group and again, read permissions for everyone else. Let’s
convert 644 octal number to binary, this will be 110100100 , or (in groups of 3 bits) 110 100 100 .
Now we see that each triplet describe permissions for owner/group/others: first is rw- , second is r--
and third is r-- .
Octal numeral system was also popular on old computers like PDP-8, because word there could be 12,
24 or 36 bits, and these numbers are divisible by 3, so octal system was natural on that environment.
Nowadays, all popular computers employs word/address size of 16, 32 or 64 bits, and these numbers are
divisible by 4, so hexadecimal system is more natural here.
Octal numeral system is supported by all standard C/C++ compilers. This is source of confusion some-
times, because octal numbers are encoded with zero prepended, for example, 0377 is 255. And some-
times, you may make a typo and write ”09” instead of 9, and the compiler would report error. GCC may
report something like that:
error: invalid digit "9" in octal constant .
9 GNU Compiler Collection
10 https://gcc.gnu.org/onlinedocs/gcc/Binary-constants.html
11 Uniform Resource Locator

4
1.3. EMPTY FUNCTION
Divisibility

When you see a decimal number like 120, you can quickly deduce that it’s divisible by 10, because the
last digit is zero. In the same way, 123400 is divisible by 100, because two last digits are zeros.
Likewise, hexadecimal number 0x1230 is divisible by 0x10 (or 16), 0x123000 is divisible by 0x1000 (or
4096), etc.
Binary number 0b1000101000 is divisible by 0b1000 (8), etc.
This property can be used often to realize quickly if a size of some block in memory is padded to some
boundary. For example, sections in PE12 files are almost always started at addresses ending with 3 hex-
adecimal zeros: 0x41000, 0x10001000, etc. The reason behind this is in the fact that almost all PE
sections are padded to boundary of 0x1000 (4096) bytes.

Multi-precision arithmetic and radix

Multi-precision arithmetic can use huge numbers, and each one may be stored in several bytes. For
example, RSA keys, both public and private, are spanning up to 4096 bits and maybe even more.
In [Donald E. Knuth, The Art of Computer Programming, Volume 2, 3rd ed., (1997), 265] we can find
the following idea: when you store multi-precision number in several bytes, the whole number can be
represented as having a radix of 28 = 256, and each digit goes to corresponding byte. Likewise, if you store
multi-precision number in several 32-bit integer values, each digit goes to each 32-bit slot, and you may
think about this number as stored in radix of 232 .

Pronouncement

Numbers in non-decimal base are usually pronounced by one digit: “one-zero-zero-one-one-...”. Words
like “ten”, “thousand”, etc, are usually not pronounced, because it will be confused with decimal base
then.

Floating point numbers

To distinguish floating point numbers from integer ones, they are usually written with “.0” at the end, like
0.0, 123.0, etc.

1.3 Empty function

The simplest possible function is arguably one that does nothing:

Listing 1.1: C/C++ Code
void f()
{
return;
};

Lets compile it!

1.3.1 x86

Here’s what both the optimizing GCC and MSVC compilers produce on the x86 platform:
Listing 1.2: Optimizing GCC/MSVC (assembly output)
f:
ret

There is just one instruction: RET , which returns execution to the caller.
12 Portable Executable

5
1.3. EMPTY FUNCTION
1.3.2 ARM

Listing 1.3: Optimizing Keil 6/2013 (ARM mode) assembly output

f PROC
BX lr
ENDP

The return address is not saved on the local stack in the ARM ISA, but rather in the link register, so the
BX LR instruction causes execution to jump to that address—effectively returning execution to the caller.

1.3.3 MIPS

There are two naming conventions used in the world of MIPS when naming registers: by number (from $0
to $31) or by pseudo name ($V0, $A0, etc.).
The GCC assembly output below lists registers by number:

Listing 1.4: Optimizing GCC 4.4.5 (assembly output)

j $31
nop

…while IDA13 does it—by their pseudo names:

Listing 1.5: Optimizing GCC 4.4.5 (IDA)

j $ra
nop

The first instruction is the jump instruction (J or JR) which returns the execution flow to the caller, jumping
to the address in the $31 (or $RA) register.
This is the register analogous to LR14 in ARM.
The second instruction is NOP15 , which does nothing. We can ignore it so far.

A note about MIPS instruction/register names

Register and instruction names in the world of MIPS are traditionally written in lowercase. However, for
the sake of consistency, we’ll stick to using uppercase letters, as it is the convention followed by all other
ISAs featured this book.

1.3.4 Empty functions in practice

Despite the fact empty functions are useless, they are quite frequent in low-level code.
First of all, debugging functions are quite popular, like this one:

Listing 1.6: C/C++ Code

void dbg_print (const char *fmt, ...)
{
#ifdef _DEBUG
// open log file
// write to log file
// close log file
#endif
};

void some_function()
{
...
13 Interactive Disassembler and debugger developed by Hex-Rays
14 LinkRegister
15 No OPeration

6
1.4. RETURNING VALUE

dbg_print ("we did something\n");

...
};

In non-debug build (e.g., “release”), _DEBUG is not defined, so dbg_print() function, despite still being
called during execution, will be empty.
Another popular way of software protection is make several builds: one for legal customers, and a demo
build. Demo build can lack some important functions, like this:
Listing 1.7: C/C++ Code
void save_file ()
{
#ifndef DEMO
// actual saving code
#endif
};

save_file() function can be called when user click File->Save menu. Demo version may be delivered
with this menu item disabled, but even if software cracker will enable it, empty function with no useful
code will be called.
IDA marks such functions with names like nullsub_00 , nullsub_01 , etc.

1.4 Returning value

Another simple function is the one that simply returns a constant value:
Listing 1.8: C/C++ Code
int f()
{
return 123;
};

Lets compile it.

1.4.1 x86

Here’s what both the optimizing GCC and MSVC compilers produce on the x86 platform:
Listing 1.9: Optimizing GCC/MSVC (assembly output)
f:
mov eax, 123
ret

There are just two instructions: the first places the value 123 into the EAX register, which is used by
convention for storing the return value and the second one is RET , which returns execution to the caller.

The caller will take the result from the EAX register.

1.4.2 ARM

There are a few differences on the ARM platform:

Listing 1.10: Optimizing Keil 6/2013 (ARM mode) ASM Output
f PROC
MOV r0,#0x7b ; 123
BX lr
ENDP

7
1.5. HELLO, WORLD!
ARM uses the register R0 for returning the results of functions, so 123 is copied into R0 .

It is worth noting that MOV is a misleading name for the instruction in both x86 and ARM ISAs.
The data is not in fact moved, but copied.

1.4.3 MIPS

The GCC assembly output below lists registers by number:

Listing 1.11: Optimizing GCC 4.4.5 (assembly output)

j $31
li $2,123 # 0x7b

…while IDA does it by their pseudo names:

Listing 1.12: Optimizing GCC 4.4.5 (IDA)

jr $ra
li $v0, 0x7B

The $2 (or $V0) register is used to store the function’s return value. LI stands for “Load Immediate” and
is the MIPS equivalent to MOV .
The other instruction is the jump instruction (J or JR) which returns the execution flow to the caller.
You might be wondering why positions of the load instruction (LI) and the jump instruction (J or JR) are
swapped. This is due to a RISC feature called “branch delay slot”.
The reason this happens is a quirk in the architecture of some RISC ISAs and isn’t important for our
purposes—we just must keep in mind that in MIPS, the instruction following a jump or branch instruction
is executed before the jump/branch instruction itself.
As a consequence, branch instructions always swap places with the instruction which must be executed
beforehand.

1.4.4 In practice

Functions which merely returns 1 (true) or 0 (false) are very frequent in practice.
Smallest ever standard UNIX utilities, true and false returns 0 and 1 respectively, as an exit code (zero as
an exit code usually means success, non-zero means error).

1.5 Hello, world!

Let’s use the famous example from the book [Brian W. Kernighan, Dennis M. Ritchie, The C Programming
Language, (1988)]:
#include <stdio.h>

int main()
{
printf("hello, world\n");
return 0;
}

1.5.1 x86

MSVC

Let’s compile it in MSVC 2010:

8
1.5. HELLO, WORLD!

cl 1.cpp /Fa1.asm

( /Fa option instructs the compiler to generate assembly listing file)

Listing 1.13: MSVC 2010

CONST SEGMENT
$SG3830 DB 'hello, world', 0AH, 00H
CONST ENDS
PUBLIC _main
EXTRN _printf:PROC
; Function compile flags: /Odtp
_TEXT SEGMENT
_main PROC
push ebp
mov ebp, esp
push OFFSET $SG3830
call _printf
add esp, 4
xor eax, eax
pop ebp
ret 0
_main ENDP
_TEXT ENDS

MSVC produces assembly listings in Intel-syntax. The difference between Intel-syntax and AT&T-syntax
will be discussed in 1.5.1 on page 11.
The compiler generated the file, 1.obj , which is to be linked into 1.exe . In our case, the file contains
two segments: CONST (for data constants) and _TEXT (for code).

The string hello, world in C/C++ has type const char[] [Bjarne Stroustrup, The C++ Programming
Language, 4th Edition, (2013)p176, 7.3.2], but it does not have its own name. The compiler needs to deal
with the string somehow so it defines the internal name $SG3830 for it.
That is why the example may be rewritten as follows:
#include <stdio.h>

const char $SG3830[]="hello, world\n";

int main()
{
printf($SG3830);
return 0;
}

Let’s go back to the assembly listing. As we can see, the string is terminated by a zero byte, which is
standard for C/C++ strings. More about C/C++ strings: 5.4.1 on page 705.
In the code segment, _TEXT , there is only one function so far: main() . The function main() starts with
16
prologue code and ends with epilogue code (like almost any function) .
After the function prologue we see the call to the printf() function:
CALL _printf . Before the call, a string address (or a pointer to it) containing our greeting is placed on
the stack with the help of the PUSH instruction.

When the printf() function returns the control to the main() function, the string address (or a pointer
to it) is still on the stack. Since we do not need it anymore, the stack pointer (the ESP register) needs to
be corrected.
ADD ESP, 4 means add 4 to the ESP register value.
16 You can read more about it in the section about function prologues and epilogues ( 1.6 on page 30).

9
1.5. HELLO, WORLD!
Why 4? Since this is a 32-bit program, we need exactly 4 bytes for address passing through the stack.
If it was x64 code we would need 8 bytes. ADD ESP, 4 is effectively equivalent to POP register but
without using any register17 .
For the same purpose, some compilers (like the Intel C++ Compiler) may emit POP ECX instead of ADD
(e.g., such a pattern can be observed in the Oracle RDBMS code as it is compiled with the Intel C++ com-
piler). This instruction has almost the same effect but the ECX register contents will be overwritten. The
Intel C++ compiler supposedly uses POP ECX since this instruction’s opcode is shorter than ADD ESP, x
(1 byte for POP against 3 for ADD ).

Here is an example of using POP instead of ADD from Oracle RDBMS:

Listing 1.14: Oracle RDBMS 10.2 Linux (app.o file)

.text:0800029A push ebx
.text:0800029B call qksfroChild
.text:080002A0 pop ecx

After calling printf() , the original C/C++ code contains the statement return 0 —return 0 as the
result of the main() function.

In the generated code this is implemented by the instruction XOR EAX, EAX .

XOR is in fact just “eXclusive OR”18 but the compilers often use it instead of MOV EAX, 0 —again because
it is a slightly shorter opcode (2 bytes for XOR against 5 for MOV ).

Some compilers emit SUB EAX, EAX , which means SUBtract the value in the EAX from the value in EAX .
That in any case will results in zero.
The last instruction RET returns the control to the caller. Usually, this is C/C++ CRT19 code which in turn
returns control to the OS.

GCC

Now let’s try to compile the same C/C++ code in the GCC 4.4.1 compiler in Linux: gcc 1.c -o 1 . Next,
with the assistance of the IDA disassembler, let’s see how the main() function was created. IDA, like
MSVC, uses Intel-syntax20 .

Listing 1.15: code in IDA

main proc near

var_10 = dword ptr -10h

push ebp
mov ebp, esp
and esp, 0FFFFFFF0h
sub esp, 10h
mov eax, offset aHelloWorld ; "hello, world\n"
mov [esp+10h+var_10], eax
call _printf
mov eax, 0
leave
retn
main endp

The result is almost the same. The address of the hello, world string (stored in the data segment) is
loaded in the EAX register first and then it is saved onto the stack.
In addition, the function prologue has AND ESP, 0FFFFFFF0h —this instruction aligns the ESP register
value on a 16-byte boundary. This results in all values in the stack being aligned the same way (The CPU
17 CPU flags, however, are modified
18 wikipedia
19 C runtime library
20 We could also have GCC produce assembly listings in Intel-syntax by applying the options -S -masm=intel .

10
1.5. HELLO, WORLD!
performs better if the values it is dealing with are located in memory at addresses aligned on a 4-byte or
16-byte boundary)21 .
SUB ESP, 10h allocates 16 bytes on the stack. Although, as we can see hereafter, only 4 are necessary
here.
This is because the size of the allocated stack is also aligned on a 16-byte boundary.
The string address (or a pointer to the string) is then stored directly onto the stack without using the PUSH
instruction. var_10 —is a local variable and is also an argument for printf() . Read about it below.

Then the printf() function is called.

Unlike MSVC, when GCC is compiling without optimization turned on, it emits MOV EAX, 0 instead of a
shorter opcode.
The last instruction, LEAVE —is the equivalent of the MOV ESP, EBP and POP EBP instruction pair —in
other words, this instruction sets the stack pointer ( ESP ) back and restores the EBP register to its initial
state. This is necessary since we modified these register values ( ESP and EBP ) at the beginning of the
function (by executing MOV EBP, ESP / AND ESP, … ).

GCC: AT&T syntax

Let’s see how this can be represented in assembly language AT&T syntax. This syntax is much more
popular in the UNIX-world.

Listing 1.16: let’s compile in GCC 4.7.3

gcc -S 1_1.c

We get this:

Listing 1.17: GCC 4.7.3

.file "1_1.c"
.section .rodata
.LC0:
.string "hello, world\n"
.text
.globl main
.type main, @function
main:
.LFB0:
.cfi_startproc
pushl %ebp
.cfi_def_cfa_offset 8
.cfi_offset 5, -8
movl %esp, %ebp
.cfi_def_cfa_register 5
andl $-16, %esp
subl $16, %esp
movl $.LC0, (%esp)
call printf
movl $0, %eax
leave
.cfi_restore 5
.cfi_def_cfa 4, 4
ret
.cfi_endproc
.LFE0:
.size main, .-main
.ident "GCC: (Ubuntu/Linaro 4.7.3-1ubuntu1) 4.7.3"
.section .note.GNU-stack,"",@progbits

The listing contains many macros (beginning with dot). These are not interesting for us at the moment.
21 Wikipedia: Data structure alignment

11
1.5. HELLO, WORLD!
For now, for the sake of simplification, we can ignore them (except the .string macro which encodes a
null-terminated character sequence just like a C-string). Then we’ll see this 22 :

Listing 1.18: GCC 4.7.3

.LC0:
.string "hello, world\n"
main:
pushl %ebp
movl %esp, %ebp
andl $-16, %esp
subl $16, %esp
movl $.LC0, (%esp)
call printf
movl $0, %eax
leave
ret

Some of the major differences between Intel and AT&T syntax are:
• Source and destination operands are written in opposite order.
In Intel-syntax: <instruction> <destination operand> <source operand>.
In AT&T syntax: <instruction> <source operand> <destination operand>.
Here is an easy way to memorize the difference: when you deal with Intel-syntax, you can imagine
that there is an equality sign (=) between operands and when you deal with AT&T-syntax imagine
there is a right arrow (→) 23 .
• AT&T: Before register names, a percent sign must be written (%) and before numbers a dollar sign
($). Parentheses are used instead of brackets.
• AT&T: A suffix is added to instructions to define the operand size:
– q — quad (64 bits)
– l — long (32 bits)
– w — word (16 bits)
– b — byte (8 bits)
Let’s go back to the compiled result: it is identical to what we saw in IDA. With one subtle difference:
0FFFFFFF0h is presented as $-16 . It is the same thing: 16 in the decimal system is 0x10 in hexadec-
imal. -0x10 is equal to 0xFFFFFFF0 (for a 32-bit data type).

One more thing: the return value is to be set to 0 by using the usual MOV , not XOR . MOV just loads a
value to a register. Its name is a misnomer (data is not moved but rather copied). In other architectures,
this instruction is named “LOAD” or “STORE” or something similar.

String patching (Win32)

We can easily find “hello, world” string in executable file using Hiew:
22 This GCC option can be used to eliminate “unnecessary” macros: -fno-asynchronous-unwind-tables
23 By the way, in some C standard functions (e.g., memcpy(), strcpy()) the arguments are listed in the same way as in Intel-syntax:
first the pointer to the destination memory block, and then the pointer to the source memory block.

12
1.5. HELLO, WORLD!

Figure 1.1: Hiew

And we can try to translate our message to Spanish language:

Figure 1.2: Hiew

Spanish text is one byte shorter than English, so we also add 0x0A byte at the end ( \n ) and zero byte.
It works.
What if we want to insert longer message? There are some zero bytes after original English text. Hard to
say if they can be overwritten: they may be used somewhere in CRT code, or maybe not. Anyway, you
can overwrite them only if you really know what you are doing.

String patching (Linux x64)

Let’s try to patch Linux x64 executable using rada.re:

Listing 1.19: rada.re session

dennis@bigbox ~/tmp % gcc hw.c

dennis@bigbox ~/tmp % radare2 a.out

-- SHALL WE PLAY A GAME?
[0x00400430]> / hello
Searching 5 bytes from 0x00400000 to 0x00601040: 68 65 6c 6c 6f
Searching 5 bytes in [0x400000-0x601040]
hits: 1
0x004005c4 hit0_0 .HHhello, world;0.

[0x00400430]> s 0x004005c4

[0x004005c4]> px
- offset - 0 1 2 3 4 5 6 7 8 9 A B C D E F 0123456789ABCDEF
0x004005c4 6865 6c6c 6f2c 2077 6f72 6c64 0000 0000 hello, world....
0x004005d4 011b 033b 3000 0000 0500 0000 1cfe ffff ...;0...........
0x004005e4 7c00 0000 5cfe ffff 4c00 0000 52ff ffff |...\...L...R...
0x004005f4 a400 0000 6cff ffff c400 0000 dcff ffff ....l...........
0x00400604 0c01 0000 1400 0000 0000 0000 017a 5200 .............zR.

13
1.5. HELLO, WORLD!
0x00400614 0178 1001 1b0c 0708 9001 0710 1400 0000 .x..............
0x00400624 1c00 0000 08fe ffff 2a00 0000 0000 0000 ........*.......
0x00400634 0000 0000 1400 0000 0000 0000 017a 5200 .............zR.
0x00400644 0178 1001 1b0c 0708 9001 0000 2400 0000 .x..........$...
0x00400654 1c00 0000 98fd ffff 3000 0000 000e 1046 ........0......F
0x00400664 0e18 4a0f 0b77 0880 003f 1a3b 2a33 2422 ..J..w...?.;*3$"
0x00400674 0000 0000 1c00 0000 4400 0000 a6fe ffff ........D.......
0x00400684 1500 0000 0041 0e10 8602 430d 0650 0c07 .....A....C..P..
0x00400694 0800 0000 4400 0000 6400 0000 a0fe ffff ....D...d.......
0x004006a4 6500 0000 0042 0e10 8f02 420e 188e 0345 e....B....B....E
0x004006b4 0e20 8d04 420e 288c 0548 0e30 8606 480e . ..B.(..H.0..H.

[0x004005c4]> oo+
File a.out reopened in read-write mode

[0x004005c4]> w hola, mundo\x00

[0x004005c4]> q

dennis@bigbox ~/tmp % ./a.out

hola, mundo

What I do here: I search for “hello” string using / command, then I set cursor (or seek in rada.re terms) to
that address. Then I want to be sure that this is really that place: px dumps bytes there. oo+ switches
rada.re to read-write mode. w writes ASCII string at the current seek. Note \00 at the end—this is zero
byte. q quits.

Software localization of MS-DOS era

The way I described was a common way to translate MS-DOS software to Russian language back to 1980’s
and 1990’s. Russian words and sentences are usually slightly longer than its English counterparts, so that
is why localized software has a lot of weird acronyms and hardly readable abbreviations.
Perhaps, this also happened to other languages during that era, in other countries.

1.5.2 x86-64

MSVC: x86-64

Let’s also try 64-bit MSVC:

Listing 1.20: MSVC 2012 x64

$SG2989 DB 'hello, world', 0AH, 00H

main PROC
sub rsp, 40
lea rcx, OFFSET FLAT:$SG2989
call printf
xor eax, eax
add rsp, 40
ret 0
main ENDP

In x86-64, all registers were extended to 64-bit and now their names have an R- prefix. In order to use
the stack less often (in other words, to access external memory/cache less often), there exists a popular
way to pass function arguments via registers (fastcall) 6.1.3 on page 735. I.e., a part of the function
arguments is passed in registers, the rest—via the stack. In Win64, 4 function arguments are passed in
the RCX , RDX , R8 , R9 registers. That is what we see here: a pointer to the string for printf() is
now passed not in the stack, but in the RCX register. The pointers are 64-bit now, so they are passed in
the 64-bit registers (which have the R- prefix). However, for backward compatibility, it is still possible
to access the 32-bit parts, using the E- prefix. This is how the RAX / EAX / AX / AL register looks like in
x86-64:

14
1.5. HELLO, WORLD!
Byte number:
7th 6th 5th 4th 3rd 2nd 1st 0th
RAXx64
EAX
AX
AH AL

The main() function returns an int-typed value, which is, in C/C++, for better backward compatibility
and portability, still 32-bit, so that is why the EAX register is cleared at the function end (i.e., the 32-bit
part of the register) instead of RAX . There are also 40 bytes allocated in the local stack. This is called
the “shadow space”, about which we are going to talk later: 1.10.2 on page 101.

GCC: x86-64

Let’s also try GCC in 64-bit Linux:

Listing 1.21: GCC 4.4.6 x64
.string "hello, world\n"
main:
sub rsp, 8
mov edi, OFFSET FLAT:.LC0 ; "hello, world\n"
xor eax, eax ; number of vector registers passed
call printf
xor eax, eax
add rsp, 8
ret

A method to pass function arguments in registers is also used in Linux, *BSD and Mac OS X is [Michael Matz,
Jan Hubicka, Andreas Jaeger, Mark Mitchell, System V Application Binary Interface. AMD64 Architecture
Processor Supplement, (2013)] 24 .
The first 6 arguments are passed in the RDI , RSI , RDX , RCX , R8 , R9 registers, and the rest—via the
stack.
So the pointer to the string is passed in EDI (the 32-bit part of the register). But why not use the 64-bit
part, RDI ?

It is important to keep in mind that all MOV instructions in 64-bit mode that write something into the lower
32-bit register part also clear the higher 32-bits (as stated in Intel manuals: 11.1.4 on page 1003).
I.e., the MOV EAX, 011223344h writes a value into RAX correctly, since the higher bits will be cleared.
25
If we open the compiled object file (.o), we can also see all the instructions’ opcodes :
Listing 1.22: GCC 4.4.6 x64
.text:00000000004004D0 main proc near
.text:00000000004004D0 48 83 EC 08 sub rsp, 8
.text:00000000004004D4 BF E8 05 40 00 mov edi, offset format ; "hello, world\n"
.text:00000000004004D9 31 C0 xor eax, eax
.text:00000000004004DB E8 D8 FE FF FF call _printf
.text:00000000004004E0 31 C0 xor eax, eax
.text:00000000004004E2 48 83 C4 08 add rsp, 8
.text:00000000004004E6 C3 retn
.text:00000000004004E6 main endp

As we can see, the instruction that writes into EDI at 0x4004D4 occupies 5 bytes. The same instruction
writing a 64-bit value into RDI occupies 7 bytes. Apparently, GCC is trying to save some space. Besides,
it can be sure that the data segment containing the string will not be allocated at the addresses higher
than 4GiB.
We also see that the EAX register has been cleared before the printf() function call. This is done
because according to ABI!26 standard mentioned above, the number of used vector registers is passed
in EAX in *NIX systems on x86-64.
24 Also available as https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf
25 Thismust be enabled in Options → Disassembly → Number of opcode bytes
26 ABI!

15
1.5. HELLO, WORLD!
Address patching (Win64)

If our example compiled in MSVC2013 using \MD switch (meaning smaller executable due to MSVCR*.DLL
file linkage), the main() function came first and can be easily found:

Figure 1.3: Hiew

As an experiment, we can increment address by 1:

16
1.5. HELLO, WORLD!

Figure 1.4: Hiew

Hiew shows “ello, world” string. And when we run patched executable, this very string is printed.

Pick another string from binary image (Linux x64)

The binary file I’ve got when I compile our example using GCC 5.4.0 on Linux x64 box has many other text
strings: they are mostly imported function names and library names.
I run objdump to get contents of all sections of the compiled file:
$ objdump -s a.out

a.out: file format elf64-x86-64

Contents of section .interp:

400238 2f6c6962 36342f6c 642d6c69 6e75782d /lib64/ld-linux-
400248 7838362d 36342e73 6f2e3200 x86-64.so.2.
Contents of section .note.ABI-tag:
400254 04000000 10000000 01000000 474e5500 ............GNU.
400264 00000000 02000000 06000000 20000000 ............ ...
Contents of section .note.gnu.build-id:
400274 04000000 14000000 03000000 474e5500 ............GNU.
400284 fe461178 5bb710b4 bbf2aca8 5ec1ec10 .F.x[.......^...
400294 cf3f7ae4 .?z.

...

It’s not a problem to pass address of the text string “/lib64/ld-linux-x86-64.so.2” to printf() call:
#include <stdio.h>

17
1.5. HELLO, WORLD!
int main()
{
printf(0x400238);
return 0;
}

Hard to believe, this code prints mentioned string.

Change the address to 0x400260 , and the “GNU” string will be printed. The address is true for my specific
GCC version, GNU toolset, etc. On your system, executable may be slightly different, and all addresses
will also be different. Also, adding/removing code to/from this source code will probably shift all addresses
back or forth.

1.5.3 GCC—one more thing

The fact that an anonymous C-string has const type ( 1.5.1 on page 9), and that C-strings allocated in
constants segment are guaranteed to be immutable, has an interesting consequence: the compiler may
use a specific part of the string.
Let’s try this example:
#include <stdio.h>

int f1()
{
printf ("world\n");
}

int f2()
{
printf ("hello world\n");
}

int main()
{
f1();
f2();
}

Common C/C++-compilers (including MSVC) allocate two strings, but let’s see what GCC 4.8.1 does:

Listing 1.23: GCC 4.8.1 + IDA listing

f1 proc near

s = dword ptr -1Ch

sub esp, 1Ch

mov [esp+1Ch+s], offset s ; "world\n"
call _puts
add esp, 1Ch
retn
f1 endp

f2 proc near

s = dword ptr -1Ch

sub esp, 1Ch

mov [esp+1Ch+s], offset aHello ; "hello "
call _puts
add esp, 1Ch
retn
f2 endp

aHello db 'hello '

s db 'world',0xa,0

18
1.5. HELLO, WORLD!
Indeed: when we print the “hello world” string these two words are positioned in memory adjacently and
puts() called from f2() function is not aware that this string is divided. In fact, it’s not divided; it’s
divided only “virtually”, in this listing.
When puts() is called from f1() , it uses the “world” string plus a zero byte. puts() is not aware that
there is something before this string!
This clever trick is often used by at least GCC and can save some memory. This is close to string interning.
Another related example is here: 3.2 on page 468.

1.5.4 ARM

For my experiments with ARM processors, several compilers were used:

• Popular in the embedded area: Keil Release 6/2013.
27
• Apple Xcode 4.6.3 IDE with the LLVM-GCC 4.2 compiler .
• GCC 4.9 (Linaro) (for ARM64), available as win32-executables at http://go.yurichev.com/17325.
32-bit ARM code is used (including Thumb and Thumb-2 modes) in all cases in this book, if not mentioned
otherwise. When we talk about 64-bit ARM here, we call it ARM64.

Non-optimizing Keil 6/2013 (ARM mode)

Let’s start by compiling our example in Keil:

armcc.exe --arm --c90 -O0 1.c

The armcc compiler produces assembly listings in Intel-syntax, but it has high-level ARM-processor related
macros 28 , but it is more important for us to see the instructions “as is” so let’s see the compiled result in
IDA.

Listing 1.24: Non-optimizing Keil 6/2013 (ARM mode) IDA

.text:00000000 main
.text:00000000 10 40 2D E9 STMFD SP!, {R4,LR}
.text:00000004 1E 0E 8F E2 ADR R0, aHelloWorld ; "hello, world"
.text:00000008 15 19 00 EB BL __2printf
.text:0000000C 00 00 A0 E3 MOV R0, #0
.text:00000010 10 80 BD E8 LDMFD SP!, {R4,PC}

.text:000001EC 68 65 6C 6C+aHelloWorld DCB "hello, world",0 ; DATA XREF: main+4

In the example, we can easily see each instruction has a size of 4 bytes. Indeed, we compiled our code
for ARM mode, not for Thumb.
29
The very first instruction, STMFD SP!, {R4,LR} , works as an x86 PUSH instruction, writing the values
of two registers ( R4 and LR) into the stack.
Indeed, in the output listing from the armcc compiler, for the sake of simplification, actually shows the
PUSH {r4,lr} instruction. But that is not quite precise. The PUSH instruction is only available in Thumb
mode. So, to make things less confusing, we’re doing this in IDA.
This instruction first decrements the SP31 so it points to the place in the stack that is free for new entries,
then it saves the values of the R4 and LR registers at the address stored in the modified SP.

This instruction (like the PUSH instruction in Thumb mode) is able to save several register values at once
which can be very useful. By the way, this has no equivalent in x86. It can also be noted that the STMFD
instruction is a generalization of the PUSH instruction (extending its features), since it can work with any
register, not just with SP. In other words, STMFD may be used for storing a set of registers at the specified
memory address.
27 Itis indeed so: Apple Xcode 4.6.3 uses open-source GCC as front-end compiler and LLVM code generator
28 e.g. ARM mode lacks PUSH / POP instructions
29 STMFD30
31 stack pointer. SP/ESP/RSP in x86/x64. SP in ARM.

19
1.5. HELLO, WORLD!
The ADR R0, aHelloWorld instruction adds or subtracts the value in the PC32 register to the offset where
the hello, world string is located. How is the PC register used here, one might ask? This is called
“position-independent code”33 .
Such code can be executed at a non-fixed address in memory. In other words, this is PC-relative addressing.
The ADR instruction takes into account the difference between the address of this instruction and the
address where the string is located. This difference (offset) is always to be the same, no matter at what
address our code is loaded by the OS. That’s why all we need is to add the address of the current instruction
(from PC) in order to get the absolute memory address of our C-string.
34
BL __2printf instruction calls the printf() function. Here’s how this instruction works:

• store the address following the BL instruction ( 0xC ) into the LR;

• then pass the control to printf() by writing its address into the PC register.

When printf() finishes its execution it must have information about where it needs to return the control
to. That’s why each function passes control to the address stored in the LR register.
That is a difference between “pure” RISC-processors like ARM and CISC35 -processors like x86, where the
return address is usually stored on the stack. Read more about this in next section ( 1.7 on page 30).
By the way, an absolute 32-bit address or offset cannot be encoded in the 32-bit BL instruction because
it only has space for 24 bits. As we may recall, all ARM-mode instructions have a size of 4 bytes (32 bits).
Hence, they can only be located on 4-byte boundary addresses. This implies that the last 2 bits of the
instruction address (which are always zero bits) may be omitted. In summary, we have 26 bits for offset
encoding. This is enough to encode current_P C ± ≈ 32M .
36
Next, the MOV R0, #0 instruction just writes 0 into the R0 register. That’s because our C-function
returns 0 and the return value is to be placed in the R0 register.
37
The last instruction LDMFD SP!, R4,PC . It loads values from the stack (or any other memory place) in
order to save them into R4 and PC, and increments the stack pointer SP. It works like POP here.
N.B. The very first instruction STMFD saved the R4 and LR registers pair on the stack, but R4 and PC
are restored during the LDMFD execution.
As we already know, the address of the place where each function must return control to is usually saved
in the LR register. The very first instruction saves its value in the stack because the same register will be
used by our main() function when calling printf() . In the function’s end, this value can be written
directly to the PC register, thus passing control to where our function has been called.
Since main() is usually the primary function in C/C++, the control will be returned to the OS loader or
to a point in a CRT, or something like that.
All that allows omitting the BX LR instruction at the end of the function.

DCB is an assembly language directive defining an array of bytes or ASCII strings, akin to the DB directive
in the x86-assembly language.

Non-optimizing Keil 6/2013 (Thumb mode)

Let’s compile the same example using Keil in Thumb mode:

armcc.exe --thumb --c90 -O0 1.c

We are getting (in IDA):

Listing 1.25: Non-optimizing Keil 6/2013 (Thumb mode) + IDA

.text:00000000 main
.text:00000000 10 B5 PUSH {R4,LR}
32 Program Counter. IP/EIP/RIP in x86/64. PC in ARM.
33 Read more about it in relevant section ( 6.4.1 on page 748)
34 Branch with Link
35 Complex instruction set computing
36 Meaning MOVe
37 LDMFD38 is an inverse instruction of STMFD

20
1.5. HELLO, WORLD!
.text:00000002 C0 A0 ADR R0, aHelloWorld ; "hello, world"
.text:00000004 06 F0 2E F9 BL __2printf
.text:00000008 00 20 MOVS R0, #0
.text:0000000A 10 BD POP {R4,PC}

.text:00000304 68 65 6C 6C+aHelloWorld DCB "hello, world",0 ; DATA XREF: main+2

We can easily spot the 2-byte (16-bit) opcodes. This is, as was already noted, Thumb. The BL instruction,
however, consists of two 16-bit instructions. This is because it is impossible to load an offset for the
printf() function while using the small space in one 16-bit opcode. Therefore, the first 16-bit instruction
loads the higher 10 bits of the offset and the second instruction loads the lower 11 bits of the offset.
As was noted, all instructions in Thumb mode have a size of 2 bytes (or 16 bits). This implies it is impossible
for a Thumb-instruction to be at an odd address whatsoever. Given the above, the last address bit may
be omitted while encoding instructions.
In summary, the BL Thumb-instruction can encode an address in current_P C ± ≈ 2M .

As for the other instructions in the function: PUSH and POP work here just like the described STMFD / LDMFD
only the SP register is not mentioned explicitly here. ADR works just like in the previous example. MOVS
writes 0 into the R0 register in order to return zero.

Optimizing Xcode 4.6.3 (LLVM) (ARM mode)

Xcode 4.6.3 without optimization turned on produces a lot of redundant code so we’ll study optimized
output, where the instruction count is as small as possible, setting the compiler switch -O3 .

Listing 1.26: Optimizing Xcode 4.6.3 (LLVM) (ARM mode)

__text:000028C4 _hello_world
__text:000028C4 80 40 2D E9 STMFD SP!, {R7,LR}
__text:000028C8 86 06 01 E3 MOV R0, #0x1686
__text:000028CC 0D 70 A0 E1 MOV R7, SP
__text:000028D0 00 00 40 E3 MOVT R0, #0
__text:000028D4 00 00 8F E0 ADD R0, PC, R0
__text:000028D8 C3 05 00 EB BL _puts
__text:000028DC 00 00 A0 E3 MOV R0, #0
__text:000028E0 80 80 BD E8 LDMFD SP!, {R7,PC}

__cstring:00003F62 48 65 6C 6C+aHelloWorld_0 DCB "Hello world!",0

The instructions STMFD and LDMFD are already familiar to us.

The MOV instruction just writes the number 0x1686 into the R0 register. This is the offset pointing to
the “Hello world!” string.
The R7 register (as it is standardized in [iOS ABI Function Call Guide, (2010)]39 ) is a frame pointer. More
on that below.
The MOVT R0, #0 (MOVe Top) instruction writes 0 into higher 16 bits of the register. The issue here is
that the generic MOV instruction in ARM mode may write only the lower 16 bits of the register.
Keep in mind, all instruction opcodes in ARM mode are limited in size to 32 bits. Of course, this limitation
is not related to moving data between registers. That’s why an additional instruction MOVT exists for
writing into the higher bits (from 16 to 31 inclusive). Its usage here, however, is redundant because
the MOV R0, #0x1686 instruction above cleared the higher part of the register. This is supposedly a
shortcoming of the compiler.
The ADD R0, PC, R0 instruction adds the value in the PC to the value in the R0 , to calculate the abso-
lute address of the “Hello world!” string. As we already know, it is “position-independent code” so this
correction is essential here.
The BL instruction calls the puts() function instead of printf() .
39 Also available as http://go.yurichev.com/17276

21
1.5. HELLO, WORLD!
GCC replaced the first printf() call with puts() . Indeed: printf() with a sole argument is almost
analogous to puts() .
Almost, because the two functions are producing the same result only in case the string does not contain
printf format identifiers starting with %. In case it does, the effect of these two functions would be different
40
.
41
Why did the compiler replace the printf() with puts() ? Presumably because puts() is faster .
Because it just passes characters to stdout without comparing every one of them with the % symbol.
Next, we see the familiar MOV R0, #0 instruction intended to set the R0 register to 0.

Optimizing Xcode 4.6.3 (LLVM) (Thumb-2 mode)

By default Xcode 4.6.3 generates code for Thumb-2 in this manner:

Listing 1.27: Optimizing Xcode 4.6.3 (LLVM) (Thumb-2 mode)

__text:00002B6C _hello_world
__text:00002B6C 80 B5 PUSH {R7,LR}
__text:00002B6E 41 F2 D8 30 MOVW R0, #0x13D8
__text:00002B72 6F 46 MOV R7, SP
__text:00002B74 C0 F2 00 00 MOVT.W R0, #0
__text:00002B78 78 44 ADD R0, PC
__text:00002B7A 01 F0 38 EA BLX _puts
__text:00002B7E 00 20 MOVS R0, #0
__text:00002B80 80 BD POP {R7,PC}

...

__cstring:00003E70 48 65 6C 6C 6F 20+aHelloWorld DCB "Hello world!",0xA,0

The BL and BLX instructions in Thumb mode, as we recall, are encoded as a pair of 16-bit instructions. In
Thumb-2 these surrogate opcodes are extended in such a way so that new instructions may be encoded
here as 32-bit instructions.
That is obvious considering that the opcodes of the Thumb-2 instructions always begin with 0xFx or
0xEx .
But in the IDA listing the opcode bytes are swapped because for ARM processor the instructions are
encoded as follows: last byte comes first and after that comes the first one (for Thumb and Thumb-2
modes) or for instructions in ARM mode the fourth byte comes first, then the third, then the second and
finally the first (due to different endianness).
So that is how bytes are located in IDA listings:
• for ARM and ARM64 modes: 4-3-2-1;
• for Thumb mode: 2-1;
• for 16-bit instructions pair in Thumb-2 mode: 2-1-4-3.
So as we can see, the MOVW , MOVT.W and BLX instructions begin with 0xFx .

One of the Thumb-2 instructions is MOVW R0, #0x13D8 —it stores a 16-bit value into the lower part of
the R0 register, clearing the higher bits.

Also, MOVT.W R0, #0 works just like MOVT from the previous example only it works in Thumb-2.

Among the other differences, the BLX instruction is used in this case instead of the BL .

The difference is that, besides saving the RA42 in the LR register and passing control to the puts()
function, the processor is also switching from Thumb/Thumb-2 mode to ARM mode (or back).
This instruction is placed here since the instruction to which control is passed looks like (it is encoded in
ARM mode):
40 It has also to be noted the puts() does not require a ‘\n’ new line symbol at the end of a string, so we do not see it here.
41 ciselant.de/projects/gcc_printf/gcc_printf.html
42 Return Address

22
1.5. HELLO, WORLD!

__symbolstub1:00003FEC _puts ; CODE XREF: _hello_world+E

__symbolstub1:00003FEC 44 F0 9F E5 LDR PC, =__imp__puts

This is essentially a jump to the place where the address of puts() is written in the imports’ section.

So, the observant reader may ask: why not call puts() right at the point in the code where it is needed?
Because it is not very space-efficient.
Almost any program uses external dynamic libraries (like DLL in Windows, .so in *NIX or .dylib in Mac
OS X). The dynamic libraries contain frequently used library functions, including the standard C-function
puts() .
In an executable binary file (Windows PE .exe, ELF or Mach-O) an import section is present. This is a list
of symbols (functions or global variables) imported from external modules along with the names of the
modules themselves.
The OS loader loads all modules it needs and, while enumerating import symbols in the primary module,
determines the correct addresses of each symbol.
In our case, __imp__puts is a 32-bit variable used by the OS loader to store the correct address of the
function in an external library. Then the LDR instruction just reads the 32-bit value from this variable and
writes it into the PC register, passing control to it.
So, in order to reduce the time the OS loader needs for completing this procedure, it is good idea to write
the address of each symbol only once, to a dedicated place.
Besides, as we have already figured out, it is impossible to load a 32-bit value into a register while using
only one instruction without a memory access.
Therefore, the optimal solution is to allocate a separate function working in ARM mode with the sole goal of
passing control to the dynamic library and then to jump to this short one-instruction function (the so-called
thunk function) from the Thumb-code.
By the way, in the previous example (compiled for ARM mode) the control is passed by the BL to the
same thunk function. The processor mode, however, is not being switched (hence the absence of an “X”
in the instruction mnemonic).

More about thunk-functions

Thunk-functions are hard to understand, apparently, because of a misnomer. The simplest way to under-
stand it as adaptors or convertors of one type of jack to another. For example, an adaptor allowing the
insertion of a British power plug into an American wall socket, or vice-versa. Thunk functions are also
sometimes called wrappers.
Here are a couple more descriptions of these functions:

“A piece of coding which provides an address:”, according to P. Z. Ingerman, who invented

thunks in 1961 as a way of binding actual parameters to their formal definitions in Algol-60
procedure calls. If a procedure is called with an expression in the place of a formal parameter,
the compiler generates a thunk which computes the expression and leaves the address of
the result in some standard location.
…
Microsoft and IBM have both defined, in their Intel-based systems, a “16-bit environment”
(with bletcherous segment registers and 64K address limits) and a “32-bit environment”
(with flat addressing and semi-real memory management). The two environments can both
be running on the same computer and OS (thanks to what is called, in the Microsoft world,
WOW which stands for Windows On Windows). MS and IBM have both decided that the
process of getting from 16- to 32-bit and vice versa is called a “thunk”; for Windows 95,
there is even a tool, THUNK.EXE, called a “thunk compiler”.

( The Jargon File )

Another example we can find in LAPACK library—a “Linear Algebra PACKage” written in FORTRAN. C/C++
developers also want to use LAPACK, but it’s insane to rewrite it to C/C++ and then maintain several

23
 1.5. HELLO, WORLD!
versions. So there are short C functions callable from C/C++ environment, which are, in turn, call FORTRAN
functions, and do almost anything else:
double Blas_Dot_Prod(const LaVectorDouble &dx, const LaVectorDouble &dy)
{
assert(dx.size()==dy.size());
integer n = dx.size();
integer incx = dx.inc(), incy = dy.inc();

return F77NAME(ddot)(&n, &dx(0), &incx, &dy(0), &incy);

}

Also, functions like that are called “wrappers”.

ARM64

GCC

Let’s compile the example using GCC 4.8.1 in ARM64:

Listing 1.28: Non-optimizing GCC 4.8.1 + objdump

1 0000000000400590 <main>:
2 400590: a9bf7bfd stp x29, x30, [sp,#-16]!
3 400594: 910003fd mov x29, sp
4 400598: 90000000 adrp x0, 400000 <_init-0x3b8>
5 40059c: 91192000 add x0, x0, #0x648
6 4005a0: 97ffffa0 bl 400420 <puts@plt>
7 4005a4: 52800000 mov w0, #0x0 // #0
8 4005a8: a8c17bfd ldp x29, x30, [sp],#16
9 4005ac: d65f03c0 ret
10
11 ...
12
13 Contents of section .rodata:
14 400640 01000200 00000000 48656c6c 6f210a00 ........Hello!..

There are no Thumb and Thumb-2 modes in ARM64, only ARM, so there are 32-bit instructions only. The
Register count is doubled: .2.4 on page 1031. 64-bit registers have X- prefixes, while its 32-bit parts—
W- .

The STP instruction (Store Pair) saves two registers in the stack simultaneously: X29 and X30 .
Of course, this instruction is able to save this pair at an arbitrary place in memory, but the SP register is
specified here, so the pair is saved in the stack.
ARM64 registers are 64-bit ones, each has a size of 8 bytes, so one needs 16 bytes for saving two registers.
The exclamation mark (“!”) after the operand means that 16 is to be subtracted from SP first, and only
then are values from register pair to be written into the stack. This is also called pre-index. About the
difference between post-index and pre-index read here: 1.32.2 on page 441.
Hence, in terms of the more familiar x86, the first instruction is just an analogue to a pair of PUSH X29
and PUSH X30 . X29 is used as FP43 in ARM64, and X30 as LR, so that’s why they are saved in the
function prologue and restored in the function epilogue.
The second instruction copies SP in X29 (or FP). This is made so to set up the function stack frame.

ADRP and ADD instructions are used to fill the address of the string “Hello!” into the X0 register, because
the first function argument is passed in this register. There are no instructions, whatsoever, in ARM that
can store a large number into a register (because the instruction length is limited to 4 bytes, read more
about it here: 1.32.3 on page 442). So several instructions must be utilized. The first instruction ( ADRP )
writes the address of the 4KiB page, where the string is located, into X0 , and the second one ( ADD ) just
adds the remainder to the address. More about that in: 1.32.4 on page 444.
43 Frame Pointer

24
1.5. HELLO, WORLD!
0x400000 + 0x648 = 0x400648 , and we see our “Hello!” C-string in the .rodata data segment at this
address.
puts() is called afterwards using the BL instruction. This was already discussed: 1.5.4 on page 21.

MOV writes 0 into W0 . W0 is the lower 32 bits of the 64-bit X0 register:

High 32-bit part low 32-bit part
X0
W0

The function result is returned via X0 and main() returns 0, so that’s how the return result is prepared.
But why use the 32-bit part?
Because the int data type in ARM64, just like in x86-64, is still 32-bit, for better compatibility.
So if a function returns a 32-bit int, only the lower 32 bits of X0 register have to be filled.

In order to verify this, let’s change this example slightly and recompile it. Now main() returns a 64-bit
value:

Listing 1.29: main() returning a value of uint64_t type

#include <stdio.h>
#include <stdint.h>

uint64_t main()
{
printf ("Hello!\n");
return 0;
}

The result is the same, but that’s how MOV at that line looks like now:

Listing 1.30: Non-optimizing GCC 4.8.1 + objdump

4005a4: d2800000 mov x0, #0x0 // #0

LDP (Load Pair) then restores the X29 and X30 registers.
There is no exclamation mark after the instruction: this implies that the values are first loaded from the
stack, and only then is SP increased by 16. This is called post-index.
A new instruction appeared in ARM64: RET . It works just as BX LR , only a special hint bit is added,
informing the CPU that this is a return from a function, not just another jump instruction, so it can execute
it more optimally.
Due to the simplicity of the function, optimizing GCC generates the very same code.

1.5.5 MIPS

A word about the “global pointer”

One important MIPS concept is the “global pointer”. As we may already know, each MIPS instruction has
a size of 32 bits, so it’s impossible to embed a 32-bit address into one instruction: a pair has to be used
for this (like GCC did in our example for the text string address loading). It’s possible, however, to load
data from the address in the range of register − 32768...register + 32767 using one single instruction (because
16 bits of signed offset could be encoded in a single instruction). So we can allocate some register for
this purpose and also allocate a 64KiB area of most used data. This allocated register is called a “global
pointer” and it points to the middle of the 64KiB area. This area usually contains global variables and
addresses of imported functions like printf() , because the GCC developers decided that getting the
address of some function must be as fast as a single instruction execution instead of two. In an ELF file
this 64KiB area is located partly in sections .sbss (“small BSS44 ”) for uninitialized data and .sdata (“small
data”) for initialized data. This implies that the programmer may choose what data he/she wants to
be accessed fast and place it into .sdata/.sbss. Some old-school programmers may recall the MS-DOS
44 Block Started by Symbol

25
 1.5. HELLO, WORLD!
memory model 10.6 on page 993 or the MS-DOS memory managers like XMS/EMS where all memory was
divided in 64KiB blocks.
This concept is not unique to MIPS. At least PowerPC uses this technique as well.

Optimizing GCC

Lets consider the following example, which illustrates the “global pointer” concept.

Listing 1.31: Optimizing GCC 4.4.5 (assembly output)

1 $LC0:
2 ; \000 is zero byte in octal base:
3 .ascii "Hello, world!\012\000"
4 main:
5 ; function prologue.
6 ; set the GP:
7 lui $28,%hi(__gnu_local_gp)
8 addiu $sp,$sp,-32
9 addiu $28,$28,%lo(__gnu_local_gp)
10 ; save the RA to the local stack:
11 sw $31,28($sp)
12 ; load the address of the puts() function from the GP to $25:
13 lw $25,%call16(puts)($28)
14 ; load the address of the text string to $4 ($a0):
15 lui $4,%hi($LC0)
16 ; jump to puts(), saving the return address in the link register:
17 jalr $25
18 addiu $4,$4,%lo($LC0) ; branch delay slot
19 ; restore the RA:
20 lw $31,28($sp)
21 ; copy 0 from $zero to $v0:
22 move $2,$0
23 ; return by jumping to the RA:
24 j $31
25 ; function epilogue:
26 addiu $sp,$sp,32 ; branch delay slot + free local stack

As we see, the $GP register is set in the function prologue to point to the middle of this area. The RA
register is also saved in the local stack. puts() is also used here instead of printf() . The address of
the puts() function is loaded into $25 using LW the instruction (“Load Word”). Then the address of the
text string is loaded to $4 using LUI (“Load Upper Immediate”) and ADDIU (“Add Immediate Unsigned
Word”) instruction pair. LUI sets the high 16 bits of the register (hence “upper” word in instruction name)
and ADDIU adds the lower 16 bits of the address.

ADDIU follows JALR (haven’t you forgot branch delay slots yet?). The register $4 is also called $A0,
which is used for passing the first function argument 45 .
JALR (“Jump and Link Register”) jumps to the address stored in the $25 register (address of puts() )
while saving the address of the next instruction (LW) in RA. This is very similar to ARM. Oh, and one
important thing is that the address saved in RA is not the address of the next instruction (because it’s in
a delay slot and is executed before the jump instruction), but the address of the instruction after the next
one (after the delay slot). Hence, P C + 8 is written to RA during the execution of JALR , in our case, this
is the address of the LW instruction next to ADDIU .

LW (“Load Word”) at line 20 restores RA from the local stack (this instruction is actually part of the function
epilogue).
MOVE at line 22 copies the value from the $0 ($ZERO) register to $2 ($V0).
MIPS has a constant register, which always holds zero. Apparently, the MIPS developers came up with
the idea that zero is in fact the busiest constant in the computer programming, so let’s just use the $0
register every time zero is needed.
45 The MIPS registers table is available in appendix .3.1 on page 1032

26
 1.5. HELLO, WORLD!
Another interesting fact is that MIPS lacks an instruction that transfers data between registers. In fact,
MOVE DST, SRC is ADD DST, SRC, $ZERO (DST = SRC + 0), which does the same. Apparently, the MIPS
developers wanted to have a compact opcode table. This does not mean an actual addition happens at
each MOVE instruction. Most likely, the CPU optimizes these pseudo instructions and the ALU46 is never
used.
J at line 24 jumps to the address in RA, which is effectively performing a return from the function. ADDIU
after J is in fact executed before J (remember branch delay slots?) and is part of the function epilogue.
Here is also a listing generated by IDA. Each register here has its own pseudo name:

Listing 1.32: Optimizing GCC 4.4.5 (IDA)

1 .text:00000000 main:
2 .text:00000000
3 .text:00000000 var_10 = -0x10
4 .text:00000000 var_4 = -4
5 .text:00000000
6 ; function prologue.
7 ; set the GP:
8 .text:00000000 lui $gp, (__gnu_local_gp >> 16)
9 .text:00000004 addiu $sp, -0x20
10 .text:00000008 la $gp, (__gnu_local_gp & 0xFFFF)
11 ; save the RA to the local stack:
12 .text:0000000C sw $ra, 0x20+var_4($sp)
13 ; save the GP to the local stack:
14 ; for some reason, this instruction is missing in the GCC assembly output:
15 .text:00000010 sw $gp, 0x20+var_10($sp)
16 ; load the address of the puts() function from the GP to $t9:
17 .text:00000014 lw $t9, (puts & 0xFFFF)($gp)
18 ; form the address of the text string in $a0:
19 .text:00000018 lui $a0, ($LC0 >> 16) # "Hello, world!"
20 ; jump to puts(), saving the return address in the link register:
21 .text:0000001C jalr $t9
22 .text:00000020 la $a0, ($LC0 & 0xFFFF) # "Hello, world!"
23 ; restore the RA:
24 .text:00000024 lw $ra, 0x20+var_4($sp)
25 ; copy 0 from $zero to $v0:
26 .text:00000028 move $v0, $zero
27 ; return by jumping to the RA:
28 .text:0000002C jr $ra
29 ; function epilogue:
30 .text:00000030 addiu $sp, 0x20

The instruction at line 15 saves the GP value into the local stack, and this instruction is missing mysteri-
ously from the GCC output listing, maybe by a GCC error 47 . The GP value has to be saved indeed, because
each function can use its own 64KiB data window. The register containing the puts() address is called
$T9, because registers prefixed with T- are called “temporaries” and their contents may not be preserved.

Non-optimizing GCC

Non-optimizing GCC is more verbose.

Listing 1.33: Non-optimizing GCC 4.4.5 (assembly output)

1 $LC0:
2 .ascii "Hello, world!\012\000"
3 main:
4 ; function prologue.
5 ; save the RA ($31) and FP in the stack:
6 addiu $sp,$sp,-32
7 sw $31,28($sp)
8 sw $fp,24($sp)
9 ; set the FP (stack frame pointer):
10 move $fp,$sp
11 ; set the GP:
46 Arithmetic logic unit
47 Apparently, functions generating listings are not so critical to GCC users, so some unfixed errors may still exist.

27
 1.5. HELLO, WORLD!
12 lui $28,%hi(__gnu_local_gp)
13 addiu $28,$28,%lo(__gnu_local_gp)
14 ; load the address of the text string:
15 lui $2,%hi($LC0)
16 addiu $4,$2,%lo($LC0)
17 ; load the address of puts() using the GP:
18 lw $2,%call16(puts)($28)
19 nop
20 ; call puts():
21 move $25,$2
22 jalr $25
23 nop ; branch delay slot
24
25 ; restore the GP from the local stack:
26 lw $28,16($fp)
27 ; set register $2 ($V0) to zero:
28 move $2,$0
29 ; function epilogue.
30 ; restore the SP:
31 move $sp,$fp
32 ; restore the RA:
33 lw $31,28($sp)
34 ; restore the FP:
35 lw $fp,24($sp)
36 addiu $sp,$sp,32
37 ; jump to the RA:
38 j $31
39 nop ; branch delay slot

We see here that register FP is used as a pointer to the stack frame. We also see 3 NOPs. The second
and third of which follow the branch instructions. Perhaps the GCC compiler always adds NOPs (because
of branch delay slots) after branch instructions and then, if optimization is turned on, maybe eliminates
them. So in this case they are left here.
Here is also IDA listing:
Listing 1.34: Non-optimizing GCC 4.4.5 (IDA)
1 .text:00000000 main:
2 .text:00000000
3 .text:00000000 var_10 = -0x10
4 .text:00000000 var_8 = -8
5 .text:00000000 var_4 = -4
6 .text:00000000
7 ; function prologue.
8 ; save the RA and FP in the stack:
9 .text:00000000 addiu $sp, -0x20
10 .text:00000004 sw $ra, 0x20+var_4($sp)
11 .text:00000008 sw $fp, 0x20+var_8($sp)
12 ; set the FP (stack frame pointer):
13 .text:0000000C move $fp, $sp
14 ; set the GP:
15 .text:00000010 la $gp, __gnu_local_gp
16 .text:00000018 sw $gp, 0x20+var_10($sp)
17 ; load the address of the text string:
18 .text:0000001C lui $v0, (aHelloWorld >> 16) # "Hello, world!"
19 .text:00000020 addiu $a0, $v0, (aHelloWorld & 0xFFFF) # "Hello, world!"
20 ; load the address of puts() using the GP:
21 .text:00000024 lw $v0, (puts & 0xFFFF)($gp)
22 .text:00000028 or $at, $zero ; NOP
23 ; call puts():
24 .text:0000002C move $t9, $v0
25 .text:00000030 jalr $t9
26 .text:00000034 or $at, $zero ; NOP
27 ; restore the GP from local stack:
28 .text:00000038 lw $gp, 0x20+var_10($fp)
29 ; set register $2 ($V0) to zero:
30 .text:0000003C move $v0, $zero
31 ; function epilogue.
32 ; restore the SP:
33 .text:00000040 move $sp, $fp

28
 1.5. HELLO, WORLD!
34 ; restore the RA:
35 .text:00000044 lw $ra, 0x20+var_4($sp)
36 ; restore the FP:
37 .text:00000048 lw $fp, 0x20+var_8($sp)
38 .text:0000004C addiu $sp, 0x20
39 ; jump to the RA:
40 .text:00000050 jr $ra
41 .text:00000054 or $at, $zero ; NOP

Interestingly, IDA recognized the LUI / ADDIU instructions pair and coalesced them into one LA (“Load
Address”) pseudo instruction at line 15. We may also see that this pseudo instruction has a size of 8 bytes!
This is a pseudo instruction (or macro) because it’s not a real MIPS instruction, but rather a handy name
for an instruction pair.
Another thing is that IDA doesn’t recognize NOP instructions, so here they are at lines 22, 26 and 41. It is
OR $AT, $ZERO . Essentially, this instruction applies the OR operation to the contents of the $AT register
with zero, which is, of course, an idle instruction. MIPS, like many other ISAs, doesn’t have a separate
NOP instruction.

Role of the stack frame in this example

The address of the text string is passed in the register. Why setup a local stack anyway? The reason
for this lies in the fact that the values of registers RA and GP have to be saved somewhere (because
printf() is called), and the local stack is used for this purpose. If this was a leaf function, it would have
been possible to get rid of the function prologue and epilogue, for example: 1.4.3 on page 8.

Optimizing GCC: load it into GDB

Listing 1.35: sample GDB session

root@debian-mips:~# gcc hw.c -O3 -o hw
root@debian-mips:~# gdb hw
GNU gdb (GDB) 7.0.1-debian
...
Reading symbols from /root/hw...(no debugging symbols found)...done.
(gdb) b main
Breakpoint 1 at 0x400654
(gdb) run
Starting program: /root/hw

Breakpoint 1, 0x00400654 in main ()

(gdb) set step-mode on
(gdb) disas
Dump of assembler code for function main:
0x00400640 <main+0>: lui gp,0x42
0x00400644 <main+4>: addiu sp,sp,-32
0x00400648 <main+8>: addiu gp,gp,-30624
0x0040064c <main+12>: sw ra,28(sp)
0x00400650 <main+16>: sw gp,16(sp)
0x00400654 <main+20>: lw t9,-32716(gp)
0x00400658 <main+24>: lui a0,0x40
0x0040065c <main+28>: jalr t9
0x00400660 <main+32>: addiu a0,a0,2080
0x00400664 <main+36>: lw ra,28(sp)
0x00400668 <main+40>: move v0,zero
0x0040066c <main+44>: jr ra
0x00400670 <main+48>: addiu sp,sp,32
End of assembler dump.
(gdb) s
0x00400658 in main ()
(gdb) s
0x0040065c in main ()
(gdb) s
0x2ab2de60 in printf () from /lib/libc.so.6
(gdb) x/s $a0

29
1.6. FUNCTION PROLOGUE AND EPILOGUE
0x400820: "hello, world"
(gdb)

1.5.6 Conclusion

The main difference between x86/ARM and x64/ARM64 code is that the pointer to the string is now 64-bits
in length. Indeed, modern CPUs are now 64-bit due to both the reduced cost of memory and the greater
demand for it by modern applications. We can add much more memory to our computers than 32-bit
pointers are able to address. As such, all pointers are now 64-bit.

1.5.7 Exercises

• http://challenges.re/48
• http://challenges.re/49

1.6 Function prologue and epilogue

A function prologue is a sequence of instructions at the start of a function. It often looks something like
the following code fragment:
push ebp
mov ebp, esp
sub esp, X

What these instruction do: save the value in the EBP register, set the value of the EBP register to the
value of the ESP and then allocate space on the stack for local variables.

The value in the EBP stays the same over the period of the function execution and is to be used for local
variables and arguments access. For the same purpose one can use ESP , but since it changes over time
this approach is not too convenient.
The function epilogue frees the allocated space in the stack, returns the value in the EBP register back
to its initial state and returns the control flow to the caller:
mov esp, ebp
pop ebp
ret 0

Function prologues and epilogues are usually detected in disassemblers for function delimitation.

1.6.1 Recursion

Epilogues and prologues can negatively affect the recursion performance.

More about recursion in this book: 3.4.3 on page 480.

1.7 Stack
48
The stack is one of the most fundamental data structures in computer science . AKA49 LIFO50 .
Technically, it is just a block of memory in process memory along with the ESP or RSP register in x86 or
x64, or the SP register in ARM, as a pointer within that block.
48 wikipedia.org/wiki/Call_stack
49 Also Known As
50 Last In First Out

30
1.7. STACK
The most frequently used stack access instructions are PUSH and POP (in both x86 and ARM Thumb-
mode). PUSH subtracts from ESP / RSP /SP 4 in 32-bit mode (or 8 in 64-bit mode) and then writes the
contents of its sole operand to the memory address pointed by ESP / RSP /SP.

POP is the reverse operation: retrieve the data from the memory location that SP points to, load it into
the instruction operand (often a register) and then add 4 (or 8) to the stack pointer.
After stack allocation, the stack pointer points at the bottom of the stack. PUSH decreases the stack
pointer and POP increases it. The bottom of the stack is actually at the beginning of the memory allocated
for the stack block. It seems strange, but that’s the way it is.
ARM supports both descending and ascending stacks.
For example the STMFD/LDMFD, STMED51 /LDMED52 instructions are intended to deal with a descending
stack (grows downwards, starting with a high address and progressing to a lower one). The STMFA53 /LDMFA54 ,
STMEA55 /LDMEA56 instructions are intended to deal with an ascending stack (grows upwards, starting from
a low address and progressing to a higher one).

1.7.1 Why does the stack grow backwards?

Intuitively, we might think that the stack grows upwards, i.e. towards higher addresses, like any other
data structure.
The reason that the stack grows backward is probably historical. When the computers were big and
occupied a whole room, it was easy to divide memory into two parts, one for the heap and one for the
stack. Of course, it was unknown how big the heap and the stack would be during program execution, so
this solution was the simplest possible.

Start of heap Start of stack

Heap Stack

In [D. M. Ritchie and K. Thompson, The UNIX Time Sharing System, (1974)]57 we can read:

The user-core part of an image is divided into three logical segments. The program text
segment begins at location 0 in the virtual address space. During execution, this segment
is write-protected and a single copy of it is shared among all processes executing the same
program. At the first 8K byte boundary above the program text segment in the virtual ad-
dress space begins a nonshared, writable data segment, the size of which may be extended
by a system call. Starting at the highest address in the virtual address space is a stack
segment, which automatically grows downward as the hardware’s stack pointer fluctuates.

This reminds us how some students write two lecture notes using only one notebook: notes for the first
lecture are written as usual, and notes for the second one are written from the end of notebook, by flipping
it. Notes may meet each other somewhere in between, in case of lack of free space.

1.7.2 What is the stack used for?

Save the function’s return address

x86

51 Store Multiple Empty Descending (ARM instruction)

52 Load Multiple Empty Descending (ARM instruction)
53 Store Multiple Full Ascending (ARM instruction)
54 Load Multiple Full Ascending (ARM instruction)
55 Store Multiple Empty Ascending (ARM instruction)
56 Load Multiple Empty Ascending (ARM instruction)
57 Also available as http://go.yurichev.com/17270

31
1.7. STACK
When calling another function with a CALL instruction, the address of the point exactly after the CALL
instruction is saved to the stack and then an unconditional jump to the address in the CALL operand is
executed.
The CALL instruction is equivalent to a
PUSH address_after_call / JMP operand instruction pair.

RET fetches a value from the stack and jumps to it —that is equivalent to a POP tmp / JMP tmp instruc-
tion pair.
Overflowing the stack is straightforward. Just run eternal recursion:
void f()
{
f();
};

MSVC 2008 reports the problem:

c:\tmp6>cl ss.cpp /Fass.asm
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.21022.08 for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

ss.cpp
c:\tmp6\ss.cpp(4) : warning C4717: 'f' : recursive on all control paths, function will cause ⤦
Ç runtime stack overflow

…but generates the right code anyway:

?f@@YAXXZ PROC ; f
; File c:\tmp6\ss.cpp
; Line 2
push ebp
mov ebp, esp
; Line 3
call ?f@@YAXXZ ; f
; Line 4
pop ebp
ret 0
?f@@YAXXZ ENDP ; f

…Also if we turn on the compiler optimization ( /Ox option) the optimized code will not overflow the
stack and will work correctly58 instead:
?f@@YAXXZ PROC ; f
; File c:\tmp6\ss.cpp
; Line 2
$LL3@f:
; Line 3
jmp SHORT $LL3@f
?f@@YAXXZ ENDP ; f

GCC 4.4.1 generates similar code in both cases without, however, issuing any warning about the problem.

ARM

ARM programs also use the stack for saving return addresses, but differently. As mentioned in “Hello,
world!” ( 1.5.4 on page 19), the RA is saved to the LR (link register). If one needs, however, to call another
function and use the LR register one more time, its value has to be saved. Usually it is saved in the
function prologue.
Often, we see instructions like PUSH R4-R7,LR along with this instruction in epilogue POP R4-R7,PC —
thus register values to be used in the function are saved in the stack, including LR.
58 irony here

32
 Another Random Document on
Scribd Without Any Related Topics
obtuse than the other. They are two in number, but, as Mr. Audubon
states, occasionally the nest contains three. Two broods are raised in
a season.
In the vicinity of Charleston these birds were observed to remain all
the year, though the greater proportion retired south or to the sea-
islands.
In the Florida Keys Mr. Audubon met with them among the islands
resorted to by the Zenaida Doves, and also on Sandy Island, near
Cape Sable. In the latter place they were so gentle that he
approached to within two yards of them. Their nest was on the top
of a cactus, not more than two feet from the ground.
Their food, in a wild state, consists of grass-seeds and various small
berries, with which they swallow a large proportion of gravel to
assist digestion. They are extremely fond of dusting themselves in
the sand, lying down in it in the manner of various gallinaceous
birds.
The eggs of this species are of a uniform bright white color, are
slightly more pointed at one end than at the other, and measure .85
of an inch in length by .63 in breadth.
This species was found in abundance at Cape St. Lucas by Mr.
Xantus. They were nesting from April 15 until August 29, and
evidently had two or more broods in a season. Their nests were
usually placed in low cactuses, near the ground, or in small shrubs.
Their nests, eggs, and general habits, so far as we can gather them
from the meagre notes of Mr. Xantus, are in no wise different from
those of the more eastern birds.

PLATE LVIII.
 1. Oreopeleia martinica. ♂ Jamaica.
2. Zenaidura carolinensis. ♂ N. C., 55569.
3. Zenaida amabilis. ♂ Jamaica, 24406.
4. Melopeleia leucoptera. ♂ Mazatlan, 34009.
5. Starnoæna cyanocephalus. ♂ Jamaica, ? 12541.
6. Chamæpelia passerina. ♂ 28281.
7. Scardafella inca. ♂ Texas, 45465.

Genus OREOPELEIA, Reichenbach.

Oreopeleia, Reichenbach, Handbuch der speciellen Ornithol. I, i, 1851, page xxiv.

(Type, Columba martinica, L.)
Gen. Char. Bill lengthened, slender; culmen half the rest of the head from the
frontal feathers. Feet large, stout; tarsi longer than the middle toe and claw,
covered anteriorly by transverse scutellæ. Inner lateral claw longer than outer;
reaching beyond the base of the middle one, the outer falling short of it. Hind toe
and claw more than half the middle. Quills and tail-feathers very broad; the wings
rounded; second and third quills longest, the first intermediate between the fourth
and fifth. Tail suborbicular, the shafts convex outwardly; the feathers rounded, and
a little graduated.

Of this genus, which is peculiar to America, two well-marked

species, may be distinguished.
 O. martinica. Above chestnut-rufous, the crown and nape with
purplish-green, the lower part of nape with golden-green, the
back with violet, the other upper parts with bright purplish-red
reflections; beneath pinkish-white, more purplish on the jugulum.
A distinctly marked light stripe on the cheek, bordered below by
one of purplish-red. Length, 10.70; wing, 6.20; tail, 5.75. Hab. Key
West, Florida (?) Cuba, and Martinique.

O. montana.111 Above deep orange-rufous, without bright

reflections, but with an opaque gloss of reddish-purple on the
back and nape. Beneath ochraceous, inclining to vinaceous on the
jugulum. Cheeks without distinct whitish bar bordered below by
reddish. Wing, 5.70. Hab. Atlantic region of Middle America from
Xalapa to Brazil; West Indies.

Oreopeleia martinica, Reich.

KEY WEST PIGEON.

Columba martinica, Gmelin, I, 1788, 781 (not of Temminck). Geotrygon martinica,

Bonap. Consp. Av. II, 1854, 74.—Cab. Jour. IV, 1856, 108. Oreopeleia
martinicana, Reich. Syst. Av. 1851, page xxv.—Ib. “Icones Avium, tab. 257, fig.
1431.” Columba montana, Aud. Orn. Biog. II, 1834, 382, pl. clxvii.—Ib. Syn.
1839, 191.—Ib. Birds Am. V, 1842, 14, pl. cclxxxii.—Nuttall, Man. I, (2d ed.,)
1840, 756 (not of Linnæus). Zenaida montana, Bonap. Geog. & Comp. List,
1838. “Columbigallina montana, Temminck.” “Columba mystacea, Lembeye,” Bonap.
(not of Temminck). Oreopeleia martinica, Baird, Birds N. Am. 1858, 607.—Cab.
J. IV, 109 (Cuba).—Gundl. Repert. Cub. I, 1866, 299 (Cuba).—Reich. Handb.
Taub. 31, tab. 257, fig. 1432.
Sp. Char. Ground-color of the upper parts, including wing (both surfaces), and tail-
feathers, chestnut-rufous; the upper part of head and neck with metallic
reflections of green and purple; the back, rump, and wing-coverts, with reflections
of metallic light-purplish or violet. There is a white band from the lower mandible
along side of the head, bordered below by purplish-red, like the forehead, and a
similar band through the eyes, which are without metallic lustre. The breast is very
light purplish-red, fading to white towards the tail and chin. The feathers of the
under tail-coverts are dusky-brown at the base. Length, 10.70; wing, 6.00; tail,
5.75.
Hab. Key West, Florida; Cuba and Martinique, perhaps elsewhere in the West
Indies.
 41876 ♂ ½ ½
Oreopeleia martinica.

Habits. The Key West Pigeon is found within the fauna of the United
States only in the extreme southern portion of Florida, and, so far as
known, only on the island of Key West, where Mr. Audubon met with
them, and enjoyed a limited opportunity of observing their habits.
He describes the flight as low, swift, and protracted, as he saw them
passing from Cuba to Key West. They moved in loose flocks of from
five or six to a dozen, and so very low as to almost seem to touch
the surface. They were fond of going out early in the morning from
their thickets to cleanse their plumage in the shelly sand, but on the
least approach of danger would fly back to the thickest part of the
woods, throw themselves on the ground, and run off with great
rapidity. Their movements of the tail and neck are similar to those of
the Carolina Dove. Their coo is said to be neither so soft nor so
prolonged as that of the common Dove, and may be represented by
the syllable whoe-whoe-oh-oh-oh. When suddenly approached, they
utter a guttural gasping sound. They are said to alight on the lower
branches of shrubby trees, and to delight in the neighborhood of
shady ponds, always inhabiting by preference the darkest solitudes.
Whatever may have been their abundance on Key West, in Mr.
Audubon’s time, it is certain that they are very rare there now, as I
am not aware of their having been taken of late years by any of the
numerous collectors who have visited South Florida since Mr.
Audubon’s time.

Oreopeleia martinica.

The nest is described as formed of light dry twigs, in shape much

resembling that of the Carolina Dove. Occasionally it is placed on the
ground, and is then less elaborate. Some are placed on large
branches near the ground, while others are built among slender
twigs.
Towards the middle of July, according to Mr. Audubon, they become
so abundant that sportsmen are able to shoot a score or more in a
day. They feed on berries and the seeds of various plants, and are
especially fond of the fruit of the sea-grape.

Genus STARNŒNAS, Bonaparte.

Starnœnas, Bonaparte, Geog. & Comp. List, 1838. (Type, Columba cyanocephala,
L.)
Gen. Char. Bill short; culmen about one third the rest of head, measured from the
frontal feathers. Legs very stout and large; tarsus bare on the entire tibial joint,
and covered with hexagonal scales, largest anteriorly, longer than the middle toe
and claw. Inner lateral claw the larger, reaching the base of the middle claw; all
the claws short, thick, and blunt. Hind toe and claw short; half the middle. Wings
short, broad, and concave; much rounded. Tail short, broad, nearly even, but
slightly vaulted.

The single species of Dove composing the genus in many respects

resembles the Partridges or Quails, both in external appearance and
in manners.

Starnœnas cyanocephala, Bon.

BLUE-HEADED PIGEON.

Columba cyanocephala, Linn. Syst. Nat. I, 1766, 282.—Gmelin, Syst. I, 1788, 778.—
Wagler, Syst. Avium, 1827, Columba, No. 112.—Aud. Orn. Biog. II, 1834, 441;
V. 1839, 557, pl. clxxii. Starnœnas cyanocephala, Bonap. List, 1838.—Ib. Consp.
II, 1854, 69.—Aud. Syn. 1839, 193.—Ib. Birds Amer. V, 1842, 23, pl. cclxxxiv.—
Gundlach, Cab. Journ. IV, 1856, 108.—Baird, Birds N. Am. 1858, 608.—Cab. J.
IV, 108 (Cuba).—Gundl. Repert. Cub. I, 1866, 299.—Reichenb. Handb. Taub. 30,
tab. 257, f. 1431; 266, f. 2879–81. Starnœnas cyanocephala, Reichenbach,
Systema Av. 1851, p. xxv, pl. xxiii.—Ib. Icones Av. tab. 260 and 266. Geophilus?
cyanocephala, Selby, Pigeons, Jard. Nat. Lib. V, 216, pl. xxvii. Columba
(Lophyrus) cyanocephala, Nuttall, Man. I, (2d. ed.,) 1840, 769. Columba
tetraoides, (Scopoli,) Gmelin, I, 772. Blue-headed Turtle, Latham, Syn. II, II, 651.
Sp. Char. Bill blue, the fleshy part at the base carmine. Iris brown, scales of feet
carmine, the interspaces white. Above and on sides glossy dark chocolate-
olivaceous; beneath brownish-red, lighter centrally. Chin and throat black, with a
narrow border of white below. A white line begins in the chin, and passes under
the eye to the occiput. Sides of head above this and forehead black; crown blue.
Length, 10.70; wing, 5.40; tail, 4.35.
Hab. West India Islands; according to Audubon found occasionally at Key West,
Florida, and other southern keys.
 2827 ♂ ½ ½
Starnœnas cyanocephala.

The axillars and under surface of the wings are like the belly. The
crissum is most like the back. The outer tail-feathers have a bluish
tinge above.
The hind toe in this species is not strictly in the same plane with the
others, but placed a little above their point of insertion.
Habits. This handsome Pigeon belongs to the fauna of the West India
Islands, and is only an occasional visitant of Key West and other
southern keys of Florida. They are a common species in Cuba, from
which island a few are stated by Mr. Audubon to migrate each year
to certain of the keys of Florida, where, however, they are rarely
seen on account of their living only in the most tangled thickets. Mr.
Audubon saw a pair on the western side of Key West. They were
near the water picking gravel, but they would not suffer a near
approach. He saw a pair, also, that had been taken, when young, on
“Mule Keys.” These fed well on cracked corn and rice, but he was
unable to obtain any further information in respect to them.
Though abundant in Cuba this species does not appear to have been
found in Jamaica, except as an imported bird from the former island,
contrary to the assertions of various writers, as Temminck, Brisson,
and others. Mr. Gosse was not able to trace its presence, though its
existence among the precipitous woods on the north side of that
island he regards as quite possible.

Starnœnas cyanocephala.

Like Oreopeleia martinica and Zenaida amabilis, this species, though

described by Audubon as not being rare on the keys of South
Florida, has not been met with in that State by later explorers.
An egg of this species laid in confinement in the aviary of Dr.
Bachman, in Charleston, S. C., is of a rounded-oval shape, and of a
uniform creamy-white color; it measures 1.43 inches in length by
1.10 in breadth.
 Family CRACIDÆ.—The Curassows.
Char. Body large, but rather slender; bill more or less arched; tail lengthened; legs
long, robust, without any spur. Toes moderate, slender, the hinder scarcely
elevated. Naked spaces frequently occurring on the head and throat.

Messrs. Sclater and Salvin, in their masterly and model monograph

of Cracidæ (Pr. Zoöl. Soc. 1870, 504), define the subfamilies as
follows:—
A. Post-acetabular area narrow; upper mandible higher than
broad; culmen compressed … I. Cracinæ.
B. Post-acetabular area broad; upper mandible broader than high;
culmen depressed.
Top of head covered with feathers; space between the nostrils
naked; nostrils exposed … II. Penelopinæ.
Top of head with a bony tubercle; internasal space densely
feathered; nostrils concealed … III. Oreophasinæ.
By the term “post-acetabular area” is understood that portion of the
dorsal aspect of the pelvis which is bounded in front by a line drawn
through the acetabula.
The Cracinæ, or Curassows, are found in Mexico, in Central and in
South America; the Oreophasinæ are represented by a single
species, Oreophasis derbianus, a bird nearly as large as a Turkey,
occurring in the wooded region of the Volcan de Fuego, Guatemala,
at an altitude of 10,000 feet. Of the Penelopinæ one species only is
found in the United States.

Subfamily PENELOPINÆ.
This is the most extensive section of Cracidæ, embracing, according
to Sclater and Salvin, no less than thirty-nine species. The genera
indicated are as follows:—
A. A central fold of skin on the throat.
Outer quills narrow, but entire.
Throat feathered … 1. Stegnolæma.
Throat naked.
Sexes similar … 2. Penelope.
Sexes different … 3. Penelopina.
Outer quills emarginated.
Gular fold short … 4. Pipile.
Gular fold lengthened; linear … 5. Aburria.
B. No central gular fold.
Throat feathered; outer quills emarginated … 6. Chamæpetes.
Throat naked; with a central line of bristly feathers; outer quills
entire … 7. Ortalida.

Genus ORTALIDA, Merrem.

Ortalida, Merrem, Av. rar. Icones et Desc. II, 1786, 40 (Gray). (Type, Phasianus
motmot, L.)
 37977 ♂ ⅓ ⅓
Ortalida maccalli.

Of Ortalida, as characterized above, Messrs. Sclater and Salvin

enumerate eighteen species; like the rest of the family, all American.
Of these only one has so far been detected within our limits,
although it is by no means improbable that the O. poliocephala,
Wagler (Sclater and Salvin, Pr. Zoöl. Soc. 1870, 537), may yet be
detected in New Mexico or Arizona.112

Ortalida vetula, var. maccalli, Baird.

THE TEXAS CHACALACCA.

Ortalida vetula, Lawrence, Ann. N. Y. Lyc. V, 1851, 116. (Not Penelope vetula,
Wagler, Isis, 1830, 1112, and 1831, 517.)—Scl. & Salv. P. Z. S. 1870, 538.
(Considers it the same as P. vetula, Wagler). Ortalida poliocephala, Cassin,
Illust. I, ix, 1855, 267, pl. xliv. (Not Penelope poliocephala, Wagler, Isis, 1830,
1112.) Ortalida maccalli, Baird, Birds N. Am. 1858, 611.—Ib. M. Bound. II,
Birds, 22.—Dresser, Ibis, 1866, 24 (S. E. Texas, breeding).—Lawr. Ann. N. Y. IX,
209 (Yucatan).—Scl. & Salv. P. Z. S. 1870, 538 (Honduras, Vera Cruz,
Guatemala).—Reichenb. Handb. der sp. Orn. Lief, viii, 145. (Describes more
adult specimens.)
Sp. Char. Body above dark greenish-olive; beneath brownish-yellow, tinged with
olive. Head and upper part of neck plumbeous. Tail-feathers lustrous green, all
tipped with white, except the middle one. Feathers along the middle of the throat
black; outer edge of primaries tinged with gray. Eyes brown. Bill and feet lead-
colored. Length, 23.50; wing, 8.50; tail, 11.00.
Hab. Valley of the Rio Grande, and southward to Guatemala.

This form is distinguishable from O. vetula, as restricted, of which it

is the northern representative, by the paler and less fulvous colors,
and lighter—often nearly white—tips to the tail-feathers, besides
other minor differences in coloration. The two cannot be separated
specifically, however, since they undoubtedly grade into each other.
Habits. This very remarkable bird, belonging as it does to a form
peculiar to this continent, is the only species found within the limits
of the United States, and only within a quite restricted area in the
valley of the Rio Grande. Numerous species of this family are found
in the warmer countries of America, especially Mexico and Central
America, all or nearly all of which appear to be capable of
domestication, and some of which, including the present species,
have, in repeated instances, been quite as completely domesticated
as our common Turkey.

Ortalida maccalli.
Birds of the family to which the Texan species belongs differ in a
very marked manner, in habits, from most Gallinaceæ, inasmuch as
they not only live almost exclusively in deep forests, but are also
remarkable for habitually frequenting trees, feeding upon their
foliage, and building their nests within their branches, more in the
manner of the smaller birds. They are all said to have loud and
discordant voices, and are generally of a black or dark plumage.
Specimens of this bird were taken at Boquillo, in New Leon, in the
spring of 1853, by Lieutenant Couch, who speaks of them as
gregarious and as seeking their food wholly or in part on trees.
According to Mr. Clark, they do not occur higher up the Rio Grande
than the vicinity of Ringgold Barracks, inhabiting the deepest
chaparrals, which they never quit. They are inactive, and for the
most of the time sit about in flocks in these thickets, feeding on
leaves. The Mexican name of Chacalacca is supposed to be derived
from the noise with which at times they make the valleys ring, and
which may be well imitated in kind, but not in strength, by putting
the most stress upon the last two syllables. No sooner does one take
up the song than others chime in from all quarters, till, apparently
exhausted, the noise gradually dies off into an interlude, only to be
again renewed. These concerts take place in the morning and
evening. The birds are quite gentle, are easily tamed, and are said
to cross with the common domestic fowl.
Mr. Dresser states that the Chacalacca is very common near
Matamoras and Brownsville, and that in the autumn great numbers
are exposed for sale in the market of the latter place. The Mexicans
are said to hold it in high esteem for its fighting qualities, and often
keep it in a domesticated state and cross it with the common fowl,
making use of the hybrid for cock-fighting. Mr. Dresser was so
informed by many Mexicans, upon whose word he placed reliance,
and was an eyewitness of a fight in which one of these hybrids was
engaged. Mr. Dresser had a tame one, when at Matamoras, that
became so familiar that he could hardly keep it out of his room. This
bird would occasionally go away for a day or two, and pay a visit to
the poultry belonging to a neighbor; whenever he missed it, he had
only to go to a poultry-yard near the house, where it could generally
be found.
This species was first taken within the United States by Colonel
McCall, who obtained it in Texas, and who enjoyed and improved
unusually good opportunities to observe the habits and manners of
this bird. From his notes, quoted by Mr. Cassin, we give the
following:—
“This very gallant-looking and spirited bird I saw for the first time
within our territory in the extensive forests of chaparral which
envelop the Resaca de la Palma. Here, and for miles along the Lower
Rio Grande, it was abundant; and throughout this region the
remarkable and sonorous cry of the male bird could not fail to attract
and fix the attention of the most obtuse or listless wanderer who
might chance to approach its abode. By the Mexicans it is called
Chiac-chia-lacca, an Indian name, without doubt derived from the
peculiar cry of the bird, which strikingly resembles a repetition of
these syllables. And when I assure you that its voice, in compass, is
equal to that of the Guinea-fowl, and in harshness but little inferior,
you may form some idea of the chorus with which the forest is made
to ring at the hour of sunrise. At that hour, in the month of April, I
have observed a proud and stately fellow descend from the tree on
which he had roosted, and, mounting upon an old log or stump,
commence his clear, shrill cry. This was soon responded to in a lower
tone by the female, the latter always taking up the strain as soon as
the importunate call of her mate had ceased. Thus alternating, one
pair after another would join in the matutinal chorus, and, before
the rising sun had lighted up their close retreat, the woods would
ring with the din of a hundred voices, as the happy couples met
after the period of separation and repose. When at length all this
clatter had terminated, the parties quietly betook themselves to their
morning meal. If surprised while thus employed, they would fly into
the trees above, and, peering down with stretched necks, and heads
turned sideways to the ground, they would challenge the intruder
with a singular and oft-repeated croaking note, of which it would be
difficult to give any adequate idea with words alone.”
Colonel McCall adds that the volubility and singularity of its voice is
its most striking and remarkable trait. While on his march from
Matamoras to Tampico he had encamped, on the 30th of December,
at the spring of Encinal, whence, a short time before sunset, he rode
out in search of game. Passing through a woodland near the stream,
his ears were saluted with a strange sound that resembled
somewhat the cry of the panther (Felis onca). He was at a loss to
what animal to ascribe it, and, dismounting, crawled cautiously
through the thicket for some distance, until he came upon an
opening where there were some larger trees, from the lower
branches of one of which he ascertained that the sound proceeded.
There he discovered a large male bird of this species, ascending
towards the top of the tree, and uttering this hitherto unheard
sound, as he sprang from branch to branch in mounting to his roost.
In a few moments his call was answered from a distance, and soon
after he was joined by a bird of the year. Others followed, coming in
from different quarters, and there were in a little while five or six
upon the tree. One of these discovered the intruder and gave the
alarm. The singular cry of the old bird ceased, and they all began to
exhibit uneasiness and a disposition to fly, whereupon Colonel McCall
shot the old bird.
Colonel McCall also states that the eye is a remarkable feature in the
living birds of this species, being full of courage and animation,
equal, in fact, in brilliancy to that of the finest gamecock. He
frequently noticed this bird domesticated by the Mexicans at
Matamoras, Monterey, etc., and going at large about their gardens.
He was assured that in that condition it not unfrequently crossed
with the common fowl.
In the wild state the eggs are said to be from six to eight, never
exceeding the last number. They are white, without spots, and rather
smaller than a pullet’s egg. The nest is usually on the ground, at the
root of a large tree or at the side of an old log, where a hole several
inches deep is scratched in the ground; this is lined with leaves, and
the eggs are always carefully covered with the same when the
female leaves them for the purpose of feeding. If disturbed while on
her nest, she flies at the intruder with great spirit and determination.
Eggs of this species, from Matamoras, are of an oblong-oval shape,
equally pointed at either end, and measure 2.35 inches in length by
1.65 in breadth. They are of a dirty-white color with a light tint of
buff, and have a slightly roughened or granulated surface.
 Family MELEAGRIDÆ.—The Turkeys.
Char. Bill moderate; the nasal fossæ bare. Head and neck without feathers, but
with scattered hairs, and more or less carunculated. An extensible fleshy process
on the forehead, but no development of the bone. Tarsus armed with spurs in the
male. Hind toe elevated. Tail nearly as long as the wing, truncate, of more than
twelve feathers.

The family Meleagridæ, or Turkeys, as at present known, is entirely

confined to North and Middle America, and represented only by the
genus Meleagris. It forms, in combination with the Guinea-fowls
(Numididæ), the Pheasants and common fowls (Phasianidæ), and
the Grouse and Partridges (Tetraonidæ), a peculiar group, to which
the name Alecteropodes has been given by Professor Huxley; this
group is well distinguished from the Cracidæ and the Megapodidæ
(which form together an opposed group, called Peristeropodes), in
addition to the characters enumerated under the family names, by
salient characters developed in the sternum. In the present family
and its relations, as all may recall from experience at the dinner-
table, the sternum, or breast-bone, is divided into a long narrow keel
(lophosteon) extending far backwards; while towards the front, from
each side, and separated by a very deep notch from the median
portion, a wing (pleurosteon) originates obliquely, and, soon splitting
in two, extends also far backwards; in front, two processes (called
costal) project well forwards. In the Cracidæ and Megapodidæ, on
the contrary, the sternum is not so split, the keel and wing, as
above, being more continuous and the notch comparatively shallow;
the costal processes are also comparatively small and obtuse.
Externally the Turkeys have considerable resemblance to the Guinea-
fowls (Numididæ), but they differ from them in having a backward
process of the second metacarpal bone, and in the form of the costal
processes of the sternum and of the acromial process of the
scapular; while they are distinguished from the Guinea-fowls and all
others by the form of the pelvis (the post-acetabular area is greater
than the pre-acetabular, and is also longer than broad), and by the
furcula (wish-bone), which is very weak and straight, with its point
(hypocleidium) straight and rod-like. To Professor Huxley we are
indebted for having first pointed out most of these characters.
Although the number of known species of Meleagridæ as we
understand them, is limited to two now living, the family was
apparently well represented in former geological periods, no less
than three having been already described from more or less perfect
remains; of these, two have been found in the post-pleiocene of
New Jersey, one of which (Meleagris altus, Marsh, or M. superbus,
Cope) was taller than the common Turkey, while the other (Meleagris
celer, Marsh) was much smaller. The third species (Meleagris
antiquus, Marsh) lived at a still earlier date, its remains having been
obtained in the miocene beds of Colorado.

Genus MELEAGRIS, Linnæus.

Meleagris, Linnæus, Syst. Nat. 1735. (Type, Meleagris gallopavo, Linn.)

 ¼¼¼
Meleagris gallopavo.

Gen. Char. Legs with transverse scutellæ before and behind; reticulated laterally.
Tarsi with spurs. Tail rounded, rather long, usually of eighteen feathers. Forehead
with a depending fleshy cone. Head and the upper half of the neck without
feathers. Breast of male in most species with a long tuft of bristles.

Species and Varieties.

M. gallopavo. Head livid blue, legs red, general color copper-

bronze, with copper and green reflections, each feather with a
 velvet-black margin; all the quills brown, closely barred with white.
Tail-feathers chestnut, narrowly barred with black; the tip with a
very broad, subterminal black bar.
Tail-coverts dark purplish-chestnut throughout, with the tips not
lighter. Tip of tail-feathers scarcely paler chestnut than the
ground-color. Hab. Eastern Province of United States … var.
gallopavo.
Tail-coverts chestnut, the tips much paler, sometimes almost
white. Tip of tail-feathers light brownish-yellow or white;
sometimes with the coverts broadly whitish. Hab. Southern
portion of Western Province of United States, from Texas to
Arizona. Table-lands of Mexico, south to Orizaba, Mirador,
etc. … var. mexicanus.

The M. ocellatus113 of Honduras and Yucatan is a very distinct

species, and one which vies with the Phasianidæ of Asia in the
brilliancy of its coloring. It is very rare in collections, and has a very
restricted distribution.

Meleagris gallopavo, var. gallopavo, Linn.

WILD TURKEY.

Meleagris gallopavo, Linnæus, Syst. Nat. I, 1758, 156.—Gmelin, I, 1788, 732.—

Latham, Ind. Orn. II, 1790, 618.—Stephens, in Shaw’s Zoöl. XI, i, 1819, 156
(domestic bird).—Bonap. Am. Orn. I, 1825, 79, pl. ix.—Aud. Orn. Biog. I, 1831,
1 and 33; V, 1839, 559, pl. i.—Ib. Birds Amer. V, 1842, 42, pl. cclxxxvii,
cclxxxviii.—Nuttall, Man. I, 1832, 630.—Reichenbach, Systema Av. 1851, pl. xxvi.
—Ib. Icones Av. tab. 289.—Baird, Birds N. Am. 1858, 615.—Dresser, Ibis, 1866,
25 (Southeastern Texas, breeds).—Max. Cab. J. VI, 1858, 426. Meleagris
americana, Bartram, Travels, 1791, 290. Meleagris sylvestris, Vieill. Nouv. Dict.
IX, 447. Gallopavo sylvestris, Leconte, Pr. A. N. Sc. Phil. 1857, 179. Meleagris
fera, Vieillot, Galerie Ois. II, 1824, 10, pl. x.—Gray. Cat. Gall. V, 42, 1867.—
Wild Turkey, Clayton, Philos. Trans. XVII, 1693, 992.—Pennant, Philos. Trans.
LXXI, 1781, 67.—Ib. Arctic Zoöl. No. 178. American Turkey, Latham, Syn. II, ii,
676. Gallopavo sylvestris, Novæ Angliæ, Ray, Syn. 51. Gallopavo sylvestris,
Catesby, Carol. I, 1730, App. p. xliv.—Brisson, Orn. V, 1760, 162.
 Meleagris gallopavo.

Sp. Char. The naked skin of the head and neck is blue; the excrescences purplish-
red. The legs are red. The feathers of the neck and body generally are very broad,
abruptly truncate, and each one well defined and scale-like; the exposed portion
coppery-bronze, with a bright coppery reflection in some lights, in the specimens
before us chiefly on the under parts. Each feather is abruptly margined with
velvet-black, the bronze assuming a greenish or purplish shade near the line of
junction, and the bronze itself sometimes with a greenish reflection in some lights.
The black is opaque, except along the extreme tip, where there is a metallic gloss.
The feathers of the lower back and rump are black, with little or no copper gloss.
The feathers of the sides behind, and the coverts, upper and under, are of a very
dark purplish-chestnut, with purplish-metallic reflections near the end, and a
subterminal bar of black; the tips are of the opaque purplish-chestnut referred to.
The concealed portion of the coverts is dark chestnut barred rather finely with
black; the black wider than the interspaces. The tail-feathers are dark brownish-
chestnut, with numerous transverse bars of black, which, when most distinct, are
about a quarter of an inch wide and about double their interspaces; the extreme
tip for about half an inch is plain chestnut, lighter than the ground-color; and
there is a broad subterminal bar of black about two inches wide on the outer
feathers, and narrowing to about three quarters of an inch to the central ones.
The innermost pair scarcely shows this band, and the others are all much broken
and confused. In addition to the black bars on each feather, the chestnut
interspaces are sprinkled with black. The black bands are all most distinct on the
inner webs; the interspaces are considerably lighter below than above.
There are no whitish tips whatever to the tail or its coverts. The feathers on the
middle of the belly are downy, opaque, and tipped obscurely with rusty whitish.
The wing-coverts are like the back; the quills, however, are blackish-brown, with
numerous transverse bars of white, half the width of the interspaces. The exposed
surfaces of the wing, however, and most of the inner secondaries, are tinged with
brownish-rusty, the uppermost ones with a dull copper or greenish gloss.
The female differs in smaller size, less brilliant colors, absence generally of bristles
on the breast and of spur, and a much smaller fleshy process above the base of
the bill.
Male. Length, 48.00 to 50.00; extent, 60.00; wing, 21.00; tail, 18.50. Weight, 16
to 35 lbs. Female. Weight about 12 lbs.; measurements smaller in proportion.
Hab. Eastern Province of the United States, and Canada. West along the timbered
river-valleys towards the Rocky Mountains; south to the Gulf coast.

There is some question as to the names to be applied to the two

races of Northern Meleagris, and especially as to which is entitled to
bear the name of gallopavo. The original description of M. gallopavo
quotes the New England Turkey as described by Ray, but as far as
the characters given go refers rather to the domestic form, which is
equivalent to M. mexicana of Gould. In this state of the case we
therefore think it as well to use gallopavo for the eastern race,
although the arguments of Major Leconte and others in favor of
applying it to the wild Mexican, and its derivative the domestic
variety, are not without much weight.
Habits. The Wild Turkey is found throughout eastern North America,
from South Carolina northward, and from the Atlantic to Texas and
Arkansas. It has probably become an extinct species in New
England, though within a few years individuals have been shot in
Montague, Mass., and in other towns in Franklin County. The
construction of railroads, however, and the settlement of the country,
have probably led to their final extermination; at least, I have known
of none being taken within the limits of Massachusetts for several
years.
In the unsettled portions of the Southern and Western States, and in
the country watered by the Mississippi and the Missouri Rivers and
their affluents, these birds are comparatively plentiful, though the
question of their final extinction is probably only one of time, and
that not very distant. In Audubon’s day they were to be found along
the whole line of the Alleghanies, where they still occur, but have
become very wary and to be approached only with the greatest
difficulty. In Louisiana and in Kentucky, Audubon found them most
abundant, and in these States he enjoyed the most favorable
opportunities for observing their very remarkable habits in regions
then comparatively undisturbed by the intrusion of civilized man.
They are said to be not uncommon in Virginia, and are not
unfrequently met with even in the vicinity of Washington.
Dr. Woodhouse found this species abundant throughout the wooded
portions of the Indian Territory and Texas. While in the Creek
country his party killed numbers of them daily. Many of them were
very large, and weighed upwards of nineteen pounds each, although
at that time they were in poor condition. They were quite abundant
along the Rio San Pedro in Texas.
Mr. Dresser found the Wild Turkey common in all the portions of
Texas and Mexico that he visited, and particularly so on the rivers
between San Antonio and the Rio Grande. His first Turkey hunt was
on the Upper Medina River, about forty miles from San Antonio. It
proved to be wary and difficult to approach in the daytime; but by
watching to see where they roosted, and visiting them by moonlight,
one or two could generally be secured. They generally preferred
roosting in high cottonwood-trees, on the banks of a stream,
perching as high up as possible. He once saw eleven Turkeys on one
large bough of a cottonwood-tree on the Medina. When the pecan-
nuts are ripe the Turkeys become very fat, as they are extremely
fond of these nuts, which are very oily. One very plump bird was
found, after it had been dressed, to weigh sixteen pounds. Mr.
Dresser was informed by the hunters, that, for a nest, the Turkeys
scratch a hole in the ground, or make a sort of nest in the grass
under a bush, and that the eggs resemble those of the tame Turkey,
except in being smaller and more elongated in form. The Mexicans,
on the Upper Rio Grande, sometimes domesticate the Wild Turkey,
and at Piedras Negras Mr. Dresser saw two that had been caught
when quite young and had become very tame. The female was then
sitting, and the eggs, when examined, were found to agree with the
account given him by the hunters.
Mr. Audubon, in his very full and minute account of their habits,
speaks of them as irregularly migratory and gregarious, their
migrations having reference only to the abundance of food, and the
meeting together in the same localities being to a large degree
caused by the same source of attraction,—the supply of mast in
certain regions. In this way they desert sections where the supply is
exhausted, and advance towards those where it is more plentiful.
Late in October these birds assemble in flocks in the rich bottom-
lands of the Western rivers, the male birds associating in parties of
from ten to a hundred, and keeping apart from the females. The
latter are simultaneously moving into the same regions, but only in
small family groups, each leading its own flock, then nearly grown.
Gradually they unite with other families, forming at length parties of
seventy or eighty. They are said to avoid very carefully the old
males, who have the very unparental disposition to destroy the
young birds even when nearly grown. These migrations are made on
foot except when they are compelled to cross a stream. On their first
coming to the banks of a river they are said to make a pause there
of one or two days before they attempt to cross, the old males
strutting about up and down the banks, making a loud gobbling, and
calling to one another as if to raise their courage to a befitting point.
Even the females and the young assume something of the same
pompous demeanor, spreading out their tails, running round one
another, and making a loud purring noise. At length, after this
prolonged preparation for the passage, they all mount to the top of
a high tree, and, at a signal given by their leader, take flight for the
opposite shore. Occasionally some fall into the water, when these
bring the wings close to the body, spread out the tail, and plying
their legs with great vigor move rapidly towards the shore, where,
by a violent effort, they extricate themselves from the water. After
thus crossing a stream of any magnitude, they are often found in a
bewildered state, and fall an easy prey to the hunter.
Where their food occurs abundantly they separate into smaller
flocks, composed of birds of all ages and sexes. At times they are
known to approach farmhouses, associate with the domesticated
fowl, and enter the corn-cribs in quest of food, passing the fall and
the winter in this manner.
Early in February the love-season is said to commence, the first
demonstrations being made by the males, but for some time
persistently avoided by the females. At this period the sexes roost
apart. When a female utters a call-note, the male birds within
hearing return the cry, uttering notes similar to those with which the
domestic Turkey greets any very unusual sound. If the call-note has
been uttered by a female on the ground, the males fly to the place,
spreading and erecting their tails, drawing their heads back on their
shoulders, depressing their wings with a quivering motion, and
strutting pompously about. At the same time they emit from their
lungs a succession of very peculiar puffs. On these occasions the
males often encounter each other, and desperate contests ensue,
which frequently have a fatal termination, caused by furious blows
inflicted on the head. When one Cock-Turkey has thus destroyed its
rival, it is said to caress the dead body in an apparently affectionate
manner.
When the Turkeys have mated, the connection is supposed to last
for that season, though a male Turkey is often known to have more
than a single mate; and the hens are said also to keep apart from
the males while they are laying their eggs, for the cock would
inevitably destroy them. At the end of the love-season the males
become emaciated, and cease to gobble. They then separate entirely
from the females, and keep apart by themselves until they recover
their strength, when they reunite in small flocks.
The female is said to begin to deposit her eggs about the middle of
April, selecting for that purpose a place as much concealed as
possible from her many enemies. The nest, always on the ground,
consists of a few withered leaves in a hollow scratched out by the
side of a fallen log, or the top of a prostrate tree, or under a thicket,
or within the edge of a cane-brake, but always in a dry place. The
eggs sometimes amount to twenty in number, though there are
usually from ten to fifteen. They are described as of a dull cream-
color, sprinkled with reddish dots. When the female leaves her nest,
she is said to be very careful to cover them with leaves, so that it is
always difficult for any one to find them. Mr. Audubon observed that
Turkey-hens not unfrequently selected small islands in which to
deposit their eggs, apparently on account of the great masses of
drift-timber which accumulated at their heads, in which they could
seek protection and shelter.
If a female is approached while sitting on her eggs, she rarely moves
unless she is discovered. Mr. Audubon has frequently approached
within a few paces of a nest, the female remaining undisturbed.
They seldom abandon their nest when it has been discovered by
man, but forsake it if any of the eggs have been destroyed by any
kind of animal. If the eggs are taken or destroyed, the female
prepares for another nest, but otherwise has only one brood in a
season. Audubon also states that he has known several hens
associate together, deposit their eggs in the same nest, and rear
their broods together, having once found three hens sitting on forty-
two eggs in a single nest, one female at least being always present
to protect it. When the eggs are near hatching, the female will not
leave her eggs under any circumstances, and will suffer herself to be
made a prisoner rather than abandon them. The mother assists the
young birds to extricate themselves from the egg-shell, caresses and
dries them with her bill, and aids them in their first efforts to totter
out of the nest. As the brood follow her, she is very watchful against
Hawks or other enemies, spreads her wings a little to protect them,
and calls them close to her side, keeping them on dry ground and
carefully guarding them from wet, which is very injurious to them
Welcome to Our Bookstore - The Ultimate Destination for Book Lovers
Are you passionate about books and eager to explore new worlds of
knowledge? At our website, we offer a vast collection of books that
cater to every interest and age group. From classic literature to
specialized publications, self-help books, and children’s stories, we
have it all! Each book is a gateway to new adventures, helping you
expand your knowledge and nourish your soul
Experience Convenient and Enjoyable Book Shopping Our website is more
than just an online bookstore—it’s a bridge connecting readers to the
timeless values of culture and wisdom. With a sleek and user-friendly
interface and a smart search system, you can find your favorite books
quickly and easily. Enjoy special promotions, fast home delivery, and
a seamless shopping experience that saves you time and enhances your
love for reading.
Let us accompany you on the journey of exploring knowledge and
personal growth!

ebookball.com