Download Complete Foundations of Linux Debugging, Disassembling, and Reversing: Analyze Binary Code, Understand Stack Memory Usage, and Reconstruct C/C++ Code with Intel x64 1st Edition Dmitry Vostokov PDF for All Chapters
Download Complete Foundations of Linux Debugging, Disassembling, and Reversing: Analyze Binary Code, Understand Stack Memory Usage, and Reconstruct C/C++ Code with Intel x64 1st Edition Dmitry Vostokov PDF for All Chapters
com
OR CLICK HERE
DOWLOAD NOW
https://ebookmass.com/product/foundations-of-arm64-linux-debugging-
disassembling-and-reversing-dmitry-vostokov/
ebookmass.com
https://ebookmass.com/product/comprehensive-materials-finishing-1st-
edition-saleem-hashmi/
ebookmass.com
Oxide Electronics 1st Edition Asim K. Ray
https://ebookmass.com/product/oxide-electronics-1st-edition-asim-k-
ray/
ebookmass.com
https://ebookmass.com/product/behavioral-and-mental-health-care-
policy-and-practice-a-biopsychosocial-perspective-ebook-pdf-version/
ebookmass.com
https://ebookmass.com/product/how-things-count-as-the-same-memory-
mimesis-and-metaphor-adam-b-seligman/
ebookmass.com
https://ebookmass.com/product/practitioners-guide-to-operationalizing-
data-governance-mary-anne-hopper/
ebookmass.com
https://ebookmass.com/product/p-o-w-e-r-learning-foundations-of-
student-success-4th-edition-robert-feldman/
ebookmass.com
Wrongful Convictions and Forensic Science Errors: Case
Studies and Root Causes John Morgan
https://ebookmass.com/product/wrongful-convictions-and-forensic-
science-errors-case-studies-and-root-causes-john-morgan/
ebookmass.com
Foundations of Linux
Debugging, Disassembling,
and Reversing
Analyze Binary Code, Understand
Stack Memory Usage, and Reconstruct
C/C++ Code with Intel x64
—
Dmitry Vostokov
Foundations of Linux
Debugging,
Disassembling, and
Reversing
Analyze Binary Code,
Understand Stack Memory
Usage, and Reconstruct C/C++
Code with Intel x64
Dmitry Vostokov
Foundations of Linux Debugging, Disassembling, and Reversing: Analyze
Binary Code, Understand Stack Memory Usage, and Reconstruct C/C++
Code with Intel x64
Dmitry Vostokov
Dublin, Ireland
Preface����������������������������������������������������������������������������������������������xiii
iii
Table of Contents
Chapter 4: Pointers�����������������������������������������������������������������������������33
A Definition���������������������������������������������������������������������������������������������������������33
“Pointers” Project: Memory Layout and Registers����������������������������������������������34
“Pointers” Project: Calculations��������������������������������������������������������������������������36
Using Pointers to Assign Numbers to Memory Cells�������������������������������������������36
Adding Numbers Using Pointers�������������������������������������������������������������������������42
Incrementing Numbers Using Pointers���������������������������������������������������������������45
Multiplying Numbers Using Pointers�������������������������������������������������������������������48
Summary������������������������������������������������������������������������������������������������������������51
iv
Table of Contents
Registers Revisited���������������������������������������������������������������������������������������������65
NULL Pointers�����������������������������������������������������������������������������������������������������65
Invalid Pointers���������������������������������������������������������������������������������������������������65
Variables As Pointers������������������������������������������������������������������������������������������66
Pointer Initialization��������������������������������������������������������������������������������������������67
Initialized and Uninitialized Data�������������������������������������������������������������������������67
More Pseudo Notation�����������������������������������������������������������������������������������������68
“MemoryPointers” Project: Memory Layout�������������������������������������������������������68
Summary������������������������������������������������������������������������������������������������������������79
v
Table of Contents
vi
Table of Contents
vii
Table of Contents
Index�������������������������������������������������������������������������������������������������167
viii
About the Author
Dmitry Vostokov is an internationally
recognized expert, speaker, educator, scientist,
and author. He is the founder of the pattern-
oriented software diagnostics, forensics,
and prognostics discipline and Software
Diagnostics Institute (DA+TA: DumpAnalysis.
org + TraceAnalysis.org). Vostokov has also
authored more than 50 books on software
diagnostics, anomaly detection and analysis,
software and memory forensics, root cause analysis and problem solving,
memory dump analysis, debugging, software trace and log analysis,
reverse engineering, and malware analysis. He has more than 25 years
of experience in software architecture, design, development, and
maintenance in various industries, including leadership, technical, and
people management roles. Dmitry also founded Syndromatix, Anolog.
io, BriteTrace, DiaThings, Logtellect, OpenTask Iterative and Incremental
Publishing (OpenTask.com), Software Diagnostics Technology and
Services (former Memory Dump Analysis Services; PatternDiagnostics.
com), and Software Prognostics. In his spare time, he presents various
topics on Debugging TV and explores Software Narratology, its further
development as Narratology of Things and Diagnostics of Things (DoT),
and Software Pathology. His current areas of interest are theoretical
software diagnostics and its mathematical and computer science
foundations, application of artificial intelligence, machine learning and
ix
About the Author
x
About the Technical Reviewer
Vikas Talan is a senior engineer at Qualcomm
(an American multinational corporation). He is
the founder of S.M.A.R.T Solutions, a technical
company. He also worked at MediaTek and
Cadence in core technical domains. He has
in-depth experience in Linux kernel
programming, Linux device drivers, ARM 64,
ARM, and porting of Android OS and Linux
drivers on chipsets. He hails from Delhi
NCR, India.
xi
Preface
The book covers topics ranging from Intel x64 assembly language
instructions and writing programs in assembly language to pointers, live
debugging, and static binary analysis of compiled C and C++ code.
Diagnostics of core memory dumps, live and postmortem debugging
of Linux applications, services, and systems, memory forensics, malware,
and vulnerability analysis require an understanding of x64 Intel assembly
language and how C and C++ compilers generate code, including
memory layout and pointers. This book is about background knowledge
and practical foundations that are needed to understand internal Linux
program structure and behavior, start working with the GDB debugger, and
use it for disassembly and reversing. It consists of practical step-by-step
exercises of increasing complexity with explanations and many diagrams,
including some necessary background topics.
By the end of the book, you will have a solid understanding of how
Linux C and C++ compilers generate binary code. In addition, you will be
able to analyze such code confidently, understand stack memory usage,
and reconstruct original C/C++ code.
The book will be useful for
• Software testers
xiii
Preface
This book can also be used as an x64 assembly language and Linux
debugging supplement for relevant undergraduate-level courses.
Source Code
All source code used in this book can be downloaded from github.com/
apress/linux-debugging-disassembling-reversing.
xiv
CHAPTER 1
Memory, Registers,
and Simple Arithmetic
emory and Registers Inside
M
an Idealized Computer
Computer memory consists of a sequence of memory cells, and each cell
has a unique address (location). Every cell contains a “number.” We refer
to these “numbers” as contents at addresses (locations). Because memory
access is slower than arithmetic instructions, there are so-called registers
to speed up complex operations that require memory to store temporary
results. We can also think about them as stand-alone memory cells. The
name of a register is its address. Figure 1-1 illustrates this concept.
2
Chapter 1 Memory, Registers, and Simple Arithmetic
3
Chapter 1 Memory, Registers, and Simple Arithmetic
static int a, b;
4
Chapter 1 Memory, Registers, and Simple Arithmetic
5
Chapter 1 Memory, Registers, and Simple Arithmetic
If we use the C or C++ language, “a” is called “the variable a,” and we
write the assignment as
a = 1;
mov $1, a
In the GDB disassembly output, we see the following code where the
variable “a” and address are shown in comments:
Notice movl instructions instead of mov. This is because “a” and “b”
can point to both 32-bit (like %EAX or %EDX registers) and 64-bit memory
cells (like %RAX and %RDX registers). In the registers’ case, it is clear from
their names whether we use 64-bit %RAX or 32-bit %EAX. But in the case
of memory addresses “a” and “b,” it is not clear whether they refer to 64-bit
or 32-bit cells. We use movl to disambiguate and show that we use 32-bit
memory cells that are enough to hold integers from 0 to 4294967295.
0x2ef2(%rip) is how the compiler generates code to calculate the
address “a” instead of specifying it directly. Such code requires less
memory space. We explain this in later chapters.
6
Chapter 1 Memory, Registers, and Simple Arithmetic
Literal constants have the $ prefix, for example, $0x1. The 0x prefix
means the following number is hexadecimal. The leading four zeroes of
the address are also omitted in the comment. We explain such numbers in
Chapter 3. Please also notice that the movement direction is the same in
both the disassembly output and the pseudo-code: from left to right.
After executing the first two assembly language instructions, we have
the memory layout shown in Figure 1-4.
Figure 1-4. Memory layout after executing the first two assembly
language instructions
7
Chapter 1 Memory, Registers, and Simple Arithmetic
1 -> register
(a) -> register
mov $0x0,%eax
8
Chapter 1 Memory, Registers, and Simple Arithmetic
b = b + a;
b += a;
mov a, %eax
add %eax, b
or we can add two registers and move the result to the memory cell b:
mov b, %edx
mov a, %eax
add %edx, %eax
mov %eax, b
9
Chapter 1 Memory, Registers, and Simple Arithmetic
10
Chapter 1 Memory, Registers, and Simple Arithmetic
Incrementing/Decrementing Numbers
in Memory and Registers
In pseudo-code, it looks simple and means increment (decrement) a
number stored at the location (address) “a”:
11
Chapter 1 Memory, Registers, and Simple Arithmetic
In the C or C++ language, we can write this using three possible ways:
a = a + 1;
++a;
a++;
b = b – 1;
--b;
b--;
incl a
inc %eax
decl a
dec %eax
inc %eax
or
12
Chapter 1 Memory, Registers, and Simple Arithmetic
13
Chapter 1 Memory, Registers, and Simple Arithmetic
Multiplying Numbers
In pseudo-code, we write
b = b * a;
b *= a;
14
Chapter 1 Memory, Registers, and Simple Arithmetic
mov a, %eax
imul b, %eax
mov %eax, b
mov a, %eax
mov b, %edx
imul %edx, %eax
mov %eax, b
15
Chapter 1 Memory, Registers, and Simple Arithmetic
16
Chapter 1 Memory, Registers, and Simple Arithmetic
Figure 1-7. Memory layout after the execution of IMUL and MOV
instructions
Summary
This chapter introduced CPU registers and explained the memory layout
of a simple arithmetic program. We learned basic x64 instructions and
manually translated simple C and C++ code to assembly language.
The next chapter looks at assembly language code produced by a
debugger via disassembling binary code. Then, we reverse it to C and C++
code. We also compare the disassembly output of nonoptimized code to
optimized code.
17
CHAPTER 2
Code Optimization
“Arithmetic” Project: C/C++ Program
Let’s rewrite our “Arithmetic” program in C/C++. Corresponding assembly
language instructions are put in comments:
int a, b;
Downloading GDB
We used WSL2 and "Debian GNU/Linux 10 (buster)" as a working
environment. We chose Debian because we used it for the “Accelerated
Linux Core Dump Analysis” training course.1 After installing Debian, we
need to install essential build tools and GDB:
1
www.dumpanalysis.org/accelerated-linux-core-dump-analysis-book
20
Chapter 2 Code Optimization
coredump@DESKTOP-IS6V2L0:~/pflddr/x64/Chapter2$ gcc
ArithmeticProjectC.cpp -o ArithmeticProjectC
we get the binary executable module we can load in GDB and inspect
assembly code.
First, we run GDB with the program as a parameter:
coredump@DESKTOP-IS6V2L0:~/pflddr/x64/Chapter2$ gdb ./
ArithmeticProjectC
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/
licenses/gpl.html>
This is free software: you are free to change and
redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources
online at:
<http://www.gnu.org/software/gdb/documentation/>.
21
Chapter 2 Code Optimization
Then we start execution of the program (let it run). The program then
stops at the previously set breakpoint:
(gdb) run
Starting program: /home/coredump/pflddr/x64/Chapter2/
ArithmeticProjectC
22
Chapter 2 Code Optimization
0x0000555555555152 <+45>: mov %eax,0x2ed8(%rip)
# 0x555555558030 <b>
0x0000555555555158 <+51>: mov 0x2ece(%rip),%eax
# 0x55555555802c <a>
0x000055555555515e <+57>: add $0x1,%eax
0x0000555555555161 <+60>: mov %eax,0x2ec5(%rip)
# 0x55555555802c <a>
0x0000555555555167 <+66>: mov 0x2ec3(%rip),%edx
# 0x555555558030 <b>
0x000055555555516d <+72>: mov 0x2eb9(%rip),%eax
# 0x55555555802c <a>
0x0000555555555173 <+78>: imul %edx,%eax
0x0000555555555176 <+81>: mov %eax,0x2eb4(%rip)
# 0x555555558030 <b>
0x000055555555517c <+87>: mov $0x0,%eax
0x0000555555555181 <+92>: pop %rbp
0x0000555555555182 <+93>: retq
End of assembler dump.
0x0000555555555130 <+11>: movl $0x1,0x2ef2(%rip)
# 0x55555555802c <a>
0x000055555555513a <+21>: movl $0x1,0x2eec(%rip)
# 0x555555558030 <b>
0x0000555555555144 <+31>: mov 0x2ee6(%rip),%edx
# 0x555555558030 <b>
0x000055555555514a <+37>: mov 0x2edc(%rip),%eax
# 0x55555555802c <a>
0x0000555555555150 <+43>: add %edx,%eax
23
Chapter 2 Code Optimization
0x0000555555555152 <+45>: mov %eax,0x2ed8(%rip)
# 0x555555558030 <b>
0x0000555555555158 <+51>: mov 0x2ece(%rip),%eax
# 0x55555555802c <a>
0x000055555555515e <+57>: add $0x1,%eax
0x0000555555555161 <+60>: mov %eax,0x2ec5(%rip)
# 0x55555555802c <a>
0x0000555555555167 <+66>: mov 0x2ec3(%rip),%edx
# 0x555555558030 <b>
0x000055555555516d <+72>: mov 0x2eb9(%rip),%eax
# 0x55555555802c <a>
0x0000555555555173 <+78>: imul %edx,%eax
0x0000555555555176 <+81>: mov %eax,0x2eb4(%rip)
# 0x555555558030 <b>
24
Chapter 2 Code Optimization
(gdb) q
A debugging session is active.
Quit anyway? (y or n) y
coredump@DESKTOP-IS6V2L0:~/pflddr/x64/Chapter2$
coredump@DESKTOP-IS6V2L0:~/pflddr/x64/Chapter2$ gcc
ArithmeticProjectC.cpp -O1 -o ArithmeticProjectC
and after repeating the same steps in GDB, we get the following output:
Please note that the compiler also chose to put memory cell “b” first
(000055555555802c) and then memory cell “a” (0000555555558030).
25
Chapter 2 Code Optimization
What happened to all our assembly code in this executable? This code
seems to be directly placing the end result into the “b” memory cell if we
observe. Why is this happening? The answer lies in compiler optimization.
When the code is compiled in optimization mode, the compiler can
calculate the final result from the simple C/C++ source code itself and
generate only the necessary code to update corresponding memory
locations.
Summary
In this chapter, we looked at assembly language code produced by a
debugger via disassembling binary code. Then, we reversed it to C and C++
code. We also compared the disassembly output of nonoptimized code to
optimized code and understood why.
The next chapter refreshes number representations, especially the
hexadecimal one.
26
CHAPTER 3
Number
Representations
Numbers and Their Representations
Imagine a herder in ancient times trying to count his sheep. He has a
certain number of stones (twelve):
However, he can only count up to three and arranges the total into
groups of three:
The American traveller can also agree in part with Dr. Johnson
that “there is nothing which has yet been contrived by man by which
so much happiness is produced as by a good tavern or inn,” yet who,
except Leighton, that sunny soul, the principal of Edinburgh
University,—who was “not militant enough to please his fierce co-
presbyters,”—could say “that if he were to choose a place to die in, it
should be an inn”?
Our experiences of Scottish hotels, “temperance,” “hydro,”
ordinary, fashionable, rural, rustic, and what not, were almost
invariably pleasant, and our hosts were honest in their dealings; yet
in the many British homes in which we were guests the welcome
was so warm and the care taken of us so thorough, thoughtful, and
minute, that Shenstone’s verse seems to have in it more verbal
music than experimental reality. Leighton’s wish seems strange,
indeed, especially in the light of memory, when sickness, apparently
nigh unto the bourne of life, proved the depth of Scottish hospitality
and friendship.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
ebookmass.com