Lehrstuhl für Systemsicherheit

Virtual Machine-based Fingerprints

SPRING 9 Bochum, 31.07 - 01.08.2014
Table of Contents

1. Background
1. Fingerprinting
2. Virtual Machines

2. Implemented Schemes
1. Permutation-based Fingerprints
2. Dynamic branch-based Fingerprints
3. Fingerprints based on Encoding Choice

3. Conclusion

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
 Background

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
Fingerprinting I

●
Two phases:

1. Embed an unique identifier (“mark”) into object

2. Identify the object by extracting the fingerprint
mark

●
Fingerprint mark identifies party that uses the object

●
In contrast to watermarking (claim ownership)
●
Software use case: given a copy of the software, find
out who it has been sold to

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
Fingerprinting II

●
Three types of fingerprints, determined by extraction
phase:

1. Static
2. Dynamic
3. Abstract

●
Balance properties:

1. Stealth
2. Data Rate
3. Resilience

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
Virtual Machines I

●
Structure commonly used in software protection systems

●
Basic idea: Translate (parts of) native code into a custom
architecture and embed interpreter (VM)

●
breaks existing tools
●
non-trivial to attack generically
●
hides original semantic and tamper-proofs

●
Set of handlers describe semantics

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
Virtual Machines II
bytecode VM context

opcode parameters entry value

5A 0x0f00 0xdeadbeef vIP [pointer]

FE 0xbeef 0x0f00 handler tbl [pointer]
32 native eax 0xdeadbeef
5A 0x0f00 0xcafebabe native ecx 0x1badc0de
FE 0xdead 0x0f00 ... ...
07 0xbeef 0xdead

5A 0xb00b 0x1badf00d
32
FE 0xdead 0x0f00

00 vm_and_reg_reg vm_mov_reg_imm
... ... handler
5A vm_mov_reg_imm fetch operands
... ... calculate
7F vm_add_reg_reg update ctx
80 vm_xor_reg_reg dispatch next
... ...
FF vm_mov_reg_reg

handler table handler code

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
 Implemented Schemes

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
Permutation-based Fingerprints

●
Based on patent by Davidson and Myhrvold (1996)

●
Embeds the mark in order of basic blocks of a
function
●
Mark extracted by comparing order in binary to
canonical ordering
●
But: Prone to subsequent application!

●
Approach here: Embed mark in permutation of handler
table
●
Subsequent application results in non-functional
program!

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
Permutation-based Fingerprints
Extracted
Canonical Form Perm.
Handler Table

0040AFC4 00 0040640A FE
00407513 01 0040645A 39
0040645A 02 004064AB 01
0040699E 03 004064FF 12
canonical
004070A1 04 0040654F 2A
0040640A 05 004065A0 00
... ... ... ...
00407F72 FF 0040AF72 42

lookup handler index

0040AFC4

00407513

0040645A

0040699E

004070A1

Code 0040640A

...

00407F72

Fingerprinted Binary

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
Branch-based Fingerprints

●
Based on method by Linn et al., extension by Collberg et
al.
●
Mark encoded in (unstealthy!) series of unconditional
branches
●
Branch direction encodes one bit
●
Extraction using Execution Trace

●
Approach here: Transferred verbatim, but extraction phase
problematic due to VM layer

●
Circumvent VM layer without lowering its security?

●
VM Trapdooring: constant (secret) seed when generating
components
Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
 Branch-based Fingerprints
handler table

00 jmp 35 vm_mov_reg_imm
01 jmp 07 vm_mov_reg_reg
02 jmp 08 0040645A
0 1
... ... 0040699E
1
07 jmp target vm_mov_reg_imm
08 jmp 00 0040640A
0 ... ... 1
...
0
12 jmp 24 vm_mov_reg_reg
... ...
1
23 ...
24 jmp 02 ...
... ...
35 jmp 01
VM code

virtualized code encoding fingerprint 0b1010101

intercept handler execution

jmp target (IA-32)

verify vIP update verify VM sequence

mov_reg_imm tmp, target track target immediate track dst register

mov_reg_reg vIP, tmp

vm_mov_reg_imm observer vm_mov_reg_reg observer

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
FPs based on Encoding Choice

●
Handler Duplication: duplicate handler code

●
Multiple handlers encode same semantics
●
Multiple opcodes per virtual instruction
●
We have a choice when encoding bytecode

●
Approach here: Group equivalent handlers and assign
values to each member in a group (cf. Monden et al.)

●
Every encoded virtual instruction embeds a few bits
based on the handler it chooses
●
Embed mark in all emitted instructions

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
FPs based on Encoding Choice

handler table bytecode

enc.
bits
opcode semantics opcode parameters

00 00 vm_mov_reg_imm ?? 0x0f00 0xdeadbeef

... 01 vm_mov_reg_reg ?? 0xbeef 0x0f00

... 02 vm_add_reg_imm ??
... 03 vm_and_reg_reg ?? 0x0f00 0xcafebabe
01 04 vm_mov_reg_imm ?? 0xdead 0x0f00

10 05 vm_mov_reg_imm ?? 0xbeef 0xdead

... 06 vm_add_reg_imm ?? 0xb00b 0x1badf00d

... ... ... ??
11 FF vm_mov_reg_imm ?? 0xdead 0x0f00

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
 Conclusion

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
Conclusion

●
Schemes draw from resilience provided by VM

●
Exploit specific VM traits, tied to VM layer

●
Comes at the cost of increased time/space complexity

●
Refrain from protecting performance-critical sections

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
Bibliography

●
Robert I. Davidson and Nathan Myhrvold. Method and system for generating and
auditing a signature for a computer program, September 24 1996. US Patent
5,559,884.

●
Cullen Linn, Saumya Debray, and John Kececioglu. Enhancing Software Tamper-
Resistance via Stealthy Address Computations. In Proceedings of the 19th Annual
Computer Security Applications Conference (ACSAC 2003). Citeseer, 2003.

●
Akito Monden, Hajimu Iida, K-i Matsumoto, Katsuro Inoue, and Koji Torii. A Practical
Method for Watermarking Java Programs. In Computer Software and Applications
Conference, 2000. COMPSAC 2000. The 24th Annual International, pages 191-197.
IEEE, 2000.

●
Christian Collberg and Jasvir Nagra. Surreptitious Software. Upper Saddle River, NJ:
Addision-Wesley Professional, 2010.

●
Patrick Cousot and Radhia Cousot. An Abstract Interpretation-Based Framework for
Software Watermarking. In ACM SIGPLAN Notices, volume 39, pages 173-185. ACM,
2004.

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9
 Thank you for your attention!

Any questions?

@dwuid

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9