BHUSA2015 Unicorn

Unicorn: Next Generation CPU Emulator Framework
www.unicorn-engine.org
NGUYEN Anh Quynh <aquynh -at- gmail.com>

DANG Hoang Vu <danghvu -at- gmail.com>
BlackHat USA, August 5th 2015
1 / 39 NGUYEN Anh Quynh, DANG Hoang Vu Unicorn: Next Generation CPU Emulator Framework
Self-introduction
Nguyen Anh Quynh (aquynh -at- gmail.com)

I PhD in Computer Science, security researcher
I Operating System, Virtual Machine, Binary analysis, Forensic, etc
I Capstone disassembly framework (capstone-engine.org)
Dang Hoang Vu (danghvu -at- gmail.com)

I PhD candidate in Computer Science at UIUC, security hobyist
I Member of VNSecurity.NET, casual CTF player, exploit writer
I Capstone, Peda contributor
Agenda
1 CPU Emulator
Background
Problems of existing CPU emulators
2 Unicorn engine: demands, ideas, design & implementation

Goals of Unicorn
Design & implementation
Write applications with Unicorn API
3 Live demo
4 Conclusions
CPU Emulator
Definition
Emulate physical CPU - using software only.
Focus on CPU operations only, but ignore machine devices.
Applications
Emulate the code without needing to have a real CPU.
I Cross-architecture emulator for console game.
Safely analyze malware code, detect virus signature.
Verify code semantics in reversing.
Example
Emulate to understand code semantics.
Internals of CPU emulator
Given input code in binary form

Decode binary into separate instructions
Emulate exactly what each instruction does
I Instruction-Set-Architecture manual referenced is needed
I Handle memory access & I/O upon requested
Update CPU context (regisers/memory/etc) after each step
Example of emulating X86 32bit instructions
Ex: 50 → push eax

I load eax register
I copy eax value to stack bottom
I decrease esp by 4, and update esp
Ex: 01D1 → add eax, ebx

I load eax & ebx registers
I add values of eax & ebx, then copy result to eax
I update flags OF, SF, ZF, AF, CF, PF accordingly
Challenges of building CPU emulator
Huge amount of works!

Good understanding of CPU architecture
Good understanding of instruction set
Instructions with various side-effect (sometimes undocumented, like
ex: Intel X86)
Tough to support all kind of code existed
Good CPU emulator?
Multi-arch?
I X86, Arm, Arm64, Mips, PowerPC, Sparc, etc
Multi-platform?
I *nix, Windows, Android, iOS, etc
Updated?
I Keep up with latest CPU extensions
Independent?
I Support to build independent tools
Good performance?
I Just-In-Time (JIT) compiler technique vs Interpreter
Existing CPU emulators
Features libemu PyEmu IDA-x86emu libCPU Dream

Multi-arch X X X X1 X
Updated X X X X X
Independent X2 X3 X4 X X
JIT X X X X X
Multi-arch: existing tools only support X86

Updated: existing tools do not supports X86_64
1
Possible by design, but nothing actually works
2
Focus only on detecting Windows shellcode
3
Python only
4
For IDA only
Dream a good emulator
Multi-architectures
I Arm, Arm64, Mips, PowerPC, Sparc, X86 (+X86_64) + more
Multi-platform: *nix, Windows, Android, iOS, etc
Updated: latest extensions of all hardware architectures
Independent with multiple bindings
I Low-level framework to support all kind of OS and tools
I Core in pure C, and support multiple binding languages
Good performance with JIT compiler technique
I Dynamic compilation vs Interpreter
Allow instrumentation at various levels
I Single-step/isntruction/memory access
Problems
No reasonable CPU emulator even in 2015!

Apparently nobody wants to fix the issues
No light at the end of the dark tunnel
Until Unicorn was born!
Unicorn == Next Generation CPU Emulator
Goals of Unicorn
Multi-architectures
I Arm, Arm64, Mips, PowerPC, Sparc, X86 (+X86_64) + more
Multi-platform: *nix, Windows, Android, iOS, etc
Updated: latest extensions of all hardware architectures
Core in pure C, and support multiple binding languages
Good performance with JIT compiler technique
Allow instrumentation at various levels
I Single-step/instruction/memory access
Unicorn vs others
Features libemu PyEmu IDA-x86emu libCPU Unicorn

Multi-arch X X X X X
Updated X X X X X
Independent X X X X X
JIT X X X X X
Multi-arch: existing tools only support X86

Updated: existing tools do not supports X86_64
Challenges to build Unicorn engine
Huge amount of works!

Too many hardware architectures
Too many instructions
Instructions with various side-effect (sometimes undocumented, like
Intel X86)
Hard to to support all kind of code existed
Limited resource
I Started as a personal for-fun in-spare-time project
Unicorn design
Ambitions & ideas
Have all features in months, not years!

Stand on the shoulders of the giants at the initial phase.
Open source project to get community involved & contributed.
Idea: Qemu!
Introduction on Qemu
Qemu project
Open source project (GPL license) on system emulator:
http://www.qemu.org
Huge community & highly active
Multi-arch
I X86, Arm, Arm64, Mips, PowerPC, Sparc, etc (18 architectures)
Multi-platform
I Compile on *nix + cross-compile for Windows
Qemu architecture
Courtesy of cmchao
Why Qemu?
Support all kind of architectures and very updated

Already implemented in pure C, so easy to immplement Unicorn core
on top
Already supported JIT in CPU emulation
Are we done?
Challenges to build Unicorn (1)
Qemu codebase is a challenge

Not just emulate CPU, but also device models & ROM/BIOS to fully
emulate physical machines
Qemu codebase is huge and mixed like spaghetti :-(
Difficult to read, as contributed by many different people
Unicorn job
Keep only CPU emulation code & remove everything else (devices,
ROM/BIOS, migration, etc)
Keep supported subsystems like Qobject, Qom
Rewrites some components but keep CPU emulation code intact (so
easy to sync with Qemu in future)
Qemu is set of emulators

Set of emulators for individual architecture
I Independently built at compile time
I All archs code share a lot of internal data structures and global
variables
Unicorn wants a single emulator that supports all archs :-(
Unicorn job
Isolated common variables & structures
I Ensured thread-safe by design
Refactored to allow multiple instances of Unicorn at the same time
Modified the build system to support multiple archs on demand
Qemu has no instrumentation

Instrumentation for static compilation only
JIT optimizes for performance with lots of fast-path tricks, making
code instrumenting extremely hard :-(
Unicorn job
Build dynamic fine-grained instrumentation layer from scratch
Support various levels of instrumentation
I Single-step or on particular instruction (TCG level)
I Intrumentation of memory accesses (TLB level)
I Dynamically read and write register or memory during emulation.
I Handle exception, interrupt, syscall (arch-level) through user provided
callback.
Qemu is leaking memory

Objects is open (malloc) without closing (freeing) properly everywhere
Fine for a tool, but unacceptable for a framework
Unicorn job
Find and fix all the memory leak issues
Refactor various subsystems to keep track and cleanup dangling
pointers.
Unicorn vs Qemu
Forked Qemu, but go far beyond it

Independent framework
Much more compact in size, lightweight in memory
Thread-safe with multiple architectures supported in a single binary
Provide interface for dynamic instrumentation
More resistant to exploitation (more secure)
I CPU emulation component is never exploited!
I Easy to test and fuzz as an API.
Qemu vulnerabilities
Write applications with Unicorn
Introduce Unicorn API
Clean/simple/lightweight/intuitive architecture-neutral API.

The core provides API in C
I open & close Unicorn instance
I start & stop emulation (based on end-address, time or instructions
count)
I read & write memory
I read & write registers
I memory management: hook memory events, dynamically map memory
at runtime
F hook memory events for invalid memory access
F dynamically map memory at runtime (handle invalid/missing memory)
I instrument with user-defined callbacks for
instructions/single-step/memory event, etc
Python binding built around the core
Sample code in C
Sample code in Python
Live demo
Status & future works
Status
Support Arm, Arm64, Mips, M68K, PowerPC, Sparc, X86 (+X86_64)
Python binding available
Based on Qemu 2.3
Future works
Support all the rest architectures of Qemu
(alpha/s360x/microblaze/sh4/etc - totally 18)
Stripping more ultility code from Qemu e.g. improve the disassembler
(with potential integration with Capstone).
More bindings promised by community!
Synchronize with Qemu 2.4 (released soon)
I Future of Unicorn is guaranteed by Qemu active development!
Conclusions
Unicorn is an innovative next generation CPU emulator

I Multi-arch + multi-platform
I Clean/simple/lightweight/intuitive architecture-neutral API
I Implemented in pure C language, with bindings for Python available.
I High performance with JIT compiler technique
I Support fine-grained instrumentation at various levels.
I Thread-safe by design.
I Open source GPL license.
I Future update guaranteed for all archs.
We are seriously committed to this project to make it the best CPU
emulator.
Call for beta testers
Run beta test before official release

Willing to help? If you can code, contact us!
I Unicorn homepage: http://www.unicorn-engine.org
I Unicorn twitter: @unicorn_engine
I Unicorn mailing list:
http://www.freelists.org/list/unicorn-engine
First public version to be released after the beta phase - in GPL
license.
Questions and answers
Unicorn: Next Generation CPU Emulator Framework
NGUYEN Anh Quynh <aquynh -at- gmail.com>
DANG Hoang Vu <danghvu -at- gmail.com>
References
Qemu: http://www.qemu.org
libemu: http://libemu.carnivore.it
PyEmu: http://code.google.com/p/pyemu
libcpu: https://github.com/libcpu/libcpu
IDA-x86emu: http://www.idabook.com/x86emu/index.html
Unicorn engine
I Homepage: http://www.unicorn-engine.org
I Mailing list: http://www.freelists.org/list/unicorn-engine
I Twitter: @unicorn_engine
Acknowledgement
Nguyen Tan Cong for helped with the shellcode demo!

Other beta testers helped to improve our code!

BHUSA2015 Unicorn

Uploaded by

Copyright:

Available Formats

BHUSA2015 Unicorn

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

BHUSA2015 Unicorn

Uploaded by

Copyright:

Available Formats

Unicorn: Next Generation CPU Emulator Framework

NGUYEN Anh Quynh <aquynh -at- gmail.com>

BlackHat USA, August 5th 2015

Nguyen Anh Quynh (aquynh -at- gmail.com)

Dang Hoang Vu (danghvu -at- gmail.com)

2 Unicorn engine: demands, ideas, design & implementation

Given input code in binary form

Ex: 50 → push eax

Ex: 01D1 → add eax, ebx

Huge amount of works!

Features libemu PyEmu IDA-x86emu libCPU Dream

Multi-arch: existing tools only support X86

No reasonable CPU emulator even in 2015!

Features libemu PyEmu IDA-x86emu libCPU Unicorn

Multi-arch: existing tools only support X86

Huge amount of works!

Have all features in months, not years!

Support all kind of architectures and very updated

Qemu codebase is a challenge

Qemu is set of emulators

Qemu has no instrumentation

Qemu is leaking memory

Forked Qemu, but go far beyond it

Clean/simple/lightweight/intuitive architecture-neutral API.

Unicorn is an innovative next generation CPU emulator

Run beta test before official release

NGUYEN Anh Quynh <aquynh -at- gmail.com>

DANG Hoang Vu <danghvu -at- gmail.com>

Nguyen Tan Cong for helped with the shellcode demo!

You might also like