Hacking Erlang: Building Strange and Magical Creations
Hacking Erlang: Building Strange and Magical Creations
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
Things Worth Trying:
•code injection
•meta programming
•reverse engineering byte code
•anything that makes Ericsson cringe...
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
Step 1
understanding the abstract format
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
The Abstract Format
• a tree-like structure representing parsed Erlang code
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
The Abstract Format
• a tree-like structure representing parsed Erlang code
• comprised of a list of forms
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
The Abstract Format
• a tree-like structure representing parsed Erlang code
• comprised of a list of forms
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
The Abstract Format
• a tree-like structure representing parsed Erlang code
• comprised of a list of forms
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
The Abstract Format
• a tree-like structure representing parsed Erlang code
• comprised of a list of forms
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
The Abstract Format
• a tree-like structure representing parsed Erlang code
• comprised of a list of forms
form
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
The Abstract Format
• a tree-like structure representing parsed Erlang code
• comprised of a list of forms
form
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
The Abstract Format
• a tree-like structure representing parsed Erlang code
• comprised of a list of forms
form
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
The Abstract Format
• a tree-like structure representing parsed Erlang code
• comprised of a list of forms
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
The Abstract Format
• a tree-like structure representing parsed Erlang code
• comprised of a list of forms
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
Scanning Source Code
the first step in compiling
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
Scanning Source Code
the first step in compiling
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
Scanning Source Code
the first step in compiling
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
erl_scan
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
erl_scan
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
token
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
token
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
token
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
erl_parse
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
erl_parse
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
erl_parse
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
erl_parse
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
compile
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
compile
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
compile
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
compile
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
compile
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
dynamic_compile
The dynamic_compile module performs
the actions we’ve just seen, plus takes care
of macro expansion and inclusion of
external header files.
http://github.com/JacobVorreuter/dynamic_compile
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
dynamic_compile
moving on...
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
the parse_transform
debate...
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
Programmers
are strongly advised
NOT to engage in parse
transformations
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
yeah, you can do
everything with macros
anyway
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
wait! parse_transforms are
cool and have their place in the
language...in moderation.
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
How do parse_transforms work?
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
How do parse_transforms work?
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
How do parse_transforms work?
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
a pizza example
encode pizza
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
a pizza example
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
a pizza example
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
a pizza example
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
expand_records.erl
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
intermission
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
Act II
compiling custom syntax
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
Compiling Custom Syntax
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
Compiling Custom Syntax
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
Compiling Custom Syntax
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
leex
The leex module takes a definition file with the
extension .xrl as input and generates the source
code for a lexical analyzer as output.
<Header>
Definitions.
<Macro Definitions>
Rules.
<Token Rules>
Erlang Code.
<Erlang Code>
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_scanner.xrl
<Header>
Definitions.
<Macro Definitions>
Rules.
<Token Rules>
Erlang Code.
<Erlang Code>
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_scanner.xrl
Definitions.
A = [a-z][0-9a-zA-Z_]*
I = [0-9]+
WS = ([\000-\s]|%.*)
Rules.
<Token Rules>
Erlang Code.
<Erlang Code>
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_scanner.xrl
Definitions.
A = [a-z][0-9a-zA-Z_]*
I = [0-9]+
WS = ([\000-\s]|%.*)
Rules.
\➸
:
{token,{module,TokenLine}}.
\➽
:
{token,{function,TokenLine}}.
\✓
:
{token,{'->',TokenLine}}.
\◤
:
{token,{'[',TokenLine}}.
\◥
:
{token,{']',TokenLine}}.
{A} : {token,{atom,TokenLine,list_to_atom(TokenChars)}}.
{I} : {token,{integer,TokenLine,list_to_integer(TokenChars)}}.
\➟
:
{token,{'<-',TokenLine}}.
\✈
:
{token,{'||',TokenLine}}.
\❤
:
{token,{heart,TokenLine}}.
\✂{WS}
:
{end_token,{dot,TokenLine}}.
{WS}+ : skip_token.
Erlang Code.
<Erlang Code>
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_scanner.xrl
Definitions.
A = [a-z][0-9a-zA-Z_]*
I = [0-9]+
WS = ([\000-\s]|%.*)
Rules.
\➸
:
{token,{module,TokenLine}}.
\➽
:
{token,{function,TokenLine}}.
\✓
:
{token,{'->',TokenLine}}.
\◤
:
{token,{'[',TokenLine}}.
\◥
:
{token,{']',TokenLine}}.
{A} : {token,{atom,TokenLine,list_to_atom(TokenChars)}}.
{I} : {token,{integer,TokenLine,list_to_integer(TokenChars)}}.
\➟
:
{token,{'<-',TokenLine}}.
\✈
:
{token,{'||',TokenLine}}.
\❤
:
{token,{heart,TokenLine}}.
\✂{WS}
:
{end_token,{dot,TokenLine}}.
{WS}+ : skip_token.
Erlang code.
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_scanner.xrl
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
yecc
The yecc module takes a BNF* grammar
definition as input, and produces the source
code for a parser.
<Header>
<Non-terminals>
<Terminals>
<Root Symbol>
<End Symbol>
<Erlang Code>
* Backus–Naur Form (BNF) is a metasyntax used to express context-free grammars: that is, a formal way to describe formal languages
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
<Header>
<Non-terminals>
<Terminals> The header
<Root Symbol> provides a chance
<End Symbol> to add
<Erlang Code> documentation
before the module
declaration in your
parser
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
Header "%% Copyright (C)"
"%% @Author Jacob Vorreuter"
We could do
<Non-terminals> something like
<Terminals> this, but
<Root Symbol>
whatever
<End Symbol>
<Erlang Code>
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
<Non-terminals>
<Terminals>
<Root Symbol>
<End Symbol>
<Erlang Code>
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
<Non-terminals>
<Terminals> Terminal symbols are literal
<Root Symbol> strings forming the input of a
<End Symbol> formal grammar and cannot be
<Erlang Code> broken down into smaller units
without losing their literal
meaning
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
<Non-terminals>
Terminals atom integer heart module function '[' ']' '->' '<-' '||'.
<Root Symbol>
<End Symbol>
<Erlang Code>
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
<Non-terminals>
Terminals atom integer heart module function '[' ']' '->' '<-' '||'.
<Root Symbol>
<End Symbol>
<Erlang Code> These terminal symbols are the
products of the regular expressions
in our lexical analyzer
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
<Non-terminals>
Terminals atom integer heart module function '[' ']' '->' '<-' '||'.
<Root Symbol>
<End Symbol> Nonterminal symbols are the rules
<Erlang Code> within the formal grammar consisting
of a sequence of terminal symbols or
nonterminal symbols. Nonterminal
symbols may self reference to specify
recursion.
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
Nonterminals element module_declaration function_declaration function_body
comprehension.
Terminals atom integer heart module function '[' ']' '->' '<-' '||'.
<Root Symbol>
<End Symbol>
<Erlang Code>
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
Nonterminals element module_declaration function_declaration function_body
comprehension.
Terminals atom integer heart module function '[' ']' '->' '<-' '||'.
<Root Symbol> Here we are declaring symbols
<End Symbol> that will be further defined as
<Erlang Code> descendants of the root symbol
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
Nonterminals element module_declaration function_declaration function_body
comprehension.
Terminals atom integer heart module function '[' ']' '->' '<-' '||'.
<Root Symbol>
<End Symbol> The root symbol is the most
<Erlang Code> general syntactic category
which the parser ultimately will
parse every input string into.
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
Nonterminals element module_declaration function_declaration function_body
comprehension.
Terminals atom integer heart module function '[' ']' '->' '<-' '||'.
Rootsymbol element.
element -> module_declaration : '$1'.
element -> function_declaration : '$1'.
module_declaration -> module atom :
{attribute,line_of('$2'),module,value_of('$2')}.
function_declaration -> function atom '->' function_body :
{function,line_of('$2'),value_of('$2'),0,[{clause,line_of('$2'),[],[],'$4'}]}.
function_body -> comprehension : ['$1'].
comprehension -> '[' ']' : nil.
comprehension -> '[' integer '<-' integer '||' heart ']' :
{lc,line_of('$2'),{var,line_of('$2'),'A'},[{generate,line_of('$2'),
{var,line_of('$2'),'A'},
{call,line_of('$2'),{remote,line_of('$2'),{atom,line_of('$2'),lists},
{atom,line_of('$2'),seq}},['$2','$4']}}]}.
<End Symbol>
<Erlang Code>
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
Nonterminals element module_declaration function_declaration function_body
comprehension.
Terminals atom integer heart module function '[' ']' '->' '<-' '||'.
Rootsymbol element.
element -> module_declaration : '$1'.
element -> function_declaration : '$1'.
module_declaration -> module atom :
{attribute,line_of('$2'),module,value_of('$2')}.
function_declaration -> function atom '->' function_body :
{function,line_of('$2'),value_of('$2'),0,[{clause,line_of('$2'),[],[],'$4'}]}.
function_body -> comprehension : ['$1'].
comprehension -> '[' ']' : nil.
comprehension -> '[' integer '<-' integer '||' heart ']' :
{lc,line_of('$2'),{var,line_of('$2'),'A'},[{generate,line_of('$2'),
{var,line_of('$2'),'A'},
{call,line_of('$2'),{remote,line_of('$2'),{atom,line_of('$2'),lists},
{atom,line_of('$2'),seq}},['$2','$4']}}]}.
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
Nonterminals element module_declaration function_declaration function_body
comprehension.
Terminals atom integer heart module function '[' ']' '->' '<-' '||'.
Rootsymbol element.
element -> module_declaration : '$1'.
element -> function_declaration : '$1'.
module_declaration -> module atom :
{attribute,line_of('$2'),module,value_of('$2')}.
function_declaration -> function atom '->' function_body :
{function,line_of('$2'),value_of('$2'),0,[{clause,line_of('$2'),[],[],'$4'}]}.
function_body -> comprehension : ['$1'].
comprehension -> '[' ']' : nil.
comprehension -> '[' integer '<-' integer '||' heart ']' :
{lc,line_of('$2'),{var,line_of('$2'),'A'},[{generate,line_of('$2'),
{var,line_of('$2'),'A'},
{call,line_of('$2'),{remote,line_of('$2'),{atom,line_of('$2'),lists},
{atom,line_of('$2'),seq}},['$2','$4']}}]}.
Endsymbol dot.
<Erlang Code>
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
Nonterminals element module_declaration function_declaration function_body
comprehension.
Terminals atom integer heart module function '[' ']' '->' '<-' '||'.
Rootsymbol element.
element -> module_declaration : '$1'.
element -> function_declaration : '$1'.
module_declaration -> module atom :
{attribute,line_of('$2'),module,value_of('$2')}.
function_declaration -> function atom '->' function_body :
{function,line_of('$2'),value_of('$2'),0,[{clause,line_of('$2'),[],[],'$4'}]}.
function_body -> comprehension : ['$1'].
comprehension -> '[' ']' : nil.
comprehension -> '[' integer '<-' integer '||' heart ']' :
{lc,line_of('$2'),{var,line_of('$2'),'A'},[{generate,line_of('$2'),
{var,line_of('$2'),'A'},
{call,line_of('$2'),{remote,line_of('$2'),{atom,line_of('$2'),lists},
{atom,line_of('$2'),seq}},['$2','$4']}}]}.
Endsymbol dot. The Erlang code section
can contain any functions
<Erlang Code> that we need to call from
our symbol definitions
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
Nonterminals element module_declaration function_declaration function_body
comprehension.
Terminals atom integer heart module function '[' ']' '->' '<-' '||'.
Rootsymbol element.
element -> module_declaration : '$1'.
element -> function_declaration : '$1'.
module_declaration -> module atom :
{attribute,line_of('$2'),module,value_of('$2')}.
function_declaration -> function atom '->' function_body :
{function,line_of('$2'),value_of('$2'),0,[{clause,line_of('$2'),[],[],'$4'}]}.
function_body -> comprehension : ['$1'].
comprehension -> '[' ']' : nil.
comprehension -> '[' integer '<-' integer '||' heart ']' :
{lc,line_of('$2'),{var,line_of('$2'),'A'},[{generate,line_of('$2'),
{var,line_of('$2'),'A'},
{call,line_of('$2'),{remote,line_of('$2'),{atom,line_of('$2'),lists},
{atom,line_of('$2'),seq}},['$2','$4']}}]}.
Endsymbol dot.
Erlang code.
value_of(Token) -> element(3, Token).
line_of(Token) -> element(2, Token).
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example_parse.yrl
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
example4.erl
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
custom syntax in
the wild...
• Lisp Flavored Erlang
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter
END
http://jacobvorreuter.com/hacking-erlang http://github.com/JacobVorreuter