A Matlab-Like Environment For Machine Learning
A Matlab-Like Environment For Machine Learning
development of numerical algorithms, (2) it should be easily extended (including the use of other
libraries), and (3) it should be fast.
We found that a scripting (interpreted) language with a good C API appears as a convenient solution
to “satisfy” the constraint (2). First, a high-level language makes the process of developing a
program simpler and more understandable than a low-level language. Second, if the programming
language is interpreted, it becomes also easier to quickly try various ideas in an interactive manner.
And finally, assuming a good C API, the scripting language becomes the “glue” between heterogeneous
libraries: different structures of the same concept (coming from different libraries) can
be hidden behind a unique structure in the scripting language, while keeping all the functionalities
coming from all the different libraries.
Among existing scripting languages1 finding the ones that satisfy condition (3) severely restricted
our choice. We chose Lua, the fastest interpreted language (with also the fastest Just In Time (JIT)
compiler2) we could find. Lua as also the advantage to have been designed to be easily embedded in
a C application, and provides a great C API, based on a virtual stack to pass values to and from C.
This unifies the interface to C/C++ and makes library wrapping trivial.
Lua is intended to be used as a powerful, light-weight scripting language for any program that
needs one. Lua is implemented as a library, written in clean C (that is, in the common subset of
ANSI C and C++). Quoting Lua webpage3,
1For e.g. on http://shootout.alioth.debian.org.
2http://luajit.org/
3http://www.lua.org.
1
Lua combines simple procedural syntax with powerful data description constructs
based on associative arrays and extensible semantics. Lua is dynamically typed,
runs by interpreting bytecode for a register-based virtual machine, and has automatic
memory management with incremental garbage collection, making it ideal
for configuration, scripting, and rapid prototyping.
Lua offers good support for object-oriented programming, functional programming, and data-driven
programming. Lua’s main type is the table, which implements associative arrays in a very efficient
manner. An associative array is an array that can be indexed not only with numbers, but also with
strings or any other value of the language. Tables have no fixed size, can be resized dynamically,
and can be used as “virtual tables” over another table, to simulate various object-oriented paradigms.
Tables are the only data structuring mechanism in Lua4, yet a powerful one. We use tables to
represent ordinary arrays, symbol tables, sets, records, queues, and other data structures, in a simple,
uniform, and efficient way. Lua uses tables to represent packages as well.
Why not Python? It is hard to talk about a language without starting a flame war. While Lua is
well known in the gaming programmer community (because of its speed advantage and great embedding
capabilities), Python is more popular in a more general public. With no doubt, Python ships
with more libraries. However, with no doubt5, “Integrating Lua with C is so easy a child could do
it. Lua was designed for this also, from the beginning, and it shows6. This means that with a few
hours’ work, any C or C++ library can become a Lua library.”. Another key advantage of Lua is its
embedding capabilities: once code has been prototyped, it can be turned into a final system/product
with very little extra work. Extra performance can be obtained using LuaJIT, yielding C-like performance
for most of the pure Lua code. Lua being written in pure ANSI C, it can be easily compiled
for arbitrary targets (cell-phones, embedded CPUs in FPGAs, DSP processors, ...). Adding Lua’
speed advantage, the choice was a “no brainer”.
Lua satisfied our initial constraints (2) and (3). In the next section, we show how we satisfied
constraint (1), that is, how we made easy to develop numerical algorithms, by developing a Tensor
object, which serves as a “glue brick” between all our library interfaces.
Torch5 is our previous version of Torch7, and was already leveraging Lua, as we do in Torch7. In
contrast, Torch7 allows the user to switch easily (on the fly) between floating types (float, doubles,