Maltego

TRX
Writing Python transforms (for use with the TDS)
RT

TableofContents
)ntroduction why use the TDS? ......................................................................................................................................
What is TRX? .........................................................................................................................................................................

Architecture of the TDS .........................................................................................................................................................

Developing transforms for the TDS .................................................................................................................................

Publically available TDS ...................................................................................................................................................

)nternal/private TDS .........................................................................................................................................................

The future of the TDS .............................................................................................................................................................

Functionality of the NTDS ...............................................................................................................................................

Pricing model of the NTDS ..............................................................................................................................................

Development environments ...............................................................................................................................................

P(P ...........................................................................................................................................................................................

Python .....................................................................................................................................................................................

Prepping a server for use with the TDS Python ......................................................................................................

)nstall Apache ....................................................................................................................................................................

)nstall mod_wsgi ..................................................................................................................................................................

)nstall bottle ..........................................................................................................................................................................

Copy & extract TRX files ...................................................................................................................................................

Edit Apache configuration ...............................................................................................................................................

)nstall TRX files ....................................................................................................................................................................

Restart Apache ..................................................................................................................................................................

Setting up the TDS to use the example transform DNS )P .................................................................................

Setting up a seed .................................................................................................................................................................

Setting up the transform ................................................................................................................................................

Setting up Maltego to use the sample transform .................................................................................................

Under the hood .......................................................................................................................................................................

Debugging transforms .........................................................................................................................................................

Changing to the bottle server ..................................................................................................................................

Deploying transforms with Apache ......................................................................................................................

Multiple seeds ...............................................................................................................................................................

Components of TRX ..............................................................................................................................................................

debugTRX_Server.py .......................................................................................................................................................

Transform library e.g. DNSTRANSFORMS.py ....................................................................................................

TRX

Page2

Maltego library maltego.py .......................................................................................................................................

Using TRX ..................................................................................................................................................................................

Basic use ...............................................................................................................................................................................

Using Display )nfo .............................................................................................................................................................

Link properties, bookmarks and notes ....................................................................................................................

Entity properties ...............................................................................................................................................................

Static vs. dynamic properties ..................................................................................................................................

A quick note on entity design ..................................................................................................................................

Calculated properties .................................................................................................................................................

Property types ...............................................................................................................................................................

Reading properties ...........................................................................................................................................................

Adding dynamic properties ..........................................................................................................................................
Using transform settings ...............................................................................................................................................

)ntroduction ...................................................................................................................................................................
Transform setting types ............................................................................................................................................

Using transform settings in TRX ............................................................................................................................

Conclusion ...........................................................................................................................................................................

Entity reference ......................................................................................................................................................................

V /V Confusion ...............................................................................................................................................................
Calculated properties ......................................................................................................................................................

)nheritance ..........................................................................................................................................................................

Why are we stuck with V properties? ....................................................................................................................

Entity reference guide ....................................................................................................................................................

The TRX Maltego.py AP).................................................................................................................................................

MaltegoMsg class ..............................................................................................................................................................

MaltegoTransform class .................................................................................................................................................

MaltegoEntity class ..........................................................................................................................................................

TRX

Page3

IntroductionwhyusetheTDS?
There are several ways to add custom transforms to Maltego. One option is to use local transforms.
Local transforms are an easy way to get going but has several drawbacks in the long term:

. Code base needs to be installed on local machine.

a. )n many cases this means installing the development environment on local machine
think Python, P(P, PERL .
b. This makes it difficult to install on the client PC and means more moving parts.
c. Every transform needs to be manually registered on the Maltego GU) or Maltego
configuration files needs to be patched. While this is feasible it might not be
backward compatible with future releases of Maltego.
d. The code needs to be maintained for every installation of Maltego. A change in the
transform code means it has to be reinstalled on each Maltego client..
. Limited functionality / access to Maltego features. The following features are not exposed to
local transforms:
a. Slider value the user cannot limit how many results are returned.
b. Transform settings cannot access any transform settings. Transform settings are
the popups asking the user to configure a transform prior to running it.
c. Notes / link labels / bookmarking. Access to these fields will likely be removed from
local transforms in future.
d. Meta data such as disclaimers etc.
. When transforms depend on rd party software these need to be installed and maintained
across all clients. )t is not always feasible to install this type of software on each type of
platform e.g. if their Operating Systems are different .
. No access control or auditing. Future releases of the TDS will include access control and
auditing.
. Transforms such as SocialNet from Packetninjas are provided as a commercial product
as pay per transform. With local transforms it's more difficult for transform writers to offer
their transforms as a service.
. (ard to control intellectual property. As transforms run locally on the client it becomes
difficult to protect the code against reverse engineering.
. Paterva currently has no plans to provide future enhancements.

Most of these limitations can be somehow bypassed but these shortcuts are clunky, hardly
maintainable and not elegant at all.

(aving said that local transforms have the advantage that:

. They are simple to write in any language.

. They can interact with software that requires the use of a GU). Think here of Skype that s
difficult to use headless.
. )s infrastructureless it can run straight off your workstation.
. Data does not travel over the wire everything is contained in a single workstation.

TRX

Page4

Local transforms are a good start for proof of concept code, rapid development or where
interaction with GU) based software is required but in enterprise models or more serious
deployments it does not scale well or provide the necessary control mechanisms. These limitations
of local transforms are well known to Paterva. The more elegant solution to the problem would be
to centralize the transforms on a server and provide access to run them remotely, similar to
running the builtin transforms. Such a solution has been implemented using the Transform
Distribution Server TDS .

WhatisTRX?

TRX is a framework although this perhaps a bit presumptuous that facilitates coding Python
transforms for Maltego using the TDS. )t assumes the transform writer has access to of a web facing
Ubuntu server. )t uses Apache with mod_WSG), Bottle and Python.

ArchitectureoftheTDS
The following diagram shows how the Maltego clients, the transforms themselves and the TDS fit
together:

TRX

Page5

The TDS acts like a proxy for transforms. )t hides several of the complexities of seed management,
entity metadata, transform settings, transform discovery etc. from the transform writer and allows
him/her to concentrate on developing the actual transform. The transform code is hosted on the
end user s infrastructure and is served from a web server.

Configuration of transforms, seeds, transform settings and metadata is performed via a web
interface on the TDS.

DevelopingtransformsfortheTDS
There are currently two ways to develop transforms for the TDS.

PublicallyavailableTDS

This is a TDS that is located on the )nternet and is free for all to use. )t s a convenient way to
immediately start writing transforms. Since this server is located on Paterva s infrastructure data
will be flowing from the Maltego GU) to this server and finally to your transform code hosted on a
web server of your choice. Note that your transforms need to be hosted on an )nternet facing
server. This is not the preferred way to deploy transforms in a production environment especially
if the transform content is sensitive but it allows developers to get a feel for how the system works.
The public TDS requires registration and transforms and seeds are defined per user. The server
interface lives at https://cetas.paterva.com/TDS/

Internal/privateTDS

The internal TDS functions in the same way as the public TDS but can be hosted on the enduser s
private infrastructure. )t is delivered as Virtual Machine VMWare image and is commercial.
Several high profile companies use the TDS in its current format.
Clients with a valid license for the TDS will automatically be upgraded to the NTDS see next
section .

ThefutureoftheTDS
The TDS was first released in
and has seen very little changes since then. )nitially Paterva was
reluctant to sell the TDS to clients but since mid
have lifted this restriction. Several high
profile clients now use the TDS to centralize and distribute their custom enterprise wide
transforms. This renewed attention to the TDS caused us to rethink the future of this server.

FunctionalityoftheNTDS

The following key areas will be developed for the NTDS New TDS :

TRX

. Distribution of elements including:

a. Transforms currently the only functionality of the TDS
b. Entity definitions
c. Transform sets

Page6

d. Machine scripts
e. Viewlets
. Access control and auditing
. Transform stats, system health
. Future proof for implementation of protocol .

PricingmodeloftheNTDS

Currently the TDS when used as a private server is sold at a fixed price but with unlimited amount
of transforms and users. This all or nothing model has proved problematic for small enterprises
that wish to share a handful of transforms amongst a small number of users. )n future the NTDS
pricing model will be based on users and transforms and should be priced more attractively for
smaller enterprises.
The public TDS will also be upgraded to the new NTDS and will remain free for use.

Developmentenvironments
PHP
An extensive P(P library is provided for the TDS. The library and documentation can be found here:
https://www.paterva.com/web /documentation/developertds.php#

Python

The rest of this document focuses around the Python development environment.

)n the past Paterva only provided a CG) based Python library. This is easy to work with but has
obvious limitations for every transform that runs an instance of Python was started. This can
easily drain resources on the host and is not very efficient. The more efficient method is using
WSG).

PreppingaserverforusewiththeTDS(Python)
We recommend using Ubuntu Server with Apache , Python . , mod_wsgi and bottle. Assuming a
stock standard Ubuntu server with Python installed the following is a recipe for quickly building
your environment:

InstallApache2

sudo apt-get update

sudo apt-get install apache2

Installmod_wsgi
sudo apt-get install libapache2-mod-wsgi

Installbottle
sudo apt-get install python-setuptools
TRX

Page7

sudo easy_install bottle

Copy&extractTRXfiles
cd /tmp
wget http://www.paterva.com/TRX_Ubuntu.tgz
tar -xvzf TRX_Ubuntu.tgz

EditApacheconfiguration
Edit the file /etc/apache2/ports.conf and add the line:
Listen 9001

Add it just below the line that reads Listen 80 so the file looks like so:

NameVirtualHost *:80
Listen 80
Listen 9001

Copy the TRX Apache configuration file:

sudo cp /tmp/etc.apache2.sites-available/TRX /etc/apache2/sitesavailable/

Now enable the site:

sudo a2ensite TRX

InstallTRXfiles
sudo mkdir /var/www/TRX
cd /var/www/TRX
sudo cp /tmp/var.www.TRX.tgz .
sudo tar -xvzf var.www.TRX.tgz
An example transform DNS )P is provided it s a function defined in DNSTRANSFORMS.py, but
more about that later . )n the next section we will see how to configure the TDS to use this
transform.

RestartApache2

sudo /etc/init.d/apache2 restart

Feel free to inspect the configuration of the TRX site defined in /etc/apache2/sitesavailable and customize this to your liking. The configuration will route all traffic on port

to the WSG) script TRX.wsgi located in /var/www/TRX.
TRX

Page8

SettinguptheTDStousetheexampletransform(DNS2IP)
Next we are going to configure the TDS to use the sample DNS2IP transform. Start by browsing to
the TDS. )f you are using an internal/private TDS it will ask you for your SSL certificate provided by
Paterva. On the server portal you ll find instructions on how to install the supplied client certificate
and use it.
)f you are using the public TDS server you need to first log in with the user/password used at
registration and complete that horrible reCAPTC(A .
Once authenticated to the TDS you ll see three sections in the interface:

Settingupaseed

A seed is really just a URL that points to a group of transforms. )t s this URL that Maltego will use to
load all of the transforms contained in the seed into its internal configuration. Thus, the first step
we want to do is create a new seed container . Once we ve done that we ll create a new transform
and put the transform inside that seed.
Click on Seeds . You ll see a list of seeds if you have any . Click on the Add Seed button . You need
to give a seed name, a seed URL and select which transforms you want to add into the seed. You can
complete the form as follows:

When you re done, click on the Add Seed button.

TRX

Page9

Now our seed is configured. )t s empty there are no transforms in the seed, but that s OK we ll
populate it in a bit.

Settingupthetransform

You can easily navigate around with the bread > crumb at the top of the screen:

Click on the very first item it will navigate to the root of the TDS.

Click on Transforms on the internal/private TDS you see a screen with a couple of P(P sample
transforms. On the public TDS there won t be any transforms configured unless you ve added them
there of course!

We want to add a transform, so click on Add Transform . You are now going to configure the
transform. (ere is a section of that webpage:

TRX

Page10

Change the fields as follows:

Transform name:

http://<Your_DevServer_Here>:9001/DNS2IP

)nput Entity:

maltego.DNSName from the dropdown

Description:

My first sample transform or get creative

Debug:

Checked

Seeds:

Click on MyTDSSeed to highlight.

Transform URL:
Disclaimer:
Version:

Transform settings:

DNS )P

Leave empty unless you feel like putting something there

Leave as is we ll get to this later

)f everything worked you should see Successfully I nsert ed 'DNS2I P' at t he t op of t he screen:

We now successfully added a transform to the seed and can use this seed in Maltego.

SettingupMaltegotousethesampletransform

We now want to tell Maltego to discover transforms from our seed. Don t worry most of this is a
onetime configuration and once it s all set up we never have to touch it again.

We re lazy and so we are going to copy and paste the seed from the TDS. Go to Seeds and right click
on the URL of the seed. Copy it to your clipboard:

TRX

Page11

Open Maltego. )nside of Maltego go to the Manage tab and click on the bottom of Discover
Transforms . You ll see a dropdown. Select Advanced :

The following screen should appear:

TRX

Page12

(opefully yours would not be that blurry . )n the Name section type MyTDS it could be anything
you want and in the URL section paste the URL from the clipboard. Click on the Add button and
you ll see it appears in the list of Seeds below:

Now all that s left is to discover the seed. You may choose to uncheck your other seeds it won t do
you any harm, but it will ensure that you just discover from the TDS seed. Follow the wizard next
>next .
Finally, you ll see:
TRX

Page13

Don t worry about the new entities were installed . That s a legacy bug actually it s not really,
but that s an entire other chapter . Now you are ready to use the transform.

Open a new graph, drag a DNSName from the palette to the graph and right click. Voila your
transform shows up complete with description :

The transform is fully functional and will resolve the DNS name to an )P address:

TRX

Page14

)f you want to share this transform with anyone you can simply give them the URL and guide them
through the discovery process in the same way as which you did. And if you change the code on
the server the change will transparently happen on all clients as the transform runs on the server .
You can now add more transforms to the seed. Maltego periodically checks for new transforms in
the seeds which mean you never need to touch the client side anymore. Joy!

Underthehood
Let s see what really happened here. These are the steps:

After you discovered from the TDS server Maltego knows that there s an additional
transform called DNS2IP available on the entity type Maltego.DNSName.
When you right click on a DNS Name it makes this transform visible
)t sends the transform request to the TDS server
The TDS server knows that the transform really lives on
http://<Your_DevServer_Here>:9001/DNS2IP
The TDS makes a connection to port
on the dev server and sends some XML in a POST
6) The Apache WSG) configuration on the dev server send all requests on port
to a file
located in /var/www/TRX/TRX.wsgi
TRX.wsgi sees that it s destined for /DNS2IP and routes it to the right piece of code
The Maltego Python library interprets the XML to something that s easy to work with in
Python and passes it along
A DNS lookup is done
The reply is wrapped back into XML by the library and the data flows back the way it came
in.

Seems like a mouthful. The actual transform code is quite simple and looks like this:

TRX

Page15

def trx_DNS2IP(m):
TRX = MaltegoTransform()
DNSName=None
try:
DNSName = socket.gethostbyname(m.Value)
TRX.addEntity("maltego.IPv4Address",DNSName)
except socket.error as msg:
TRX.addUIMessage("Problem:"+str(msg),UIM_PARTIAL)
return TRX.returnOutput()

(ardly rocket science.

Debuggingtransforms
Changingtothebottleserver
One of the painful realities of working with WSG) is that you need to restart the Apache server
every time you make a change to the code. There are some scripts on the )nternet available to make
this easier but this method of coding is altogether unintuitive for most coders. Luckily there s a way
around it.

Bottle has its own web server built in and this server can be set to reload whenever a change in
code happens this goes for changes to the stub as well as any modules/libraries that it depends
on. Let s look a little closer.

Go to /var/www/TRX. )n here you ll see a file called debugTRX_Server.py. This file is a near
exact copy of TRX.wsgi but it can run as a standalone server. You ll see that we ve uncommented
the bottle (TTP server and commented the WSG) server at the end of the script . )n this case it s
listening on the )P . . .
and port
. Before you can run this script you need to disable
the Apache server because it is grabbing port
:
/etc/init.d/apache2 stop

Now you can start the sub optimal, but great for debugging server:
sudo python debugTRX_Server.py
)t starts up as follows:

Bottle v0.11.6 server starting up (using WSGIRefServer())...

Listening on http://0.0.0.0:9001/
Hit Ctrl-C to quit.
What s great about this server is that it automatically reloads every time you make a change to the
code. Better still, it verbosely complains about any mistake you might have made in your code.
(ere s a screenshot of that happening:
TRX

Page16

)n the example above we made a silly mistake in a transform by using m.slider and not
m.Slider note the difference in case . The bottle server complained bitterly about it. We fixed
the code and as soon as we saved the file the server reloaded with the changes. )f you wondering
we had our server listen on . . .
and not on . . . .
DeployingtransformswithApache
The main reason why we don t want to run the bottle server in production is because it s not
optimized for heavy load and we don t want to have to struggle with startup scripts.

Once you are happy with your transforms you should stop the bottle server, simply uncomment the
first couple of lines, change the server to run as a WSG) and start Apache again:
/etc/init.d/apache2 start

Multipleseeds
Another way to do this is to run the bottle server on a different port. You might want to add another
seed DEV ? and register the transforms on this port. Once you want to move it into production
you can simple change the port and insert it into the production seed.

ComponentsofTRX
There are basically three files that are of interest. All of them are located in /var/www/TRX/.

debugTRX_Server.py

This is the dispatcher TRX.wsgi and debugTRXServer.py is essentially the same script the
former used with Apache, the latter when debugging . Let s look at the file in a little bit more detail:

TRX

Page17

Each transform has a small router stub in here that defines the URL of the transform and to which
function it routes to. The request.body.getvalue() reads the entire POST from the message.
This XML is sent to the function MaltegoMsg which is defined in the Maltego library. The
function returns an object which can be used to easily extract all the information from the XML.
This object is passed along to the actual transform. The transform does its work and returns XML
this is returned back to the TDS. We ll look into the transform code in more detail shortly.

)n the example we ve included the library DNSTRANSFORMS . This is the transform code library
e.g. where the transform itself is defined. )n the router for transform DNS )P case it s called
trx_DNS2IP. Of course, as you go along you may choose to add more libraries.

Transformlibrary(e.g.DNSTRANSFORMS.py)
You can include as many libraries as you want. For our example we used DNSTRANSFORMS.py.
Let s take a look at what s really happening:

TRX

Page18

This is the code that actually performs the work of the transform. As input it gets an object that
contains all the details about the request this is called m in this case. Next, it creates a vessel that
will eventually contain the response here it s called TRX . )t then does the DNS lookup and adds
an entity of type maltego.IPv4Address with the value of the lookup to the vessel. )t also adds
some UIMessages. Finally the transform returns the vessel s XML to the router which in turn
sends it back to the TDS.

Maltegolibrary(maltego.py)

This is the library that shuffles XML to object and back. You are more than welcome to optimize,
tinker with this library should you feel inclined to do so. But keep in mind that should we change
the library you ll need to redo any changes you ve made to it.

UsingTRX
Basicuse
Let s look at a couple of examples. You might want to refer to the TRX reference guide at the end of
this document every now and again to follow we are using the library.

Let s say we want to create a transform that runs on a Phrase entity. )t will assume that the phrase
is a number and it will return as many entities and just as a test we ll return these as AS number
entities.
Let s begin really simple:

Our stub looks like this in TRX.wsgi or debugTRX_Server.py :

@route('/EnumAS', method='ANY')
def EnumAS():
if request.body.len>0:
return(trx_EnumAS(MaltegoMsg(request.body.getvalue())))

Our transform looks like this in DNSTRANSFORMS.py too lazy to put in another library :
def trx_EnumAS(m):
TRX = MaltegoTransform()
howmany = int(m.Value)
for i in range(1,howmany+1):
TRX.addEntity('maltego.AS', str(i))
return TRX.returnOutput()

We call it EnumAS on the TDS and point it to a URL http://server:9001/EnumAS .

Remember that if you used Apache as a server you will need to restart the server to register the
changes (/etc/init.d/apache2 reload . )f you used the bottle server and it is running you
shouldn t have to do a thing.
TRX

Page19

(ere is the output:

This code is really nasty. What happens if we put the value

in the graph? What if it s not
really a number but a phrase like test ? So, as a start, let s make sure we don t go over the limit set
by the user using the result slider:

We change the code to read the slider:

def trx_EnumAS(m):
TRX = MaltegoTransform()
howmany = int(m.Value)
if (howmany > m.Slider):
howmany=m.Slider
for i in range(1,howmany+1):
TRX.addEntity('maltego.AS', str(i))
return TRX.returnOutput()

Now we at least honor the user s wishes on how many entities to return. Next, we want to make
sure it s a number and if it s not, return a nice message to the user saying he/she is not
understanding the transform. The code now changes to this:
def trx_EnumAS(m):
TRX = MaltegoTransform()

TRX

Page20

if (not m.Value.isdigit()):
TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole
number',UIM_PARTIAL)
return TRX.returnOutput()
#here we know we're good to go.
howmany = int(m.Value)
if (howmany > m.Slider):
howmany=m.Slider
for i in range(1,howmany+1):
TRX.addEntity('maltego.AS', str(i))
return TRX.returnOutput()

Right! When we now run our transform on an entity that s not a number we get the following:

)t seems that everything is mostly sorted out, but there s one more thing. You ll notice that the
entities seem to be laid out on the graph in a random fashion. Maltego actually lays out entities
according to their weight top left to bottom right . Since we didn t set the weight all the entities
have the same weight the default of
. Let s see if we can fix this:
def trx_EnumAS(m):
TRX = MaltegoTransform()

if (not m.Value.isdigit()):
TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole
number', UIM_PARTIAL)
return TRX.returnOutput()
#here we know we're good to go.
howmany = int(m.Value)
if (howmany > m.Slider):
howmany=m.Slider
for i in range(1,howmany+1):
Ent = TRX.addEntity('maltego.AS', str(i))
Ent.setWeight(howmany-i)
return TRX.returnOutput()

TRX

Page21

(ere you see that the addEntity method actually returns a MaltegoEntity object which we can
modify. )n this case we use it to set the weight and now the graph looks as follows:

The first node with value has a weight of and the last node
has a weight of . You can
see this in the Property View. The following screenshot shows the property view when the AS node
with value was selected:

UsingDisplayInfo

Let s next assume we want to create some fancy (TML in the Detail View to go with our AS entities.
The addDisplayInformation method is used to do this. The content could be plain text or
(TML. Let s start with something really simple. We want to display the words This is number X on
each entity returned. We want this to be rendered in a label called AS Number . (ere is how the
script looks now:
TRX

Page22

def trx_EnumAS(m):
TRX = MaltegoTransform()
if (not m.Value.isdigit()):
TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole
number',UIM_PARTIAL)
return TRX.returnOutput()
#here we know we're good to go.
howmany = int(m.Value)
if (howmany > m.Slider):
howmany=m.Slider
for i in range(1,howmany+1):
Ent = TRX.addEntity('maltego.AS', str(i))
Ent.setWeight(howmany-i)
Ent.addDisplayInformation("This is number "+str(i),"AS
Number")
return TRX.returnOutput()

When you run the transform you ll see that each node rendered the label and content:

You can easily add more labels. Display information is merged when other transforms creates the
same entity instance. Maltego will expand the label by default, but Detail View labels can be
collapsed and expanded by the user and the client will remember the choice for future operations.
TRX

Page23

Let s see how it works with (TML. Let s assume we want to make a clickable link in (TML. We will
create the link so that it is dynamic and based on the node number. Consider the following code:
def trx_EnumAS(m):
TRX = MaltegoTransform()

if (not m.Value.isdigit()):
TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole
number',UIM_PARTIAL)
return TRX.returnOutput()
#here we know we're good to go.
howmany = int(m.Value)
if (howmany > m.Slider):
howmany=m.Slider
for i in range(1,howmany+1):
Ent = TRX.addEntity('maltego.AS', str(i))
Ent.setWeight(howmany-i)
Ent.addDisplayInformation("This is number "+str(i),"AS
Number")
Ent.addDisplayInformation('Click <a
href="https://www.ultratools.com/tools/asnInfoResult?domainName=AS'+st
r(i)+'"> here </a> to get more information about this AS','AS Info
Link')
return TRX.returnOutput()
Now each node has a link that will open the browser to a specific webpage:

TRX

Page24

Clicking on the links gives us:

)t important to note that display information is never sent to the server when you run subsequent
transforms on the node it s read only info and is kept in the client.

Linkproperties,bookmarksandnotes

You can set the link color, label, thickness and style really easily. Consider this code snippet used in
the same transform . We are not going to discuss each method in detail because it s rather obvious
from the code and the resultant graph:
if (i%2==0):
Ent.setLinkColor('0x00FF00')

TRX

Page25

Ent.setNote('Even')
Ent.setLinkLabel('Even link')
Ent.setLinkStyle(LINK_STYLE_NORMAL)
Ent.setLinkThickness(1)
Ent.setBookmark(BOOKMARK_COLOR_GREEN)
else:
Ent.setLinkColor('0xFF0000')
Ent.setNote('Odd')
Ent.setLinkLabel('Odd link')
Ent.setLinkStyle(LINK_STYLE_DASHED)
Ent.setLinkThickness(2)
Ent.setBookmark(BOOKMARK_COLOR_RED)

Although we recommend you keep it a little simpler this code illustrates the concept. The resultant
graph ugly looks like this layout in organic mode else it s REALLY messy :

The only thing to keep in mind is that entity notes stack on top of each other that means that
when a subsequent transform writes to a note it will append to the note rather than deleting the
previous note.
TRX

Page26

Entityproperties
Entities can have additional properties. For example a person might have a property called SSN .

Staticvs.dynamicproperties
When you design an entity you ll see that you can add additional properties to the entity. These are
called static properties. )t means that should a user drag an entity from the palette to the graph
these properties will be available for editing. As an example, here is the property view for Person:

Transforms can create new properties for entities they return. The transform simply adds a
property to the entity when it s returned. These are called dynamic properties. )f a transform
returns a property that had been defined as static in the entity definition it will store the value as a
static property, if it s not defined in the entity definition it then becomes dynamic.

The only real difference between static and dynamic properties is that users cannot edit dynamic
properties on a fresh node from the palettesimply because the property does not exist yet a
transform needed to create it.
Aquicknoteonentitydesign
One of the most important questions to ask yourself when doing entity design is if data should be
stored in a property of an entity or if it should become a new entity type. Similarly you should
consider if some information should only be display information or stored as a property within an
entity.

Usually you can answer these questions as follows. )f you can think of a transform that will
regularly use a property of an entity it would be better to make it a separate entity type. For
instance it would be better to have SSN as an entity type and have a Person to SSN transform than
to only have SSN as a property in the Person entity you might still store the SSN as a property

TRX

Page27

within the Person entity, but it makes sense to have it as a separate entity too . Similarly if you
cannot decide if something should be a property or simply display info consider if another
transform would need to read that property to work properly or if it s only shown to the user for
information because display info is never passed back to the server . )f it s just useful as display
info don t bloat the entity by adding it as a property.
Calculatedproperties
)n some cases properties are calculated from other properties. )n the case of a person the property
FullName is created from FirstName and LastName. This does not mean you cannot set the
property, but you don t need to. For a full list of entity names, properties and calculated properties
refer to the Entity definition reference at the end of this document.

Propertytypes
The current version of the Python library only supports string types but subsequent releases might
extend this to include all the other types of properties that Maltego support. When dealing with
entities that have property types other than strings, make sure you format the string value properly
so that the Maltego client can parse the correct type from the string value.

Readingproperties

This is fairly straightforward. Consider the Website entity. )t has a static property called ports with
a default of see Entity definition reference for a list of static entity properties . )nside of Maltego
this is really defined as a list of integers but we ll see how we deal with it in TRX.

Let s start a new transforms. We ll call it Mangle , register it on the TDS running on a website
entity and in our TRX.wsgi stub.
Our transform code looks like this:
def trx_Mangle(m):

#define response
TRX = MaltegoTransform()
TRX.addUIMessage("Property value is:"+m.getProperty("ports"))
TRX

Page28

return TRX.returnOutput()
We use the getProperty method to get the property s value. You will see that we do not return
any entities that s
% OK , we simply write a message to the output with the Output screen. On
a default website entity the output looks like this:

)f we add a port to the list

and run the transform again we get:

)n other words the list is returned as a comma delimited list.

Addingdynamicproperties

Let s see how we can write properties using TRX. Each property has

TRX

Displayname
Name
Value
Matching rule
merged

:
:
:
:

how it s rendered in Maltego

the actual name used in code
he value of the property
if entities with the same value, type and property value should be
Page29

Let s assume we want to create a transform that runs on an )P address and generates a netblock. )t
will always assume a class C netblock
)Ps but it will add a dynamic property to the netblock to
indicate if it s a private netblock or not. We ll do this with a dynamic property called private and it
will be set to yes or no .
We start by building a stub in TRX.wsgi for Apache deployment or debugTRX_Server.py
when debugging and registering our transform on the TDS. We begin our transform without input
validation it s not the point like this:
def trx_NetblocksRUs(m):
TRX=MaltegoTransform()

start=m.Value[0:m.Value.rfind('.')]
netblock=start+".0-"+start+".255"
Ent = TRX.addEntity('maltego.Netblock')
Ent.setValue(netblock)
return TRX.returnOutput()
Note that at this point we haven t added any dynamic properties to the netblock. We need to first
have something that checks if an )P address is private or not. A quick search on the )nternet finds
this useful piece of code:
>>> from IPy import IP
>>> ip = IP('10.0.0.1')
>>> ip.iptype()
'PRIVATE'

This function relies on a Python module called IPy. On Ubuntu installing this is as simple as:
sudo apt-get install python-ipy

This seems perfect for our transform. We add a line importing this library to our
DNSTRANSFORMS.py:
from IPy import IP
And the transform now looks like this:
def trx_NetblocksRUs(m):
TRX=MaltegoTransform()

start=m.Value[0:m.Value.rfind('.')]
netblock=start+".0-"+start+".255"
Ent = TRX.addEntity('maltego.Netblock')
ip = IP(m.Value)
TRX

Page30

if (ip.iptype()=='PRIVATE'):
Ent.addProperty('private','Private network','strict','yes')
else:
Ent.addProperty('private','Private network','strict','no')
Ent.setValue(netblock)
return TRX.returnOutput()
Now running the transform on two different )P addresses we get:

Perfect! Of course we can create a transform that uses netblocks as input and simply outputs a
phrase public or private this is not very useful, but hopefully will show how to use additional
properties .
def trx_PublicPrivate(m):
TRX=MaltegoTransform()
Ent = TRX.addEntity('maltego.Phrase')
if m.getProperty('private')=='yes':
Ent.setValue('Private')
else:
Ent.setValue('Public')
return TRX.returnOutput()

TRX

Page31

The resultant graph when populated with some )Ps>Netblock>Public/Private and graph set to
Bubble view block layout :

Usingtransformsettings
Introduction
Sometimes you want the user to be able to send you information about how you should run your
transform. These could be settings that could change every time the transform is executed, or could
be settings that the user wants to customize and have Maltego remember. We call these transform
inputs or settings.
Almost all of the standard transforms have settings most of the time users don t really bother to
change them because the defaults are working as expected. A good example of this can be seen in
the reverse DNS transform where the (TTP timeout to Serversniff and Robtex can be configured:

TRX

Page32

To get to the transform manager click on the Manage ribbon and go to Manage transforms . Any
transform setting can be configured to popup. This is useful when the user would want to change
the setting prior to the transform running. Such is the case with )P to Netblock natural
boundaries . The user might choose to select class C
or sub class Cs
, , or perhaps
even class Bs
.
This transform setting looks as follows in the manager:

TRX

Page33

The small dialog icon next to the setting indicates that this transform will pop up a setting prior to
executing the transform:

The user might choose to fix the value for subsequent transforms by checking the Remember
these settings checkbox. )n order to get the transform to pop up the setting dialog again the Popup
checkbox needs to be selected in the transform manager.
TRX

Page34

To quickly get to each transform s settings you can click on the small gear in the context menu:

Transformsettingtypes
The Maltego GU) client supports many different popup types integer, dates, lists of strings and
integers but the TDS currently only support the use of strings. This will be extended in the NTDS.

UsingtransformsettingsinTRX
Now that we know what transform settings are, let s see how to use them with custom transforms.

Because transform settings are prompted for prior to the transform running their definitions are
stored in the transform itself they are loaded when the transforms are discovered. This means that,
should you change transform settings, you need to rediscover from the seed so that their definitions
are updated. Luckily background discovery makes this really painless:

TRX

Page35

Transform settings are defined on the TDS and can be reused across any number of transforms.
The TDS only supports string based transform settings, but this might change in the near future.
Basically a transform setting requires a
Name

Display Dialog :

As used in the code

(ow the user is asked for it

Default value :

)nitially setting will have this value choose carefully

Optional

)s the setting optional or mandatory?

Popup state :

Will it pop up?

The current specification says that a transform setting will only pop up if it s mandatory and there
is no default value. This is something Paterva is aware of and will fix in the NTDS.

Let s go back to our initial transform the one with the phrase as a number and ASes as results .
We want to add a transform setting that will obtain a number prior to the transform running and
only return AS numbers that are divisible by that setting if that sounds silly, it is, but again
hopefully is shows the concept . First off we need to define a transform setting on the TDS. We go to
the root, click on Settings and click Add setting . We fill out the form as such:

Our variable will be called ISDIV. Once you ve added the transform setting click on Add Transform
Setting . Navigate to transforms, click on the EnumAS transform which we created at the start of
this document and scroll down. At Transform Settings highlight the )SD)V item you can shift
click to select multiple settings :
TRX

Page36

This tells the TDS that you wish to use the )SD)V transform setting in your transform. Transforms
could have multiple settings and the same settings can be used on multiple transforms.

Next let s look what s needed in the code. Our previous EnumAS ended making a graph that
looked like pea soup so we ll start with fresh code:
def trx_EnumAS(m):
TRX = MaltegoTransform()

#read the value

if (not m.Value.isdigit()):
TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole
number',UIM_PARTIAL)
return TRX.returnOutput()
#read the setting
isdiv = m.getTransformSetting('ISDIV')
if (not isdiv.isdigit()):
TRX.addUIMessage('Silly! We need a number',UIM_FATAL)
return TRX.returnOutput()

###here we know we're good to go.

howmany = int(m.Value);
# how many have accumulated?
accum=0;
for i in range(1,howmany+1):
if (i % int(isdiv) == 0):
Ent = TRX.addEntity('maltego.AS', str(i))
Ent.setWeight(howmany-i)
accum=accum+1;
if accum>=m.Slider:
break

TRX

Page37

return TRX.returnOutput()

Let s see what happens when we run the transform. First up we get a popup asking us some
questions:

)n the code we do:

#read the setting

isdiv = m.getTransformSetting('ISDIV')
if (not isdiv.isdigit()):
TRX.addUIMessage('Silly! We need a number',UIM_FATAL)
return TRX.returnOutput()
This checks if the value is indeed a digit if it s not we complain bitterly with UIM_FATAL :

)t s important to remember that the TDS only supports strings. As such we need to convert our
string to an integer after checking that it really is one! and make sure we give the right amount of
nodes back to the user.
TRX

Page38

The output of the transform is as expected:

Conclusion

The combination of all of these components makes Maltego very flexible and powerful in the hands
of a crafty developer. TRX makes it very easy to wield this type of power and the developer is
properly shielded from any of the Maltego internal complexities.

TRX

Page39

Entityreference
V2/V3Confusion
With the release of Maltego version around
a new data model was developed. This was
done to ensure scalability of entities. Where Maltego v just knew about a Person we started to
understand that other users would want to perhaps create their own Person entity. Thus we
moved to maltego.Person. The same principle was applied to properties. Where we had
firstname in V we now understood that we should have person.firstname. Furthermore
the model was built in such a way that entities could have any number of properties but that there
would be a mapping between any of these properties and what s displayed underneath the entity in
the graph the display value . A good example of this is the maltego.Document entity it makes
more sense to display the title of the document rather than the URL where the document is located.
As such the display value for a Document entity is the title. This can be easily seen on the Display
Settings when editing an entity within the Maltego client. (ere the maltego.Document entity s
Display Settings is shown:

)n the same way the value that s edited when the user clicks on the entity and the )conURL can be
mapped to any property of the entity.

Calculatedproperties

Another concept that was introduced in version was the use of calculated properties. A person s
fullname for instance is calculated by the concatenation of the firstnames and the lastname.
This is exposed in the Maltego client:

TRX

Page40

The only entities that use calculated properties are

maltego.Person
maltego.Location
maltego.PhoneNumber

Inheritance
CaseFile offered many more entities than Maltego. )n CaseFile you can have a Judge, Criminal and
Officer that are essentially all Persons. When importing a graph made in CaseFile into Maltego you
would want to be able to run the Person transforms on all of these but the early data model did not
support it.

We added the concept of inheritance for the standard Maltego installation this meant that the
MXRecord, NSRecord and Website entities were really just specialized DNSNames. The upside of it
is that one transform DNSName )PAddress worked on all of them this saved a lot of transform
configuration. For example if you specify on the TDS that a transform will run on a DNSName it
will also run on all entities down the family tree MXRecord, Website and NSRecord.
At the top of the tree is maltego.Unknown. This means that if you configure a transform to run
on this base entity type it will be available when you right click on any entity.

WhyarewestuckwithV2properties?

The trouble with this new data model was that the server was still expecting the old property
names for incoming values and was still supplying the old property names as results. With many
different versions of Maltego in the wild changing the server to use the new properties would mean
that many clients would have a nonfunctional Maltego client.
TRX

Page41

As such a conversion layer was implemented that mapped the old property names internally to the
new names. )t meant that when setting the value of a property it would internally be setting a
specific property of the entity. )t also meant that when the GU) supplied entities to the server it
would convert the properties to the old names.

This worked fine until other developers started to code transforms. The Maltego GU) exposed the
internal property names and there was no way to know what these properties translated to when it
was supplied to the transforms. That is unless you followed the specification that was last
updated in
.

As such we provide here both the internal V property names as well as their V equivalents.
When coding transforms it is best to stick with the V equivalents. )t would be trivial to change the
server CTAS to use the new properties but it will almost certainly result in some clients stuck with
a nonfunctional GU). )t seems that these legacy properties are here to stay.
When working with your own custom entities this legacy problem is a nonissue of course. )t only
comes into play when you want to leverage the transforms that are located on the CTAS then you
need to make sure you are mapping to the right V equivalent properties.

Entityreferenceguide

maltego.

Domain
PropertyName
fqdn
whois-info

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
string

Domainname
WHOISinfo

V2equivalent
Value
whois

maltego.

DNSName
PropertyName
fqdn

Inheritsfrom:
maltego.Unknown
Type

Displayname

string

DNSName

V2equivalent
Value

maltego.

MXRecord
PropertyName
fqdn
TRX

Inheritsfrom:
Maltego.DNSName
Type

Displayname

string

MXRecord

V2equivalent
Value
Page42

mxrecord.priority

integer

Priority

mxrecord.priority

maltego.

Inheritsfrom:
Maltego.DNSName
Type

Displayname

string

MXRecord

V2equivalent
Value

NSRecord
PropertyName
fqdn

maltego.

IPv4Address
PropertyName
ipv4-address
ipaddress.internal

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
boolean

IPAddress
Internal

V2equivalent
Value
ipaddress.internal

maltego.

Netblock
PropertyName
ipv4-range

Inheritsfrom:
maltego.Unknown
Type

Displayname

string

IPRange

V2equivalent
Value

maltego.

AS
PropertyName
as.number

Inheritsfrom:
maltego.Unknown
Type

Displayname

integer

ASNumber

V2equivalent
Value

maltego.

Website
PropertyName
fqdn
TRX

Inheritsfrom:
maltego.DNSName
Type

Displayname

string

Website

V2equivalent
Value
Page43

website.ssl-enabled
ports

boolean
int[]

SSLEnabled
Ports

website.ssl-enabled
ports

maltego.

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
URL
string

ShortTitle
Value
Title

V2equivalent
Value
theurl
fulltitle

URL
PropertyName
short-title
url
title

maltego.

Phrase
PropertyName
text

Inheritsfrom:
maltego.Unknown
Type

Displayname

string

Text

V2equivalent
Value

maltego.

Document
PropertyName
url
Title
document.meta-data

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
String
String

URL
Title
MetaData

V2equivalent
link
Value
metainfo

maltego.

Person
PropertyName
*
person.fullname
+
person.firstnames
+
person.lastname

TRX

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
string
string

FullName
FirstNames
Surname

V2equivalent
Value
firstname
lastname

Page44


maltego.

EMailAddress
PropertyName
email

Inheritsfrom:
maltego.Unknown
Type

Displayname

string

EmailAddress

V2equivalent
Value

maltego.

Location
PropertyName
*
location.name
+
country
+
city
location.area
countrycode
longitude
latitude

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
string
string
string
string
float
float

Name
Country
City
Area
CountryCode
Longitude
Latitude

V2equivalent
Value
country
city
area
countrysc
long
lat

maltego.

PhoneNumber
PropertyName
*
Phonenumber
+
phonenumber.countrycode
+
phonenumber.citycode
+
phonenumber.areacode
+
phonenumber.lastnumbers

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
string
string
string
string

PhoneNumber
CountryCode
CityCode
AreaCode
LastDigits

V2equivalent
Value
countrycode
citycode
areacode
lastnumbers

maltego.

Alias
PropertyName
alias

Inheritsfrom:
maltego.Unknown
Type

Displayname

string

Alias

V2equivalent
Value

TRX

Page45

maltego.

Inheritsfrom:
maltego.Unknown

Image
PropertyName
description
url

Type

Displayname

string
URL

Description
URL

V2equivalent
Value
url

maltego.

Inheritsfrom:
maltego.Unknown

Twit
PropertyName
twit.name
id
author
author_uri
content
img_link
pubdate
title

Type

Displayname

string
string
string
string
string
string
string
string

Twit
TwitID
Author
AuthorURI
Content
ImageLink
Datepublished
Title

V2equivalent
value
id
author
author_uri
content
img_link
pubdate
title

maltego.affiliation.

Twitter
PropertyName
person.name
affiliation.network
affiliation.uid
affiliation.profile-url
twitter.number
twitter.screen-name
twitter.friendcount
person.fullname

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
string
string
string
int
string
int
string

Name
Network
UID
ProfileURL
TwitterNumber
ScreenName
FriendCount
RealName

V2equivalent
value
network
uid
affiliation.profile-url
twitter.number
twitter.screen-name
twitter.friendcount
person.fullname

* Calculated properties

+ Used to calculate property

TRX

Page46

TheTRXMaltego.pyAPI
MaltegoMsgclass
This is used to read the Maltego request. )t is passed along to each transform.
Member name
Value String

Weight )nteger
Slider )nteger
Type String

Properties List

TransformSettings List

Description
The value of the node as displayed on the graph. Note
that this is not necessarily the value you want work with
see URL entity .
The weight of the node.
The slider s value e.g. how many results should be
returned.
The type of the input node. See entity definitions for
possible values.
A list of properties of the node. Name, value pairs as
strings
The settings for the transform. A list of name, value pairs
as strings.

The following methods are defined to read entity properties and transform settings. You can read it
straight out of the Property and TransformSettings list, but it s nicer to use these functions
as they do some error checking:
Method name
getProperty
String key)
getTransformSetting
String key)

Description
Returns
Returns the value of the key, Value
None if not defined
Returns the value of the key, Value
None if not defined

MaltegoTransformclass
This is used to construct the reply to the TDS. All values are strings.
Method name
addEntity
([String Type,
String Value])
addUIMessage
(String msg,
Const type)
addException
(String msg)
TRX

Description
Adds an entity to the return vessel with type
Type and value Value . Note that these can be
set using functions in MaltegoEntity library.
Shows a message msg in the Maltego GU).
Types could be
U)M_FATAL pop up window
U)M_PART)AL yellow
U)M_)NFORM default
U)M_DEBUG light gray
Throws a transform exception

Returns
MaltegoEntity
none

none
Page47

returnOutput()

MaltegoEntityclass

Returns the XML of the vessel

none

This is the object that defines a single entity within Maltego. An entity can be created using the
addEntity method in the MaltegoTransform class.

Method name
setType

(String Type)
setValue
(String Value)
setWeight
()nteger weight)
addDisplayInformation
(String Value,
[String Label])
addProperty
(String propertyname,
String displayname,
String matchingrule
String value)
setIconURL
(String URL)
setLinkColor
(String HexColor)
setLinkStyle
(Const Style)

setLinkThickness
)nteger thickness)
setLinkLabel
String value)
setBookmark
Const Color)

setNote
String note)
TRX

Description
Sets the type of the entity see list of Entity definitions for possible
values
Sets the value of the entity
Sets the weight of the entity

Adds display information for label named Label . This field is

rendered as (TML within Maltego. Default label is )nfo .

Adds a property to the entity. Each property has a name, value and a
display name. The display name is how it will be represented within
Maltego. The matching rule determines how entities will be matched
and could be strict default or loose

)f set it will change the appearance of the icon. The URL should point
to a PNG or JPG file. Maltego will size to fit but lots of large files will
drain resources.
Sets the color of the link to the node. Colors are in hex for example
xff ff
Sets the style of the link to the node. The following constants should
be used:
L)NK_STYLE_NORMAL
L)NK_STYLE_DAS(ED
L)NK_STYLE_DOTTED
L)NK_STYLE_DAS(DOT
Sets the thickness of the link to the node. Value is in pixels.
Sets the label of the link to the node.

Sets the bookmark color of the node. Keep in mind that these are
chosen from a set number of colors. Use the following constants:
BOOKMARK_COLOR_NONE
BOOKMARK_COLOR_BLUE
BOOKMARK_COLOR_GREEN
BOOKMARK_COLOR_YELLOW
BOOKMARK_COLOR_ORANGE
BOOKMARK_COLOR_RED

Creates an annotation to the node with value note . )f a subsequent
transform sets an annotation on the node it will appended to the
note.

Page48

TRX

Page49