TRX Transform Library Guide

This page provides explanations and examples of the methods that are available in the TRX Transform library.

This page provide explanations and examples of the methods that are available in the TRX Transform library.

Basic use

Let’s look at a couple of examples. You might want to refer to the TRX reference guide  every now and again as we go along.

Let’s say we want to create a transform that runs on a Phrase Entity. It will assume that the phrase is a number and it will return that number of Entities. Just as a test we’ll return these as AS number Entities.

Let’s begin with a simple example:

Our stub looks like this (in TRX.wsgi or debugTRX_Server.py):

@route('/EnumAS', method='ANY') 
def EnumAS(): 
    if request.body.len>0:
        return(trx_EnumAS(MaltegoMsg(request.body.getvalue())))

Our Transform looks like this:

def trx_EnumAS(m):
    TRX = MaltegoTransform() 
    howmany = int(m.Value) 
    for i in range(1,howmany+1):
        TRX.addEntity('maltego.AS', str(i)) 
    return TRX.returnOutput()

We call it ‘EnumAS’ on the iTDS and point it to a URL https://server:9001/EnumAS . Remember that if you used an Apache server you will need to restart the server to register the changes (/etc/init.d/apache2 reload). If you used the bottle server and it is running you shouldn’t have to do a thing.

Here is the output:

This code is really nasty. What happens if we put the value ‘250000’ in the graph? What if it’s not really a number but a phrase like ‘test’? So, as a start, let’s make sure we don’t go over the limit set by the user using the result slider:

We change the code to read the slider:

def trx_EnumAS(m):
    TRX = MaltegoTransform()

    howmany = int(m.Value)

    if (howmany > m.Slider):
        howmany=m.Slider

    for i in range(1,howmany+1):
        TRX.addEntity('maltego.AS', str(i))
    return TRX.returnOutput()

Now we at least honor the user’s wishes on how many entities to return. Next, we want to make sure it’s a number and if it’s not, return a nice message to the user saying he/she is not understanding the transform. The code now changes to this:

def trx_EnumAS(m): 
    TRX = MaltegoTransform() 
     if (not m.Value.isdigit()): 
        TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole number',UIM_PARTIAL)
        return TRX.returnOutput() 
    #here we know we're good to go. 
    howmany = int(m.Value) 
     if (howmany > m.Slider):
         howmany=m.Slider 
     for i in range(1,howmany+1): 
        TRX.addEntity('maltego.AS', str(i))
    return TRX.returnOutput() 

Right! When we now run our transform on an entity that’s not a number we get the following:

It seems that everything is mostly sorted out, but there’s one more thing. You’ll notice that the entities seem to be laid out on the graph in a random fashion. Maltego actually lays out entities according to their weight (top left to bottom right). Since we didn’t set the weight all the entities have the same weight – the default of 100. Let’s see if we can fix this:

def trx_EnumAS(m): 
    TRX = MaltegoTransform() 
     if (not m.Value.isdigit()): 
        TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole number', UIM_PARTIAL)
        return TRX.returnOutput() 
    #here we know we're good to go. 
    howmany = int(m.Value) 
     if (howmany > m.Slider):
        howmany=m.Slider 
     for i in range(1,howmany+1): 
        Ent = TRX.addEntity('maltego.AS', str(i))
        Ent.setWeight(howmany-i)
    return TRX.returnOutput()

Here you see that the addEntity method actually returns a MaltegoEntity object which we can modify. In this case we use it to set the weight and now the graph looks as follows:

The first node (with value ‘1’) has a weight of 19 and the last node (’20’) has a weight of 0. You can see this in the Property View. The following screenshot shows the property view when the AS node with value ‘1’ was selected:

Using Display Info

Next let’s assume we want to create some fancy HTML in the Detail View to go with our AS entities. The ‘addDisplayInformation’ method is used to do this. The content could be plain text or HTML. Let’s start with something really simple. We want to display the words ‘This is number X’ on each entity returned. We want this to be rendered in a label called ‘AS Number’. Here is how the script looks now:

def trx_EnumAS(m): 
    TRX = MaltegoTransform() 
     if (not m.Value.isdigit()): 
        TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole number',UIM_PARTIAL)
        return TRX.returnOutput() 
    #here we know we're good to go. 
    howmany = int(m.Value) 
     if (howmany > m.Slider):
        howmany=m.Slider 
     for i in range(1,howmany+1): 
        Ent = TRX.addEntity('maltego.AS', str(i)) 
        Ent.setWeight(howmany-i) 
        Ent.addDisplayInformation("This is number "+str(i),"AS Number")
    return TRX.returnOutput()

When you run the Transform you’ll see that each node rendered the label and content:

You can easily add more labels. Display information is merged when other transforms creates the same entity instance. Maltego will expand the label by default, but Detail View labels can be collapsed and expanded by the user and the client will remember the choice for future operations.

Let’s see how it works with HTML. Let’s assume we want to make a click-able link in HTML. We will create the link so that it is dynamic and based on the node number. Consider the following code:

def trx_EnumAS(m): 
    TRX = MaltegoTransform() 
     if (not m.Value.isdigit()): 
        TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole number',UIM_PARTIAL)
        return TRX.returnOutput() 
    #here we know we're good to go. 
    howmany = int(m.Value) 
     if (howmany > m.Slider):
        howmany=m.Slider 
     for i in range(1,howmany+1): 
        Ent = TRX.addEntity('maltego.AS', str(i)) 
        Ent.setWeight(howmany-i) 
        Ent.addDisplayInformation("This is number "+str(i),"AS Number") 
        Ent.addDisplayInformation('Click <a href="https://www.ultratools.com/tools/asnInfoResult?domainName=AS'+st r(i)+'"> here </a> to get more information about this AS','AS Info Link')
    return TRX.returnOutput()

Now each node has a link that will open the browser to a specific webpage:

Clicking on the link gives us:

It’ important to note that display information is never sent to the server (when you run subsequent transforms on the node) – it’s read only info and is kept in the client.

Link properties, bookmarks and notes

You can set the link color, label, thickness and style really easily. Consider this code snippet (used in the same transform). We are not going to discuss each method in detail because it’s rather obvious from the code and the resultant graph:

if (i%2==0): 
    Ent.setLinkColor('0x00FF00') 
    Ent.setNote('Even') 
    Ent.setLinkLabel('Even link') 
    Ent.setLinkStyle(LINK_STYLE_NORMAL) 
    Ent.setLinkThickness(1) 
    Ent.setBookmark(BOOKMARK_COLOR_GREEN)        
else: 
    Ent.setLinkColor('0xFF0000')            
    Ent.setNote('Odd') 
    Ent.setLinkLabel('Odd link') 
    Ent.setLinkStyle(LINK_STYLE_DASHED) 
    Ent.setLinkThickness(2) 
    Ent.setBookmark(BOOKMARK_COLOR_RED) 

The only thing to keep in mind is that entity notes stack on top of each other – that means that when a subsequent transform writes to a note it will append to the note rather than deleting the previous note.

Entity properties

Entities can have additional properties. For example – a person might have a property called ‘SSN’.

Static vs. dynamic properties

When you design an entity you’ll see that you can add additional properties to the entity. These are called static properties. It means that should a user drag an entity from the palette to the graph these properties will be available for editing. As an example, here is the property view for Person:

Transforms can create new properties for entities they return. The transform simply adds a property to the entity when it’s returned. These are called dynamic properties. If a transform returns a property that had been defined as static in the entity definition it will store the value as a static property, if it’s not defined in the entity definition it then becomes dynamic.

The only real difference between static and dynamic properties is that users cannot edit dynamic properties on a fresh node from the palette…simply because the property does not exist yet - a transform needed to create it.

A quick note on entity design

One of the most important questions to ask yourself when doing entity design is if data should be stored in a property of an entity or if it should become a new entity type. Similarly you should consider if some information should only be display information or stored as a property within an entity.

Usually you can answer these questions as follows. If you can think of a transform that will regularly use a property of an entity it would be better to make it a separate entity type. For instance – it would be better to have SSN as an entity type and have a Person to SSN transform than to only have SSN as a property in the Person entity (you might still store the SSN as a property within the Person entity, but it makes sense to have it as a separate entity too).

Similarly if you cannot decide if something should be a property or simply display info consider if another transform would need to read that property to work properly or if it’s only shown to the user for information (because display info is never passed back to the server). If it’s just useful as display info don’t bloat the entity by adding it as a property.

Calculated properties

In some cases properties are calculated from other properties. In the case of a person the property FullName is created from FirstName and LastName. This does not mean you cannot set the property, but you don’t need to. For a full list of entity names, properties and calculated properties refer to the Entity definition documentation.

Property types

The current version of the Python library only supports string types but subsequent releases might extend this to include other property types. When dealing with entities that have property types other than strings, make sure you format the string value properly so that the Maltego client can parse the correct type from the string value.

Reading properties

This is fairly straightforward. Consider the Website entity. It has a static property called ‘ports’ with a default of 80 (see Entity definition reference for a list of static entity properties). Inside of Maltego this is really defined as a list of integers but we’ll see how we deal with it in TRX.

Let’s start a new transform. We’ll call it ‘Mangle’, register it on the TDS (running on a website entity) and in our TRX.wsgi stub.

Our transform code looks like this:

def trx_Mangle(m): 
    #define response 
    TRX = MaltegoTransform() 
    TRX.addUIMessage("Property value is:"+m.getProperty("ports"))     
    return TRX.returnOutput() 

We use the ‘getProperty’ method to get the property’s value. You will see that we do not return any entities (that’s 100% OK), we simply write a message to the output with the Output screen. On a default website entity the output looks like this:

If we add a port to the list…

…and run the transform again we get:

In other words – the list is returned as a comma delimited list.

Adding dynamic properties

Let’s see how we can write properties using TRX. Each property has

Let’s assume we want to create a transform that runs on an IP address and generates a netblock. It will always assume a class C netblock (255 IPs) but it will add a dynamic property to the netblock to indicate if it’s a private netblock or not. We’ll do this with a dynamic property called ‘private’ and it will be set to ‘yes’ or ‘no’.

We start by building a stub (in TRX.wsgi for Apache deployment or debugTRX_Server.py when debugging) and registering our transform on the TDS. We begin our transform (without input validation – it’s not the point) like this:

def trx_NetblocksRUs(m):
    TRX=MaltegoTransform() 
    start=m.Value[0:m.Value.rfind('.')]
    netblock=start+".0-"+start+".255" 
    Ent = TRX.addEntity('maltego.Netblock') 
    Ent.setValue(netblock) 
    return TRX.returnOutput()

Note that at this point we haven’t added any dynamic properties to the netblock. We need to first have something that checks if an IP address is private or not. A quick search on the Internet finds this useful piece of code:

>>> from IPy import IP
>>> ip = IP('10.0.0.1') 
>>> ip.iptype() 
'PRIVATE'

This function relies on a Python module called IPy. On Ubuntu installing this is as simple as:

sudo apt-get install python-ipy

This seems perfect for our transform. We add a line importing this library to our DNSTRANSFORMS.py:

from IPy import IP

And the transform now looks like this:

def trx_NetblocksRUs(m):     
   TRX=MaltegoTransform() 
    start=m.Value[0:m.Value.rfind('.')]
    netblock=start+".0-"+start+".255"  
    Ent = TRX.addEntity('maltego.Netblock') 
    ip = IP(m.Value)
    if (ip.iptype()=='PRIVATE'): 
        Ent.addProperty('private','Private network','strict','yes')
    else: 
        Ent.addProperty('private','Private network','strict','no') 
    Ent.setValue(netblock)
    return TRX.returnOutput()

Now running the transform on two different IP addresses we get:

Perfect! Of course we can create a transform that uses netblocks as input and simply outputs a phrase ‘public’ or ‘private’ (this is not very useful, but hopefully will show how to use additional properties).

def trx_PublicPrivate(m): 
    TRX=MaltegoTransform() 
    Ent = TRX.addEntity('maltego.Phrase') 
    if m.getProperty('private')=='yes':
        Ent.setValue('Private')
    else: 
        Ent.setValue('Public') 
    return TRX.returnOutput() 

The resultant graph (when populated with some IPs->Netblock->Public/Private and graph set to Bubble view block layout):

Using Transform Settings

Introduction

Sometimes you want the user to be able to send you information about how you should run your transform. These could be settings that could change every time the transform is executed, or could be settings that the user wants to customize and have Maltego remember. We call these transform inputs or settings.

Almost all of the standard transforms have settings – most of the time users don’t really bother to change them because the defaults are working as expected. A good example of this can be seen in the reverse DNS transform where the HTTP timeout to Serversniff and Robtex can be configured:

To get to the transform manager click on the ‘Manage’ ribbon and go to ‘Manage transforms’. Any transform setting can be configured to pop-up. This is useful when the user would want to change the setting prior to the transform running. Such is the case with IP to Netblock (natural boundaries). The user might choose to select class C (256) or sub class Cs (16,32,64) or perhaps even class Bs (65535).

This transform setting looks as follows in the manager:

The small dialog icon next to the setting indicates that this transform will pop up a setting prior to executing the transform:

The user might choose to fix the value for subsequent transforms – by checking the ‘Remember these settings’ checkbox. In order to get the transform to pop up the setting dialog again the ‘Popup’ checkbox needs to be selected in the transform manager.

To quickly get to each transform’s settings you can click on the small gear in the context menu:

Transform setting types

The Maltego GUI client supports many different pop-up types (integer, dates, lists of strings and integers) but the TDS currently only supports the use of strings.

Using Transform settings in TRX

Now that we know what transform settings are, let’s see how to use them with custom transforms.

Because transform settings are prompted for prior to the transform running, their definitions are loaded when the transforms are discovered. This means that, should you change transform settings, you need to re-install from the seed so that their definitions are updated. This can easily be done from the Data Hub.

Transform settings are defined on the TDS and can be re-used across any number of Transforms. The TDS currently only supports string based transform settings.

Basically a transform setting requires:

The current specification says that a transform setting will only pop up if it’s mandatory and there is no default value. 

Let’s go back to our initial transform (the one with the phrase as a number and AS-es as results). We want to add a transform setting that will obtain a number prior to the transform running and only return AS numbers that are divisible by that setting (if that sounds silly, it is, but again – hopefully is shows the concept). First off we need to define a transform setting on the TDS. We go to the root, click on Settings and click ‘Add setting’. We fill out the form as such:

Our variable will be called ISDIV. Once you’ve added the transform setting click on ‘Add Transform Setting’. Navigate to transforms, click on the EnumAS transform (which we created at the start of this document) and scroll down. At ‘Transform Settings’ highlight the ‘ISDIV’ item (you can shift click to select multiple settings):

This tells the TDS that you wish to use the ISDIV transform setting in your transform. Transforms could have multiple settings and the same settings can be used on multiple transforms.

Next – let’s look what’s needed in the code. Our previous EnumAS ended making a graph that looked like pea soup – so we’ll start with fresh code:

def trx_EnumAS(m): 
    TRX = MaltegoTransform() 
    #read the value     
    if (not m.Value.isdigit()): 
        TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole number',UIM_PARTIAL)
        return TRX.returnOutput() 
    #read the setting
    isdiv = m.getTransformSetting('ISDIV')
    if (not isdiv.isdigit()): 
        TRX.addUIMessage('Silly! We need a number',UIM_FATAL)
        return TRX.returnOutput() 
    ###here we know we're good to go. 
    howmany = int(m.Value); 
    # how many have accumulated? 
    accum=0; 
    for i in range(1,howmany+1):
        if (i % int(isdiv) == 0): 
            Ent = TRX.addEntity('maltego.AS', str(i)) 
            Ent.setWeight(howmany-i) 
            accum=accum+1;
            if accum>=m.Slider: 
                break               
    return TRX.returnOutput() 

Let’s see what happens when we run the transform. First up we get a pop-up asking us some questions:

In the code we do as follows:

#read the setting
isdiv = m.getTransformSetting('ISDIV') 
if (not isdiv.isdigit()):
    TRX.addUIMessage('Silly! We need a number',UIM_FATAL) 
    return TRX.returnOutput()

This checks if the value is indeed a digit – if it’s not we complain bitterly (with UIM_FATAL):

It’s important to remember that the TDS only supports strings. As such we need to convert our string to an integer (after checking that it really is one!) and make sure we give the right amount of nodes back to the user.

The output of the transform is as expected:

Conclusion

The combination of all of these components makes Maltego very flexible and powerful in the hands of a crafty developer. TRX makes it very easy to wield this type of power and the developer is properly shielded from any of the Maltego internal complexities.

More TRX transforms examples can be found under the example page here:

TDS Transforms Code Examples - Python

And the transform library reference guide can be found here:

TRX Library Reference