These iTDS Code snippets utilise Paterva's recommended TRX Python transfrom library and show some basic functionality in an easy to copy and paste way. If you are not familiar with the TRX framework we recommend taking a look at TRX library guide first. You will also need to already have set up your transform host server, instructions for doing this can be found on the Transform Host Server Setup page.
Returning a Complex Entity
This transform shows how to read input, create UI messages, use transform settings. It takes a phrase as input, requires a transform setting called 'ISDIV' and return AS numbers. It also sets the link properties based on the odd/even state of the AS number.
Transform Stub (TRX.wsgi):
@route('/EnumAS', method='ANY') def EnumAS(): if request.body.len>0: return(trx_EnumAS(MaltegoMsg(request.body.getvalue())))
from Maltego import * # Input - phrase entity # Output - AS numbers # NB: You need ISDIV defined as transform setting def trx_EnumAS(m): # construct a return vessel TRX = MaltegoTransform() #read the value, make sure its a digit if (not m.Value.isdigit()): # if not - complain TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole number',UIM_PARTIAL) return TRX.returnOutput() #read the setting - you need ISDIV defined as transform setting in the TDS isdiv = m.getTransformSetting('ISDIV') #check if its a digit - else complain even more bitterly if (not isdiv.isdigit()): TRX.addUIMessage('Silly! We need a number',UIM_FATAL) return TRX.returnOutput() #here we know we're good to go. #read the value of the node howmany = int(m.Value); # how many have accumulated? accum=0; for i in range(1,howmany+1): if (i % int(isdiv) == 0): # add an AS entity with the index as a value... Ent = TRX.addEntity('maltego.AS', str(i)) # ... and set the weight Ent.setWeight(howmany-i) # add a property called 'div' Ent.addProperty('div','Divisible by','strict',str(isdiv)) # see it's odd or even and set the link/note/bookmark properties # this makes for a very ugly graph..but..ya if (i%2==0): Ent.setLinkColor('0x00FF00') Ent.setNote('Even') Ent.setLinkLabel('Even link') Ent.setLinkStyle(LINK_STYLE_NORMAL) Ent.setLinkThickness(1) Ent.setBookmark(BOOKMARK_COLOR_GREEN) else: Ent.setLinkColor('0xFF0000') Ent.setNote('Odd') Ent.setLinkLabel('Odd link') Ent.setLinkStyle(LINK_STYLE_DASHED) Ent.setLinkThickness(2) Ent.setBookmark(BOOKMARK_COLOR_RED) accum=accum+1; if accum>=m.Slider: break # return the XML to the TDS server return TRX.returnOutput()
The following code samples provide example transforms that query different types of databases. They includes examples for querying MySQL, PostgreSQL and MSSQL using an ODBC connector.
MySQL Transform Code
To access a MySQL database from our Python code we first need to install the MySQLdb Python library on our server which can be done by running the following command:
$ sudo apt-get update $ sudo apt-get install python-mysqldb
The MySQL transform example below will simply query a sample database containing customer information for a made-up company. The database includes columns for the customer's company name and the location of the customer. The transforms will take the customer's company name in as the input entity, lookup the location for that customer in the MySQL database and then return the location of the customer's company as a location entity in Maltego.
#import required libraries import MySQLdb from Maltego import * def trx_mysqlExamplTransform(m): TRX = MaltegoTransform() #Read the value of the input entity. customerName = m.Value #Read database password from file. try: f = open('dbConfig.txt', 'r') except: TRX.addUIMessage('Could not find dbConfig file.', UIM_FATAL) return TRX.returnOutput() #Create variables to store database connection details. server = "LocalHost" username = "root" password = f.read() database = "classicmodels" #Connect to database using the details above. try: conn = MySQLdb.connect(server,username,password,database) except: TRX.addUIMessage('Could not access database, please check that the connection details are correct.', UIM_FATAL) return TRX.returnOutput() #Define the cursor for the database connection c = conn.cursor() #Run the execute method using a placeholder for the input and enter its value as the second parameter of the method. c.execute("SELECT city, country FROM customers WHERE customerName = %s", customerName) rows = c.fetchall() #Create a new entity for each row that is returned from our SQL query. for i,eachRow in enumerate(rows): if eachRow and eachRow: TRX.addEntity('maltego.Location',eachRow+', '+eachRow) #Read the slider value and break the loop when the number of entities exceeds the slider value if i>=m.Slider: break return TRX.returnOutput()
Running this simple transform on a few customers from our sample database will result with the following graph output:
PostgreSQL Transform Code
To access a PostgreSQL database from Python we will first need to install the library python-psycopg2 which can be done as follows:
$ sudo apt-get update $ sudo apt-get install python-psycopg2
This transform example will query another sample database containing information about the domain paterva.com. The database includes columns for the DNS names and the IP addresses related to the domain. The transform example below will query the sample database for DNS names that resolve to the IP address specified as the input entity. This transform also shows how transform settings can be used to specify additional pieces of information when a transform is run. In this case a transform setting is used to specify a pattern that must be matched in the DNS names before it can be returned from the transform.
#import the PostgreSQL python library import psycopg2 def trx_postgresExampleTransform(m): TRX = MaltegoTransform() #Read the value of the input entity. IP = m.Value #Read transform setting for pattern that needs to be match by returned DNS names. DnsPattern = m.getTransformSetting('DnsPattern') #Read database password from file. try: f = open('dbConfig.txt', 'r') except: TRX.addUIMessage('Could not find dbConfig file.', UIM_FATAL) return TRX.returnOutput() #Create variables to store database connection details. server = "LocalHost" username = "postgres" password = f.read() database = "postgres" #Connect to database using the details from the dbConfig file. try: conn = psycopg2.connect(database, user, host, password) except: TRX.addUIMessage('Could not access database, please ensure that your database details are correct.', UIM_FATAL) return TRX.returnOutput() #Define the cursor for the database connection c = conn.cursor() #Run the execute method using a placeholders for the inputs and enter their values in a list in the second parameter of the execute method. c.execute("SELECT DNSNAME FROM iptodns WHERE IP = %s AND WHERE DNSNAME = %s;", [IP, '*'+DnsPattern+'*']) rows = c.fetchall() #Create a new entity for each row that is returned. for i,eachRow in enumerate(rows): if eachRow: TRX.addEntity('maltego.DNSName',eachRow) #Read the slider value and break the loop when the number of entities exceeds the slider value if i>=m.Slider: break return TRX.returnOutput()
Running this transform on the IP address "126.96.36.199" which belongs to Paterva and specifying the pattern to match "paterva" in the transform setting results in the graph on the right below:
MSSQL [using ODBC] Transform Code
ODBC (Open Database Connectivity) is a standard programming language middleware API for accessing DBMS. In this transform example we will be connecting to a MSSQL database using the FreeTDS ODBC Driver. With the correct ODBC drivers installed on your server this transform could be modified to work with a range of database types by changing the connection string to use the appropriate ODBC driver.
ODBC and FreeTDS setup
On our server we first need to install all the pre-requisites for using ODBC with MSSQL and then configure them correctly. First install unixODBC and FreeTDS with the following command:
$ sudo apt-get update $ sudo apt-get install unixodbc unixodbc-dev freetds-bin tdsodbc freetds-dev
Microsoft SQL Server uses Tabular Data Stream (TDS) to transfer data between a database server and a client. Freetds is an open source TDS implementation. Next we need to configure a FreeTDS file to include server details of the SQL server you want to connect to. In the case of this example our transform will be connecting to a MSSQL server hosted on Microsoft Azure. :
[global] tds version = 7.1 [SERVERNAME] host = HOSTNAME.database.windows.net port = 1433
Next we need to configure ODBC to include the details of the FreeTDS driver:
[FreeTDS] Description = FreeTDS Driver Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
And then configure the ODBC data sources names (DSN) by adding your SQL server details to the file as shown below:
[DATA_SOURCE_NAME] Driver = FreeTDS Servername = SERVERNAME # This server name must be the same as the one configured in freetds.conf Port = 1433 Database = DBNAME
The last thing we need to install is the Python library for working with ODBC connectors, this can be done with the command below:
$ pip install pyodbc
After following the steps above you will be able to access a MSSQL database using the ODBC connection string from within your Python code with pyodbc.
MSSQL [using ODBC] Transform
The transform in the example below will connect to a sample MSSQL database hosted on Microsoft Azure using the ODBC driver that has been configured above. The database being queried has information from the Bitcoin Blockchain and the transform will take a Bitcoin Address in as the input entity and return addresses that have received Bitcoins from the input address:
import pyodbc def trx_odbcExampleTransform(m): TRX = MaltegoTransform() #Read the input address BTC_Addresses = m.Value #You choose the ODBC driver using the DRIVER keyword. The other settings are then separated by semicolons. try: conn = pyodbc.connect('DRIVER=FreeTDS;SERVER=mdviumqix9.database.windows.net; PORT=1433;DATABASE=AdventureWorks;UID=testDB@mdviumqix9;PWD=******;TDS_Version=8.0;') except: TRX.addUIMessage('Could not access database, please ensure that your database details are correct.', UIM_FATAL) return TRX.returnOutput() c = conn.cursor() #ODBC supports parameters using a question mark as a place holder in the SQL. This is safer than putting the values into the string because #the parameters are passed to the database separately, protecting against SQL injection attacks. c.execute("SELECT SentAddresses FROM BTCTransactions WHERE Address = ?", BTC_Addresses) rows = c.fetchall() #Create a new entity for each row that is returned. for i,eachRow in enumerate(rows): if eachRow: TRX.addEntity('PR.BtcAddress',eachRow) if i>=m.Slider: break return TRX.returnOutput()
Running this transform on the Bitcoin address "16Fg2yjwrbtC6fZp61EV9mNVKmwCzGasw5" with the transform slider set to 12 results in the graph below: