Production Transform Server using Gunicorn

Modified on: Wed, 10 May, 2023 at 12:42 PM

Overview

The instructions in this section serve as an example only. Your specific environment and requirements might require another solution. This article provides instructions for setting up a Transform host server  with Python’s Gunicorn.


Gunicorn Server

The following section covers how to use Gunicorn as a Transform host server. This setup guide was tested on Ubuntu 18.04 LTS using Gunicorn and Python 3.


Note: This is only an example and requires additional design and configuration to serve as a secure and reliable production Transform Server. Generally Gunicorn runs behind a reverse proxy such as NGINX. Configuring NNX is not explained in this guide.


Note: Gunicorn is only supported on Python 3.



Install system and Python dependencies:

Refresh APT repositories:


sudo apt update


Install Python3:


 sudo apt install python3 -y


Install Python3 pip:

Configure Python3 and pip as the default for your system:


echo "alias python=python3" >> ~/.bash_aliases
echo "alias pip=pip3" >> ~/.bash_aliases
source ~/.bash_aliases


Install Maltego Python Transform Library:


sudo -H pip3 install maltego-trx


Install Gunicorn:


sudo -H pip3 install gunicorn


Prepare the Transform project. In this basic setup guide, it is assumed that the Transform project was created in the following folder:


$ sudo mkdir -p /var/www/    

$ cd /var/www/

$ sudo maltego-trx start TRX


This will create the Transform project folder "/var/www/TRX" with the recommend subfolder structure.


The Gunicorn web server will run as the “www-data” user. Set the “www-data” user as the owner of Transform project folder. Use the following command to change folder ownership:


$ sudo chown -R www-data:www-data /var/www/TRX/


Configure Gunicorn     

It is possible to run Gunicorn once-off from the command line to test basic functionality. Change to the Transform project folder and execute the following command:


$ cd /var/www/TRX/

$ gunicorn --bind=0.0.0.0:8081 --threads=25 --workers=2 project:app


The Gunicorn process will start and bind to all host IP addresses on port 8081.

Use control + z to terminate Gunicorn process.


Note: The default (host) port can be configured here to use any other open port on which you prefer to expose the Transform.


To run Gunicorn as a persistent service, create the service manually as follows:


$ sudo nano /etc/systemd/system/TRX.service


The basic content of the service configuration file should be:


[Unit]
Description=Gunicorn instance to serve as transform server
After=network.target

[Service]
User=www-data
Group=www-data
WorkingDirectory=/var/www/TRX/
ExecStart= /usr/bin/python3 /usr/local/bin/gunicorn --bind=0.0.0.0:8081 --threads=25 --workers=2 project:app

[Install]
WantedBy=multi-user.target


Note that the "ExecStart" requires the absolute paths for the python and Gunicorn installations. Use the following command to determine the absolute path to python and Gunicorn:


$ which python3

$ which gunicorn


Note: In this case ‘python3’ must be specified.


Note: If you tested from the command line, please ensure that all running instances are stopped by using the following command:


$ ps aux | grep gunicorn | awk '{print $2;}' | xargs kill -9 2


Start and persist the Gunicorn service as follows:


$ sudo systemctl start TRX

$ sudo systemctl enable TRX


Confirm the status of the service as follows:


$ sudo systemctl status TRX


Test your configuration

Test the configuration by browsing to the following URL:


http://<your server name or IP>:8081/

* Update <your server name or IP> with appropriate values.


A valid and correct configuration will return the following response:


You have reached a Maltego Transform Server.


Retrieve the Transform links by running the following command from the Transform project folder:


$ cd /var/www/TRX/

$ python project.py list


The output of the “list” command will contain Transform server URLs:


= Transform Server URLs =
/run/dnstoip/: DNSToIP
/run/greetperson/: GreetPerson


Test the Transform server URLs by concatenating them to the Transform server URL:


http://<your server name or IP>:8081/run/dnstoip/

* Update <your server name or IP> with appropriate values.


A valid and correct configuration will return the following response:


Transform found with name 'dnstoip', you will need to send a POST request to run it.


Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.