Docker Transform Development Setup

Oftentimes, developers or internal policies may prefer to keep Transforms separate based on their functionalities or the data sources they query. Using Docker containers is a popular way of doing this and also keeps the Maltego servers deployment requirements consistent.

The steps below detail how to build a separate Docker image for each Maltego-TRX based project and run the container in development mode or using Gunicorn.

Step 1: Write Maltego-TRX Transforms on your local machine

You can write Maltego-TRX Transforms on your local machine and build a Docker image from the Maltego-TRX project. This Docker image can then be run on any Docker host.

You can read more about the Transform Development Server in our Development guide.

For simplicity, we will create a new Maltego-TRX project and build the Docker image with the demo Transforms included. You can copy any of your existing Transforms to the Transforms directory.

$ maltego-trx start TRX
Successfully created a new project in the 'TRX' folder.

This will be the default directory and file listing of the 'TRX' project.

TRX/
├── project.py
└── transforms
    ├── DNSToIP.py
    └── GreetPerson.py

Step 2: Create Python requirements.txt

maltego-trx
gunicorn

Gunicorn is not required in development mode. See Convert to Production Docker image for its usage.

Step 2: Create a Dockerfile

Create a file called Dockerfile inside the TRX directory using the following instructions.

FROM python:3.8-slim-buster
LABEL Name=maltego-trx Version=0.0.1
EXPOSE 8080
RUN mkdir -p /var/www/TRX/
WORKDIR /var/www/TRX/
# Copy python requirements file
COPY requirements.txt requirements.txt
# Install maltego-trx and gunicorn
RUN pip3 install -r requirements.txt
# Copy project file and transforms
COPY project.py /var/www/TRX/
COPY transforms /var/www/TRX/transforms/
RUN chown -R www-data:www-data /var/www/TRX/
CMD ["python", "project.py", "runserver"]

Step 3: Build the Docker image

$ docker build -t local/maltego-trx:0.0.1 .

$ docker image ls
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
local/maltego-trx   0.0.1              632150c6a966        5 seconds ago       175MB

$ docker run --rm -p 8081:8080 --name maltego-trx -d local/maltego-trx:0.0.1

Note: The above docker run command will run the docker image in De-attached mode (-d) and will remove (--rm) the container if the container Exits or is Stopped (manually or on host machine reboot).

This helps keep the development environment clean. Make changes as required.

$ curl http://<docker-host>:8081
You have reached a Maltego Transform Server.

docker logs -f maltego-trx

version: '3'
services:
  maltego-trx:
    build: .
    image: local/maltego-trx:0.0.1

Note: For more information on customizing our docker compose file (itds.yml), please have a look at Customizing Docker.

This tells Docker to add an additional "maltego-trx" container that is built using our local Dockerfile created in the previous steps.

You can start the server in De-attached mode (-d) using the following command:

$ docker-compose -f itds.yml -f custom-itds.yml up -d

The hostname for communication between the iTDS and the Transform server will be the same as the name of the service defined in the Docker Compose YAML file. In this case, we defined an additional service called "maltego-trx". This means we can proxy traffic from the iTDS to the Transform server using the hostname "maltego-trx" on port 8080 inside the Docker network instead of exposing a host port.

Convert to Production Docker image

When you are done developing and testing, you can also create a Docker image running Gunicorn in production mode. To do this, make the following changes.

Remove the following line from the previously created Dockerfile

CMD ["python", "project.py", "runserver"]

CMD ["gunicorn", "--bind=0.0.0.0:8080", "--threads=25", "--workers=2", "project:app"]

$ gunicorn --bind=0.0.0.0:8080 --threads=25 --workers=2 project:app

Note: The working directory should be kept in the same place as the project.py file.