Docker Transform Development Setup

Modified on: Mon, 22 Jun, 2020 at 10:25 PM

Oftentimes, developers or internal policies may prefer to keep Transforms separate based on their functionalities or the data sources they query. Using Docker containers is a popular way of doing this and also keeps the Maltego servers deployment requirements consistent.


The steps below detail how to build a separate Docker image for each Maltego-TRX based project and run the container in development mode or using Gunicorn.


Step 1: Write Maltego-TRX Transforms on your local machine

You can write Maltego-TRX Transforms on your local machine and build a Docker image from the Maltego-TRX project. This Docker image can then be run on any Docker host.


You can read more about the Transform Development Server in our Development guide.

For simplicity, we will create a new Maltego-TRX project and build the Docker image with the demo Transforms included. You can copy any of your existing Transforms to the Transforms directory.


$ maltego-trx start TRX
Successfully created a new project in the 'TRX' folder.


Generic

This will be the default directory and file listing of the 'TRX' project.


TRX/
├── project.py
└── transforms
    ├── DNSToIP.py
    └── GreetPerson.py


Step 2: Create Python requirements.txt

This step is optional but recommended as it makes it easier to maintain your Python Transform dependencies. Create a requirements.txt file under the TRX directory with the following content:


maltego-trx
gunicorn


Gunicorn is not required in development mode. See Convert to Production Docker image for its usage.


Generic

Step 2: Create a Dockerfile

Create a file called Dockerfile inside the TRX directory using the following instructions.


FROM python:3.8-slim-buster
LABEL Name=maltego-trx Version=0.0.1
EXPOSE 8080
RUN mkdir -p /var/www/TRX/
WORKDIR /var/www/TRX/
# Copy python requirements file
COPY requirements.txt requirements.txt
# Install maltego-trx and gunicorn
RUN pip3 install -r requirements.txt
# Copy project file and transforms
COPY project.py /var/www/TRX/
COPY transforms /var/www/TRX/transforms/
RUN chown -R www-data:www-data /var/www/TRX/
CMD ["python", "project.py", "runserver"]


Generic\



Step 3: Build the Docker image

Run the following command to build the maltego-trx Docker image containing Transforms and their dependencies.


$ docker build -t local/maltego-trx:0.0.1 .


You can view the newly created maltego-trx image. This Docker image can now pushed to a Docker Registry or saved and shared with other Docker hosts.


$ docker image ls
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
local/maltego-trx   0.0.1              632150c6a966        5 seconds ago       175MB


Step 4: Start the Docker container

Run the following command to run the image as a container on host port 8081 mapped to port 8080 of the container.


$ docker run --rm -p 8081:8080 --name maltego-trx -d local/maltego-trx:0.0.1


Note: The above docker run command will run the docker image in De-attached mode (-d) and will remove (--rm) the container if the container Exits or is Stopped (manually or on host machine reboot).

This helps keep the development environment clean. Make changes as required.


Step 5: Verify Docker container deployment

Use a web browser or tool to get the following response


$ curl http://<docker-host>:8081
You have reached a Maltego Transform Server.


To view the live logs when using the Deattached mode (-d), use the following command:


docker logs -f maltego-trx


Deploy with iTDS

You can also create a custom-itds.yml Docker compose YAML file which contains the additional service. This is useful if you want to build and run the Docker Transform host server with iTDS. The file should have the following contents:


Generic
version: '3'
services:
  maltego-trx:
    build: .
    image: local/maltego-trx:0.0.1


Note: For more information on customizing our docker compose file (itds.yml), please have a look at Customizing Docker.


This tells Docker to add an additional "maltego-trx" container that is built using our local Dockerfile created in the previous steps.


You can start the server in De-attached mode (-d) using the following command:


$ docker-compose -f itds.yml -f custom-itds.yml up -d


Generic

The hostname for communication between the iTDS and the Transform server will be the same as the name of the service defined in the Docker Compose YAML file. In this case, we defined an additional service called "maltego-trx". This means we can proxy traffic from the iTDS to the Transform server using the hostname "maltego-trx" on port 8080 inside the Docker network instead of exposing a host port.



Convert to Production Docker image

When you are done developing and testing, you can also create a Docker image running Gunicorn in production mode. To do this, make the following changes.


Remove the following line from the previously created Dockerfile


CMD ["python", "project.py", "runserver"]


Either add the following command to the Docker file at the end


CMD ["gunicorn", "--bind=0.0.0.0:8080", "--threads=25", "--workers=2", "project:app"]


Or when deploying the image as a container, provide the following command:


$ gunicorn --bind=0.0.0.0:8080 --threads=25 --workers=2 project:app


Note: The working directory should be kept in the same place as the project.py file.



Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.