Transform Meta Info
|Display Name||To Website [using Search Engine]|
|Short Description||This Transform shows sites where various permutations of the person's name was found|
This Transform shows sites where various permutations of a person's name was found. You’ll see a pop up asking for a Domain or TLD and an additional search term.
General to all search engine Transforms
There are a few Transforms that use search engines - all of them are very similar. The basic recipe for these Transforms is as follows:
- Expand the question. The question is the input from the GUI - be that a persons name, a domain or an phone number. When looking at a persons name, for instance, the name "Kosie Kramer", will be expanded to searches like Kosie Kramer, K Kramer, Kramer Kosie etc. In the case of a telephone number the search will be expanded to include most telephone notations used.
- Assign confidence levels. Because a search for Kosie Kramer is more likely to return good results - rather than a search for KramerK the confidence level for the first search would be higher. The confidence levels are also used to assign preference to certain file types when doing searches on documents (these are configurable within the Transform itself). In the same way an XLS file containing the word is likely more interesting than a PDF file.
- Perform each search. The searches are performed and the snippets are obtained. It is important to note that only snippets are parsed. For parsing the entire page you need to dump to URL and process the URLs separately. Various search engines have various snippet lengths.
- Parse for output Entities. Depending on what output is required the snippets are parsed for Entities - in some cases the web site name is all that's required.
- Calculate weight. The weight is calculated from various factors - the confidence of the search, the frequency of the result, the importance of the originating website,, and in some cases, a correlation to the input.
- Normalise. The weights are now normalised using a fairly interesting algorithm that involves the mean and standard deviation of the spread of weights. It is important to understand that a search result with an equal spread of weights are often of no use.
General notes when using search engine Transforms
Maltego will sometimes give you results that seem incorrect. You need to keep in mind that the application will get desperate when it does not have results to return. So when you are searching for a person called "Vaxynutus Grabounill" and that person simply left no marks on the Internet, Maltego will eventually go after a search term "VG" despite a low confidence. However, you will still get some results. These results could seem completely off the mark, but should have very low weights. Always consider the weights.
Problems with parsing results
Some Entities are difficult to parse. Telephone numbers are notoriously hard to parse. There is always a trade-off between missing numbers and parsing non-telephone numbers as phone numbers. With the current Transform we hope to reach the optimal balance.
Staring with a person named "John Doe" we can run our Transform. Choosing the domain as "pastebin.com" the Transform will only return results with that domain. The Transfrom returns a website Entity for each domain it finds. The individual URLs are stored as a property of the website entity. We can use the "To URLs" Transform to create a URL entity for each URL belonging to the website.