Open navigation

Mirror: External links found

Modified on: Wed, 24 Jun, 2020 at 11:06 AM

Transform Meta Info

Display Name
Mirror: External links found
Transform Name
Short Description
This Transform uses Gary's Ruby website mirror to spider the site and extract links
Data Source


This Transform will make a (partial) mirror of the web site and extract all external links found on the site - these will be returned as website entities. The slider plays a big role in this transform as it set the time-out for the mirroring process. The higher (to the right) the slider is set, the deeper the mirroring process will go, and hopefully, the more results you'll get. The process runs via a caching server (that is local on the box) which means that you won’t be doing the data transfer to the site twice (if you run the transform again) - expect of course if the first round did not manage to get the entire site. Also keep in mind that not all sites are mirror friendly. Flash based sites will give problems as will sites with exotic JavaScript menus and redirects.

Typical Use Case

Extract other websites mentioned in the target website.


Starting with the URL for our homepage, we can convert the URL entity into a website entity. From the website entity we can run this transform to find the external websites that we link to.

Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.