The trendiction bot crawls public websites, including newssites, messageboards and blogs, including their comments.

If our spider doesn't behave well (e.g. too many requests, recursive urls) please contact us so we can fix this issue!

Why do you access my website?

We crawl your website to integrate your website into our public search engine.

Additionaly, we process and filter this data so our clients can access this data through our web service apis. Among our clients are market research companies, marketing agencies, search engines and ohter web applications who use us to outsource their crawling in order to save ressources (imagine every media company crawling your site).

More informations on our products are located here.

How much bandwith do we use?

We try to be as efficient as possible when we request your website.

We try to keep the bandwith and requests as low as possible:

How can I recognize that your bot accesses my web site?

We use the following user agent:

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.0; trendictionbot0.5.0; trendiction search;; please let us know of any problems; web at Gecko/20071127 Firefox/

How can I instruct the trendiction bot from not accessing my web site?

If you would like to exclude a part or the entire website from crawlers accessing your site, you can do so by creating a file called "robots.txt" in the root directory of your website:

User-Agent: *
Disallow: /

Or if you would only like to exclude our spider from accessing your site:

User-Agent: trendictionbot
Disallow: /

The robots.txt file is a widely recognized format to instruct crawlers on what they can crawl and what not. More informations regarding robots.txt are available on the site

Due to our internal caching procedures, it can take up to 5 days for the updated robots.txt file to be active.

However, most people would like to have their website indexed because it can be found in search engines later on and generate traffic.

Trendiction S.A. - 12-16, avenue Monterey - L-2163 Luxembourg - Tel.: +352 20 33 35 30