ScoutJet is the web crawler for a new Silicon Valley based search engine company created by the founders of DMOZ and Topix.

We are developing next generation search technology, and kindly request that you permit ScoutJet access to your site so that we may refine our relevance algorithms with the broadest variety of content available from the Internet.

ScoutJet obeys robots.txt

You can prevent ScoutJet from indexing all or part of your site by including the following lines in your http://www.yoursite.com/robots.txt file:

# Allow only specific directories
User-agent: ScoutJet
Disallow: /
Allow: /public


ScoutJet tries its best to crawl politely. But if you do experience a problem with ScoutJet, please let us know at crawler (at) blekko (dot) com.

More about who we are

Skrentablog
Marksonland
TechCrunch article