User-agent: * Disallow: /test/robots/noindex/ Disallow: /test/robots/disal/ Disallow: /test/robots/partial Disallow: /test/relativelinks/2ndlevel/http:// Disallow: /test/relativelinks/rtestprob/http://searchtools/about/ Disallow: /test/relativelinks/rtestprob/http://searchtools/analysis/ Disallow: /test/relativelinks/rtestprob/http://searchtools/guide/ Disallow: /test/relativelinks/rtestprob/http://searchtools/info/ Disallow: /test/relativelinks/rtestprob/http://searchtools/pub/ Disallow: /test/relativelinks/rtestprob/http://searchtools/robots/ Disallow: /test/relativelinks/rtestprob/http://searchtools/search/ Disallow: /test/relativelinks/rtestprob/http://searchtools/site/ Disallow: /test/relativelinks/rtestprob/http://searchtools/slides/ Disallow: /test/relativelinks/rtestprob/http://searchtools/surveys/ Disallow: /test/relativelinks/rtestprob/http://searchtools/tools/ Disallow: /background/ Disallow: /blog/ Disallow: /cgi-bin/ Disallow: /info/articles/ Disallow: /info/conferences-past.html Disallow: /info/meetings/examples/ Disallow: /info/meetings/thunderlizard/examples/ Disallow: /info/robots/ Disallow: /info/slides/ Disallow: /lists/ Disallow: /related/ Disallow: /reviews/ Disallow: /searchtools/ Disallow: /site/ Disallow: /slides/examples/ Disallow: /ST/ Disallow: /st/ Disallow: /St/ Disallow: /wr/ # updated 2002-03-22 (disallow rtestprob links) # updated 2002-06-25 (disallow info/slides links, info/robots/) # updated 2002-07-25 (disallow /searchtools/ which is an alias) # updated 2005-09-09 (rearranged as per Enrico's advice) # updated 2007-01-24 (added site directory to disallow list) # updated 2007-01-25 (removed rss disallow, file is gone)