# # checked with http://tool.motoricerca.info/robots-checker.phtml # User-agent: Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html) Disallow: / User-agent: Btsearch/1.1 (Btsearch; http://www.baotongsoft.com/search.html; Btsearch-agent@baotongsoft.com) Disallow: / User-agent: Baiduspider+(+http://www.baidu.com/search/spider.htm) Disallow: / User-agent: Cuil Twiceler - Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html) Disallow: / User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1; aggregator:Spinn3r (Spinn3r 3.1); http://spinn3r.com/robot) Gecko/20021130 Disallow: / User-agent: Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm.*) Disallow: / User-agent: Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/) Disallow: / User-agent: Gigabot/3.0 (http://www.gigablast.com/spider.html) Disallow: / User-agent: Morfeus Fucking Scanner Disallow: / User-agent: SheenBot/SheenBot-1.0.2 (Sheen web crawler) Disallow: / User-agent: Yandex/1.01.001 (compatible; Win16; I) Disallow: / User-agent: * Disallow: /nowhere/ Disallow: /NoRoBoTs/ Disallow: /PSP/ Disallow: /NBS/ Disallow: /tools/ Disallow: /.bugzilla/ Disallow: /blackwhite/ Disallow: /cgi-bin/ Disallow: /BlogPix/ Disallow: /images/ Disallow: /TTDEV/ Disallow: /plogger/ Disallow: /TagCloudLongTail.php Disallow: /pop3h.php