# Robots.txt file for domain: http://ksforum.inboxrevenge.com # A larger sitemap to add guidance to the key pages [TB] March 12 2009 Sitemap: http://ksforum.inboxrevenge.com/sitemap2.xml # styles are not relevant [TB] Feb 15 User-Agent: * Disallow: /styles/ Disallow: /style* # nobody gets into the images Feb 15 User-Agent: * Disallow: /images/ # nobody needs the Avatars [TB] Feb 15 User-Agent: * Disallow: /download/ # get everyone out of the member list [TB] Feb 15 probably will not work User-agent: * Disallow: /memberlist.php Disallow: /memberlist* Disallow: /viewonline.php Disallow: /viewonline* # idiot becomebot User-agent: BecomeBot Disallow: / # this guy is a spammer. User-Agent: 63.148.99.234 Disallow: / # so is this guy # [Performance systems international inc.] User-Agent: 38.118.42.38 Disallow: / # Another, Mar. 13, 2006 User-Agent: 38.118.25.59 Disallow: / # OmniExplorer Bot # Nov. 28th, 2005 # Performed 2800 scans # http://www.omni-explorer.com User-Agent: OmniExplorer_Bot Disallow: # SemanticDiscovery morons # Dec. 6th, 2005 # http://www.semanticdiscovery.com User-Agent: 71.213.8.209 Disallow: / # Yet another Semantic Discovery idiot # SemanticDiscovery morons # Dec. 6th, 2005 # http://www.semanticdiscovery.com User-Agent: 71.213.8.140 Disallow: / # Any other SemanticDiscovery bots... User-Agent: semanticdiscovery Disallow: / # Nov. 25, 2008: I've decided - if you're small enough and do # not feature a prominent search engine, you are out. # Not allowed. I don't care. User-Agent: Charlotte Disallow: / User-agent: psbot Disallow: / User-agent: gigabot Disallow: / User-agent: ScoutJet Disallow: / # Jan. 9, 2009 # A new one: VoilaBot User-Agent: VoilaBot BETA 1.2 Disallow: / # And another: User-Agent: WebDataCentreBot Disallow: / # And another: User-Agent: MJ12bot Disallow: / # And another: User-Agent: Twiceler Disallow: / # And yet another: User-Agent: Charlotte Disallow: / # And still another: User-Agent: DotBot Disallow: / # Another dubious search bot User-agent: MLBot Disallow: /