#Инструкции для всех роботов User-Agent: * Disallow:/cgi-bin Disallow:/images Disallow:/style Disallow:/banex.gif Disallow:/form # For AllTheWeb User-agent: Fast-Webcrawler Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Alexa User-agent: ia_archiver Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For AltaVista User-agent: Scooter Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Exactseek.com User-agent: exactseek-pagereaper Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Htdig.org User-agent: htdig Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Google User-agent: Googlebot Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Google Mediapartners User-agent: Mediapartners-Google Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For GoForIt.com User-agent: GoForIt.com Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Gigabot User-agent: Gigabot Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Jeeves User-agent: Jeeves Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Inktomi Slurp User-agent: Inktomi-Slurp Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Kellysearch User-agent: semanticdiscovery Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For LinkWalker User-agent: LinkWalker Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Looksmart User-agent: WISENutbot Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For MSN User-agent: MSNBot Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Mirago [UK] User-agent: mirago Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Mozdex User-agent: mozDex Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Nameprotect User-agent: NPBot Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Nutch User-agent: NutchCVS Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For ObjectsSearch User-agent: ObjectsSearch Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Picsearch User-agent: psbot Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Turnitin.com User-agent: TurnitinBot Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Voila.com France Telecom search engine User-agent: Voila Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Walhello.com User-agent: appie Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Wget User-agent: Wget Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Wotbox.com User-agent: Wotbox Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form # For Yahoo User-agent: Yahoo-MMCrawler Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form #-------------------- RU -------------------- #Инструкции для робота Апорта User-Agent: Aport Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/http://cn.tiwatiwa.com # Инструкции для робота Иван Сусанин User-Agent: Ivan-Susanin-validator Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/http://cn.tiwatiwa.com # Инструкции для робота Рамблера User-Agent: StackRambler Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/http://cn.tiwatiwa.com # Инструкции для робота Яндекса  User-Agent: yandex Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/http://cn.tiwatiwa.com #------------------- BY ------------------------ # Инструкции для робота All.by  User-Agent: All.by Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/http://it.tiwatiwa.com Disallow:/http://cz.tiwatiwa.com Disallow:/http://cn.tiwatiwa.com #------------------------- CN+TW+KR --------------------- # For Baidu [cn]   User-Agent: Baiduspider+ Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/russia Disallow:/belarus Disallow:/italy Disallow:/germany Disallow:/offer Disallow:/other Disallow:/china Disallow:/czech Disallow:/tanzania Disallow:/by Disallow:/http://it.tiwatiwa.com Disallow:/http://cz.tiwatiwa.com # For Naver.com [kr]   User-Agent: NaverBot Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/russia Disallow:/belarus Disallow:/italy Disallow:/germany Disallow:/offer Disallow:/other Disallow:/china Disallow:/czech Disallow:/tanzania Disallow:/by Disallow:/http://it.tiwatiwa.com Disallow:/http://cz.tiwatiwa.com # For Openfind [tw] User-Agent: Openbot Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/russia Disallow:/belarus Disallow:/italy Disallow:/germany Disallow:/offer Disallow:/other Disallow:/china Disallow:/czech Disallow:/tanzania Disallow:/by Disallow:/http://it.tiwatiwa.com Disallow:/http://cz.tiwatiwa.com # For Sohu [cn]   User-Agent: sohu-search Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/russia Disallow:/belarus Disallow:/italy Disallow:/germany Disallow:/offer Disallow:/other Disallow:/china Disallow:/czech Disallow:/tanzania Disallow:/by Disallow:/http://it.tiwatiwa.com Disallow:/http://cz.tiwatiwa.com #-------------------------- IT ---------------------- # For Iltrovatore User-Agent: IlTrovatore-Setaccio Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/russia Disallow:/belarus Disallow:/italy Disallow:/germany Disallow:/offer Disallow:/other Disallow:/china Disallow:/czech Disallow:/tanzania Disallow:/by Disallow:/http://cn.tiwatiwa.com Disallow:/http://cz.tiwatiwa.com #-------------------------- NO [Норвегия]---------------------- # For www.boitho.com User-Agent: boitho.com-dc Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/russia Disallow:/belarus Disallow:/italy Disallow:/germany Disallow:/offer Disallow:/other Disallow:/china Disallow:/czech Disallow:/tanzania Disallow:/by Disallow:/http://it.tiwatiwa.com Disallow:/http://cn.tiwatiwa.com Disallow:/http://cz.tiwatiwa.com #-------------------------- de ---------------------- # For www.peerbot.com User-Agent: PEERbot Disallow:/cgi-bin Disallow:/images Disallow:/script Disallow:/style Disallow:/banex.gif Disallow:/form Disallow:/russia Disallow:/belarus Disallow:/italy Disallow:/germany Disallow:/offer Disallow:/other Disallow:/china Disallow:/czech Disallow:/tanzania Disallow:/by Disallow:/http://it.tiwatiwa.com Disallow:/http://cn.tiwatiwa.com Disallow:/http://cz.tiwatiwa.com