#Exclude SOCS areas that either don't need indexed or just have too many cross-links: User-agent: * Disallow: /survey Disallow: /vfeedback Disallow: /vcalendar # Disallow this poorly behaved bot that creates invalid session ID's User-agent: findfiles.org Disallow: / # Disallow this bot that sucks bandwidth at all times of day: # http://www.searchme.com/support/spider/ User-Agent: Charlotte Disallow: / # Disallow this bandwidth-consuming Russian spider: spider36.yandex.ru #(oh noes - russians may not find our schools...) User-Agent: Yandex Disallow: /