User-agent: * Disallow: /cgi-bin/ Disallow: /images/ Disallow: /group/ Disallow: /photogallery/ Disallow: /fuelman/ Disallow: /fpdb/ Disallow: /MastList_interface/ Disallow: /catalog/ Disallow: /AAHCDirectory_interface/ Disallow: /documents/ User-agent: msnbot Crawl-delay: 10 User-agent: Teoma Crawl-delay: 10 User-agent: Slurp Crawl-delay: 10 User-agent: aipbot Disallow: / User-agent: BecomeBot Disallow: / User-agent: psbot Disallow: / --------> End of robots.txt file