# This disallows all bots to several directories and files: # User-agent: * # Disallow: /gw/ # Disallow: /images/ # Disallow: /cgi-bin/ # Disallow: /membersonly/ # PARTIAL access (Alta Vista) User-agent: Scooter Disallow: /15-12.php Disallow: /cgi-bin/ Disallow: /conf_files/ Disallow: /graphics/ Disallow: /gw/ Disallow: /membersonly/ Disallow: /stuff/ Disallow: /home/music/ # PARTIAL access (Excite) User-agent: ArchitextSpider Disallow: /15-12.php Disallow: /cgi-bin/ Disallow: /conf_files/ Disallow: /graphics/ Disallow: /gw/ Disallow: /membersonly/ Disallow: /stuff/ Disallow: /home/music/ # PARTIAL access (Google) User-agent: Googlebot Disallow: /15-12.php Disallow: /cgi-bin/ Disallow: /conf_files/ Disallow: /graphics/ Disallow: /gw/ Disallow: /membersonly/ Disallow: /stuff/ Disallow: /home/music/ # PARTIAL access (Inktomi) User-agent: Slurp Disallow: /15-12.php Disallow: /cgi-bin/ Disallow: /conf_files/ Disallow: /graphics/ Disallow: /gw/ Disallow: /membersonly/ Disallow: /stuff/ Disallow: /home/music/ # PARTIAL access (Lycos) User-agent: Lycos_Spider_(T-Rex) Disallow: /15-12.php Disallow: /cgi-bin/ Disallow: /conf_files/ Disallow: /graphics/ Disallow: /gw/ Disallow: /membersonly/ Disallow: /stuff/ Disallow: /home/music/