[Bio] / Sprout / robots.txt Repository:
ViewVC logotype

Diff of /Sprout/robots.txt

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.1, Mon Nov 5 21:29:45 2007 UTC revision 1.2, Mon May 5 19:27:08 2008 UTC
# Line 4  Line 4 
4  Disallow: /FIG/Html/gbrowse/    # gbrowse.cgi is okay, but not the support files  Disallow: /FIG/Html/gbrowse/    # gbrowse.cgi is okay, but not the support files
5  Disallow: /includes             # These are pulled in on the main page anyway  Disallow: /includes             # These are pulled in on the main page anyway
6  Disallow: /next                 # Skip the development site  Disallow: /next                 # Skip the development site
7  Disallow: /prev                 # Skip the old site  Disallow: /FIG/Html/SetPassword.html
8  Disallow: /cur                  # Only allow the main entrance                          # Stay off the password page
9  Disallow: /FIG/protein.cgi      # Skip the protein pages  Disallow: /FIG/SproutScript.cgi
10    Disallow: /FIG/TestForms.cgi
11    Disallow: /FIG/SetObject.cgi
12                            # Don't try the test scripts.
13    Crawl-delay: 4

Legend:
Removed from v.1.1  
changed lines
  Added in v.1.2

MCS Webmaster
ViewVC Help
Powered by ViewVC 1.0.3