[Bio] / Sprout / robots.txt Repository:
ViewVC logotype

View of /Sprout/robots.txt

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.4 - (download) (annotate)
Tue Sep 16 18:50:33 2008 UTC (10 years, 11 months ago) by parrello
Branch: MAIN
CVS Tags: mgrast_dev_08112011, mgrast_dev_08022011, rast_rel_2014_0912, rast_rel_2008_12_18, mgrast_dev_04082011, rast_rel_2010_0928, rast_2008_0924, mgrast_version_3_2, mgrast_dev_12152011, mgrast_dev_06072011, rast_rel_2008_09_30, rast_rel_2009_0925, rast_rel_2010_0526, rast_rel_2014_0729, rast_rel_2009_05_18, rast_rel_2010_1206, mgrast_release_3_0, mgrast_dev_03252011, rast_rel_2010_0118, mgrast_rel_2008_0924, mgrast_rel_2008_1110_v2, rast_rel_2009_02_05, rast_rel_2011_0119, mgrast_rel_2008_0625, mgrast_release_3_0_4, mgrast_release_3_0_2, mgrast_release_3_0_3, mgrast_release_3_0_1, mgrast_dev_03312011, mgrast_release_3_1_2, mgrast_release_3_1_1, mgrast_release_3_1_0, mgrast_dev_04132011, rast_rel_2008_10_09, mgrast_dev_04012011, rast_release_2008_09_29, mgrast_rel_2008_0923, mgrast_rel_2008_0919, rast_rel_2009_07_09, rast_rel_2010_0827, mgrast_rel_2008_1110, myrast_33, rast_rel_2011_0928, rast_rel_2008_09_29, mgrast_rel_2008_0917, rast_rel_2008_10_29, mgrast_dev_04052011, rast_rel_2009_03_26, mgrast_dev_10262011, rast_rel_2008_11_24, HEAD
Changes since 1.3: +5 -8 lines
Removed some obsolete stuff.

User-agent: *
Disallow: /FIG/p2p 		# Don't touch the peer-to-peer update.
Disallow: /FIG-Tmp/		# These are temporary files.
Disallow: /next			# Skip the development site
Disallow: /prev			# Skip the archive site.
Disallow: /FIG/Emergency.cgi
Disallow: /FIG/Attributes.cgi
Disallow: /FIG/FigSQL.cgi
				# Don't try the test and maintenance scripts.
Crawl-delay: 6

MCS Webmaster
ViewVC Help
Powered by ViewVC 1.0.3