User-Agent: *
Disallow: /directory.bml

User-agent: *
Crawl-delay: 5

#
# Blocked journals aren't listed here because robots.txt files
# can't be above 50k or so, depending on the spider.
# 
# Instead, blocked journals have HTML inserted in them which
# should prevent behaved spiders from indexing it.
#
# Note that http://username.blurty.com journals have an
# autogenerated robots.txt, since it can be small.
#