# Excluded spiders
User-agent: AhrefsBot
User-agent: Yandex
User-agent: CCBot
User-agent: MJ12bot
User-agent: dotbot
User-agent: spbot
User-agent: Baiduspider
User-agent: psbot
User-agent: Exabot
User-agent: JamesBOT
User-agent: SemrushBot
User-agent: MegaIndex.ru
User-agent: BLEXBot
Disallow: /
Noindex: /

# Throttling
User-agent: *
Crawl-delay: 2

# Coldfusion specific
Disallow: /*?*CFTOKEN*
Disallow: /*?*CFID*