Scribd
Upload
35 views1 page

User-Agent: Bingbot Crawl-Delay: 60 User-Agent: Msnbot Crawl-Delay: 60 User-Agent: Msnbot-Udiscovery/2.0B Crawl-Delay: 60 User-Agent: Crawl-Delay: 60

This document provides crawl-delay settings for various bots including bingbot, msnbot, and msnbot-UDiscovery, instructing them to wait 60 seconds between requesting pages on this site. Any other bots are also given a crawl delay of 60 seconds to prevent overloading the site.

Uploaded by

subashini udayakumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
35 views1 page

User-Agent: Bingbot Crawl-Delay: 60 User-Agent: Msnbot Crawl-Delay: 60 User-Agent: Msnbot-Udiscovery/2.0B Crawl-Delay: 60 User-Agent: Crawl-Delay: 60

This document provides crawl-delay settings for various bots including bingbot, msnbot, and msnbot-UDiscovery, instructing them to wait 60 seconds between requesting pages on this site. Any other bots are also given a crawl delay of 60 seconds to prevent overloading the site.

Uploaded by

subashini udayakumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
You are on page 1/ 1

User-agent: bingbot

Crawl-delay: 60

User-agent: msnbot
Crawl-delay: 60

User-agent: msnbot-UDiscovery/2.0b
Crawl-delay: 60

User-agent: *
Crawl-delay: 60

You might also like