RED
bot
Type in a URI here and press 'return' to check it. You can also specify request headers by clicking 'add a request header.'
news:
REDbot and robots.txt