Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Doesn't matter.

Even by the typical robots.txt definition, a bot has to operate more or less fully autonomously. If the user initiated the request, it doesn't count, and it shouldn't follow robots.txt.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: