Description:
Disallows robots from indexing your site unless explicitly allowed.
(Robots link in Visitors tab)
(Example of Robots page)
(Example of Options page)
(Example of User group permissions)
Features:
All phrases start with robots_ for your convenience.
User group permissions:
Normally only the Administrative user group is set the 'Yes' in order to see the Robots link in the visitor tab menu.
Questions and Answers:
Q: What is the purpose of this add-on?
A: Lots of robots crawling your forum will slow it down.
Q: I noticed a robot crawling my site but it's identified as a guest.
A: You need to add the name to the Blacklist field in the Options page.
Q: What happens if a robot which is not allowed comes to my site?
A: It receives a blank page.
Q: Why not use the robots.txt file to restrict robots?
A: Because robots ignore this file.
Database modifications:
This add-on will create a table called xf_robots in your database.
Disallows robots from indexing your site unless explicitly allowed.
(Robots link in Visitors tab)
(Example of Robots page)
(Example of Options page)
(Example of User group permissions)
Features:
All phrases start with robots_ for your convenience.
User group permissions:
Normally only the Administrative user group is set the 'Yes' in order to see the Robots link in the visitor tab menu.
Questions and Answers:
Q: What is the purpose of this add-on?
A: Lots of robots crawling your forum will slow it down.
Q: I noticed a robot crawling my site but it's identified as a guest.
A: You need to add the name to the Blacklist field in the Options page.
Q: What happens if a robot which is not allowed comes to my site?
A: It receives a blank page.
Q: Why not use the robots.txt file to restrict robots?
A: Because robots ignore this file.
Database modifications:
This add-on will create a table called xf_robots in your database.