MobileMoxie Glossary
XRobots Tag
The XRobots Tag programmatically set various robot.txt-related directives in the headers of a HTTP response, normally done in the .htaccess file. This can be done in Nginx and Apache server configurations. This means of providing search engines crawling instructions can be quite destructive if errors are included, so proceed with caution when using the XRobots Tag.