"Robots" meta tag


The robots meta tag has instructions for crawlers in it. Crawlers look for this file and read it. Hello Engines! checks if the meta tag allows indexing your website because this is how your website gets listed in search engines.