Meta tags can contain instructions for crawlers. The most important instructions are INDEX or NOINDEX and FOLLOW or NOFOLLOW.

INDEX/NOINDEX: Tells if the page should be indexed by search engines. Indexed pages are displayed in the SERPs. If the webmaster doesn't want particular pages to be displayed in the search results, he can leave the “NOINDEX” instruction in the meta tags.

FOLLOW/NOFOLLOW: Tells crawlers if they should follow the links on the page. When crawlers follow a link, they take a small part of the page's page rank with them to the linked page.


<meta name="robots" content="INDEX,NOFOLLOW"/>


  1. You can also leave instructions for the crawlers in the robots.txt. These instructions will cover the entire website.
  2. Use the Metatag Generator to automatically generate the meta tags. This way you don't need to worry about the correct HTML syntax anymore.