The meta tag “robots” is useful to manage the indexing or rather the non-indexation of certain web pages.
This tag application to search engines not to index the page, that is to say, never to be included in search results.The which allows indexing is unnecessary because indexing is allowed by default.
This tag application to search engines not to follow links in the page. Must still agree on what it means to “follow the links.”
Officially, the search engines include those links in the list of links that indexing is allowed, but they give no weight (or PageRank or “link juice”) in the presence of links on the page.The which allows tracking links is unnecessary since this “monitoring” is allowed by default.
This tag combines the effects of the “noindex” and “nofollow”. It is exactly equivalent to whose use can be confusing.It is also preferable to avoid using multiple meta tags “spam” like this:
Tags for a particular search engine
The same kinds of tags can be utlises to give instructions to a particular search engine instead whether for all search engines:
* only for Google.
* only for Yahoo.
* only affects Microsoft search engines.
Some general remarks
META tags are always placed in the first part of the source, that is to say between the and . Some search engines ignore the meta tag “robots”. This is particularly true of the Internet Archive Wayback Machine.
The meta tag “robots” is totally insensitive. The use of uppercase or lowercase is no particular effect.
If multiple values are specified for CONTENT, they must be separated by a comma, possibly followed by one or more spaces.
The allows indexing and tracking links. It is useless because indexing and “monitoring” are allowed by default.
If you ban a page in robots.txt, it will not be visited by robots. The content of any meta tag “robots” on this page will never be read by search engine robots. They can therefore be ignored.
This tag requires search engines to never display a “cache”. Any search engine must keep secret the contents of the page as it was archived. This tag does not prohibit search engines utliser otherwise the page content, or the appearance of the page in search results.
This tag is used to force visitors to go to the website itself, rather than a cached version of the site. It is sometimes used to conceal the cloaking (a technique that shows a different page to search engines and visitors).
This tag requires search engines to not show a “snippet” in the search results. The “snippet” is the short description or excerpt of the page that is displayed in search results.
For Google, this tag also prohibits the display of “cache”.
This tag application to search engines not to use the title and description from DMOZ in search results. It is intended for search engines Google, Yahoo and Microsoft.
This tag application to search engines not to use the title and description from the Yahoo Directory Directory in search results. It addresses the search engine Yahoo.