Update: I would recommend you to use Yoast WordPress Seo plugin as it have better option with extra setting.
<Meta> Tag is a Special tag which tell robots not to index any web page, and/or not to follow it. It is a method to tell search engine which is to be indexed,followed or no-index,no-follow any web page of your site.
<html> <head> <title>...</title> <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"> </head>
Which search engine supports which robots meta tag values ?
< Image Source:Yoast >
Other meta tags are noarchive, nocache, nosnippet, noodp, noydir ,which have different function to do. Using meta tags in all pages is a lengthy and difficult task. However if you use WordPress as Cms then there is plugin called Robots Meta developed by Yoast will do the work for you.
Features of this Robots Meta Plugin?
Robots Meta Plugin makes it possible (and very easy) to:
- Prevent indexing of your page in search engine.
- Prevent indexing of the subpages to your homepage, by adding nofollow tag.
- Prevent indexing login page and other admin section form your site.
- Nofollow the category listings on single posts and pages to prevent PageRank leakage.
- You can Edit your robots.txt and .htaccess from within your WordPress.
- Prevent indexing of just your comment feeds.
- Nofollow tag links.
Why you should use Robots Meta Plugin?
Robots Meta Plugin is very useful and important for your site as it helps in Search engine Optimization by making site friendly.It is also important to use it after Google’s Panda Update , so that your site doesn’t get affected.
How to install and configure Robots Meta Plugin?
First of all install and activate Robots Meta Plugin from your WordPress admin area. Now for setting Go to WordPress Admin >Setting>Robots Meta.
Here are the Best Configuration for Robots Meta Plugin .
- First option is of verbose explanations of settings,which show or hide information about each setting.
- RSS Feeds: From here you can prevent the search engines from indexing your comment feeds and Rss Feeds.It is highly Discouraged.
- Prevent indexing :From this section you can prevent site’s search result pages, login and register pages, Sub-pages of the homepage, Author archives and Date-based archives from indexing in Search Engine.It is highly recommended to add no index in above pages as not to loose link juice.
Other option is for Category archives, Tag archives and noarchive meta tag which is optional and is completely depend upon on Webmaster. However it is now Recommended to add noindex in Categories archives and Tag archives to stay safe for Google’s Panda (Content Farming) Updates.
- Domz and Yahoo Directory
Noodp meta tag, prevents all search engines from using the DMOZ description for your site in the search results and Noydir meta robots tag prevents Yahoo! from using the Yahoo! directory description for your site in the search results.It is completely optional and you can leave this to default.
- Permalink Setting
It will enforce a trailing slash on all category and tag URL’s ,if you choose a permalink for your posts with .html, or anything else but a / on the end, this will force WordPress to add a trailing slash to non-post pages nonetheless.It is good for Seo and site load faster by doing this.So it is Recommended.
- Archive Setting
It has following option Disable the author archives, and Redirect search results pages when referrer is external.It is recommended to check all option for better SEO result.
- Internal Nofollow Setting
It is very important part of your blog as Internal linking is considered as one of the SEO factor through which site rank well in Google.Form my views and Knowledge check all option for better result.
You can Also Watch Video by Yoast.
Disclaimer: All Views and suggestion are my personal views and this site or even Author is not responsible for any Unfortunate happenings or Problems.