In Blogger, Crawlers and Indexing status settings stay immaculate naturally.
It doesn't mean your blog will not get recorded, yet it will require generally greater investment to file your blog entries. Since Crawlers will slither everything on your site including superfluous pages like filed pages.
In this way, you need to empower the Crawlers and ordering settings and redo it. It will assist the Crawler with understanding which presents and pages on creep, and your posts will get filed a lot quicker.
10 point Custom robots header tags you have to know
When you empower the choice for custom robots header labels, you will see that Home page labels, Archive and Search page labels, and Post and Page labels choices are accessible. Allow me just to make sense of the implications of those labels.
- all: it means there aren't any restrictions for crawlers to crawl or index any page.
- noindex: if this option is selected for any page, then the crawler can't crawl or index that page.
- nofollow: blog posts contain both internal and external links. Crawlers automatically crawl all the links on any webpage, but if you don't want crawlers to crawl any links on any page, then you have to enable this option.
- none: it means both "noindex" and "nofollow" at once.
- noarchive: it will remove any cached link from search.
- nosnippet: it will stop search engines to show snippets of your web pages.
- noodp: it will remove titles and snippets from open directory projects like DMOZ.
- notranslate: it will stop crawlers to show translation on pages.
- noimageindex: in this case, your post will be crawled and indexed, but all the images will not be indexed.
- unavailable_after: this option helps to noindex any post after a particular time.
- Custom robot tags for the home page: all, noodp
- Custom robot tags for archive and search pages: noindex, noodp
- Custom robot tags for posts and pages: all, noodp