How to Custom Robots Txt Generator and setup for Blogger [Update guide 2022 ]

Agami Computer Education is the best computer training institute in Tangail. We are training a special course like Office Management, Database Design, Graphic design, Web Design, and Outsourcing.

If you want to get your blog listed and crawl your pages quickly then you must add this custom robots.txt file to your blogger blog. as well as it's a section of search engine optimization so you should be aware of the terms.  

Now we describe How to Custom Robots Txt Generator and set up for Blogger [Update guide 2022 ]

Custom Robots Txt Generator and setup for Blogger [ Update Guide 2022]

How It Works?Robots.txt is a command for search engine robots to explore or browse pages of our blog. Robots.txt has arguably filtered our weblog from search engines.

Let's say robotic wants to visit a Webpage URL, for example, http://www.example.com/about.html. Before it does so, it will take a look at http://www.example.com/robots.txt, and then it will get admission to the particular webpage.

All the blogs already have a robots.txt file given by means of blogger/Blogspot. By default robots.txt on blogs like this: 

User-agent: Media partners-Google

Disallow:

User-agent: *

Disallow: /search

Allow: /

Sitemap:http://yourblogname.com/atom.xml?redirect=false&start-index=1&max-results=500

Is The Meaning Of Above Codes?

User-agent: Media partners-Google

This command tells your blog to allow Adsense bots to crawl your blog. If you’re not using Google Adsense on your blog then simply remove this line.

  

Disallow: 

This command prevents Search Engine bots from crawling pages on your blog. 

User-agent: *

All Robot Search Engines / Search engine

Disallow: / search

Not allowed to crawl the search folder, like... / search/label and ... search/search? Updated ... here label is not inserted to search because the label is not a URL who estate towards one specific page. 

Example : 

http://www.tutbig.blogspot.com/search/label/SEOhttp://www.tutbig.blogspot.com/search/SEO

Allow: /

This command tells allow all pages to be crawled, except that written on Disallow above. Mark (/) or less.

Sitemap:   

Sitemap:http://yourblogname.com/atom.xml?redirect=false&start-index=1&max-results=500

How To Add Custom Robots.txt File In Blogger?

Note: Before adding a custom robots.txt file in blogger you should keep one thing in your mind if you are using the robots.txt file incorrectly then your entire blog is being ignored by search engines.

Step1

Go to your Blogger Dashboard

Step2

Now go to Settings >> Search Preferences >> Crawlers and indexing >> Custom robots.txt >> Edit >> Yes

How to Custom Robots Txt Generator and setup for Blogger [ Update Guide 2022]

Step3 

Now paste below this custom robots.txt file code in the box.

User-agent: Media partners-Google

Disallow:

User-agent: *

Disallow: /search

Allow: /

Sitemap:http://yourblogname.com/atom.xml?redirect=false&start-index=1&max-results=500

Step4 

Now click on the Save Changes button.

Step5 

That's all, You are done! now you can see your robot.txt file. To view the robots.txt fine, please type in the browser... 

http://yourblogname.com/robots.txt

How to Custom Robots Txt Generator and setup for Blogger [ Update Guide 2022]

It's okay, your blog will still be crawled by search engine robots because as I mentioned before, every blog already possessed default robots.txt. 

I hope you enjoy this post and the photos. Custom Robots Txt Generator for Blogger

Agami Computer Education is the best computer training institute in Tangail. We are training a special course like Office Management, Database Design, Graphic design, Web Design, and Outsourcing.