Bing Webmaster Tools makes it easy to edit and verify your robots.txt

Since the dawn of the search engine age, webmasters have been looking for a reliable and efficient tool to foster and control their love-hate relationship with web robots/crawlers/spiders. While Robots exclusion protocol gives the power to inform web robots and crawlers which sections of a website should not be processed or scanned, the growing number of search engines and parameters have forced webmasters to search for their Robots.txt file amongst the millions of folders on their hosting servers, editing them without guidance and finally scratching heads as the issue of that unwanted crawler still persists.

We at Bing understand that frustration of our users and hence have come up with our new and enhanced Robots.txt tester tool. The Robots.txt tester helps webmasters to not only analyse their robots.txt file and highlight the issues that would prevent them from getting optimally crawled by Bing and other robots; but, also guides them step-by-step from fetching the latest file to uploading the same file at the appropriate address.

Webmasters can submit a URL to the robots.txt Tester tool and it operates as Bingbot and BingAdsBot would, to check the robots.txt file and verifies if the URL has been allowed or blocked accordingly.

Not only this, but the test functionality checks the URL which we have submitted against the content of the editor  and hence, once changes are made in the editor, you can easily and instantly retest the URL to check for errors. The system checks for allow/disallow statements for respective user agents and displays the robots.txt file in the editor with 4 variations i.e. http://, https://, http://www, https://www. Webmaster can edit the txt file and/or download the same to be updated offline. If there have been changes to the Robots file elsewhere and updated, the webmaster can use the Fetch latest option to get the latest Robots file of the property.

The download option provides a step by step process of updating the file that includes downloading the edited file, uploading the same on the domains root which can be checked for a live version and lastly requesting Bing to update the same.
We are excited to present this feature to our users, especially now that we have migrated to a new and improved platform and experience. We are here listening to your feedback as we urge you to reach out to us on Twitter and share how you feel about the tool and its effect on the empowering webmasters to increase their productivity. If you encounter any issues, please contact our support team.
 
Regards
Kumar Snehansu
Bing Webmaster Tools team

Source: Bing Blog Feed