Once your robots.txt file has been created according to requirements and made live, if not already, it’s important to check to make sure it’s valid.
Google has a free tool to do just that, which can be found here:
https://www.google.com/webmasters/tools/robots-testing-tool
Simply submit a URL to the robots.txt Tester tool, and it will flag up any errors or warnings.