What is the robots.txt file?
The robots.txt file is a standard text file that tells search engine crawlers which pages they can access, scrape, and ultimately list in their search engine results.
The file is listed on the root URL of your site, https...
