0 votes
86 views
in Discuss by
edited
What is robots.txt ?

1 Answer

0 votes
by
selected
 
Best answer
robots.txt is a text file that instructs web robots (search engine robots) about how to crawl pages on their website. It is used to manage crawler traffic to the website.

Related questions

0 votes
1 answer 121 views
asked Aug 10, 2022 in Discuss by anonymous
0 votes
1 answer 283 views
0 votes
1 answer 131 views
asked Sep 13, 2022 in Discuss by Doubtly (98.9k points)
0 votes
1 answer 107 views
asked Aug 27, 2022 in Discuss by Doubtly (98.9k points)
0 votes
1 answer 113 views
0 votes
1 answer 127 views

Doubtly is an online community for engineering students, offering:

  • Free viva questions PDFs
  • Previous year question papers (PYQs)
  • Academic doubt solutions
  • Expert-guided solutions

Get the pro version for free by logging in!

5.7k questions

5.1k answers

108 comments

554 users

...