Ask a Question

What is robots.txt ?

0 votes
75 views
in Discuss by
edited
What is robots.txt ?

1 Answer

0 votes
by
selected
 
Best answer
robots.txt is a text file that instructs web robots (search engine robots) about how to crawl pages on their website. It is used to manage crawler traffic to the website.

Related questions

0 votes
1 answer 105 views
Which is better — Meta Robot tags or robots.txt?
asked Aug 10, 2022 in Discuss by anonymous
0 votes
1 answer 230 views
What is the extension of java code files? a) .js b) .txt c) .class d) .java
asked Jul 20, 2022 in JAVA by Doubtly (98.9k points)
0 votes
1 answer 118 views
what is Tailwind CSS ?
asked Sep 13, 2022 in Discuss by Doubtly (98.9k points)
0 votes
1 answer 89 views
What is GNU? General Public License
asked Aug 27, 2022 in Discuss by Doubtly (98.9k points)
0 votes
1 answer 103 views
What is the difference between free and paid WordPress theme?
asked Aug 27, 2022 in Discuss by Doubtly (98.9k points)
0 votes
1 answer 115 views
What is the difference between custom themes and normal themes?
asked Aug 27, 2022 in Discuss by Doubtly (98.9k points)

Doubtly is an online community for engineering students, offering:

  • Free viva questions PDFs
  • Previous year question papers (PYQs)
  • Academic doubt solutions
  • Expert-guided solutions

Get the pro version for free by logging in!

Most popular tags

jeemain geography-10th engineering physics semester 5 web computing

5.7k questions

5.1k answers

108 comments

537 users

Doubtly
...