Ask a Question

What is robots.txt ?

0 votes
91 views
in Discuss by
edited
What is robots.txt ?

1 Answer

0 votes
by
selected
 
Best answer
robots.txt is a text file that instructs web robots (search engine robots) about how to crawl pages on their website. It is used to manage crawler traffic to the website.

Related questions

0 votes
1 answer 127 views
Which is better — Meta Robot tags or robots.txt?
asked Aug 10, 2022 in Discuss by anonymous
0 votes
1 answer 292 views
What is the extension of java code files? a) .js b) .txt c) .class d) .java
asked Jul 20, 2022 in JAVA by Doubtly (98.9k points)
0 votes
1 answer 140 views
what is Tailwind CSS ?
asked Sep 13, 2022 in Discuss by Doubtly (98.9k points)
0 votes
1 answer 114 views
What is GNU? General Public License
asked Aug 27, 2022 in Discuss by Doubtly (98.9k points)
0 votes
1 answer 124 views
What is the difference between free and paid WordPress theme?
asked Aug 27, 2022 in Discuss by Doubtly (98.9k points)
0 votes
1 answer 139 views
What is the difference between custom themes and normal themes?
asked Aug 27, 2022 in Discuss by Doubtly (98.9k points)

Doubtly is an online community for engineering students, offering:

  • Free viva questions PDFs
  • Previous year question papers (PYQs)
  • Academic doubt solutions
  • Expert-guided solutions

Get the pro version for free by logging in!

Most popular tags

jeemain geography-10th engineering physics semester 5 web computing

5.7k questions

5.1k answers

108 comments

563 users

Doubtly
...