Topic on Project:Support desk

Disallow: / in robots.txt file

3
Summary by Biologically

Conditions of using Disallow: / in robots.txt file.

Biologically (talkcontribs)

If someone installed MediaWiki in the web root directory (public_html folder), and now trying to created a robots.txt file for the website; they he surely can't use "Disallow: /w/" in robots.txt file.

So, what should he put there- "Disallow: /" ? According to the guide to create robots.txt file in this section - Manual:Robots.txt#Prevent indexing of non-article pages.

Thank you.

Ciencia Al Poder (talkcontribs)

This only makes sense if you have Pretty Urls configured. For example, wiki pages are at /wiki/Page_name, and you want to prevent robots indexing /index.php?title=Page_name, you'll add Disallow: /index.php

Biologically (talkcontribs)

Thank you.