Manual:Short URL/Prevent bots from crawling index.php

Prevent bots from crawling index.php

If you are using short URLs (see note below), you can make sure that search engines only index actual wiki pages, without indexing action views (such as edit or history pages, with URLs in the form index.php?title=Main_page&action=edit).

Create a file named robots.txt in the root of your MediaWiki installation with the following content.

User-agent: *
Disallow: /index.php

Note: Creating a robots.txt file in the root directory of your MediaWiki installation with Disallow: /index.php without creating a short URL first will block all pages from being indexed. This is because your MediaWiki page will still have a index.php in the title, which the robots.txt file will "disallow".

If you are using long URLs, follow the link below.

See also

• Manual:robots.txt