MediaWiki

Extension:SpamBlacklist

This page is a translated version of the page Extension:SpamBlacklist and the translation is 19% complete.
Outdated translations are marked like this.
Languages:
Diese Erweiterung ist in MediaWiki 1.21 und darüber enthalten. Daher muss sie nicht erneut heruntergeladen werden. Allerdings müssen die folgenden Anweisungen trotzdem beachtet werden.
MediaWiki-Erweiterungen
SpamBlacklist
Freigabestatus: stabil
Einbindung Seitenaktivität
Beschreibung Stellt einen Regex-basierten Spam-Filter bereit
Autor(en) Tim StarlingDiskussion
Letzte Version Laufende Aktualisierungen
Kompatibilitätspolitik Snapshots werden zusammen mit MediaWiki veröffentlicht. Der Master ist nicht abwärtskompatibel.
MediaWiki 1.31+
Lizenz GNU General Public License 2.0 oder neuer
Herunterladen
Erweiterung herunterladen
Git [?]:

README
Parameter
  • $wgBlacklistSettings
  • $wgLogSpamBlacklistHits
Hinzugefügte Rechte
  • sboverride
  • spamblacklistlog
Verwendete Hooks
Quarterly downloads 40 (Ranked 96th)
Public wikis using 4,601 (Ranked 180th)
Übersetze die SpamBlacklist-Erweiterung, wenn sie auf translatewiki.net verfügbar ist
Probleme Offene Aufgaben · Einen Fehler melden
A proposal to rename this extension is discussed at task T254649.

Die Erweiterung SpamBlacklist verhindert Bearbeitungen in denen URLs vorkommen, deren Domains zu Regulären Ausdrücken passen, die in spezifizierten Dateien oder Wiki-Seiten definiert sind. Außerdem wird die Registrierung von Benutzern mit speziellen E-Mail Adressen gesperrt.

Wenn versucht wird, eine Seite zu speichern, prüft die Erweiterung den Text gegen eine (potentiell große) Liste illegaler Hostnamen. Wird ein Treffer gefunden, zeigt die Erweiterung dem Benutzer eine Fehlermeldung und verweigert das abspeichern der Seite.

Installation und Einrichtung

Installation

  • Die Erweiterung herunterladen und die Datei(en) in ein Verzeichnis namens SpamBlacklist im Ordner extensions/ ablegen.
    Entwickler und Code-Beitragende sollten stattdessen die Erweiterung von Git installieren, mit:cd extensions/
    git clone https://gerrit.wikimedia.org/r/mediawiki/extensions/SpamBlacklist
  • Folgenden Code am Ende deiner LocalSettings.php -Datei einfügen: 
    wfLoadExtension( 'SpamBlacklist' );
  • Konfiguration der Blockliste nach Bedarf
  •   Erledigt – Navigiere zu Special:Version in deinem Wiki, um zu überprüfen, ob die Erweiterung erfolgreich installiert wurde.

Einrichtung der Blockliste

Die folgenden lokalen Seiten werden unabhängig von zusätzlichen Quellen immer verwendet:

Die Voreinstellung für eine zusätzliche Quelle einer Blockliste ist die Wikimedia Spam-Blackliste auf Meta-Wiki (m:Spam block list). Standardmäßig nutzt die Erweiterung diese Liste und aktualisiert sie alle 10-15 Minuten. Für viele Wikis wird diese Liste ausreichen um die meisten Spam-Attacken zu verhindern. Seitdem die Blockliste von vielen großen Wikis verwendet wird, ist sie allerdings vergleichsweise konservativ in ihrem Blockierverhalten.

The Wikimedia spam blacklist can only be edited by administrators, but you can suggest modifications to the blacklist at m:Talk:Spam blacklist.

You can add other bad URLs on your wiki. List them in the global variable $wgBlacklistSettings in LocalSettings.php . See examples below.

$wgBlacklistSettings is an two level array. Top level key is spam or email. They take an array with each value containing a URL, a filename, or a database location.

If you use $wgBlacklistSettings in "LocalSettings.php", the default value of "[[m:Spam blacklist]]" will no longer be used - if you want that blacklist to be accessed, you will have to add it in manually, see examples below.

Specifying a database location allows you to draw the blacklist from a page on your wiki.

The format of the database location specifier is ">DB: [db name] [title]". [db name] should exactly match the value of $wgDBname in LocalSettings.php .

You should create the required page name [title] in the default namespace of your wiki. If you do this, it is strongly recommended that you protect the page from general editing. Besides the obvious danger that someone may add a regex that matches everything, please note that an attacker with the ability to input arbitrary regular expressions may be able to generate segfaults in the PCRE library.

Beispiele

If you want to, for instance, use the English-language Wikipedia's spam blacklist in addition to the standard Meta-Wiki one, you could call the following in LocalSettings.php , AFTER wfLoadExtension( 'SpamBlacklist' ); call: 

$wgBlacklistSettings = [
	'spam' => [
		'files' => [
			"https://meta.wikimedia.org/w/index.php?title=Spam_blacklist&action=raw&sb_ver=1",
			"https://en.wikipedia.org/w/index.php?title=MediaWiki:Spam-blacklist&action=raw&sb_ver=1"
		],
	],
];

Here is an example of an entirely local set of blacklists: the administrator is using the update script to generate a local file called "wikimedia_blacklist" that holds a copy of the Meta-Wiki blacklist and has an additional blacklist on the wiki page "My spam blacklist": 

$wgBlacklistSettings = [
	'spam' => [
		'files' => [
			"$IP/extensions/SpamBlacklist/wikimedia_blacklist", // Wikimedia's list
			// database, title
			'DB: wikidb My_spam_block_list',    
		],
	],
];

Protokollierung

By default, the extension does not log hits to a spam blacklist log. To enable logging set $wgLogSpamBlacklistHits = true;. You can use the spamblacklist user right to control access to the logs. Every signed-in user can view the logs by default.

Problemfälle

Backtrack Limit

If you encounter issues with the blacklist, you may want to increase the backtrack limit.

However, on the other hand, this can reduce your security against DOS [1] attacks, as the backtrack limit is a performance limit: 

// Bump the Perl Compatible Regular Expressions backtrack memory limit
// (PHP 5.3.x default, 1000K, is too low for SpamBlacklist)
ini_set( 'pcre.backtrack_limit', '8M' );

Hardened Wikis

The SpamBlacklist will not allow editing if the wiki is hardened. Hardening includes limiting open_basedir so that curl is not on-path, and setting allow_url_fopen=Off in php.ini.

In the hardened case, SpamBlacklist will cause an exception when Guzzle attempts to make a network request. The Guzzle exception message is, GuzzleHttp requires cURL, the allow_url_fopen ini setting, or a custom HTTP handler.

Safe list

A corresponding safe list can be maintained by editing the MediaWiki:Spam-whitelist page. This is useful if you would like to override certain entries from another wiki's blacklist that you are using. Wikimedia wikis, for instance, sometimes use the spam blacklist for purposes other than combating spam.

It is questionable how effective Wikimedia spam blacklists are at keeping spam off of third-party wikis. Some spam might be targeted only at Wikimedia wikis or only at third-party wikis, which would make Wikimedia's blacklist of little help to said third-party wikis in those cases. Some third-party wikis might prefer that users be allowed to cite sources that Wikipedia does not allow. Sometimes, what one wiki considers useless spam, another might consider useful.

Users may not always realize that when a link is rejected as spammy, it does not necessarily mean that the individual wiki they are editing has specifically chosen to ban that URL. Therefore, wiki system administrators may want to edit the Manual:System messages at MediaWiki:Spamprotectiontext and/or MediaWiki:Spamprotectionmatch on your wiki to invite users to make suggestions at MediaWiki talk:Spam-whitelist for pages that should be added by a Manual:Administratoren to the safe list. For example, you could put, for MediaWiki:Spamprotectiontext:

The spam filter blocked the text you wanted to save. This is probably caused by a link to a blacklisted external site. {{SITENAME}} maintains [[MediaWiki:Spam-blacklist|its own blacklist]]; however, most blocking is done by means of [[metawikimedia:Spam-blacklist|Meta-Wiki's blacklist]], so this block should not necessarily be construed as an indication that {{SITENAME}} made a decision to block this particular text (or URL). If you would like this text (or URL) to be added to [[MediaWiki:Spam-whitelist|the local spam safe list]], so that {{SITENAME}} users will not be blocked from adding it to pages, please make a request at [[MediaWiki talk:Spam-whitelist]]. A [[Project:Sysops|sysop]] will then respond on that page with a decision as to whether it should be listed as safe.

Notes

  • This extension examines only new external links added by wiki editors.

To check user agents, add Akismet As the various tools for combating spam on MediaWiki use different methods to spot abuse, the safeguards are best used in combination.

  • Users with the sboverride can override the blacklist and add blocked links to pages. By default, this right is only given to bots.

Verwendung

Syntax

If you would like to create a blacklist of your own or modify an existing one, here is the syntax:

Everything on a line after a '#' character is ignored (for comments). All other strings are regex fragments which will only match inside URLs.

Notes
  • Do not add "http://"; this would fail, since the regex will match after "http://" (or "https://") inside URLs.
  • Furthermore, "www" is unneeded since the regex will match any subdomains.

By giving "www\." explicitly one can match specific subdomains.

  • The (?<=//|\.) and $ anchors match the beginning and end of the domain name, not the beginning and end of the URL.

The regular anchor ^ won't be of any use.

  • Slashes don't need to be escaped by backslashes. The script will do this automatically.
Beispiel

The following line will block all URLs that contain the string "example.com", except where it is immediately preceded or followed by a letter or a number. 

\bexample\.com\b

These are blocked:

  • http://www.example.com
  • http://www.this-example.com
  • http://www.google.de/search?q=example.com

These are not blocked:

  • http://www.goodexample.com
  • http://www.google.de/search?q=example.commodity

Leistung

The extension creates a single regex statement which looks like /https?:\/\/[a-z0-9\-.]*(line 1|line 2|line 3|....)/Si (where all slashes within the lines are escaped automatically). It saves this in a small "loader" file to avoid loading all the code on every page view. Page view performance will not be affected even if you're not using a bytecode cache, although using a cache is strongly recommended for any MediaWiki installation.

The regex match itself generally adds an insignificant overhead to page saves (on the order of 100ms in our experience). However, loading the spam file from the disk or the database and constructing the regex may take a significant amount of time, depending on your hardware. If you find that enabling this extension slows down saves excessively, try installing a supported bytecode cache. This extension will cache the constructed regex if such a system exists.

If you're sharing a server and cache with several wikis, you may improve your cache performance by modifying getSharedBlacklists and clearCache in SpamBlacklist_body.php to use $wgSharedUploadDBname (or a specific DB if you do not have a shared upload DB) rather than $wgDBname . Be sure to get all references! The regexes from the separate MediaWiki:Spam-blacklist and MediaWiki:Spam-whitelist pages on each wiki will still be applied.

External blacklist servers (RBLs)

This extension requires that the blacklist be constructed manually in its standard form. While regular expression wildcards are permitted, and a blacklist originated on one wiki may be re-used by many others, some effort is still required to add new patterns in response to spam or remove patterns that generate false positives.

Much of this effort may be reduced by supplementing the spam regex with lists of known domains advertised in spam emails. The regex will catch common patterns (like "casino-" or "-viagra") while the external blacklist server will automatically update with names of specific sites being promoted through spam.

In the filter() function in includes/SpamBlacklist.php, approximately halfway between the file start and end, are the lines: 

       # Do the match
       wfDebugLog( 'SpamBlacklist', "Checking text against " . count( $blacklists ) .
           " regexes: " . implode( ', ', $blacklists ) . "\n" );

Directly above this section (which does the actual regex test on the extracted links), one could add additional code to check the external RBL servers [2]: 

        # Do RBL checks
        $retVal = false;
        $wgAreBelongToUs = ['l1.apews.org.', 'multi.surbl.org.', 'multi.uribl.com.'];
        foreach( $addedLinks as $link ) {
              $link_url=parse_url($link);
              $link_url=$link_url['host'];
              if ($link_url) {
                   foreach( $wgAreBelongToUs as $base ) {
                        $host = "$link_url.$base";
                        $ipList = gethostbynamel( $host );
                        if( $ipList ) {
                           wfDebug( "RBL match: Hostname $host is {$ipList[0]}, it's spam says $base!\n" );
                           $ip = wfGetIP();
                           wfDebugLog( 'SpamBlacklistHit', "$ip caught submitting spam: {$link_url} per RBL {$base}\n" );
                           $retVal = $link_url . ' (blacklisted by ' . $base .')';
                           wfProfileOut( $fname );
                           return $retVal;
                        }
                   }
              }
        }

        # If no match is found on the RBL server, continue normally with regex tests...

This ensures that if an edit contains URLs from already blocked spam domains, an error is returned to the user indicating which link cannot be saved due to its appearance on an external spam blacklist. If nothing is found, the remaining regex tests can run normally, so any manually specified 'suspicious pattern' in the URL may be identified and blocked.

Note that the RBL servers list just the base domain names - not the full URL path - so http://example.com/casino-viagra-lottery.html will trigger RBL only if "example.com" itself was blocked by name by the external server. The regex, however, would be able to block on any of the text in the URL and path, from "example" to "lottery" and everything in between. Both approaches carry some risk of false positives - the regex because of the use of wildcard expressions, and the external RBL as these servers are often created for other purposes - such as control of abusive spam email - and may include domains which are not engaged in forum, wiki, blog or guestbook comment spam per se.

Other spam-fighting tools

There are various helpful manuals on mediawiki.org on combating spam and other vandalism:

Other anti-spam, anti-vandalism extensions include:

References

  1. ↑ Denial of Service (DOS)
  2. ↑ Realtime Blackhole List (RBL)
Retrieved from "https://www.mediawiki.org/w/index.php?title=Extension:SpamBlacklist/de&oldid=6893501"