MediaWiki

Manual:Memcached

Translate this page
Languages:

memcached is an in-memory object store that MediaWiki can use for caching values, to reduce the need to perform expensive computations and to reduce load on the database servers.

When to use

For a small website hosted by a single server, installing Memcached might not be worth the trouble. For such cases, consider configuring MediaWiki to use PHP's APCu instead as the main object store. For large websites like Wikipedia and in general for wikis hosted by multiple web servers, Memcached is a common choice for MediaWiki's object cache.

For more about caching options in MediaWiki, see Manual:Performance tuning § Object caching

Install Memcached

Most package managers for Linux and macOS have ready-to-use packages for Memcached available (including for Debian, Fedora, and Ubuntu).

If there's no package available for your distribution, you may need to compile it from source by downloading from the memcached.org. To compile from source you will also need libevent. Memcached and libevent are open source projects released under BSD-style licenses.

For more about Memcached in general, see also "Memcached" on Wikipedia.

Security

Memcached has no security or authentication. Please ensure that your server is appropriately firewalled, and that the port(s) used for memcached servers are not publicly accessible. Otherwise, anyone on the internet can put data into and read data from your cache.

An attacker familiar with MediaWiki internals could use this to give themselves developer access and delete all data from the wiki's database, as well as getting all users' password hashes and email addresses.

PHP client for Memcached

As of this writing (MediaWiki 1.27), MediaWiki uses a pure-PHP memcached client (based on work by Ryan T. Dean). It also supports the php-memcached PECL extension. To use Memcached with MediaWiki, PHP must be compiled with --enable-sockets (this is the default).

If you want to use the PECL based memcached client, set $wgMainCacheType and related cache variables to 'memcached-pecl' instead CACHE_MEMCACHED. The PECL memcached client supports some additional features like SASL authorization which may be useful to some users.

To read more about how to select Memcached as the backend for different parts of MediaWiki, see Manual:Caching.

Setup

If you want to start small, just run one memcached on your web server: 

 memcached -d -l 127.0.0.1 -p 11211 -m 64

(to run in daemon mode, accessible only via loopback interface, on port 11211, using up to 64MB of memory)

In your LocalSettings.php file, set: 

$wgMainCacheType = CACHE_MEMCACHED;
$wgParserCacheType = CACHE_MEMCACHED; // optional
$wgMessageCacheType = CACHE_MEMCACHED; // optional
$wgMemCachedServers = [ '127.0.0.1:11211' ];

$wgSessionsInObjectCache = true; // optional -- removed in 1.33+
$wgSessionCacheType = CACHE_MEMCACHED; // optional

The wiki should then use memcached to cache various data. To use multiple servers (physically separate boxes or multiple caches on one machine on a large-memory x86 / Power box), just add more items to the array. To increase the weight of a server (say, because it has twice the memory of the others and you want to spread usage evenly), make its entry a subarray: 

$wgMemCachedServers = [
    '127.0.0.1:11211', // 1 GB on this box
    [ '192.168.0.1:11211', 2 ] // 2 GB on the other box
];

SELinux

For systems with SELinux, there are several policies for Memcached. To allow Apache (httpd) to access Memcached, you must set the following policy: 

 setsebool -P httpd_can_network_memcache 1

Troubleshooting

Loss of session data when saving

If you store session data in memcached, and users see this message intermittently when they try to save edits:

Sorry! We could not process your edit due to a loss of session data.

You might have been logged out. Please verify that you're still logged in and try again. If it still does not work, try logging out and logging back in, and check that your browser allows cookies from this site.

then one or more of your memcached servers might have a misconfigured /etc/hosts file. On each of your memcached servers, make sure the server's own hostname is mapped to localhost: 

127.0.0.1  servername.here localhost localhost.localdomain ...

Otherwise, the server might not be able to connect to its own memcached process.

Using memcached in your code

If you're writing an extension that does expensive database queries, it might be useful to cache the data in memcached. There are a few main ways to get a handle to memcached: 

  • $cache = ObjectCache::getMainWANInstance()
    ...use this if you want a memory-based shared cache with explicit purge ability in order to store values derived from persistent sources
  • $cache = ObjectCache::getLocalClusterInstance()
    ...use this if you want a memory-based ephemeral store that is not shared among datacenters
  • $cache = ObjectCache::getLocalServerInstance()
    ...use this if you want a memory-based ephemeral cache that is not shared among web servers
  • $cache = wfGetCache( CACHE_ANYTHING )
    ...use this if you want any available cache, which may or may not be per-datacenter, even an emulated one that uses a SQL database. Note that these may return handles that talk to Redis, APC, MySQL or other stores instead. The use of the word "memcached" is historically due to the API being defined around the simple commands that memcached supports and the fact that, to date, memcached is normally the best general-purpose cache store.

Extensions that have specific needs (like persistence) should define new configuration settings like $wgMyExtCache or $wgMyExtWANCache. Code using the caches can pass them to wfGetCache() and ObjectCache::getWANInstance(), respectively.

The following code snippet demonstrates how to cache the results of a database query into memcached for 15 minutes and query memcached first for the results instead of the database. 

class MyExtensionFooBars {
	public function getPopularTen() {
		$cache = ObjectCache::getMainWANInstance();
		
		return $cache->getWithSetCallback(
			// The variadic arguments to wfMemcKey() are used to construct the key for the cache in memcached. 
			// It must be unique to the query you are saving.
			// The first value is normally the extension or component name, and following values tell you what query you are saving.
			$cache->makeKey( 'myextension', 'foobars', 'popular', '10' ),
			// Cache for 15 minutes
			$cache::TTL_MINUTE * 15,
			// Function to generate the value on cache miss
			function ( $oldValue, &$ttl, &$setOpts ) {
				$dbr = MediaWikiServices::getInstance()->getConnectionProvider()-> getReplicaDatabase();
				// Adjust TTL based on DB replication lag
				$setOpts = Database::getCacheSetOptions( $dbr );

				$res = $dbr->select(
					// your database query goes here
					// see Database::select for docs
				);

				$data = array();
				foreach ( $res as $row ) {
					$data[] = array(
						// Do something with the data we just got from the database.
						// For example, if we looked up page_id from the page table, we could do this:
						'id' => $row->page_id
					);
				}

				return $data;
			}
		);
	}
}

The abstract BagOStuff and WANObjectCache classes define and document all of the available functions:

Old Development Notes

Broadly speaking, we'd like to be able to dump lots of data in the cache, use it whenever we can, and automatically expire it when changes are made.

Expiration model

  • explicit expiration times: memcached lets us set an expiration time on an object when we store it. After the time is up, another request for the object will find that it has expired and return nothing to us.
    • pro: last-ditch fallback to let data that could be updated badly eventually fall out of the cache
    • con: we have to know ahead of time when it will cease to be invalid. hard to do when we're dealing with user edits!
  • delete cached objects when we know we're doing something that will cause them to be invalid but are not in a position to update them while we're at it
    • pro: fairly simple; the item will be reloaded from the database and recached when it's next needed
    • con: if this will affect a large number of related items (for instance, creating or deleting a page invalidates the links/brokenlinks tables and rendered HTML cache of pages that link to that page) we may have to hunt them all down and do a lot of updating
  • include timestamps on cached objects and do our own expiries based on dependencies
    • pro: can expire many objects at once by updating a single node they depend on
    • con: more things to load; multiple dependencies could be trickier to work with

Questions & Answers

Q: Can I search on part of a key or a regular expression on a Memcached server?

A: No, you can only search for an exact key if you need more information on what you could possibly do you can check out the Memcached protocol

Q: Can I have multiple wikis point to the same Memcached server?

A: Yes, as long as each have different wiki-ids ($wgDBname ). Certain cache keys are intentionally shared in such a scenario, such as rate limiting stuff.

See also

Retrieved from "https://www.mediawiki.org/w/index.php?title=Manual:Memcached/en&oldid=6674990"