Extension:CentralNotice/Buckets

Overview

CentralNotice bucketing is a way of building sticky cohorts, allowing us to achieve effects across banner impressions.

Every time a user visits a CentralNotice subscribing wiki, bannerController.js checks if a valid bucket is retrievable from a cookie. If it can't get a bucket from the cookie, it randomly assigns a bucket. This happens even if the user is not currently targeted by a campaign.

The bucket is represented as an integer value greater than or equal to 0 and less than $wgNoticeNumberOfControllerBuckets. In the current WMF production setup, this means either 0 or 1 (which corrsponds to buckets A and B in the campaigns administration UI).

bannerController.js stores the bucket value in a cookie called centralnotice_bucket. The cookie expires after the number of days set in $wgNoticeBucketExpiry, 7 on WMF production. During that period, every time the user visits that wiki, they will remain in the bucket indicated in the cookie. After the cookie expires, the next time the user visits the wiki, a new bucket value will again be chosen randomly and stored.

Cookies also store the values of $wgNoticeNumberOfControllerBuckets and $wgNoticeNumberOfBuckets. Any time either of these values change, bucket cookies are invalidated and new bucket values are chosen.

The CentralNotice campaign administration UI lets you assign banners to specific buckets. The maximum number of buckets available for campaigns is set by $wgNoticeNumberOfBuckets, currently 4 on WMF production. This means that on production there are 2 additional buckets, 2 (or C) and 3 (or D) that are never assigned by bannerController.js. The purpose of these extra buckets is to allow other Javascript code to assign them based on particular circumstances. Here's an example of how to do that:

// Get/set/modify the current bucket (in-memory, current page-load)
mw.centralNotice.data.bucket = 2;

// Save the bucket in a biscuit
var expectedValidityString = mw.config.get( 'wgNoticeNumberOfBuckets' ) + '.' + mw.config.get( 'wgNoticeNumberOfControllerBuckets' );
$.cookie(
  'centralnotice_bucket', mw.centralNotice.data.bucket + '-' + expectedValidityString,
  { expires: mw.config.get( 'wgNoticeBucketExpiry' ), path: '/' }
);

After this, if the user visits the wiki again before the cookie expires, bannerController.js will retrieve the programatically set bucket value and will request a banner using that value.

Note that a mechanism is in place to prevent users from being excluded from banners if the the bucket they are assigned to is not used by a campaign that targets them. When the infrastructure wiki reviews possible banners to send to a user, it checks the number of buckets in the campaign associated with each banner. If the user's bucket number is greater than the highest bucket number available for the campaign, the user's bucket number is mapped downward to ensure that the user is not excluded. So if a campaign only has one bucket (i.e., bucketing is not enabled) all targeted users will get banners for that campaign. If a campaign has two buckets and the user is in bucket 3, that will be mapped down to 1 and the user will be treated as if they were in bucket 1.

This is important, since campaigns may easily have less buckets than the number set in $wgNoticeNumberOfControllerBuckets. It also determines the interactions among overlapping campaigns on a single subscribing wiki.

Another bucketing mechanism is implemented in MediaWiki core. Currently, CentralNotice and core buckets do not play together at all.

Issuez

Cycling bucket cookie expiries?

It appears that there is no way to control or examine the expiry date of bucket cookies. Users' bucket cookies are chosen and re-chosen on cycles of 7 days + the time elapsed between the cookie's expiry and their next visit to the wiki. It is unlikely that at the start of a campaign or an A/B test, all or even most users will at the beginning of such a cycle; most will be in the middle of one. As the campaign continues, an increasing number of users will be switched to a different bucket before the 7 days of the test complete. On average, the number of users making the switch will reach somewhat less than 50% by the end of the 7 days. (The exact number switching will be random and will also depend on how often users visit the wiki.)

(Hopefully I'm wrong about this!!! :) )

Users who delete or force-expire cookies?

Some users configure their browsers to delete all cookies at the end of browser session. Others delete them manually. This will impact on tests, but it appears that there is no way to control for this phenomenon.