Extension:EventLogging/SendBeacon

Intro

This document describes some of our recent experiments of regarding EventLogging and the new sendBeacon method [1] present in more modern browsers.

The main advantage of using sendBeacon is its true async nature, it allows us to log events without disturbing the user workflow. For details of functionality requested that might benefit from sendBeacon you can go to this bug: T44815

Experiment

Goal

Assess how sendBeacon is working in browsers that claim to support it.

Methodology

In order to know if sendBeacon works as advertised we setup a minor experiment using EventLogging itself. We designed a schema that allows us to "measure" whether a browser that says to support SendBeacon really does: [2]

If sendBeacon is available, event data is logged using both the sendBeacon wrapper (logPersistentEvent) and the img approach (logEvent). Otherwise, it only uses logEvent. Thus, if sendBeacon is working properly for the browser in question we should see two events one tagged as "logEvent" and other as "logPersistentEvent" with the same id. If "logEvent" is present but "logPersistenEvent" is not when the browser says to support sendBeacon, the browser support is faulty .

Code for our experiment events can be found here: [3] .

Conclusions

Disclaimer

These conclusions are obtained with preliminary data, we expect to have data for all our users at large from January 10th onwards when cache expires make our code available to everyone. Data thus far (about 400K records) comes from editors (which see fresh files, not cached ones) and from user visits to pages that have been evicted from the cache.

Results

The bulk majority of browsers that say that support sendBeacon really do and we see our events coming as we would expect. The number of records for browsers that seem to have faulty support is quite small. We see less than 100 faulty events (out of about 200k).

Detailed Results

Browsers that support sendBeacon as expected

Of the browser set that supports send beacon as expected these are the browsers that have more than 0.5% share on our experiment:

 0.52% Chrome Mobile 39 Android 5.0
 0.52% Firefox 34 Mac OS X 10.10
 0.54% Firefox 34 Mac OS X 10.9
 0.58% Firefox 31 Windows 7
 0.65% Chrome 39 Android 4.4
 0.67% Chrome Mobile 39 Android 4.3
 0.67% Firefox 34 Ubuntu
 0.70% Firefox 34 Windows 8
 0.86% Chrome Mobile 39 Android 4.1
 1.02% Chrome Mobile 39 Android 4.2
 1.03% Firefox 34 Windows Vista
 1.32% Firefox 33 Windows 7
 1.45% Opera 26 Windows 7
 1.65% Chrome 39 Mac OS X 10.9
 1.70% Chrome 39 Windows Vista
 1.88% Chrome 39 Mac OS X 10.10
 2.42% Chrome 39 Windows 8
 2.86% Firefox 34 Windows XP
 3.75% Firefox 34 Windows 8.1
 4.23% Chrome 39 Windows XP
 6.40% Chrome Mobile 39 Android 4.4
 10.19% Chrome 39 Windows 8.1
 13.57% Firefox 34 Windows 7
 31.57% Chrome 39 Windows 7

Note that we found a bug in the random function that is used to produce random event identifiers and that we have removed these records from analysis. The number of duplicates is small but significant, there are 2000 events with duplicate ids out of 200K. See bug: https://phabricator.wikimedia.org/T78449

We first select all user agents that have two events: 'logEvent' and 'logPersistenEvent' per eventId:

-- get all browsers that support sendBeacon, remove duplicated ids for logEvent
use log;
select count(userAgent), userAgent from SendBeaconReliability_10735916
where event_method='logPersistentEvent' and event_logId in (select  logId from (select count(event_logId) as c,
event_logId as logId from SendBeaconReliability_10735916 where event_method='logEvent' group by event_logId)
as A where c=1) group by userAgent;

We process these to see which are the most used browsers of the ones that support send beacon: https://gist.github.com/nuria/9af8c6edbf44becbc6ac

Browsers with faulty support of SendBeacon

Faulty support is present in less than 100 events of our 200K event dataset, thus we will refrain of drawing conclusions until we have a bit more data.

Update January 28th 2015

We looked at a larger dataset (2 millions of events) and still the petitions that had "faulty" send beacon support was really small, thus our position is that senBeacon should be used when it is supported by the browser API and we have done changes to that extent on the EventLogging extension.

Better SQL

select count(userAgent),
       userAgent
  from SendBeaconReliability_10735916
 where event_method='logPersistentEvent'
   and event_logId in (
            select event_logId
              from SendBeaconReliability_10735916
             where event_method='logEvent'
             group by event_logId
            having count(*) = 1
       )
 group by userAgent