Certain content doesn't necessarily need to be shipped to the user upfront and sometimes not at all. A good example is the list of references. If a mobile user never clicks on a on a reference this could make a huge difference to our end users.
Progress edit
Beta cluster tests edit
We first tested out the impact of stripping references from the HTML in the beta cluster. This suggests for the Barack Obama article in stable we could shave 4.37% off fully loaded time with a 58% reduction in bytes shipped for our HTML.
Beta edit
After some work in building this we enabled on the production cluster in beta on the 28th April 2016 (after an unsuccessful attempt on the 25th). At this point images were also being lazy loaded. The immediate effect of this was that this reduced the size of the HTML of the page from 154.4KiB to 65.8KiB for the Barack Obama and fully loaded time dropped from 24.34s to 18.82s.
Tagalog Wikipedia edit
Given our tests showed that the change to references when isolated only had a small impact on fully loaded times we pushed lazy loaded references and lazy loaded images for all users of Tagalog Wikipedia on 5th July 2016. The purpose was to compare with the Bengali lazy loaded images experiment (given that Tagalog and Bengali Wikipedia's are a similar size in terms of traffic).
See main article.
Thai Wikipedia edit
Deployed lazy loaded references + lazy loaded images on 12th July at 4.14pm PST.
See main article.
Russian Wikipedia edit
Deployed lazy loaded references + lazy loaded images on 21st July at 4.25pm PST.
See main article.
Disabling edit
All experiments were terminated on 1st September 2016. See main article.
Next steps edit
The change definitely reduced bytes (around 300gb a week on Russian Wiki) but didn't make much of a difference to first paint.
The main issue with shipping this is the performance of the API for obtaining references. See https://phabricator.wikimedia.org/T123328.