What happened
editOn Tuesday 5th July lazy loaded images was enabled on Japanese Wikipedia for all users. The immediate impact we noticed is that images served to users on w:ja:日本 (Japan) dropped from 1.443 MB to 142 kB.
Bytes saved
editThe bytes saved varies per article. Articles with more images obviously will be reduced more. We can analyze bytes saved by looking at raw access logs and looking at images with an internal referrer. We can also look at specific pages to gauge impact.
Analysis of weeks worth of data
editA database dump of the EventLogging data for Japanese Wikipedia was taken from the period 1st June to 12th July. Data 10 days before and after the change was compared and sample size was not normalized. Given the change was deployed on 5th July at 4:47pm PST the following query was used.
select * from NavigationTiming_15485142 where wiki = 'jawiki' and event_mobileMode = 'stable' and event_action ='view' and timestamp > 20160625004700 and timestamp < 20160715004700
This SQL query gave a sample of x rows of data. With the result of the following query, using a Python script the following command was run
python navtimingcsv.py jawiki.tsv 20160705004700 10
Fully loaded time
edit| Label | Sample Size | 95th percentile | median |
|---|---|---|---|
| Before change | 120906 | 8142.75 | 2183.0 |
| After change | 119855 | 7104.0 | 2146.0 |
| Before change (anons) | 120781 | 8146.0 | 2183.0 |
| After change (anons) | 119722 | 7107.0 | 2147.0 |
| Before change (http2) | 85043 | 5690.0 | 1916.0 |
| After change (http2) | 84614 | 5026.0 | 1887.5 |
| Before change (http1) | 35863 | 11638.8 | 3381.0 |
| After change (http1) | 35241 | 10036.0 | 3268.0 |
First paint
edit| Label | Sample Size | 95th percentile | median |
|---|---|---|---|
| Before change | 36279 | 3428.0 | 1042.0 |
| After change | 35978 | 3219.15 | 1042.0 |
DomInteractive
edit| Label | Sample Size | 95th percentile | median |
|---|---|---|---|
| Before change | 120906 | 3585.75 | 1000.0 |
| After change | 119855 | 3232.0 | 998.0 |
Analysis of 30 days of data
editA database dump of the EventLogging data for Japanese Wikipedia was taken from the period 5th June to 4th August. Data 30 days before and after the change was compared. Given the change was deployed on 5th July at 4:47pm PST the following query was used.
select * from NavigationTiming_15485142 where wiki = 'jawiki' and event_mobileMode = 'stable' and event_action ='view' and timestamp > 20160601004700 and timestamp < 20160804004700
This SQL query gave a sample of 1249001 rows of data. With the result of the following query, using a Python script the following command was run
python navtimingcsv.py japanwiki.tsv 20160705004700 30
Fully loaded time
edit| Label | Sample Size | 95th percentile | median |
|---|---|---|---|
| Before change | 360470 | 7924.0 | 2178.0 |
| After change | 357042 | 7475.0 | 2127.0 |
| Before change (anons) | 360068 | 7926.0 | 2178.0 |
| After change (anons) | 356604 | 7476.0 | 2128.0 |
| Before change (http2) | 251404 | 5384.85 | 1902.0 |
| After change (http2) | 254774 | 5339.35 | 1869.0 |
| Before change (http1) | 109066 | 11345.0 | 3356.0 |
| After change (http1) | 102268 | 10692.65 | 3281.0 |
First paint
edit| Label | Sample Size | 95th percentile | median |
|---|---|---|---|
| Before change | 107238 | 3367.0 | 1072.0 |
| After change | 107643 | 3377.0 | 1041.0 |
DomInteractive
edit| Label | Sample Size | 95th percentile | median |
|---|---|---|---|
| Before change | 360470 | 3435.0 | 1003.0 |
| After change | 357042 | 3467.0 | 999.0 |
Week against week comparisons and longterm data transfer graph
editRollout of lazy loaded images on ja.m.wikipedia.org suggests a nontrivial speed improvement in page fully loaded time (initial lag excluded) and a significant reduction in image bytes shipped per pageview, leading to lighter weight pages.
Note well that the x-axis in the speed graphs is presented in log2n increments.
To reduce noise, navigation timing events are filtered to ensure key fields are present, with an emphasis on anonymous, non-redirected, plain article pageviews. Logged in pageviews for the NavigationTiming queried window comprised only 0.117% of the pageviews in the set and are not used in the NavigationTiming derived calculations.
To simplify analysis for bytes shipped calculations, eligible source HTML pages are constrained as to a relative path /wiki/ (on language variant wikis this path would be different, but that's out of scope here) for a given wiki without a colon ":" character in the remainder of the path, with a restriction that responses must be HTTP 200s, in order to avoid overcounting of 30x, 40x, or other such spurious responses. This aids in narrowing down the analysis to requests likely to be plain article pageviews. Image bytes are constrained to those served from upload.wikimedia.org with an eligible Referer for the same restriction as page paths.
Changes introduced to support lazy loaded images required modified (slightly increased) JavaScript/CSS/HTML.
jawiki page load speed
editBased on weeks just before and just after the transition to lazy loaded images, pages with lazy loaded images loaded faster at the 10th, 50th (median), and 90th percentiles on both HTTP1 and HTTP2. A basic Kolomogorov-Smirnov statistical significance test suggested that the pre- and post-change cohort speed differences were statistically significant.
The following table is based on anonymous pageviews using the query below.
| Percentile | HTTP1 w/o lazy | HTTP1 LAZY | HTTP2 w/o lazy | HTTP2 LAZY | H1 LAZY Improvement | H2 LAZY Improvement |
|---|---|---|---|---|---|---|
| 10% | 818 | 783.7 | 398 | 384 | 4.19% | 3.52% |
| 50% | 2021 | 1936 | 1346.5 | 1328 | 4.21% | 1.37% |
| 90% | 5492.8 | 4561 | 3206.1 | 2968 | 16.96% | 7.43% |
jawiki data transfer
editBased on examination of two 7-day periods (24-30 June 2016 vs 8-14 July 2016), image bytes per pageview were reduced by about 51.37%. This contributed to a decrease of about 9.18% bytes shipped for the modified JavaScript/CSS/HTML plus images as compared to the baseline JavaScript/CSS/HTML and images.
The following graph shows the general trends of images bytes shipped per in-scope pageview, as well in-scope pageviews, core HTML/CSS/JavaScript bytes, and the sum of core HTML/CSS/JavaScript bytes plus image bytes per pageview.
Queries
editThe following query was used to derive the lag-excluded load time, roughLoadTimeInitialLagExcluded. Note this query also captured data on tlwiki, although the scope of the analysis on this page is jawiki mobile
select
left(timestamp,8) as ts,
event_isAnon,
event_lazyLoadImages,
event_isHttp2,
event_loadEventEnd-event_responseStart as roughLoadTimeInitialLagExcluded,
event_responseEnd-event_responseStart as roughNetworkTimeInitialLagExcluded,
event_requestStart,
event_responseStart,
event_responseEnd,
event_firstPaint,
event_domInteractive,
event_domComplete,
event_loadEventStart,
event_loadEventEnd,
webHost,
event_originCountry,
event_mediaWikiVersion
from NavigationTiming_15485142
where
timestamp > '20160623'
and timestamp < '20160721'
and event_action = 'view'
and event_mobileMode = 'stable'
and event_namespaceId = 0
and event_redirectCount is null
and event_loadEventEnd is not null
and event_domComplete is not null
and event_domInteractive is not null
and event_responseStart is not null
and wiki in ('jawiki', 'tlwiki')
order by
wiki,
event_isAnon,
event_lazyLoadImages,
event_isHttp2,
ts
;
The following queries were used to derive image bytes transferred using the constraints described above, as well as to support the longer term graph with trend lines.
select
month,
day,
substr(referer,1,26),
sum(response_size)
from
webrequest
where
year = 2016
and month = 6 and day > 4
and uri_host = 'upload.wikimedia.org'
and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'
and content_type rlike '^image'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
substr(referer,1,26)
;
select
month,
day,
substr(referer,1,26),
sum(response_size)
from
webrequest
where
year = 2016
and month = 7
and uri_host = 'upload.wikimedia.org'
and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'
and content_type rlike '^image'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
substr(referer,1,26)
;
select
month,
day,
substr(referer,1,26),
sum(response_size)
from
webrequest
where
year = 2016
and month = 8 and day < 5
and uri_host = 'upload.wikimedia.org'
and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'
and content_type rlike '^image'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
substr(referer,1,26)
;
The raw data received were as follows:
month day _c2 _c3 6 30 https://ja.m.wikipedia.org 124180180469 6 5 https://ja.m.wikipedia.org 135720072139 6 6 https://ja.m.wikipedia.org 108178906611 6 7 https://ja.m.wikipedia.org 111401108523 6 8 https://ja.m.wikipedia.org 106164037819 6 9 https://ja.m.wikipedia.org 105406723448 6 10 https://ja.m.wikipedia.org 104839062684 6 11 https://ja.m.wikipedia.org 141886296285 6 12 https://ja.m.wikipedia.org 170638150504 6 13 https://ja.m.wikipedia.org 146692782536 6 14 https://ja.m.wikipedia.org 134084883702 6 15 https://ja.m.wikipedia.org 133562013207 6 16 https://ja.m.wikipedia.org 142537562128 6 17 https://ja.m.wikipedia.org 133251860232 6 18 https://ja.m.wikipedia.org 151938833556 6 19 https://ja.m.wikipedia.org 167860625982 6 20 https://ja.m.wikipedia.org 136443890103 6 21 https://ja.m.wikipedia.org 136692664905 6 22 https://ja.m.wikipedia.org 147929702365 6 23 https://ja.m.wikipedia.org 139453947620 6 24 https://ja.m.wikipedia.org 156084306064 6 25 https://ja.m.wikipedia.org 168950872999 6 26 https://ja.m.wikipedia.org 171282566058 6 27 https://ja.m.wikipedia.org 135543126283 6 28 https://ja.m.wikipedia.org 136833152633 6 29 https://ja.m.wikipedia.org 135462758117 7 1 https://ja.m.wikipedia.org 124684212453 7 2 https://ja.m.wikipedia.org 164418677635 7 3 https://ja.m.wikipedia.org 161752597537 7 4 https://ja.m.wikipedia.org 149993526937 7 5 https://ja.m.wikipedia.org 134035063705 7 6 https://ja.m.wikipedia.org 103235144867 7 7 https://ja.m.wikipedia.org 67828813390 7 8 https://ja.m.wikipedia.org 66337742258 7 9 https://ja.m.wikipedia.org 80564160978 7 10 https://ja.m.wikipedia.org 85920579820 7 11 https://ja.m.wikipedia.org 68779837487 7 12 https://ja.m.wikipedia.org 67698073684 7 13 https://ja.m.wikipedia.org 69773350604 7 14 https://ja.m.wikipedia.org 67633588058 7 15 https://ja.m.wikipedia.org 67486541269 7 16 https://ja.m.wikipedia.org 78015793312 7 17 https://ja.m.wikipedia.org 86470922402 7 18 https://ja.m.wikipedia.org 86544561954 7 19 https://ja.m.wikipedia.org 66590125244 7 20 https://ja.m.wikipedia.org 64112347900 7 21 https://ja.m.wikipedia.org 66050339575 7 22 https://ja.m.wikipedia.org 63700245787 7 23 https://ja.m.wikipedia.org 72510674705 7 24 https://ja.m.wikipedia.org 82103567189 7 25 https://ja.m.wikipedia.org 66774017408 7 26 https://ja.m.wikipedia.org 67152344637 7 27 https://ja.m.wikipedia.org 65026808541 7 28 https://ja.m.wikipedia.org 62521007017 7 29 https://ja.m.wikipedia.org 62639495316 7 30 https://ja.m.wikipedia.org 70456033383 7 31 https://ja.m.wikipedia.org 79507268276 8 1 https://ja.m.wikipedia.org 67572125043 8 2 https://ja.m.wikipedia.org 66961618961 8 3 https://ja.m.wikipedia.org 67092175391 8 4 https://ja.m.wikipedia.org 65062330557
The following queries were used to derive page and JavaScript/CSS bytes (pre- and post-modification for lazy loading) transferred using the constraints described above, as well as to support the longer term graph with trend lines.
select
month,
day,
uri_host,
sum(response_size)
from
webrequest
where
year = 2016
and month = 6 and day > 4
and uri_host = 'ja.m.wikipedia.org'
and ((uri_path rlike '^/wiki/([^:])+$') or (uri_path = '/w/load.php' and uri_query rlike 'skins\.minerva\.icons\.images\.scripts&skin=minerva' and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'))
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host
;
select
month,
day,
uri_host,
sum(response_size)
from
webrequest
where
year = 2016
and month = 7
and uri_host = 'ja.m.wikipedia.org'
and ((uri_path rlike '^/wiki/([^:])+$') or (uri_path = '/w/load.php' and uri_query rlike 'skins\.minerva\.icons\.images\.scripts&skin=minerva' and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'))
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host
;
select
month,
day,
uri_host,
sum(response_size)
from
webrequest
where
year = 2016
and month = 8 and day < 5
and uri_host = 'ja.m.wikipedia.org'
and ((uri_path rlike '^/wiki/([^:])+$') or (uri_path = '/w/load.php' and uri_query rlike 'skins\.minerva\.icons\.images\.scripts&skin=minerva' and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'))
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host
;
The raw data received were as follows:
month day uri_host _c3 6 5 ja.m.wikipedia.org 584043509098 6 6 ja.m.wikipedia.org 453573407856 6 7 ja.m.wikipedia.org 443560946176 6 8 ja.m.wikipedia.org 445979910759 6 9 ja.m.wikipedia.org 500639200613 6 10 ja.m.wikipedia.org 488962065693 6 11 ja.m.wikipedia.org 547200189342 6 12 ja.m.wikipedia.org 593204311330 6 13 ja.m.wikipedia.org 492918438444 6 14 ja.m.wikipedia.org 447036568383 6 15 ja.m.wikipedia.org 453638629857 6 16 ja.m.wikipedia.org 479841968101 6 17 ja.m.wikipedia.org 517480911709 6 18 ja.m.wikipedia.org 551939360543 6 19 ja.m.wikipedia.org 591493939800 6 20 ja.m.wikipedia.org 458977191045 6 21 ja.m.wikipedia.org 454923880318 6 22 ja.m.wikipedia.org 492636495002 6 23 ja.m.wikipedia.org 456603025665 6 24 ja.m.wikipedia.org 490341384015 6 25 ja.m.wikipedia.org 537929362713 6 26 ja.m.wikipedia.org 545503508434 6 27 ja.m.wikipedia.org 431615776632 6 28 ja.m.wikipedia.org 448780648978 6 29 ja.m.wikipedia.org 451773538098 6 30 ja.m.wikipedia.org 423185112608 7 3 ja.m.wikipedia.org 537992180969 7 4 ja.m.wikipedia.org 448322594045 7 5 ja.m.wikipedia.org 445007695821 7 6 ja.m.wikipedia.org 435472597765 7 7 ja.m.wikipedia.org 430814756332 7 8 ja.m.wikipedia.org 461334758600 7 9 ja.m.wikipedia.org 547280892717 7 10 ja.m.wikipedia.org 600062076779 7 11 ja.m.wikipedia.org 473057269970 7 12 ja.m.wikipedia.org 477262829644 7 13 ja.m.wikipedia.org 471436878948 7 14 ja.m.wikipedia.org 472398332176 7 15 ja.m.wikipedia.org 548724119441 7 16 ja.m.wikipedia.org 616527657391 7 17 ja.m.wikipedia.org 639723642046 7 18 ja.m.wikipedia.org 638694292375 7 19 ja.m.wikipedia.org 481106407202 7 20 ja.m.wikipedia.org 473148500657 7 21 ja.m.wikipedia.org 466261658125 7 22 ja.m.wikipedia.org 465249325390 7 23 ja.m.wikipedia.org 492041513150 7 24 ja.m.wikipedia.org 563837017528 7 25 ja.m.wikipedia.org 445503057133 7 26 ja.m.wikipedia.org 447799358192 7 27 ja.m.wikipedia.org 447035047673 7 28 ja.m.wikipedia.org 432588647022 7 29 ja.m.wikipedia.org 447436227804 7 30 ja.m.wikipedia.org 503914286671 7 31 ja.m.wikipedia.org 553840973015 7 1 ja.m.wikipedia.org 424109984113 7 2 ja.m.wikipedia.org 535223235640 8 4 ja.m.wikipedia.org 463721744365 8 1 ja.m.wikipedia.org 465305749921 8 2 ja.m.wikipedia.org 459082686410 8 3 ja.m.wikipedia.org 468718936637
The following queries were used to derive pageviews using the constraints above, as well as to support the longer term graph with trend lines. Practically all matching records were qualified as pageviews, largely ruling out the possibility of image byte transfer counts with proper Referer values being derived from anything other than qualified pageviews.
select
month,
day,
uri_host,
content_type,
is_pageview,
count(1)
from
webrequest
where
year = 2016
and month = 6 and day > 4
and uri_host = 'ja.m.wikipedia.org'
and uri_path rlike '^/wiki/([^:])+$'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host,
content_type,
is_pageview
;
select
month,
day,
uri_host,
content_type,
is_pageview,
count(1)
from
webrequest
where
year = 2016
and month = 7
and uri_host = 'ja.m.wikipedia.org'
and uri_path rlike '^/wiki/([^:])+$'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host,
content_type,
is_pageview
;
select
month,
day,
uri_host,
content_type,
is_pageview,
count(1)
from
webrequest
where
year = 2016
and month = 8 and day < 5
and uri_host = 'ja.m.wikipedia.org'
and uri_path rlike '^/wiki/([^:])+$'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host,
content_type,
is_pageview
;
The raw data received were as follows. Note that the low count "text/html" and "-" content types, as well as the non-pageview records are not used in the calculations on this page. These were included in this query to gauge the magnitude of their potential effect. It would appear their effect was minimal.
6 19 ja.m.wikipedia.org text/html; charset=UTF-8 true 21683504 6 28 ja.m.wikipedia.org application/pdf false 1 6 17 ja.m.wikipedia.org - false 4 6 17 ja.m.wikipedia.org image/png false 9 6 21 ja.m.wikipedia.org text/html; charset=UTF-8 false 1 6 21 ja.m.wikipedia.org text/html; charset=UTF-8 true 17370675 6 17 ja.m.wikipedia.org text/css; charset=utf-8 false 6 6 20 ja.m.wikipedia.org text/html true 2 6 17 ja.m.wikipedia.org text/javascript false 2 6 23 ja.m.wikipedia.org text/html; charset=UTF-8 false 9 6 23 ja.m.wikipedia.org text/html; charset=UTF-8 true 17673155 6 25 ja.m.wikipedia.org text/html; charset=UTF-8 false 6 6 25 ja.m.wikipedia.org text/html; charset=UTF-8 true 20560909 6 6 ja.m.wikipedia.org text/html; charset=UTF-8 false 6 6 6 ja.m.wikipedia.org text/html; charset=UTF-8 true 17427607 6 17 ja.m.wikipedia.org image/vnd.microsoft.icon false 7 6 27 ja.m.wikipedia.org text/html; charset=UTF-8 false 3 6 27 ja.m.wikipedia.org text/html; charset=UTF-8 true 16981545 6 8 ja.m.wikipedia.org text/html; charset=UTF-8 false 3 6 8 ja.m.wikipedia.org text/html; charset=UTF-8 true 17168195 6 26 ja.m.wikipedia.org text/html true 2 6 29 ja.m.wikipedia.org text/html; charset=UTF-8 false 1 6 29 ja.m.wikipedia.org text/html; charset=UTF-8 true 17880569 6 10 ja.m.wikipedia.org text/html; charset=UTF-8 false 12 6 10 ja.m.wikipedia.org text/html; charset=UTF-8 true 17312428 6 12 ja.m.wikipedia.org text/html; charset=UTF-8 false 2 6 12 ja.m.wikipedia.org text/html; charset=UTF-8 true 21701144 6 14 ja.m.wikipedia.org text/html; charset=UTF-8 false 15 6 14 ja.m.wikipedia.org text/html; charset=UTF-8 true 16960827 6 16 ja.m.wikipedia.org text/html; charset=UTF-8 false 24 6 16 ja.m.wikipedia.org text/html; charset=UTF-8 true 17600084 6 17 ja.m.wikipedia.org application/json; charset=utf-8 false 6 6 18 ja.m.wikipedia.org text/html; charset=UTF-8 false 11 6 18 ja.m.wikipedia.org text/html; charset=UTF-8 true 19381891 6 20 ja.m.wikipedia.org text/html; charset=UTF-8 false 11 6 20 ja.m.wikipedia.org text/html; charset=UTF-8 true 17291241 6 22 ja.m.wikipedia.org text/html; charset=UTF-8 false 4 6 22 ja.m.wikipedia.org text/html; charset=UTF-8 true 18665374 6 30 ja.m.wikipedia.org text/javascript; charset=utf-8 false 1 6 24 ja.m.wikipedia.org text/html; charset=UTF-8 false 5 6 24 ja.m.wikipedia.org text/html; charset=UTF-8 true 18789507 6 5 ja.m.wikipedia.org text/html; charset=UTF-8 false 8 6 5 ja.m.wikipedia.org text/html; charset=UTF-8 true 21686129 6 26 ja.m.wikipedia.org text/html; charset=UTF-8 false 4 6 26 ja.m.wikipedia.org text/html; charset=UTF-8 true 20798241 6 7 ja.m.wikipedia.org text/html; charset=UTF-8 false 6 6 7 ja.m.wikipedia.org text/html; charset=UTF-8 true 17492334 6 17 ja.m.wikipedia.org text/javascript; charset=UTF-8 false 5 6 22 ja.m.wikipedia.org text/javascript false 1 6 28 ja.m.wikipedia.org text/html; charset=UTF-8 false 5 6 28 ja.m.wikipedia.org text/html; charset=UTF-8 true 17811344 6 9 ja.m.wikipedia.org text/html; charset=UTF-8 false 3 6 9 ja.m.wikipedia.org text/html; charset=UTF-8 true 17059342 6 17 ja.m.wikipedia.org text/javascript; charset=utf-8 false 27 6 30 ja.m.wikipedia.org text/html; charset=UTF-8 false 4 6 30 ja.m.wikipedia.org text/html; charset=UTF-8 true 16896524 6 11 ja.m.wikipedia.org text/html; charset=UTF-8 false 7 6 11 ja.m.wikipedia.org text/html; charset=UTF-8 true 19594972 6 13 ja.m.wikipedia.org text/html; charset=UTF-8 false 6 6 13 ja.m.wikipedia.org text/html; charset=UTF-8 true 18410111 6 21 ja.m.wikipedia.org text/javascript; charset=utf-8 false 1 6 30 ja.m.wikipedia.org text/css; charset=utf-8 false 1 6 15 ja.m.wikipedia.org text/html; charset=UTF-8 false 1 6 15 ja.m.wikipedia.org text/html; charset=UTF-8 true 17176324 6 17 ja.m.wikipedia.org image/svg+xml false 7 6 17 ja.m.wikipedia.org text/html; charset=UTF-8 false 5 6 17 ja.m.wikipedia.org text/html; charset=UTF-8 true 17645960 6 17 ja.m.wikipedia.org text/css; charset=UTF-8 false 1 6 19 ja.m.wikipedia.org text/html; charset=UTF-8 false 4 7 6 ja.m.wikipedia.org text/html true 3 7 28 ja.m.wikipedia.org text/html; charset=UTF-8 false 14 7 28 ja.m.wikipedia.org text/html; charset=UTF-8 true 16285305 7 9 ja.m.wikipedia.org text/html; charset=UTF-8 false 10 7 9 ja.m.wikipedia.org text/html; charset=UTF-8 true 20806034 7 30 ja.m.wikipedia.org text/html; charset=UTF-8 false 2 7 30 ja.m.wikipedia.org text/html; charset=UTF-8 true 18485500 7 11 ja.m.wikipedia.org text/html; charset=UTF-8 false 6 7 11 ja.m.wikipedia.org text/html; charset=UTF-8 true 18072054 7 13 ja.m.wikipedia.org text/html; charset=UTF-8 false 8 7 13 ja.m.wikipedia.org text/html; charset=UTF-8 true 17788981 7 3 ja.m.wikipedia.org application/pdf false 12 7 12 ja.m.wikipedia.org text/html true 1 7 15 ja.m.wikipedia.org text/html; charset=UTF-8 false 6 7 15 ja.m.wikipedia.org text/html; charset=UTF-8 true 17055753 7 14 ja.m.wikipedia.org text/html true 1 7 17 ja.m.wikipedia.org text/html; charset=UTF-8 false 11 7 17 ja.m.wikipedia.org text/html; charset=UTF-8 true 21657938 7 28 ja.m.wikipedia.org image/vnd.microsoft.icon false 1 7 7 ja.m.wikipedia.org application/pdf false 1 7 25 ja.m.wikipedia.org text/javascript; charset=utf-8 false 1 7 8 ja.m.wikipedia.org text/javascript; charset=UTF-8 false 7 7 19 ja.m.wikipedia.org text/html; charset=UTF-8 false 4 7 19 ja.m.wikipedia.org text/html; charset=UTF-8 true 17341474 7 28 ja.m.wikipedia.org application/pdf false 2 7 8 ja.m.wikipedia.org text/javascript; charset=utf-8 false 1 7 21 ja.m.wikipedia.org text/html; charset=UTF-8 false 5 7 21 ja.m.wikipedia.org text/html; charset=UTF-8 true 17240735 7 2 ja.m.wikipedia.org text/html; charset=UTF-8 false 2 7 2 ja.m.wikipedia.org text/html; charset=UTF-8 true 20680721 7 23 ja.m.wikipedia.org text/html; charset=UTF-8 false 4 7 23 ja.m.wikipedia.org text/html; charset=UTF-8 true 17927678 7 4 ja.m.wikipedia.org text/html; charset=UTF-8 false 70 7 4 ja.m.wikipedia.org text/html; charset=UTF-8 true 17534848 7 25 ja.m.wikipedia.org text/html; charset=UTF-8 false 5 7 25 ja.m.wikipedia.org text/html; charset=UTF-8 true 16528308 7 6 ja.m.wikipedia.org text/html; charset=UTF-8 false 8 7 6 ja.m.wikipedia.org text/html; charset=UTF-8 true 17320422 7 27 ja.m.wikipedia.org text/html; charset=UTF-8 false 6 7 27 ja.m.wikipedia.org text/html; charset=UTF-8 true 16801517 7 8 ja.m.wikipedia.org text/html; charset=UTF-8 false 11 7 8 ja.m.wikipedia.org text/html; charset=UTF-8 true 17586299 7 25 ja.m.wikipedia.org image/png false 1 7 29 ja.m.wikipedia.org text/html; charset=UTF-8 false 30 7 29 ja.m.wikipedia.org text/html; charset=UTF-8 true 16521987 7 4 ja.m.wikipedia.org text/javascript false 1 7 25 ja.m.wikipedia.org text/css; charset=utf-8 false 1 7 10 ja.m.wikipedia.org text/html; charset=UTF-8 false 14 7 10 ja.m.wikipedia.org text/html; charset=UTF-8 true 22014268 7 25 ja.m.wikipedia.org text/javascript false 1 7 31 ja.m.wikipedia.org text/html; charset=UTF-8 false 5 7 31 ja.m.wikipedia.org text/html; charset=UTF-8 true 20046905 7 2 ja.m.wikipedia.org image/vnd.microsoft.icon false 1 7 12 ja.m.wikipedia.org text/html; charset=UTF-8 false 5 7 12 ja.m.wikipedia.org text/html; charset=UTF-8 true 17971521 7 21 ja.m.wikipedia.org application/pdf false 1 7 10 ja.m.wikipedia.org - false 1 7 14 ja.m.wikipedia.org text/html; charset=UTF-8 false 8 7 14 ja.m.wikipedia.org text/html; charset=UTF-8 true 17191513 7 16 ja.m.wikipedia.org text/html; charset=UTF-8 false 27 7 16 ja.m.wikipedia.org text/html; charset=UTF-8 true 19963165 7 18 ja.m.wikipedia.org text/html; charset=UTF-8 false 8 7 18 ja.m.wikipedia.org text/html; charset=UTF-8 true 22374981 7 20 ja.m.wikipedia.org text/html; charset=UTF-8 false 10 7 20 ja.m.wikipedia.org text/html; charset=UTF-8 true 17164645 7 1 ja.m.wikipedia.org text/html; charset=UTF-8 false 23 7 1 ja.m.wikipedia.org text/html; charset=UTF-8 true 16583764 7 10 ja.m.wikipedia.org application/pdf false 1 7 22 ja.m.wikipedia.org text/html; charset=UTF-8 false 9 7 22 ja.m.wikipedia.org text/html; charset=UTF-8 true 16624345 7 3 ja.m.wikipedia.org text/html; charset=UTF-8 false 4 7 3 ja.m.wikipedia.org text/html; charset=UTF-8 true 20609077 7 24 ja.m.wikipedia.org text/html; charset=UTF-8 false 7 7 24 ja.m.wikipedia.org text/html; charset=UTF-8 true 20685545 7 5 ja.m.wikipedia.org text/html; charset=UTF-8 false 109 7 5 ja.m.wikipedia.org text/html; charset=UTF-8 true 17714516 7 25 ja.m.wikipedia.org application/json; charset=utf-8 false 1 7 26 ja.m.wikipedia.org text/html; charset=UTF-8 false 2 7 26 ja.m.wikipedia.org text/html; charset=UTF-8 true 16840162 7 7 ja.m.wikipedia.org text/html; charset=UTF-8 false 5 7 7 ja.m.wikipedia.org text/html; charset=UTF-8 true 16916233 8 1 ja.m.wikipedia.org text/html; charset=UTF-8 false 4 8 1 ja.m.wikipedia.org text/html; charset=UTF-8 true 17212992 8 3 ja.m.wikipedia.org text/html; charset=UTF-8 false 11 8 3 ja.m.wikipedia.org text/html; charset=UTF-8 true 17545536 8 1 ja.m.wikipedia.org application/pdf false 1 8 2 ja.m.wikipedia.org text/html; charset=UTF-8 false 9 8 2 ja.m.wikipedia.org text/html; charset=UTF-8 true 17167953 8 4 ja.m.wikipedia.org text/html; charset=UTF-8 false 5 8 4 ja.m.wikipedia.org text/html; charset=UTF-8 true 17291588
As with any data spanning time series and the myriad complexities involved with different devices and environments, data are subject to fluctuation. However, the data transfer savings are unambiguous, and the large event sampling pool lends a degree of confidence that pages are actually loading faster.
Statistical significance
The Kolomogorov-Smirnov test suggests the differences in the distributions for lazy loaded compared to non-lazy loaded image based page load times for each of HTTP/1 and HTTP/2 are material and we can reject the null hypothesis.
from scipy.stats import ks_2samp
# h1_conventional.txt contains 24-30 June 2016 HTTP/1 convetional loading rough load times
with open('h1_conventional.txt') as f:
a = [int(i) for i in f.readlines()]
# h1_lazy.txt contains 8-14 July 2016 HTTP/2 lazy loading rough load times
with open('h1_lazy.txt') as f:
b = [int(i) for i in f.readlines()]
len(a)
15517
len(b)
15898
ks_2samp(a,b)
(0.037166639427270387, 7.1355334748281838e-10)
# result is ks statistic, p-value. Null hypothesis rejected.
with open('h2_conventional.txt') as f:
c = [int(i) for i in f.readlines()]
with open('h2_lazy.txt') as f:
d = [int(i) for i in f.readlines()]
len(c)
53735
len(d)
55437
ks_2samp(c,d)
(0.020979426651792699, 7.149451368361895e-11)
# result is ks statistic, p-value. Null hypothesis rejected.
The data used for the two month line graph for bytes shipped, pageviews, etc. is provided here:
In-scope pageviews month day uri_host content_type is_pageview pvs 6 5 ja.m.wikipedia.org text/html; charset=UTF-8 true 21686129 6 6 ja.m.wikipedia.org text/html; charset=UTF-8 true 17427607 6 7 ja.m.wikipedia.org text/html; charset=UTF-8 true 17492334 6 8 ja.m.wikipedia.org text/html; charset=UTF-8 true 17168195 6 9 ja.m.wikipedia.org text/html; charset=UTF-8 true 17059342 6 10 ja.m.wikipedia.org text/html; charset=UTF-8 true 17312428 6 11 ja.m.wikipedia.org text/html; charset=UTF-8 true 19594972 6 12 ja.m.wikipedia.org text/html; charset=UTF-8 true 21701144 6 13 ja.m.wikipedia.org text/html; charset=UTF-8 true 18410111 6 14 ja.m.wikipedia.org text/html; charset=UTF-8 true 16960827 6 15 ja.m.wikipedia.org text/html; charset=UTF-8 true 17176324 6 16 ja.m.wikipedia.org text/html; charset=UTF-8 true 17600084 6 17 ja.m.wikipedia.org text/html; charset=UTF-8 true 17645960 6 18 ja.m.wikipedia.org text/html; charset=UTF-8 true 19381891 6 19 ja.m.wikipedia.org text/html; charset=UTF-8 true 21683504 6 20 ja.m.wikipedia.org text/html; charset=UTF-8 true 17291241 6 21 ja.m.wikipedia.org text/html; charset=UTF-8 true 17370675 6 22 ja.m.wikipedia.org text/html; charset=UTF-8 true 18665374 6 23 ja.m.wikipedia.org text/html; charset=UTF-8 true 17673155 6 24 ja.m.wikipedia.org text/html; charset=UTF-8 true 18789507 6 25 ja.m.wikipedia.org text/html; charset=UTF-8 true 20560909 6 26 ja.m.wikipedia.org text/html; charset=UTF-8 true 20798241 6 27 ja.m.wikipedia.org text/html; charset=UTF-8 true 16981545 6 28 ja.m.wikipedia.org text/html; charset=UTF-8 true 17811344 6 29 ja.m.wikipedia.org text/html; charset=UTF-8 true 17880569 6 30 ja.m.wikipedia.org text/html; charset=UTF-8 true 16896524 7 1 ja.m.wikipedia.org text/html; charset=UTF-8 true 16583764 7 2 ja.m.wikipedia.org text/html; charset=UTF-8 true 20680721 7 3 ja.m.wikipedia.org text/html; charset=UTF-8 true 20609077 7 4 ja.m.wikipedia.org text/html; charset=UTF-8 true 17534848 7 5 ja.m.wikipedia.org text/html; charset=UTF-8 true 17714516 7 6 ja.m.wikipedia.org text/html; charset=UTF-8 true 17320422 7 7 ja.m.wikipedia.org text/html; charset=UTF-8 true 16916233 7 8 ja.m.wikipedia.org text/html; charset=UTF-8 true 17586299 7 9 ja.m.wikipedia.org text/html; charset=UTF-8 true 20806034 7 10 ja.m.wikipedia.org text/html; charset=UTF-8 true 22014268 7 11 ja.m.wikipedia.org text/html; charset=UTF-8 true 18072054 7 12 ja.m.wikipedia.org text/html; charset=UTF-8 true 17971521 7 13 ja.m.wikipedia.org text/html; charset=UTF-8 true 17788981 7 14 ja.m.wikipedia.org text/html; charset=UTF-8 true 17191513 7 15 ja.m.wikipedia.org text/html; charset=UTF-8 true 17055753 7 16 ja.m.wikipedia.org text/html; charset=UTF-8 true 19963165 7 17 ja.m.wikipedia.org text/html; charset=UTF-8 true 21657938 7 18 ja.m.wikipedia.org text/html; charset=UTF-8 true 22374981 7 19 ja.m.wikipedia.org text/html; charset=UTF-8 true 17341474 7 20 ja.m.wikipedia.org text/html; charset=UTF-8 true 17164645 7 21 ja.m.wikipedia.org text/html; charset=UTF-8 true 17240735 7 22 ja.m.wikipedia.org text/html; charset=UTF-8 true 16624345 7 23 ja.m.wikipedia.org text/html; charset=UTF-8 true 17927678 7 24 ja.m.wikipedia.org text/html; charset=UTF-8 true 20685545 7 25 ja.m.wikipedia.org text/html; charset=UTF-8 true 16528308 7 26 ja.m.wikipedia.org text/html; charset=UTF-8 true 16840162 7 27 ja.m.wikipedia.org text/html; charset=UTF-8 true 16801517 7 28 ja.m.wikipedia.org text/html; charset=UTF-8 true 16285305 7 29 ja.m.wikipedia.org text/html; charset=UTF-8 true 16521987 7 30 ja.m.wikipedia.org text/html; charset=UTF-8 true 18485500 7 31 ja.m.wikipedia.org text/html; charset=UTF-8 true 20046905 8 1 ja.m.wikipedia.org text/html; charset=UTF-8 true 17212992 8 2 ja.m.wikipedia.org text/html; charset=UTF-8 true 17167953 8 3 ja.m.wikipedia.org text/html; charset=UTF-8 true 17545536 8 4 ja.m.wikipedia.org text/html; charset=UTF-8 true 17291588 image bytes month day _c2 _c3 6 5 https://ja.m.wikipedia.org 135720072139 6 6 https://ja.m.wikipedia.org 108178906611 6 7 https://ja.m.wikipedia.org 111401108523 6 8 https://ja.m.wikipedia.org 106164037819 6 9 https://ja.m.wikipedia.org 105406723448 6 10 https://ja.m.wikipedia.org 104839062684 6 11 https://ja.m.wikipedia.org 141886296285 6 12 https://ja.m.wikipedia.org 170638150504 6 13 https://ja.m.wikipedia.org 146692782536 6 14 https://ja.m.wikipedia.org 134084883702 6 15 https://ja.m.wikipedia.org 133562013207 6 16 https://ja.m.wikipedia.org 142537562128 6 17 https://ja.m.wikipedia.org 133251860232 6 18 https://ja.m.wikipedia.org 151938833556 6 19 https://ja.m.wikipedia.org 167860625982 6 20 https://ja.m.wikipedia.org 136443890103 6 21 https://ja.m.wikipedia.org 136692664905 6 22 https://ja.m.wikipedia.org 147929702365 6 23 https://ja.m.wikipedia.org 139453947620 6 24 https://ja.m.wikipedia.org 156084306064 6 25 https://ja.m.wikipedia.org 168950872999 6 26 https://ja.m.wikipedia.org 171282566058 6 27 https://ja.m.wikipedia.org 135543126283 6 28 https://ja.m.wikipedia.org 136833152633 6 29 https://ja.m.wikipedia.org 135462758117 6 30 https://ja.m.wikipedia.org 124180180469 7 1 https://ja.m.wikipedia.org 124684212453 7 2 https://ja.m.wikipedia.org 164418677635 7 3 https://ja.m.wikipedia.org 161752597537 7 4 https://ja.m.wikipedia.org 149993526937 7 5 https://ja.m.wikipedia.org 134035063705 7 6 https://ja.m.wikipedia.org 103235144867 7 7 https://ja.m.wikipedia.org 67828813390 7 8 https://ja.m.wikipedia.org 66337742258 7 9 https://ja.m.wikipedia.org 80564160978 7 10 https://ja.m.wikipedia.org 85920579820 7 11 https://ja.m.wikipedia.org 68779837487 7 12 https://ja.m.wikipedia.org 67698073684 7 13 https://ja.m.wikipedia.org 69773350604 7 14 https://ja.m.wikipedia.org 67633588058 7 15 https://ja.m.wikipedia.org 67486541269 7 16 https://ja.m.wikipedia.org 78015793312 7 17 https://ja.m.wikipedia.org 86470922402 7 18 https://ja.m.wikipedia.org 86544561954 7 19 https://ja.m.wikipedia.org 66590125244 7 20 https://ja.m.wikipedia.org 64112347900 7 21 https://ja.m.wikipedia.org 66050339575 7 22 https://ja.m.wikipedia.org 63700245787 7 23 https://ja.m.wikipedia.org 72510674705 7 24 https://ja.m.wikipedia.org 82103567189 7 25 https://ja.m.wikipedia.org 66774017408 7 26 https://ja.m.wikipedia.org 67152344637 7 27 https://ja.m.wikipedia.org 65026808541 7 28 https://ja.m.wikipedia.org 62521007017 7 29 https://ja.m.wikipedia.org 62639495316 7 30 https://ja.m.wikipedia.org 70456033383 7 31 https://ja.m.wikipedia.org 79507268276 8 1 https://ja.m.wikipedia.org 67572125043 8 2 https://ja.m.wikipedia.org 66961618961 8 3 https://ja.m.wikipedia.org 67092175391 8 4 https://ja.m.wikipedia.org 65062330557 page bytes month day uri_host _c3 6 5 ja.m.wikipedia.org 584043509098 6 6 ja.m.wikipedia.org 453573407856 6 7 ja.m.wikipedia.org 443560946176 6 8 ja.m.wikipedia.org 445979910759 6 9 ja.m.wikipedia.org 500639200613 6 10 ja.m.wikipedia.org 488962065693 6 11 ja.m.wikipedia.org 547200189342 6 12 ja.m.wikipedia.org 593204311330 6 13 ja.m.wikipedia.org 492918438444 6 14 ja.m.wikipedia.org 447036568383 6 15 ja.m.wikipedia.org 453638629857 6 16 ja.m.wikipedia.org 479841968101 6 17 ja.m.wikipedia.org 517480911709 6 18 ja.m.wikipedia.org 551939360543 6 19 ja.m.wikipedia.org 591493939800 6 20 ja.m.wikipedia.org 458977191045 6 21 ja.m.wikipedia.org 454923880318 6 22 ja.m.wikipedia.org 492636495002 6 23 ja.m.wikipedia.org 456603025665 6 24 ja.m.wikipedia.org 490341384015 6 25 ja.m.wikipedia.org 537929362713 6 26 ja.m.wikipedia.org 545503508434 6 27 ja.m.wikipedia.org 431615776632 6 28 ja.m.wikipedia.org 448780648978 6 29 ja.m.wikipedia.org 451773538098 6 30 ja.m.wikipedia.org 423185112608 7 1 ja.m.wikipedia.org 424109984113 7 2 ja.m.wikipedia.org 535223235640 7 3 ja.m.wikipedia.org 537992180969 7 4 ja.m.wikipedia.org 448322594045 7 5 ja.m.wikipedia.org 445007695821 7 6 ja.m.wikipedia.org 435472597765 7 7 ja.m.wikipedia.org 430814756332 7 8 ja.m.wikipedia.org 461334758600 7 9 ja.m.wikipedia.org 547280892717 7 10 ja.m.wikipedia.org 600062076779 7 11 ja.m.wikipedia.org 473057269970 7 12 ja.m.wikipedia.org 477262829644 7 13 ja.m.wikipedia.org 471436878948 7 14 ja.m.wikipedia.org 472398332176 7 15 ja.m.wikipedia.org 548724119441 7 16 ja.m.wikipedia.org 616527657391 7 17 ja.m.wikipedia.org 639723642046 7 18 ja.m.wikipedia.org 638694292375 7 19 ja.m.wikipedia.org 481106407202 7 20 ja.m.wikipedia.org 473148500657 7 21 ja.m.wikipedia.org 466261658125 7 22 ja.m.wikipedia.org 465249325390 7 23 ja.m.wikipedia.org 492041513150 7 24 ja.m.wikipedia.org 563837017528 7 25 ja.m.wikipedia.org 445503057133 7 26 ja.m.wikipedia.org 447799358192 7 27 ja.m.wikipedia.org 447035047673 7 28 ja.m.wikipedia.org 432588647022 7 29 ja.m.wikipedia.org 447436227804 7 30 ja.m.wikipedia.org 503914286671 7 31 ja.m.wikipedia.org 553840973015 8 1 ja.m.wikipedia.org 465305749921 8 2 ja.m.wikipedia.org 459082686410 8 3 ja.m.wikipedia.org 468718936637 8 4 ja.m.wikipedia.org 463721744365
Page views
editTo rule out bytes saved being due to a drop in page views, we checked the traffic to mobile site, in ja.wikipedia (i.e., ja.m.wikipedia.org) in the months of June and July. In June the user traffic was 585 millions and in July 587 millions. (normalized for a 30-day month). The two months are pretty much the same in terms of traffic for practical purposes so we concluded that the savings in bytes was not likely due to a macro change in traffic.
See also
edit- A decrease in the number of requests would make mediacounts more truthful.