Reading/Web/Projects/Performance/Inlining CSS

Hypothesis

The reading team is planning to lazy load content - navboxes, references and images.

It has been suggested that if we inline CSS there is no need to lazy load any of the HTML content, for any other reason than reducing bytes/fully load time.

First paint is important as it signals when the user can begin reading.

Fully loaded time is important given our JS will generally load around that point, meaning a user will not be able to edit or search until that time.

Prediction

My feeling was that both inlining CSS and lazy loading content would lead to improvements on first paint and fully loaded time.

My hope that was combined they would provide the lowest first paint and fully loaded time possible.

Method

I wrote a patch that allows to to configure Minerva's CSS so that it is inlined in the HTML.

I enabled this on reading web staging temporarily and initiated some web page tests.

All tests were run on the same browser on an emulated 2g connection.

Test 1

I configured the server to inline CSS in both beta and stable.

$wgMinervaInlineCSS = true;
$wgMFRemovableClasses = array(
		'stable' => array(),
		'beta' => array( '.references','.navbox' ),
}
$wgMFLazyLoadImages = array(
	'base' => false,
	'beta' => true,
);

I ran webpagetest (9 views) on beta and stable mobile channel.

Test 2

I wrote a patch to fix an issue in Extension:Gather so that it didn't load CSS unnecessarily in the head and enabled that on the testing server.

I re-ran test 1 again with this bug fix after purging the page's cache.

I include this test in the results as it's interesting to shows the impact of having that link tag in the top even if the content of the stylesheet is tiny).

Test 3

This time I disabled MinervaInlineCSS on both beta and stable and ran the tests again after purging the page's cache. The stable channel would act as the current conditions and give us a baseline.

$wgMinervaInlineCSS = false;
$wgMFRemovableClasses = array(
		'stable' => array(),
		'beta' => array( '.references','.navbox' ),
}
$wgMFLazyLoadImages = array(
	'base' => false,
	'beta' => true,
);

Test 4

For completeness, I ran a single test in stable with lazy loaded content and inlined CSS.

$wgMinervaInlineCSS = true;
$wgMFRemovableClasses = array(
		'stable' => array( '.references','.navbox' ),
		'beta' => array( '.references','.navbox' ),
}
$wgMFLazyLoadImages = array(
	'base' => true,
	'beta' => true,
);

and then with CSS not inlined.

$wgMinervaInlineCSS = false;
$wgMFRemovableClasses = array(
		'stable' => array( '.references','.navbox' ),
		'beta' => array( '.references','.navbox' ),
}
$wgMFLazyLoadImages = array(
	'base' => true,
	'beta' => true,
);

Test 5

Out of curiosity I wanted to see how much lazy loading images contributed to the fully load time

$wgMinervaInlineCSS = false;
$wgMFRemovableClasses = array(
		'stable' => array(),
		'beta' => array( '.references','.navbox' ),
}
$wgMFLazyLoadImages = array(
	'base' => true,
	'beta' => true,
);

Results

Test URL	Channel	Lazy loaded images	Lazy loaded content?	Inline CSS?	First View First paint (s)	First View Fully loaded time (s)
Test 1a	Stable	YES	NO	YES+Gather stylesheet	4.890s	50.020s
Test 1b	Beta	YES	YES	YES+Gather stylesheet	4.418s	20.015s
Test 2a	Stable	YES	No	YES	3.590s	49.962s
Test 2b	Beta	YES	Yes	Yes	3.594s	20.006s
Test 3a	Stable (baseline)	YES	No	No	5.189s	49.528s
Test 3b	Beta	YES	Yes	No	5.288s	14.820s
Test 4a	Stable	YES	Yes	Yes	3.515s	18.663s
Test 4b	Stable	YES	Yes	No	5.303s	14.681s
Test 5a	Stable	Yes	No	No	5.005s	24.686s

Analysis

In stable, without any optimisations (test 3a) the baseline showed first paint at 5.189s and fully load time at 49.528s

When just inlining CSS (test 2a) first paint dropped to 3.590s (31% improvement) and fully loaded to 49.962s.(0.9% degradation in performance). It's very important that every single stylesheet is inlined however for this to have an impact (see test 1a).

When just lazy loading content (test 4b), first paint was actually increased to 5.303s (2% degradation, possible due to the additional work on the server) but fully loaded dropped to 14.681 (70% improvement). Analysing test 5a, it seems that 50% of this improvement was due to the lazy loading of images.

Combining them both (test 4a) dropped first paint to 3.515s (32% improvement) and fully loaded to 18.663s (62% improvement in performance). It's possible that inlining the CSS will however slightly increase the time to fully load (compare results of test 4b with 4a). That would need further investigation.

Conclusions

Early predictions suggest:

• Inlining all top loaded CSS will improve first paint
• Lazy loading content should not impact first paint but will improve fully load time (which in turn you would hope would lead to increased interactions around the site due to JavaScript loading quicker) and data savings without minimal damage to the user experience.
• Doing both seems like the best of both worlds although fully loaded time will be slightly greater than if you simply lazy load content.
• Most of the savings in fully load time come from lazy loading images.

Further questions

If images are low resolution will that make any difference (given that the first image will be smaller)?

Are we aiming to drive first paint down as low as possible or fully loaded time?