Wikimedia Technology/Annual Plans/FY2019/TEC12: Developer Productivity

To make the goal of the Continuous Deployment Pipeline program a reality we must address the need for a robust local development environment; without this we lose much of the potential gain from all of the cross-departmental work put into the Streamline Services Delivery program thus far. This will improve both developer productivity and satisfaction.

The benefits from this are truly hard to over state: We'll greatly reduce our comically (and legendarily) long on-boarding time - on the order of months; we'll give our engineers fewer reasons to ever say "but it worked on my laptop"; and we'll finally address those long standing issues that subtly irritate our engineers on a daily basis, thus increasing retention.

The Release Engineering Team is already working on a streamlined pipeline that can effectively support such developer tooling, and the work we've completed over the past 2 quarters has already yielded dramatic gains in Continuous Integration speed for Mathoid (small changes passing tests in 20 seconds) and Operations puppet (an early iteration of pipeline work taking tests from > 1min to a current average of ~20 seconds with a minimum test time of 8 seconds).  The inclusion of local development in this pipeline was something that we discussed from inception but the priority and ownership of has been unclear due to resourcing needs within Technology.

We'd like to service all of engineering's (Audience, Technology, and the wider community's) development needs by extending this pipeline to directly include the local development environment.

Members of Audience management and staff are already pushing us for this need and support this request.

Why us?

Because we provide much of the developer tooling already we understand the pain points in getting code from the developer's laptop to production; we see these problems in a holistic way. We also currently are responsible for two production-like environments (Continuous Integration and the Beta Cluster) which gives us the hard won knowledge for how to unify those environments.

What we plan to do:

We would initially work on enabling developers to easily setup their development environment to essentially match what is in production; something that is impossible with our current tooling and resources.

Concurrently we will be working with all Wikimedia developers to prioritize the highest return on investment changes we can make to improve overall developer productivity. This would be informed through a survey and input from the Code Health Group. We will strive to solve the productivity issues that individual teams don't see, and do it pre-emptively.

In short: improve productivity and satisfaction of all developers.

Program outline

Teams contributing to the program

Release Engineering

Annual Plan priorities

Primary Goal: 3. Knowledge as a Service - evolve our systems and structures

How does your program affect annual plan priority?

By addressing the needs of our developers, we directly impact/improve their productivity.  This increase in efficiency and efficacy enables our developer community, staff and volunteer, to maintain and evolve the systems that we depend on to support the movement's vision of collecting and sharing the sum of all human knowledge.

Program Goal

This program will increase overall developer productivity through local development tooling and support. This will reap rewards with onboarding (both Foundation staff and new volunteer community members), retention, and development efficacy.

Outcomes

Outcome 1: Local development is unified with testing and production (task T222237)

Output 1.1

Create a production tooling compliant Dockerfile that sets up a basic LAMP stack, installs MediaWiki, and can run unit tests. (task T222205)

Output 1.2

Start an RFC on MediaWiki distribution method going forward.

Output 1.3

Create a generic method for installing extensions and skins in the image (task T222232)

Output 1.4

Support running Selenium browser tests (task T222234)

Output 1.5

A helm-based dependency chart for installing extensions and skins along with services they require (task T222233)

Outcome 2: Developer satisfaction improves year over year.

Output 2.1

A survey is created and shared with all Wikimedia developers to measure overall satisfaction and productivity as well as identifying the most sought after needs. The top needs are shared as key goals for the team.

Output 2.2

The top priority needs are addressed over the remainder of the year. Exact number is determined by difficulty and time available.

Resources

Release Engineering	FY2017–18	FY2018–19
People (OpEx)		Software Engineer (new hire)
Stuff (CapEx)		None anticipated.
Travel & Other		1 ✕ Developer Summit & team offsite 1 ✕ Wikimedia Hackathon & team offsite

Targets

Outcome 1: Local development is unified with testing and production

Target 1

A replacement for MediaWiki-Vagrant using the same tooling as production is available.

Measurement method

1. This is measured by the availability of the replacement complete with support for MediaWiki, extensions, and skins. We will also be measuring uptake through a survey conducted at the end of the fiscal year.

Outcome 2: Developer satisfaction improves year over year.

Target 2: 3 of the top needs identified by developers are addressed.

Measurement method

1. This is measured by the availability of solutions for 3 of the top issues identified.

Dependencies

• MediaWiki Platform (feedback and consultation for the development environment)