Wikimedia Engineering/2015-16 Q2 Goals

Q1

Wikimedia Engineering Goals, FY2015-16, Q2 (October–December)

Q3

Purpose of this document: Goals for the Wikimedia Product and Technology groups (collectively, Wikimedia Engineering), for the second quarter of fiscal year 2015–16 (1 October 2015 – 31 December 2015). The goal setting process owner in each section is the person responsible for coordinating completion of the section, in partnership with the team and relevant stakeholders.

Status fields can use the following templates: In progress, To do, N Postponed,  Done or Partially done

Product

Discovery

Goal setting process owner: Dan Garry

We build the anonymous path of discovery to a trusted and relevant source of knowledge.

Our focus will be on building on-top of our anonymous and trusted search stack to improve relevancy, expose additional content and leverage locally relevant knowledge.

For the below goals, the objectives are relatively finalised, but the key results and dependencies still need further definition before the goals are locked in.

Search

Objective	Key result	Dependency	ETA	Status
Improve language support for search.	Run A/B test for a feature that: Uses a library to detect the language of a user's search query. Adjusts results to match that language. Determine from A/B test results whether this feature is fit to push to production, with the aim to: Improve search user satisfaction by 10% (from 15% to 16.5%). Reduce zero results rate for non-automata search queries by 10%.	Operations: Wikimedia Labs (maintaining Labs) Discovery Analysis Team	Dec 2015	Done
Make www.wikipedia.org a portal for exploring open content on Wikimedia sites.	Measure usage of existing portal. Perform A/B tests to improve portal. Decrease time each user spends searching by Y ms (exact number pending rollout of event logging schema on the portal).	Operations: Wikimedia Labs (maintaining Labs) Community Engagement (support for communications around portal improvements) Discovery Analysis Team	Dec 2015	N Not done

Wikidata Query Service

The Wikidata Query Service Team achieved its Q1 goal to release the Wikidata Query Service to production, with a public endpoint. In Q2, development on the project will be put on hold, user feedback about what features are missing will be collected, and usage of the service will be measured over the course of Q2; this will be used to assess whether further development will be required in Q3.

Maps & Geo

The Maps Team is currently on track with its Q1 goal to release the Maps tile service to production, with a public endpoint, in Q1. In Q2, development on the project will be put on hold, user feedback about what features are missing will be collected, and usage of the service will be measured over the course of Q2; this will be used to assess whether further development will be required in Q3.

Analysis

Objective	Key result	Dependency	ETA	Status
Improve understanding of user satisfaction for search by iterating and improving on the search satisfaction metric.	Design QuickSurvey to qualitatively assess user satisfaction with search. Tie qualitative QuickSurvey results back to quantitative search satisfaction data. Validate/invalidate current approach for measuring satisfaction.	Operations: Wikimedia Labs (maintaining Labs)	Dec 2015	N Not done
Support ongoing evaluation of usage of Wikidata Query and Maps services to decide on what's next for these services.	Continue maintenance of Wikidata Query Service and Maps dashboards. Review user feedback on services. Make decision on continued level of support for WDQS and maps service.	Operations: Wikimedia Labs (maintaining Labs)	Dec 2015	Done
Create recurring performance indicator on referrer traffic from the primary search engines and determine what features may have largest impact on referrer metric.	Create dashboard which displays traffic to our sites broken down by major referrers. Based on the dashboard data, make a list of recommendations for features which could be implemented to increase traffic from prominent referrers.	Operations: Wikimedia Labs (maintaining Labs)	Dec 2015	N Not done

Notes:

• In response to a request from the Head of Research and the Vice President of Product, the Discovery Analysis Team dedicated one of its analysts at 33% capacity to the annual fundraising campaign to aid in ensuring its success. This reduction in anticipated capacity places some of the above goals, particularly the first one, at risk.

Editing

Goal setting process owner: User:Trevor Parscal (WMF)

The Editing Department build collaborative, inclusive tools for creating and editing free knowledge. In Q2, the Department will focus on rationalising technologies and strengthening engineering practices whilst delivering on the key user workflows for editors across the product range.

Editing Departmental level goals

Objective	Key result	Dependencies	ETA	Status
Focus: The visual editor and other editing improvements released to less-active accounts and logged-out users on the English Wikipedia.	The visual editor available to more accounts and logged-out users on the English Wikipedia; opt-in is visible to all. Cross-wiki notifications. Add support for inserting and editing more kinds of rich content using visual editor. Image upload as you edit from the toolbar in wikitext and the visual editors, and via drag-and-drop into the visual editor.	Community Liaisons Communications Design Research	EOQ	Part done; some parts delayed
Strengthen: Prototype and research suggestion interfaces for the visual editor and ContentTranslation.	Designs for "product"-ifying the suggested-edit features being prototyped and researched are ready to be implemented in Q3.	Design Research Research & Data	EOQ	Partially done
Experiment: Experiment with other kinds of editing and content.	Incorporate two additional extensions to VisualEditor. Research VisualEditor on mobile and use of video on Wikipedia.	Design Research	EOQ	Partially done

VisualEditor Team

Involve more people than ever before in the collaborative effort of building free knowledge through improving the usability, stability and performance of VisualEditor, integrated with the developing reading experience.

Objective	Key result	Dependencies	ETA	Status
Focus: Increase use of VisualEditor by releasing VisualEditor to more accounts and anonymous users on English Wikipedia	More editors use VisualEditor	Community Liaisons	EOQ	Delayed by scope increases
Strengthen: Improve support for math editing by improving the UX of the Math extension and researching its usability	It's easier to create and edit formulæ from within VisualEditor	–	EOQ	Done
Strengthen: Improve support for graph editing by improving the UX of the Graph extension and researching its usability	It's easier to create and edit charts from within VisualEditor	–	EOQ	Done
Experiment: Increase value of contributions by prototyping and researching integrating edit recommendations	We have an informed plan for next steps for integrating edit recommendations	Design Research Research & Data	EOQ	Delayed by other priorities
Experiment: Improve VisualEditor on mobile devices by prototyping and researching using VisualEditor on mobile devices	We have an informed plan for next steps for using VisualEditor on mobile devices	Design Research	EOQ	Partially done

Collaboration Team

Increase the participation, mentoring and awareness of activity on the site by bringing structured conversations to both talk pages and the editing experience by improving and deploying Flow where it can help most.

Objective	Key result	Dependencies	ETA	Status
Focus: Improve awareness of activity by providing a cross-wiki notifications feature	Echo can deliver notifications between many wikis, allowing users to access their notifications from any content wiki on any other content wiki	Security Community Liaisons Technical Operations	EOQ	In progress
Strengthen: Increase access to Flow by deploying and supporting an opt-in system for it	Users have the ability enable Flow on their talk page	Community Liaisons	EOQ	In progress
Experiment: Increase engagement with notifications by prototyping and researching improvements to the notifications page	We have an informed plan for the next steps for improving the notifications page	Design Research	EOQ	In progress

Multimedia Team

Enrich the content of our projects with more use of multimedia assets, by streamlining media contribution with UploadWizard, and evolving the rich editing experience to better suggest media.

Objective	Key result	Dependencies	ETA	Status
Focus: Increase media contribution by integrating media upload into VisualEditor's media dialog and drag-and-drop system	Users can upload media files directly within VisualEditor by browsing or dragging-and-dropping	–	EOQ	Done
Strengthen: Improve reliability by resolving UploadWizard bugs that cause users to start over	Users can use UploadWizard to mass-upload without interruption	–	EOQ	Partially done
Experiment: Improve multimedia editing workflow by prototyping non-destructive image editing	Users can make basic changes to images such as crop, rotation, etc. directly on the wiki	Performance	EOQ	Done
Experiment: Improve value of content by researching the learnability of content to inform future prioritisation of feature development	We have an informed plan for next steps for multimedia contribution features	Design Research	EOQ	Partially done

Language Team

Growing and accelerating projects with smaller content bases by providing and promoting enhanced Content Translation tools for translating content from projects with larger content bases.

Objective	Key result	Dependencies	ETA	Status
Focus: Improve user retention by adding additional 'suggestion' features such as customised lists and adding additional relevant notifications	Increase in the number of translations per user	Research & Data	EOQ	Done
Strengthen: Improve reliability by complying with WMF infrastructure requirements as defined by Services, Tech Ops and Security teams	ContentTranslation complies with WMF infrastructure requirements	Ops, Services	EOQ	Done
Strengthen: Improve reliability by fixing high-priority bugs that affect basic functionality	Users can use ContentTranslation without interruption	Parsing	EOQ	Done
Experiment: Increase visibility for third parties by collecting and sharing parallel corpora of translation modifications	API will be able to access the parallel corpora	Ops	EOQ	Partially done

Parsing Team

Increasing the integrity, accessibility and stability of content by providing Parsoid as both a bridge for editing wikitext documents, but also a rendering system for viewing and querying them.

Objective	Key result	Dependencies	ETA	Status
Focus: Improve editing experience and draw closer to Parsoid HTML for read views by implementing support for native Parsoid extensions	Parsoid has an extension registration mechanism and native support for galleries	–	EOQ	Done; ‎<gallery> will be implemented in Q3; ‎<translate> and testing extensions done
Strengthen: Improve mobile reading experience and draw closer to Parsoid HTML for read views by improving multimedia support in Parsoid	Parsoid's HTML5 DOM spec and HTML generation have been updated for audio and video	–	EOQ	N Postponed; Deprioritized below other requests; Will be picked up in Q3
Strengthen: Modernise parsing infrastructure to use modern Web standards by replacing Tidy with the HTML5 parser in MediaWiki core	MediaWiki has an HTML5-compliant parser for fixing PHP parser output	–	EOQ	Partially done - rollout is more complex than anticipated; Incremental roll-out of smaller pieces
Experiment: Improve editing experience and take advantage of performance optimisation opportunities by prototyping some form of opt-in / opt-out system for well-balanced template output	MediaWiki has a mechanism for templates to specify their output as balanced or not	–	EOQ	Partially done; Implementation sketch in place; Prototypes to be done; Has to go through RFC process

Fundraising Tech

Goal setting process owner: User:AGomez_(WMF), User:Khorn_(WMF)

Objective	Key result	Dependencies	ETA	Status
Focus: Support the Big English fundraiser	Fundraising creative reaches their donation goals in December	Fundraising (creative) team	21/1 - 12/31	Done
Strengthen: Get France campaign tech-ready	Fundraising creative reaches donation goals in October (in France)	3rd Party Integration Support (WorldPay) Fundraising (creative) team	10/23	Done
Experiment: Support Banner History rollout	Fundraising creative is able to see patterns and problems with banner consumption, before December	Analytics (Ellery) Fundraising (creative) Performance engineering Release engineering	10/1 - 10/31	Done
Experiment: Start investigating SmashPig as its own code project	Rough estimate of the work involved to create the open source project -OR- a decision not to move forward with independent payment libraries	Unknown	EOQ	N Postponed - Extremely disruptive conversations were endangering our mission-critical Q2 fundraising goals. Postponed for a less complicated time.

Reading

Objective	Key result	Dependencies	ETA	Status
Strengthen: Drive engagement by launching Link Preview on mobile web beta and Read More on web beta	Engagement (5% increase in links clicked) in beta or on non-EN Wikis (Link Preview T111329, Read More T94906)	Community Liaisons	EOQ	Partially done Read more is on beta, link preview is on hold
Experiment: Drive app retention via feeds and modern UX (notifications Q3)	iOS App overhaul released 7 day 15% of users retained (up from 10% in Sept ‘15) (Feeds T104415, Modernization T104510)	Design Research	EOQ	Partially done Notifications pilot in place.
Focus: Measure MediaWiki API usage, and data dump loads.	Availability of usage metrics  on Hadoop warehouse for analysis. (T102079)	Partnerships Analytics team	EOQ	In progress
Strengthen: Investigate migrating API traffic to OAuth.	Decision on if migration is desirable; then measurable impact based on existing usage (currently unknown)	MediaWiki extension developers Security team	EOQ	To do
Strengthen: Release Auth Manager: MediaWiki plug-in security architecture	Usage (# of services using new architecture) (T89459)	MediaWiki extension developers Security team	EOQ	In progress
Experiment: Prototype services based architecture for web and apps	Working web app with feature parity of current mobile web stable reading experience (not scalability, quality). (main task T113066, Wikimedia Developer Summit 2016 discussion T114542)	Services team	EOQ	Done

UI Standardization team

Goal setting process owner: Volker Eckl

Objective	Key Result	Dependency	ETA	Status
Basic stylesheet of accorded design properties	Basic stylesheet ("base.css") of generally accorded design properties/values to spec implemented in a central repo & accompanying HTML.	Designers on web-facing teams, community involvement	EOQ	In progress
Enhance accessibility of Living Style Guide (Blueprint) & WMF libraries	Color palette that is compliant to WCAG 2.0 guidelines at minimum level AA for Wikimedia projects' usage (including for use in basic stylesheet and Living Style Guide (Blueprint)). Enhance OOjs UI & mediawiki.ui library to increase reasonable WAI-ARIA best practices accomplished in design and code.	Editing (OOjs UI), community involvement	EOQ	In progress
Style Guide	Initial step: Understand what non-technical contributors need in their work	Design Research team	EOQ	In progress

Community Tech

Objective	Key results	Dependency	ETA	Status
Strengthen: Establish baseline for measuring community satisfaction with WMF tech support	Completed Tech Support survey across multiple Wikimedia projects and languages	Learning and Evaluation support	Mid-November	Done
Focus: Include Wikimedia communities in goal development and prioritization	Completed cross-project Community Wishlist survey, with community members voting on priority		EOQ	In progress
Focus: Address current top-priority community technical requests	At least 3 mid to large-sized requests from existing wishlists		EOQ	In progress

Technology

Analytics

Goal setting process owner: Kevin Leduc

In progress

Objective	Key result	Dependencies	ETA	Status
Focus: Use Hadoop infrastructure to replace computation of Wikistats	Replicate 5 of wikistats most popular pageview reports defined here.	Research (Erik Zachte)		In progress
Strengthen: Pageview API	API is operationalized and ready to be queried by outside world.  A blog post is published on Wikimedia’s blog.	Services Team		Done Blogpost
Experiment: Build infrastructure Scalable Event Bus	A service to produce to kafka with a standardized schema is running in production.	Services Team		In progress

Architecture

Goal-setting process owner:  Rob Lanphier

(previously discussed/published at WMF, but not posted here until 2015-12-11)

Objective	Key result	Dependencies	ETA	Status
Focus: Prepare WikiDev '16	Demonstrate that we can develop a modern system in an inclusive, consensus-oriented, open manner.  Large number of diverse RfCs, with ample discussion prior to WikiDev '16	Wide participation in Engineering	2016-01-04	Done - Preparation happened, summit resulted  :-)
Strengthen: Improve ArchCom utility	Set up Architecture Committee to succeed.  Maintain one public IRC meeting per week	ArchCom members at >20% time We may want/need to expand ArchCom	2015-12-31	Done - 10 IRC RFC review meetings, plus 3 “Agenda bashing” meetings for WikiDev ‘16.  Most were at the usual time (14:00 PT) Phab query for meetings
Experiment: ArchComm naming	Think through our loaded naming scheme, possibly renaming bits (e.g. “Architecture Committee” and “RfC”)	Consultation over naming	2015-12-31	N Not done - oh well

Performance

Goal-setting process owner: Ori Livneh

Speed is Wikipedia's killer feature. ("Wiki" means "quick" in Hawaiian.) As the Wikimedia Foundation’s Performance team, we want to create value for readers and editors by making it possible to retrieve and render content at the speed of thought, from anywhere in the world, on the broadest range of devices and connection profiles.

The following is a summary of the 2015–16 Q2 Performance team goals.

Objective	Key results	Dependencies	Estimated time of arrival	Status
Availability	Make MediaWiki code ready for concurrent deployment across multiple data centers. This is a milestone on the way to full deployment (T88445).	TechOps	EOD	In progress
Reporting	We'll publish weekly performance reports that provide clear and compelling insights into important performance characteristics.	None	When ready	Done, but cadence is monthly for now.
Faster painting	We want a 10% reduction in the time-to-first paint across desktop and mobile.	None	When ready	Done

Release Engineering

Goal setting process owner: Greg Grossmeier

All tracked in: #releng-201516-q2 - More details at Wikimedia_Release_Engineering_Team/Goals/201516Q2

Key Performance Indicators

Objective	Key result	Dependency	ETA	Status
STRENGTHEN Reduce CI wait time Wait time KPI	CI cluster responds to spike in queued builds by starting and registering additional jenkins slaves - task T111106 -  Done Migrate majority of CI jobs to Nodepool - task T114315 python tox - task T119141 -  Done ruby rake - task T119142 -  Done javascript npm - task T119143 - N Not done	Operations / WMF Labs	EOQ	N Not done
FOCUS Reduce number of deploy tools from 3 to 2	Migrate all Service team owned services and MW deploys to scap3 - task T109926	Services / Operations	EOQ	N Not done
FOCUS Retire Gerrit and Gitblit in favor of Phabricator	Decommission Gitblit - task T111465 - N Not done Code review RFC: creation, publication, discussion, feedback etc - task T114311 -  Done	(small) Operations / DevRel	EOQ	N Not done
FOCUS Release MediaWiki 1.26	A quality MW 1.26 successfully released - task T110486	WMF Engineering MW Technical Community	Mid-Nov	Done

Research

In Wikimedia Research we use qualitative and quantitative methods to provide strategic insights and technological solutions to the movement and the Foundation, to foster innovation and to inform the development of new products.

Research & Data

Goal setting process owner: Dario Taraborelli [ Wikimedia Research goals overview ]

Objective	Key results	Dependencies	ETA	Status
FOCUS Streamline ownership of research / data-related work	Publish a map for internal use of who is responsible for specific requests around research and analytics. (T112317)	Team / department leads	10/1-10/31	In progress
STRENGTHEN Revscoring integration	Bring revscoring to fruition to our users as a Beta Feature (score integration into RC feed) (T112856)	Ops Design Discovery	EOQ	In progress
STRENGTHEN Productize article and link recommender systems	Article creation (T112321) and link (T112322) recommendation productized as services so they can be integrated with products and community tools.	Ops Editing	EOQ	In progress
EXPERIMENT Value-added research	Deliver research on historical data to quantify who adds value  to English Wikipedia (T99172)	Altiscale cluster	EOQ	In progress
EXPERIMENT Reader segmentation research (stretch)	Deliver research on reader segments and behavioral patterns (T112326)	Analytics Dev (spiky cluster usage) Research fellows	EOQ	In progress

Design Research

Goal setting process owner: Abbey Ripstra [ Wikimedia Research goals overview ]

Objective	Key result	Dependencies	ETA	Status
STRENGTHEN Mentor people to do quality design research (those not on DR team).	One person outside the design research team does a complete design research project to DR team standards, expanding the number of people who do design research well.	Person outside the DR team having capacity to do a design research project.	EOQ	Done
STRENGTHEN Design Research workshop to build collaboration capacity with product teams.	Workshop happens, and product teams learn more about collaboration with Design Research and vice versa.	Availability of workshop participants to attend (track leads, managers and individual contributors from each product track).	MidQ	Done
STRENGTHEN Improve participant recruitment process	Implement changes to reduce steps in recruitment (of design research participants) process to make recruiting more efficient.	n/a	EOQ	Done
FOCUS Collaborate with product teams to implement design research projects.	Design Research has done design research projects in collaboration with product teams to inform human centered product development.	All product teams we collaborate with Research participants show up to research sessions (volunteers)	EOQ	Done
EXPERIMENT Persona interviews	12 persona interviews completed	Design research participants show up to research sessions (volunteers)	EOQ	Partially done

Security

Goal setting process owner: User:CSteipp (WMF)

We make life hard for the people who want to harm our sites or the people that use them.

Objective	Key result	Dependency	ETA	Status
Automated security static analysis of MediaWiki	Evaluate potential tools If a suitable tool can be found, automatically scan core and at least one extension weekly.	None	EOQ	Done
Expand developer training	Present secure code and secure SDL training for community and staff (stretch) Develop and present security training materials for DevOps and Mobile	None	EOQ	In progress
(stretch) Document and report initial metrics for security reviews	Metrics reported	None	EOQ	To do

Services

Goal setting process owner: Gabriel Wicke

For details, see the Phabricator task.

Objective	Key result	Dependencies	ETA	Status
Strengthen: Build out the REST API	Provide cached & purged REST API entry points for at least two high-traffic API use cases, and work with users to speed up page loads.	Reading, Editing	EOQ	Done
Focus: Set up an event bus and change propagation solution	Key change events like edits, renames, deletes available in a publish-subscribe event bus. Basic change propagation / pre-generation functionality for REST services operational.	Analytics	EOQ	In progress
Experiment: Prototype an API-driven web front-end	Gather information to inform our longer-term front-end architecture by prototyping a service-worker / node.js based front-end.	Reading	EOQ	Done

Technical Operations

Goal setting process owner: User:Mark Bergsma (WMF)

Objective	Key result	Dependencies	ETA	Status
Focus: Encrypt PII on cross-data center links	Encrypt cross-data center traffic for MySQL/MariaDB, Kafka, Cassandra, (Varnish) HTTP Decommission udp2log (in favor of Kafka)	Analytics (Kafka, udp2log) Fundraising (udp2log, Kafka)	EOQ	Done
Strengthen Security: Strengthen infrastructure user authentication	Establish organisation-wide workflow for creation and deactivation of user accounts in Production Implement (but don't enforce) two-factor authentication on SSH bastion hosts Migrate unmaintained OpenDJ LDAP to OpenLDAP	HR & OIT (for workflow standardization) Procurement (for 2fa hardware)	EOQ	Done
Experiment: Support bare metal servers in Labs	Allow provisioning of physical servers within the Labs network, governed by Labs Puppetmaster and LDAP user accounts		EOQ	Done
Strengthen Incident monitoring infrastructure	Upgrade incident monitoring and expand it to codfw  Done Reduce monitoring check execution lag to 0  Done Create new abstractions for incident monitoring in Puppet Partially done		EOQ	Partially done