Wikimedia Engineering/2016-17 Q4 Goals

Q3

Wikimedia Engineering Goals, FY2016-17, Q4 (April - June)

Q1 for Audiences; Q1 for Technology

Purpose of this document: Goals for the Wikimedia Technology and Product Development department, for the second quarter of fiscal year 2016–17 (April 2017 – June 2017). The goal setting process owner in each section is the person responsible for coordinating completion of the section, in partnership with the team and relevant stakeholders.

Status fields can use the following templates: In progress, To do, N Postponed,  Done or Partially done

ETA fields may use the initialism EOQ (End of Quarter).

Product

The Product department, alongside the community, builds, improves and maintains the user facing features on all the Wikimedia projects. The software, interfaces, apps, and other systems and applications the team supports are essential means by which contributors and readers connect with our sites. The work of the team’s three “audience” groupings – Discovery, Editing, and Reading – is interrelated and enables a channel for progressive engagement with the users of the Wikimedia projects.

• Annual Plan for Product 2016-2017

Discovery

Our mission is to make the wealth of knowledge and content in the Wikimedia projects easily discoverable.

Goal setting process owner: Dan Garry

Team	Objective	Key result	Dependency	ETA	Status
Search	Improve search by researching and load testing machine learning to automatically re-rank search results	Research and develop new models to determine which ones work best Build out data munging and data pipeline to develop new models Deploy automated model which matches current performance of manually-configured search result relevance Begin performing load tests on new models to make sure they can be safely deployed to production	Community Engagement, Team Practices Group, Analytics	EOQ	Partially done
	Improve support for different languages by researching and deploying new language analyzers	Perform research spikes to find new analyzers for different languages Test new analyzers to see if they are improvements Deploy new analyzers that are found to be an improvement	Community Engagement, Team Practices Group, Analytics	EOQ	Done
	Refine and test new design of search results page with occasional deployments	Test new functionality with at least 2 A/B tests Analyze tests and gather community feedback Roll out new functionality with small releases to production across all Wikipedias	Community Engagement, Analytics	EOQ	Partially done
Interactive	Stabilize maps and graphs code base	Stabilize and fix high priority bugs	Community Engagement	TBD	In progress

Wikidata Query Service goal for this quarter will be working on integrating the Mediawiki API. The service will also continue to be maintained by Stas and Guillaume to support the continued growth and use of the service.

Wikipedia.org Portal team will not have any dedicated goals this quarter other than continuing to maintain the page by fixing critical bugs and performing regular statistics and translations updates. No other work on the page is planned in Q4.

Editing

Goal setting process owner: Trevor Parscal (WMF)

The Editing Department builds collaborative, inclusive tools for creating and editing free knowledge.

Also see the more detailed progress on these goals which are based on Program 4 of the Annual Plan

Annual Goal	Team	Objective	Output (deliverable)	Outcome	Dependencies	ETA	Status
1: Maintain and incrementally improve current content creation and curation interfaces	Collaboration	Improve efficiency for edit reviewers in general while providing an edit-review process suited to the needs of new contributors.	Make improvements to the New Filters for Edit Review beta features based on user research and feedback. Spread the beta features to all wikis.	Smaller backlogs, improved retention...? more reviewers, better quality of the wikis... For the first time, reviewers will have data about user intent. More people involved? More productive.	CLs, to work with target communities. Editing Design Research to assess user satisfaction with the beta.	EOQ	Done
	Language & VisualEditor	Stabilize Content Translation for going out of beta	Replace the contenteditable editing component of Content Translation to VisualEditor and preserve the current CX functionalities	Content Translation is more stable in terms of article publishing, requiring less maintenance work from patrollers, and more ready to be enabled by default for all users.	Parsing Editing Design (for OOjs UI)	EOQ	Continues
2: Improve editing experience by reducing technical and product debt	Parsing	Support audio and video content in Parsoid	Parsoid HTML will emit markup for video and audio content according to an agreed-upon spec	Parsoid rendering for media files matches existing rendering by the PHP parser bringing us closer to the goal of using Parsoid HTML for edits as well as read views.		Deployed April 26, 2017	Done
	UI Standardization	Define and support mobile browser stack in OOjs UI	Define design requirements for widgets on mobile devices and develop an initial implementation of those design elements on some popular browsers.	A clear and documented path forward for making OOjs UI a fully responsive and unified experience across platforms	Editing Design (for OOjs UI)	EOQ	Continues

Fundraising Tech

Goal setting process owner: Katie Horn

Objective	Key result	Dependency	ETA	Status
Import Silverpop data into Civi	Unsubscribes must flow back to Civi	email team	EOQ	To do
Continue Ingenico reintegration	continue bulk of engineering work	Ingenico	EOQ	To do
Central Notice research and outreach	Multi year redesign plan	Joseph Seddon and external designer	EOQ	To do

Reading

Goal setting process owner: Jon Katz

Team	Objective	Key result	Dependency	ETA	Status
New Readers	Better support offline users by allowing them to browse and download collections of content via ZIM files in the Android app.	Requirements defined and documented in partnership with Android and Reading Services team, including consultation with the Kiwix team.	Android, Reading Services	EOQ	To do
Reading Web	Improve the quality of printed content on desktop to support offline usage	Update and improve current print styles for desktop	Community liaisons	EOQ	To do
	Increase learning by lowering the cost of exploration	Deploy page previews feature to all wikipedias	Community liaisons	EOQ	To do
	Move towards a single PDF rendering service to simplify UX and lower maintenance requirements	Evaluate current OCG functionality and expand the functionality of the Electron Rendering Service	Reading Infrastructure, Community liaisons, New Readers	EOQ	To do
	Assess the possibility of providing an offline-capable app and unifying content through various services to improve performance and development speed.	Define and scope changes to the mobile website for development in 2017-2018	Services, Community liaisons, Apps, New Readers	EOQ	To do
Reading Infrastructure	Support power users across devices by developing a Reading List Service	Design and Develop a Reading list Service. Deploy to Beta Cluster.	ArchCom (RFC)	EOQ	To do
	Finalize push notification architecture to bring relevant content and messages to our users	Make all key architecture decisions for a Push notification service. Begin development	ArchCom (RFC), Ops, Services	EOQ	To do
Android app	Support offline users, by adding support to import, read and search offline content from the ZIM format to the Android app.	Design and release to beta a version of the app which allows importing, reading and searching the ZIM format.	Reading Services. New Readers	EOQ	To do
iOS app	Bring relevant content to our users by completing integration of Trending Edits service into Explore feed and Trending Widget.	Deliver a feature in the Explore feed and related widget to base "trending content" information on edit rates, rather than previous days page view counts.	Reading Services	EOQ	To do
Multimedia	Support more expressive media formats, namely 3D models (the 11th most requested item in the 2015 wishlist).	Evaluate and address user experience improvements for 3D model support via the beta cluster, and coordinate release to production with the Commons community.	TechOps, Services, Community Liaisons, (optional) Communications	EOQ	To do

Community Tech

Goal setting process owner: Danny Horn

Team focused on meeting the needs of active Wikimedia contributors for improved, expert-focused curation and moderation tools.

Team	Objective	Key result	Dependency	ETA	Status
Anti-Harassment Tools	Improve community culture and reduce incidence of harassment	Hire and onboard all five new team members, equipped with the knowledge and initial experience to perform efficiently in FY17-18	Recruiting	EOQ	Done
	Improve the quality of existing admin and staff tools	Research, reach agreement on, and begin implementing performance and anti-spoof improvements to AbuseFilter Implement requested changes for SuSa's Trust & Safety tools	...	EOQ	In progress
	Partner with community on identifying and prioritizing needed improvements	Produce report of analysis and design research on the User Interaction History feature, based on Wikipedia community input Conduct early stakeholder meetings with admins, stewards and checkusers to gain input on AbuseFilter, ProcseeBot, and CheckUser improvements	Design Research Support and Safety	EOQ	In progress
Community Wishlist	Improve core contributors' productivity	Rewrite a stable version of XTools, a popular suite of volunteer-created curation and moderation tools		EOQ	In progress
	Simplify metrics reporting for grant recipients	Create a Grant metrics tool that automatically counts and updates important metrics	* Community Resources	EOQ	To do
	Improve editing experience for active editors	Add syntax highlighting to the classic wikitext editor	* Editing team	EOQ	To do

Wikidata

Goal Setting and process owner is Lydia Pintscher.

Wikidata is Wikimedia's structured data project and knowledge base. Our goal is to support all the other Wikimedia projects with structured data and provide a top-notch knowledge base to everyone who needs structured data about the world.

Wikidata adapt their programs for WMDE’s new annual plan in 2017. These goals are:

• • Goal 1: Continue to assure the technical and social sustainability of Wikidata (software and community) as the central, structured knowledge data base for the Wikimedia movement and beyond. (Sustainability of Wikidata)
  • Goal 2: Continue to increase the reach of Wikidata into the Wikimedia projects. (Wikidata for Commons, Wiktionary, Wikipedia)

Here you can find all objectives for next quarter.

Objective	Key result	Dependency	ETA	Status
Sustainability of Wikidata Improve the usability of Wikidata.	Researched migration to OOUI framework to be more in line with the rest of the Wikimedia projects and improved our input widgets (e.g. preview in the CommonsMedia suggester). Improved the keyboard navigability of the user interface. Created new datatype for linking to geoshapes on Wikimedia Commons. Adapted to Wikimedia’s color palette. Implemented an API for constraints checks.		EOQ	In progress
Sustainability of Wikidata Improve the documentation around Wikidata.	Organized an event to support the community in order to improve documentation and help pages for regular users at the upcoming Wikimedia hackathon in Vienna.	Community	EOQ	In progress
Sustainability of Wikidata Keep the community involved, happy and connected.	Start organizing WikidataCon, an event build for and by the Wikidata community for Wikidata’s 5th birthday.	Community	EOQ	In progress
Wikidata for Commons Provide structured data support for multimedia files to Wikimedia Commons.	Federation test system (will be published in early April)	WMF engineering and database expertise.	EOQ	In progress
Wikidata for Wiktionary Support for storing lexicographical data in order to support Wiktionary.	Worked on the Lexeme prototype in order to let the community try it out. Prepared roll-out of support for automated sitelinks to Wiktionary. Published the technical data model specification. Did user research with Wiktionary editors.		EOQ	In progress
Allow editing of Wikidata’s data directly from Wikipedia Continue to increase the reach of Wikidata into the Wikimedia projects.	Iterated on click-dummy based on community feedback.		EOQ	In progress
Make ArticlePlaceholder pages discoverable by search engines in order to help small Wikipedias grow their reach Continue to increase the reach of Wikidata into the Wikimedia projects.	Did a successful trial run on Welsh Wikipedia.	Operations team.	EOQ	In progress
Automated creation/update of list articles based on queries to Wikidata Continue to increase the reach of Wikidata into the Wikimedia projects.	Experimented with different ways to make it easier to write and edit queries - specifically query templates. Improved the documentation.		EOQ	In progress

Technology

Analytics Engineering

Goal setting process owner: Nuria Ruiz

The Analytics Engineering makes Wikimedia related data available for querying and analysis to both WMF and the different Wiki communities and stakeholders. We develop infrastructure so all our users, both within the Foundation and within the different communities, can access data in a self-service fashion that is consistent with the values of the movement.

Objective	Key result	Tech Goal	Dependency	ETA	Status
Improve tools for data display for the Foundation and community. Better tools to access data. Wikistats 2.0	Initial Launch of Wikistats 2.0 Website which will be a replacement of http://stats.wikimedia.org task T160370 . Work includes user interface but also development of a new backend on top of the mediawiki edit reconstruction dataset that exists on hive. task T156384	A. Org Level Priority. B. Serving our Audiences.	Cloud Services		N Not done
Improve data computation and release.	Global Unique Devices Counts (Preliminary vetting) task T143928 . Metric for the number of devices that access *.wikipedia.org. Helps to quantify our global user base in terms of users. Please see: https://meta.wikimedia.org/wiki/Research:Unique_Devices	A. Org Level Priority. B. Serving our Audiences.	Cloud Services		Done
Improve data computing and release Real Time	Prepare to deprecate most python RC feeds (July 7th) and help clients migrate. Better support for consumers of real time edit data, less code on server side task T156919	B. Serving our Audiences.	Ops		Done
Operational Excellence	Set of tasks needed to keep lights up and update/upgrade existing systems to maintain and improve current level of service. In order of priority. Improve purging and replication on eventlogging MySQL datastores task T124307, task T161855 To do piwik infrastructure refactoring and security upgrades task T159136  Done stat1002/3 replacement task T152712 To do Reimage cluster to debian task T157807  Done Consolidation of JVM GC default settings (mostly on the Hadoop cluster) task T159219  Done Preparation work for upgrading kafka and enabling security features in Q1 task T161637, task T161636  Done Improve the Analytics VLAN firewall rules and establish a maintainable upgrade process (could be even decide to drop the rules?) task T157435 To do	C and D. Tech Debt and improve our own feature set

Performance

Speed is Wikipedia's killer feature. ("Wiki" means "quick" in Hawaiian.) As the Wikimedia Foundation’s Performance team, we want to create value for readers and editors by making it possible to retrieve and render content at the speed of thought, from anywhere in the world, on the broadest range of devices and connection profiles.

Goal-setting process owner: Gilles Dubuc

Objective	Key result	Quadrant
Outreach	Write blog post about Thumbor migration - T159940 Write blog post about quarterly/bi-annually performance metrics changes - T110125 Improve performance.wikimedia.org - T161588	B: Customer goal
Monitoring	Set up alerts for NavigationTiming - T153169 Set up alerts for Save timing - T153170	C: Feature
Improvement	Implement Thumbor thumbnail width request limit - T150741 Serve all production traffic with Thumbor - T121388 Continue critical rendering path optimisations - T127328 Complete migration to jQuery 3 - T124742 Evaluate Dynomite on beta - T156938	D: Tech debt
Knowledge	Research isolated browser testing environment and proxy - T153360	C: Feature

Release Engineering

Goal setting process owner: Greg Grossmeier

All tracked in: #releng-201617-q4 - More details at Wikimedia Release Engineering Team/Goals/201617Q4

Objective	Key result	Dependency	Team members	ETA	Tech Goal Quad.	Status
Streamlined Services Delivery	Build a runnable image from tested and confirmed code that is suitable for promotion. define 'tested and confirmed' / determine the entry-point for building a container generate or consume a docker file for building build image, upload and  tag it in docker registry	Ops: Docker registry k8s staging cluster to run images  built via the above/our goal	Antoine Dan Tyler (Lead)	EOQ		Done
Train 2.0	Move MW+Extension deploys to scap3 - task T114313 (part 3) Integrate scap with etcd/pybal to automatically depool and repool servers - task T104352	Ops (maybe)	Chad Mukunda (Lead)	EOQ		N Not done
Release MediaWiki 1.29	A quality MediaWiki 1.29 is successfully released - task T153271	Wikimedia / MediaWiki dev community	Chad Tyler (lead)	End of May		N Not done

Research

In Wikimedia Research we use qualitative and quantitative methods to provide strategic insights and technological solutions to the movement and the Foundation, to foster innovation and to inform the development of new products.

Research & Data

Goal setting process owner: Dario Taraborelli [ Wikimedia Research goals overview ]

Objective	Key result	Dependency	ETA	Status
FOCUS AI as a service	Deploy a Wikidata item quality model in ORES. (T127470) Complete a research study to characterize and model article importance. (T155541) Conduct research to characterize the value of statements in Wikidata. (T161165) Design machine-learning methods to extract and analyze citations and their context in Wikipedia. (T161164)	Contractors/interns Formal collaborators	EOQ	Done
STRENGHTEN Research discoverability	Develop all contents for a landing page to improve the discoverability of Wikimedia Research. (T161560)	Contractors/interns Design Research	EOQ	In progress
STRENGTHEN Annual workshops and outreach	Host the annual Wiki Research Workshop at WWW '17 (Perth, April 2016) Host the WikiCite '17 event (Vienna, May 2016). (T144291)	Co-organizers Conference hosts	EOQ	Done

Design Research

Goal setting process owner: Abbey Ripstra [ Wikimedia Research goals overview ]

Objective	Key result	Dependency	ETA	Status
FOCUS Generative research	Structured Data for Commons Interviews with GLAM representatives to identify user requirements for new batch upload features on Wikimedia Commons New editor experience Analysis of prior research to bring forward what we know and don't know about new editor retention. Design and implementation of contextual inquiry in South Korea and Czech Republic to learn more about new editor retention in mid sized wikis. Author and share findings from the research Movement Strategy: Consultation and ad hoc support for contextual inquiries in Brazil and Indonesia to inform Movement Strategy, track D.	Structured Data Program Editing team and Reboot Movement Strategy Track D team and Reboot	EOQ	In progress
FOCUS Evaluative research	User evaluation of Recommender systems Identify a set of recommendation-driven features to evaluate in FY17/18, define evaluation goals, methods, and timelines New readers Move offline concepts toward usability and usefulness for new readers, consulting with Mobile Web team as needed for evaluative research. Contribute design research perspectives and collaboration with Awareness and Affordability tracks as needed.	Research & Data Scoring Platform (ORES) New Readers team	EOQ	In progress
STRENGTHEN Research communication	perform stakeholder interviews and draft key content for research landing page	Technical writer (contractor)	EOQ	In progress

Security

Goal setting process owner: Darian Partick

Objective	Key result	Dependency	Team members	ETA	Status
Update automated scanning infrastructure	Document use of existing tools Develop standard for review/update of rulesets and tool versions Cross-train staff on tool use	TBD	Darian Brian Sam	EOQ	To do

Services

Goal setting process owner: Gabriel Wicke

Objective	Key result	Dependency	ETA	Status
Modernize and speed up the storage backing the REST API.	Finish implementation of new, more efficient storage schema for recent revisions. Test Cassandra 3, and determine upgrade strategy. [Stretch] Start gradual migration, to be finished in Q1 2017/18.		EOQ	In progess
Establish requirements and identify issues for the container development environment and pipeline.	Prototype port of mediawiki-containers to the Minikube environment, focusing on the service development use case. Document pain points encountered on the way to guide the planning process for next year’s Technology-wide goal.		EOQ	In progress

Technical Operations

Team	Objective	Key result	Dependency	ETA	Status
Core Ops	Perform a datacenter switchover	Successfully switch backend traffic (MediaWiki, Swift, ElasticSearch, RESTBase, and Parsoid) to be served from codfw with no downtime and reduced read-only time Serve the site from codfw for a longer period of time than the last switchover (at least 2 weeks)	MediaWiki Performance Release Engineering Services (RESTbase) Discovery (Search) Parsing (Parsoid) Analytics (EventBus) Community Liaisons	May 5th	Done
	Prepare to serve applications from Kubernetes	Eliminate SPOFs in the existing eqiad infrastructure Expand the infrastructure to codfw Prepare and maintain base container images Define a process to keep images up-to-date on similar standards as the rest of production Design and implement a Kubernetes-based staging environment. (stretch)	Services Release Engineering Purchasing	EOQ	Done
	Reduce Swift technical debt	Support diversified storage policies such as different levels of replication and low-latency containers Upgrade the Swift infrastructure to run using Debian jessie (or later) Allow rate-limited container operations for better isolation among Swift users and use cases Delete non-used/non-requested thumbnail sizes periodically to reclaim space		EOQ	Done
Traffic	Make progress on Asia Cache DC deployment	Finalize and sign contract with the datacenter vendor Finalize system specifications and purchase the necessary hardware Prepare and deploy configuration changes needed for the new systems Allocate the datacenter's IPv4/IPv6 address space	Finance Legal	EOQ	In progress

Cloud Services

Goal setting process owner: Bryan Davis

Objective	Key result	Dependency	ETA	Status
Prototype Neutron network configuration that will eventually be applied to the main OpenStack cluster	Working Neutron networking in labtest cluster		EOQ	Continues
Make a strong technical hire to replace capacity lost in team due to staff attrition	Fully staffed team	Recruiting	EOQ	N Continues
Replace all Wikitech workflows and reporting which rely on Semantic MediaWiki features	Uninstall SMW extension from Wikitech		EOQ	Done