Wikimedia Engineering/2016-17 Q3 Goals

Q2

Wikimedia Engineering Goals, FY2016-17, Q3 (January - March)

Q4

Purpose of this document: Goals for the Wikimedia Technology and Product Development department, for the second quarter of fiscal year 2016–17 (January 2017 – March 2017). The goal setting process owner in each section is the person responsible for coordinating completion of the section, in partnership with the team and relevant stakeholders.

Status fields can use the following templates: In progress, To do, N Postponed,  Done or Partially done

ETA fields may use the initialism EOQ (End of Quarter).

Product

The Product department, alongside the community, builds, improves and maintains the user facing features on all the Wikimedia projects. The software, interfaces, apps, and other systems and applications the team supports are essential means by which contributors and readers connect with our sites. The work of the team’s three “audience” groupings – Discovery, Editing, and Reading – is interrelated and enables a channel for progressive engagement with the users of the Wikimedia projects.

• Annual Plan for Product 2016-2017

Discovery

Our mission is to make the wealth of knowledge and content in the Wikimedia projects easily discoverable.

Goal setting process owner: Dan Garry

Team	Objective	Key result	Dependency	ETA	Status
Search	Add real-time index updates to completion suggester and improve overall stability and performance of search by upgrading to ElasticSearch 5	Upgrade search cluster to ElasticSearch 5 Add real-time index updates to completion suggester	Community Engagement, Team Practices Group, Analytics	EOQ	Partially done - upgrade completed, real time updates in progress
	Improve support for different languages by researching and deploying new language analyzers	Perform research spikes to find new analyzers for different languages Test new analyzers to see if they are improvements Deploy new analyzers that are found to be an improvement	Community Engagement, Team Practices Group, Analytics	EOQ	In progress
	Prototype and test overhaul UI of search page to add cross-wiki searching	Have a working prototype in Labs Run at least 2 A/B tests on Persian, Italian, Catalan and Polish Wikipedias	Feedback from community Wikipedias: Persian, Polish, Italian and Catalan	EOQ	Done
Interactive	Stabilize maps and graphs code base	Stabilize and fix high priority bugs Note: this goal was added to the goals list mid-quarter	Community Engagement	TBD	In progress

Wikidata Query Service does not have any dedicated goals this quarter. The service will continue to be maintained by Stas and Guillaume, and Stas will work on user feature requests as time permits. Additionally, the dedicated hardware for the service will be upgraded to support the continued growth and use of the service.

The Analysis team (Mikhail and Chelsy) is in the final stage of adopting Analytics' ReportUpdater infrastructure. As part of that migration, they are starting to forecast usage of Discovery's services and products to detect when services and products might be significantly under- or over-utilized (relative to the usage our models predicted).

Wikipedia.org Portal team will not have any dedicated goals this quarter other than continuing to maintain the page by fixing critical bugs and performing regular statistics and translations updates. No other work on the page is planned in Q3.

Editing

Goal setting process owner: User:Trevor Parscal (WMF)

The Editing Department build collaborative, inclusive tools for creating and editing free knowledge.

Also see the more detailed progress on these goals which are based on Program 4 of the Annual Plan

Annual Goal	Team	Objective	Output (deliverable)	Outcome	Dependencies	ETA	Status
1: Maintain and incrementally improve current content creation and curation interfaces	VisualEditor	Visual diffs for the visual editor	Provide visual diffs for all users inside the visual editor.	Editors can see the changes they and others have made without needing to know the intricacies of wikitext.	Editing Design Community Liaisons	EOQ	Done, with more features to come
2: Improve editing experience by reducing technical and product debt	Parsing	Address technical debt in wiki content	Parsoid has a linter that can identify errors in wikitext. We are going to deploy a Linter extension to the Wikimedia cluster that collects these errors and surfaces them to editors.	Editors will be able to identify markup errors and use their existing processes to fix them. But, the Linter extension itself has client-side JS to highlight the section of wikitext with the error to make it easier for editors to fix.	Editing Design	EOQ	Partially done, deployed on small wikis
	Multimedia	3D model file support	Pick up the work on 3D file support by getting the 3d2png service and 3D extension into production, and working with Reading on MediaViewer integration.	Editors can upload and transclude 3D files on articles, which readers can view statically on-page or interactively in MediaViewer/etc..	Security Performance Services Reading Community Liaisons	EOQ	Partially done, work carried over
3: Invest in new types of content creation, and new forms of curation and collaboration tools	Collaboration	Provide an edit-review process suited to the needs of new contributors—while improving efficiency for edit reviewers in general.	Add ORES tests, optimized ORES data display, a new filtering UI and a “New User” filter to the recent changes page (as a beta). Release the ReviewStream machine-readable feed incorporating ORES and other useful edit-review data.	More powerful edit filtering generally and, in particular, the ability to identify damaging but good-faith edits by new users. Faster performance for downstream review tools and the ability to more easily incorporate newcomer-friendly features.	Analytics Services Research & Data Community Liaisons	EOQ	Partially done, deployed on some wikis
	Language	Stabilize ContentTranslation for going out of Beta	Replace the contenteditable editing component of Content Translation to VisualEditor and preserve the current CX functionalities.	Content Translation is more stable in terms of article publishing, requiring less maintenance work from patrollers, and more ready to be enabled by default for all users.	VisualEditor Parsing Editing Design (for OOjs UI)	EOQ	Partially done, work carried over

Fundraising Tech

Goal setting process owner: User:Khorn_(WMF)

Objective	Key result	Dependency	ETA	Status
Paypal Express Checkout integration	possibly campaign ready	Paypal	EOQ	Partially done, work carried over
Start Ingenico reintegration	Begin dev work	ingenico	EOQ	Done
Central Notice Banner Sequence feature	ship some version of this	To be added	EOQ	Partially done, work carried over

Reading

Team	Objective	Key result	Dependency	ETA	Status
Reading Infrastructure	Empower editors to create mobile-friendly templates by allowing <style> tags to be attached to templates.	The TemplateStyles extension is deployed in production and there are no unaddressed major community concerns.	Security, Editing, Community Liaisons, Performance, Traffic	EOQ	To do
Reading Infrastructure	Determine next steps for the ORES platform technology and its support.	A report (or reports) assessing the software architecture and software quality of the ORES platform (service, extension, and so forth), with recommendations covering areas such as code, configuration management, hosting, data persistence, security & privacy, testability, and a plan for viable human and automated support of the platform and its future intended use.	Research & Data, probably Editing	EOQ	To do
Reading Web	Provide optimal printed content	Update print styles for mobile and desktop web and deploy on all projects in preparation for using browser-based pdf rendering	New Readers (for mobile only), community liasons	EOQ	To do
	Assess the possibility of providing an offline-capable app and unifying content through various services	Create a written report planning the technical and tactical implementation of building and releasing an offline-capable app that unifies content consumption under reading services	Services	EOQ	To do
iOS App	Improve the relevancy and engagement with top read and notifications features	Integrate and evaluate trending edits API for notifications and Explore feed	Services	EOQ	To do
Android app	Improve user confidence and set a solid stage for future feature development.	Make significant improvements to offline functionality (reading lists) and offline navigation.	None	EOQ	To do
New Readers	Hand off online to offline prototypes to the web team for moving to production.	A report which summarizes research (remote and on the ground) and community feedback. Requirements defined and documented in partnership with Web team.	Design research (inc. vendor), Reading web	EOQ	To do

Community Tech

Goal setting process owner: Danny Horn

Team focused on meeting the needs of active Wikimedia contributors for improved, expert-focused curation and moderation tools.

Objective	Key result	Dependency	ETA	Status
Partner with community on identifying and prioritizing needed improvements	Report back to community on analysis of top 10 wishes from Community Wishlist Survey Host on-wiki discussions on refining the definitions of reported problems, validating proposed solutions		EOQ	Done
Improve core contributors' productivity	Build a prototype of the cross-wiki watchlist feature	Performance Operations	EOQ	N Not done
Improve core contributors' productivity	Complete development and ship features and fixes related to two wishes in the Wishlist Survey top 10		EOQ	To do
Improve workflows for Tool Labs developers	Extend Striker to allow developers to create a shared tools account and collect data to be used for right to fork policy and tool discovery.		EOQ	N Postponed
Improve workflows for Tool Labs developers	Define evaluation criteria for evaluating Platform as a Service (PaaS) solutions (carried over from Q2)	Labs team	EOQ	N Postponed
Encourage community maintenance of important tools	Help community bootstrap committee to enforce right to fork and abandoned tool policies.	Tool Labs Community	EOQ	Done
Improve community culture and reduce incidence of harassment	Hire and onboard new staff for the anti-harassment project		EOQ	To do

Team Practices Group

The Team Practices Group believes that healthy teams reliably deliver customer value and innovative products. Through dedicated resourcing, coaching, and workshops at the Wikimedia Foundation, we facilitate the use of practices that support team health, with a strong emphasis on creating conditions of clarity, trust, and continuous improvement.

Goal setting process owner: Kristen Lans

Workflow	Working with	Deliverables/Impact

Wikidata

Goal Setting and process owner is Lydia Pintscher.

Wikidata is Wikimedia's structured data project and knowledge base. Our goal is to support all the other Wikimedia projects with structured data and provide a top-notch knowledge base to everyone who needs structured data about the world.

Wikidata adapt their programms for WMDE’s new annual plan in 2017. These goals are:

• • Goal 1: Continue to assure the technical and social sustainability of Wikidata (software and community) as the central, structured knowledge data base for the Wikimedia movement and beyond. (Sustainability of Wikidata)
  • Goal 2: Continue to increase the reach of Wikidata into the Wikimedia projects. (Wikidata for Commons, Wiktionary, Wikipedia)

Here you can find all objectives for next quarter.

Objective	Key result	Dependency	ETA	Status
Sustainability of Wikidata Improve the usability of Wikidata.	Research migration to OOUI framework to be more in line with the rest of the Wikimedia projects and at the same time improve our input widgets. Improve the keyboard navigability of the user interface. Create new datatype for linking to geoshape and data pages on Wikimedia Commons.		EOQ	In progress
Sustainability of Wikidata Improve the documentation around Wikidata.	Organize an event to support the community in order to improve documentation and help pages for regular users.	Community	EOQ	In progress
Sustainability of Wikidata Keep the community involved, happy and connected.	Start organizing WikidataCon, an event build for and by the Wikidata community for Wikidata’s 5th birthday.	Community	EOQ	In progress
Wikidata for Commons Provide structured data support for multimedia files to Wikimedia Commons.	Publish Federation prototype in order to let the community try it out and work more towards Multi-Content-Revisions together with WMF.	WMF engineering and database expertise.	EOQ	In progress
Wikidata for Wiktionary Support for storing lexicographical data in order to support Wiktionary.	Publish the Lexeme prototype in order to let the community try it out. Roll out the support for automated sitelinks to Wiktionary. Publish the technical data model specification.	Community and WMF security team Input & Implementation supported by Discovery team	EOQ	In progress
Wikidata for Wikipedia Allow editing of Wikidata’s data directly from Wikipedia.	Publish next version of click-dummy to get a new round of feedback from the community.		EOQ	In progress
Wikidata for Wikipedia Make ArticlePlaceholder pages discoverable by search engines in order to help small Wikipedias grow their reach.	Trial run on at least one Wikipedia to test the impact.	Operations Team	EOQ	In progress
Wikidata for Wikipedia Make using Wikidata data on the other Wikimedia wikis easier.	Allow automatically adding authority control templates to articles via an empty system message at the end of content pages that editors can overwrite. Improve the statement parser function further (if there is time).		EOQ	In progress
Wikidata for Wikipedia Automated creation/update of list articles based on queries to Wikidata.	Experimenting with different ways to make it easier to write and edit queries (if there is time).		EOQ	In progress

Technology

Analytics

Goal setting process owner: Nuria Ruiz

Objective	Key result	Tech Goal	Dependency	ETA	Status
Improve tools for data display for the Foundation and community. Better tools to access data. Wikistats 2.0	Creation of Visual language for replacement of http://stats.wikimedia.org task T152033 . Mocks and Wireframes for UI and Information Architecture of new site when it comes to edit data.	A. Org Level Priority. B. Serving our Audiences.	Design - Community Liaisons, in order to engage with community and be able to provide early feedback.		Done Visuals can be seen here: task T157827
Improve tools for data display for the Foundation and community. Better tools to access data. Wikistats 2.0	Productionize mediawiki edit reconstruction pipeline to enable calculation of edit metrics for community and WMF for all wikis since the start of the projects task T152035	A. Org Level Priority. B. Serving our Audiences.			Done
Improve data computing and release. Real Time. PublicEventStreams Launch task T13065	Start of deprecation of RCStream with the result of less code and less resources used to publish an edit updates feed for the community	B. Serving our Audiences.	Ops		Done
Improve data computing and release. A/B testing framework design	Design Document for a more solid AB testing infrastructure. Carry along from last quarter task T135762	B. Serving our Audiences.	Ops, Research		Done Design document available in meta
Operational Excellence	Set of tasks needed to keep lights up and update/upgrade existing systems to maintain and improve current level of service. In order of priority. Install new Pageview API nodes (carry-on as we have not provisioned hardware last quarter): task T144833 Expand Cluster with 8 new Hadoop Nodes task T152713 Kafka Doc Revamp: task T150277 Deprecate old reportcard as part of removing old dashboard infrastructure from labs task T146308 CDH upgrade. Value proposition: new spark for edit reconstruction task T152714 Alarm on heap utilization task T88640	C and D. Tech Debt and improve our own feature set			To do

Performance

Speed is Wikipedia's killer feature. ("Wiki" means "quick" in Hawaiian.) As the Wikimedia Foundation’s Performance team, we want to create value for readers and editors by making it possible to retrieve and render content at the speed of thought, from anywhere in the world, on the broadest range of devices and connection profiles.

The following is a summary of the Performance Team Goals.

Objective	Key result	Quadrant
Outreach	Assist Tech Ops with second datacenter switchover. Write blog post about "hidden tabs incident" - T153048 Continue preparation work for running Active-Active datacenters. - T88445	B: Customer goal
Monitoring	Set up monitoring of performance metrics using Grafana 4 alerts and notifications - T153166	C: Feature
Improvement	Implement Thumbor rate-limiting - T151067 T151065 T151066 Continue critical rendering path optimisations. - T127328	D: Tech debt
Knowledge	Research isolated browser testing environment and proxy T153360	C: Feature

Release Engineering

Goal setting process owner: Greg Grossmeier

All tracked in: #releng-201617-q3 - More details at Wikimedia Release Engineering Team/Goals/201617Q3

See also Wikimedia Release Engineering Team/Project/Planning for more items that are not quarterly goals.

Objective	Key result	Dependency	Team members	ETA	Status	Tech Goal Quad.
Train 2.0	Move MW+Extension deploys to scap3 - task T114313 (part 2) Build fanout support in scap3 - task T121276 Assess impact of moving to `scap deploy` for all parts	Operations	Chad (Lead) Tyler Mukunda Dan	EOQ	Done	C: Tech feature A: Annual goal
Continuous Integration Infrastructure modernization (i.e., Deployment Pipeline)	Gather requirements for new CI system - task T153270	WMF Labs Operations	Tyler (Lead) Antoine Dan	EOQ	N Not done	B: Customer goal C: Tech feature
Phabricator Improvements	Re-evaluate and improve search functionality in Phabricator - task T146843	(Minimal) Discovery	Mukunda (Lead) Chad	EOQ	Done	D: Tech Debt

Research

In Wikimedia Research we use qualitative and quantitative methods to provide strategic insights and technological solutions to the movement and the Foundation, to foster innovation and to inform the development of new products.

Research & Data

Goal setting process owner: Dario Taraborelli [ Wikimedia Research goals overview ]

Objective	Key result	Dependency	ETA	Status
FOCUS Revscoring	A report (or reports) assessing the software architecture and software quality of the ORES platform (service, extension, and so forth). Complete a paper draft on ORES socio-technical AI	Reading Infrastructure	EOQ	In progress
FOCUS Stub recommendations	Design a model predicting articles that need to be expanded in English Wikipedia and expand to other languages Prototype recommender design	Formal collaborators (EPFL)	EOQ	In progress
EXPERIMENT Recommender system evaluation	Develop and pilot a methodology for comparative evaluation of search/recommendation algorithms	Reading Design Research	EOQ	In progress
STRENGTHEN Annual workshops and outreach	Host a session on "Artificial Intelligence to build and navigate content"" at the Wikimedia Dev Summit 2017 Organize the annual Wiki Research Workshop at WWW '17 Organize the WikiCite '17 event	Co-organizers Conference hosts	EOQ	In progress

Design Research

Goal setting process owner: Abbey Ripstra [ Wikimedia Research goals overview ]

Objective	Key result	Dependency	ETA	Status
FOCUS Generative research	New Readers next steps Evaluative research in country for offline concepts Concept generation and evaluation of affordability findings Consultations with Comms on awareness findings Audience project Collaborate on next steps	New Readers teams	EOQ	Done
STRENGTHEN Participant outreach	Add 1,000 new participants to the Design Research database Leverage WMF social media accounts Start a physical campaign	Communications Design researchers (Product)	EOQ	In progress
FOCUS Understanding editing culture	Document content curation workflows on Wikimedia Commons to inform development micro-contribution features T152248 Perform and publish analysis of the quality of new editor support provided at Wikipedia Teahouse T132809 Perform and publish expanded analysis of the impact of Teahouse on new editor retention T127523	Research and Data	EOQ	In progress
EXPERIMENT Recommender Evaluation	Develop and pilot a methodology for comparative evaluation of search/recommendation algorithms RelatedPages study plan T142009 Dev Summit proposal T149373	Research and Data Reading	EOQ	In progress

Security

Goal setting process owner: Darian Patrick

Objective	Key result	Dependency	Team members	ETA	Status
Improve wiki account authentication	Provide insight into active sessions Improve security messages to users (login, reset, etc.) Reduce possibility for account takeover	TBD	Darian Brian Sam	EOQ	To do

Services

Goal setting process owner: Gabriel Wicke

Objective	Key result	Dependency	ETA	Status
REST API 1.0 (B/D)	Graduate the REST API 1.0 from beta to stable production. Stabilize individual end points, improve documentation, and write a blog post announcement.		EOQ	Done
Container infrastructure (B/C)	Establish requirements for a container-based development, continuous integration, production, and third party install infrastructure. Identify issues, and investigate potential solutions.	Operations, Release Engineering	EOQ	Done

We also plan to spend significant time on:

• Change propagation and EventBus infrastructure expansion.
• RESTBase storage scaling.

Technical Operations

Team	Objective	Key result	Dependency	ETA	Status
Core Ops	Data center switchover (prep)	Audit & prepare MediaWiki application servers, Swift, ElasticSearch, RESTBase, EventBus and Parsoid for switchover to codfw Partially orchestrate the datacenter switchover, reducing the number of steps of individual merges and manual commands required Improve and implement MediaWiki cache warmup procedures from current ad-hoc method	Performance & Availability Release Engineering (Scap, MediaWiki configuration) Services (RESTbase) Discovery (Search) Parsing (Parsoid) Analytics (EventBus) Community Liaisons	EOQ	Done
	Security: Enhance and unify system user account handling	Enhance account handling to cross-validate data sources data.yaml and LDAP Simplify account/group membership handling Review & audit existing accounts and groups		EOQ	Done
	Stretch: Kubernetes	Experiment with at least one service running on Kubernetes in production		EOQ	Done
	Migrate all production systems away from Precise	Upgrade/migrate all remaining Precise production hosts to Trusty or later		April 24th 2017 (Precise End Of Support)	Done
Labs	Deprecate Precise	Precise based services in Tool Labs are discontinued Precise instances are purged from all projects		EOQ	Done
	Investigate Neutron migration path	Experiment: Initial OpenStack Neutron PoC deployment in Labtest		EOQ	In progress
Traffic	Site selection for Asia Cache DC	Consult formally with Legal on location (country) Evaluate vendor/building options in top locations Vendor review of top choices with Procurement Final selection of City, Vendor, Building for Asia Cache DC	Legal Finance (Procurement) DC Ops	EOQ	Done