Wikimedia Technology/Goals/2019-20 Q3

Q2

Wikimedia Technology Goals, FY2019–20, Q3 (January - March 2020)

Q4

Technology Department Team Goals and Status for Q3 FY19/20 in support of the Medium Term Plan (MTP) Priorities and Annual Plan for FY19/20

Analytics

Team Manager: Nuria Ruiz

Modern Event Platform

Build a reliable, scalable, and comprehensive platform for creating services, tools and user facing features that produce and consume event data

Deploy a new event stream for analytics using the new Event Platform infrastructure task T241241 N Not done

Client side Error logging enabled for 1 wiki such errors from browser clients are surfaced to developers task T226986  Done

Public schema endpoint for event schemas task T233630  Done

Smart Tools for Better Data. Make easier to understand the history of all Wikimedia projects

Design (together with core platform team) an alternative architecture for historic data endpoints used by iOS application task T241184 In progress

Move stats.wikimedia.org domain to point to Wikistats2 by default task T237752  Done

Wikistats2 UI is localized task T186122  Done

Project Newpyter: First Class Jupyter Notebook system. Deliver Project plan task T241186  Done

Enable Presto access for shell users, much faster way to query hadoop task T243312  Done

Smart Tools for Better Data. Increase Data Quality, Privacy and Security

Bots: Label high volume bot spikes in pageview data as automated traffic task T238357 In progress

Core. Operational Excellence. Increase Resilience of Systems

Deploy Eventstreams with Kubernetes task T238658  Done

Create a MySQL replica for backups for all MySQL instances we use MySQL on, like Oozie or Superset task T234826

Core. Operational Excellence. Reduce Operational Load by Phasing Out Legacy Systems/Technologies

Reimage at least one Hadoop worker to Debian Buster (to unlock the upgrade) - task T231067  Done

Spark 2.4 encryption working in Hadoop task T240934  Done

Refresh 1004 with new host and GPU task T241187  Done

Airflow as an easier job scheduling alternative, PoC for refine workflow task T241246 In progress

Status

• January 21, 2020 status - updated above
• February 28, 2020 status:

In progress

• • Project newpiter. Spike about rework of current jupyter network environment
  • Client side Error logging enabled for 1 wiki such errors from browser clients are surfaced to developers
  • Design (together with core platform team) an alternative architecture for historic data endpoints used by iOS application
  • Wikistats2 UI is localized
  • Deploy Eventstreams with Kubernetes

 Done

• • Deploy Public schema endpoint for event schemas
  • Move stats.wikimedia.org domain to point to Wikistats2
  • Enable Presto access for shell users, much faster way to query hadoop
  • Spark 2.4 encryption working in Hadoop
• March 31, 2020 status:

In progress

• • Project newpiter. Spike about rework of current jupyter network environment
  • Replacing oozie. PoC of Airflow for refine workflow
  • Design (together with core platform team) an alternative architecture for historic data endpoints used by iOS application

 Done

• • Deploy Public schema endpoint for event schemas
  • Client side Error logging enabled for 1 wiki such errors from browser clients are surfaced to developers
  • Move stats.wikimedia.org domain to point to Wikistats2
  • Wikistats2 UI is localized
  • Enable Presto access for shell users, much faster way to query hadoop
  • Deploy Eventstreams with Kubernetes
  • Create a MySQL replica for backups for all MySQL instances we use MySQL on, like Oozie or Superse
  • Reimage at least one Hadoop worker to Debian Buster (to unlock the upgrade)
  • Spark 2.4 encryption working in Hadoop
  • Refresh 1004 with new host and GPU

Core Platform

Team Manager: Corey Floyd

Core Work

Enable MultiDC Reads

IP Masking

IP Masking Implementation

Platform Evolution / Modern Event Platform

Initial modularization of MediaWiki (planning) (continued from Q2)

Push notification service

Migrate Service - changeprop (continued from Q2)

Initial modularization of MediaWiki (one component)

FAWG (desktop refresh work)

API Gateway

Tech & Product Partnerships

Developer Portal Implementation

Integrate OAuth 2.0 into API, API Keys, rate limiting

Paid API project

Safe and Secure Spaces

Image Hash Checking (in beta)

Dependencies on:

Status

• January 24, 2020 status
  • In review:
    • Migrate Service - changeprop
  • N Blocked
    • Enable MultiDC Reads
  • In progress
    • Developer Portal Implementation
  • N Postponed
    • IP Masking Implementation
• February 28, 2020 status:

In progress

• • Developer Portal Implementation
  • Integrate OAuth 2.0 into API, API Keys, rate limiting
  • Image Hash Checking (in beta)
  • FAWG (desktop refresh work)
  • API Gateway

N Postponed

• • IP Masking Implementation
  • Enable MultiDC Reads
• March 31, 2020 status:

In progress

• • API Portal Implementation
  • Integrate OAuth 2.0 into API, API Keys, rate limiting
  • FAWG (desktop refresh work)
  • API Gateway (routing layer)

 Done

• • Image Hash Checking (in beta)

N Blocked

• • IP Masking Implementation
  • Enable MultiDC Reads
  • Initial modularization of MediaWiki (one component)

Engineering Productivity Team

Performance

Team Manager: Gilles Dubuc

Core Work

Figure out the right store to use for the main stash (continued from Q1)

Publish 8 blog posts about performance (continued from Q1)

Support and maintenance of MediaWiki's object caching and data access components (continued from Q1)

Support and maintenance of WebPageTest and synthetic testing infrastructure (continued from Q1)

Support and maintenance of MediaWiki's ResourceLoader (continued from Q1)

Support and maintenance of Fresnel (continued from Q1)

Add operational monitoring for 100% of the performance-team services

Have at least 2 years of retention for ArcLamp flame graphs

Organise and run the Web Performance devroom at FOSDEM 2020 (continued from Q2)

Make seen/unseen state of watched pages reliable (all affected users who reported the issue are satisfied)

DC-shared object caches are available and replicate even if 20% of the servers fail

DC-shared temporary data is written via queues or replicated stores (continued from Q2)

Shared caching and temporary data storage use established/documented patterns

A memory-only lightweight store for temporary data exists and supports global (cross-wiki) keys (continued from Q2)

A disk-backed lightweight store for temporary data exists and supports global (cross-wiki) keys (continued from Q2)

Document how to add your own User Timing and how to see it in RUM and synthetic testing

Collect and graph First Input Delay

Document how to add your own Element Timing and how to see it in synthetic testing

Document how to add your own user journey for synthetic testing

Document the search user journeys in synthetic testing

Platform Evolution / Modern Event Platform

Provide performance expertise to FAWG outcome (continued from Q1)

Quality and Test Engineering

Team Manager: JR Branaa

Core Work

Team inception, formalization, and assessment of current organizational practices (continued from Q1)

Add all deployed to production repos to the Code Health pipeline (Code Health Metrics).

Solicit feedback from current users of CHM POC and define phase 2 enhancements (continued from Q2)

Interview engineering teams to understand their current code review practices (continued from Q2)

Relaunch the Code Review Office Hours (continued from Q2)

Put in place Code Review performance metrics (continued from Q2)

Develop Test Strategy for CPT

Platform Evolution / Modern Event Platform

Make CI warn about slow tests, and publish a collated list of slow tests

Transfer maintainership/ownership of API Test Tooling from CPT

Release Engineering

Team Manager: Tyler Cipriani

Core Work

Set up an experimental elastic search instance to store and analyze CI logs and metrics

Continuation of Phabricator and Gerrit improvement (in conjunction with SRE) (continued from Q2)

Migrate from Gerrit version 2.15 to 2.16

Platform Evolution / Modern Event Platform

Other service deployment pipeline migrations as prioritized between SRE/RelEng and relevant teams (continued from Q2)

A demonstration MediaWiki development environment hosts the full TimedMediaHandler front-end and back-end workflow

Dependencies on:

Status

• January 24, 2020 status
  • In progress
    • Reduce weekly rate of RDBMs commit/shutdown errors to less than 10
    • DC-shared temporary data is written via queues or replicated stores
    • A memory-only lightweight store for temporary data exists and supports global (cross-wiki) keys
    • A disk-backed lightweight store for temporary data exists and supports global (cross-wiki) keys
    • Migrate prod localisation cache to faster static-array distribution
    • Improve MediaWiki PHP startup time and recover from PHP7 regression
    • Add all deployed to production repos to the Code Health pipeline (Code Health Metrics)
    • Develop Test Strategy for CPT
    • Develop Test Strategy for Editing Team
    • Transfer maintainership/ownership of API Test Tooling from CPT
    • (Stretch): MobileContentService
    • Continuation of Phabricator and Gerrit improvement (in conjunction with SRE)
    • Other service deployment pipeline migrations as prioritized between SRE/RelEng and relevant teams.
    • Migrate from Gerrit version 2.15 to 2.16
    • Scap static PHP array for l10n cache
    • A demonstration MediaWiki development environment hosts the full TimedMediaHandler front-end and back-end workflow
    • Create Basic Local Development Environment for MediaWiki/Core
    • Create PipelineLib documentation to support deployment pipeline migrations
  • N Blocked
    • Reduce weekly rate of lost post-send updates to less than 10
• February 28, 2020 status:

In progress

• • Add operational monitoring for 100% of the performance-team services
  • Have at least 2 years of retention for ArcLamp flame graphs
  • Collect and graph First Input Delay
  • Migrate prod localisation cache to faster static-array distribution
  • Improve MediaWiki PHP startup time and recover from PHP7 regression
  • Performance review of the DiscussionTools extension
  • Organise and oversee org-wide frontend web performance training
  • Organise and oversee implementation of First Paint on Safari
  • Add all deployed to production repos to the Code Health pipeline (Code Health Metrics).
  • Transfer maintainership/ownership of API Test Tooling from CPT
  • Develop Test Strategy for CPT
  • Develop Test Strategy for Editing Team
  • A demonstration MediaWiki development environment hosts the full TimedMediaHandler front-end and back-end workflow
  • Migrate from Gerrit version 2.15 to 2.16
  • Migrate "integration" Cloud VPS project to stretch
  • Developer satisfaction survey

 Done

• • Document how to add your own User Timing and how to see it in RUM and synthetic testing
  • Document how to add your own Element Timing and how to see it in synthetic testing
  • Document how to add your own user journey for synthetic testing
  • Document the search user journeys in synthetic testing
  • Create Basic Local Development Environment for MediaWiki/Core
  • Create PipelineLib documentation to support deployment pipeline migrations
  • Scap static PHP array for l10n cache
• March 31, 2020 status:

In progress

• • Add operational monitoring for 100% of the performance-team services
  • Have at least 2 years of retention for ArcLamp flame graphs
  • DC-shared object caches are available and replicate even if 20% of the servers fail
  • Collect and graph First Input Delay
  • Improve MediaWiki PHP startup time and recover from PHP7 regression
  • Performance review of the KaiOS app
  • Performance review of the GrowthExperiments extension
  • Organise and oversee org-wide frontend web performance training
  • Organise and oversee implementation of First Paint on Safari
  • Propose and (if approved) help establish a stakeholder forum around TechCom
  • Add all deployed to production repos to the Code Health pipeline (Code Health Metrics).
  • Develop Test Strategy for CPT
  • Put in place Code Stewardship Review Committee
  • A demonstration MediaWiki development environment hosts the full TimedMediaHandler front-end and back-end workflow
  • Migrate zuul deployment from .deb to scap3
  • Migrate "integration" Cloud VPS project to stretch
  • Developer satisfaction survey

 Done

• • Document how to add your own User Timing and how to see it in RUM and synthetic testing
  • Document how to add your own Element Timing and how to see it in synthetic testing
  • Document how to add your own user journey for synthetic testing
  • Document the search user journeys in synthetic testing
  • Performance review of the DiscussionTools extension
  • Develop Test Strategy for Editing Team
  • Create Basic Local Development Environment for MediaWiki/Core
  • Create PipelineLib documentation to support deployment pipeline migrations
  • Scap static PHP array for l10n cache

N Blocked or N Postponed

• • Migrate prod localisation cache to faster static-array distribution
  • Shared caching and temporary data storage use established/documented patterns
  • Make seen/unseen state of watched pages reliable (all affected users who reported the issue are satisfied)
  • Transfer maintainership/ownership of API Test Tooling from CPT
  • Set up an experimental elastic search instance to store and analyze CI logs and metrics
  • Migrate from Gerrit version 2.15 to 2.16

Fundraising Tech

Team Manager: Erika Bjune

Core Work

Support Advancement in testing and planned Q3 campaigns

Make IDEAL payment processor campaign (support for Q4 campaigns)

Start Matching Gifts V2

Dependencies on:

Status

• January 24, 2020 status
  • In progress
    • Support Advancement in testing and planned Q3 campaigns
    • Make IDEAL payment processor campaign (support for Q4 campaigns)
    • Recover from the holidays
• February 28, 2020 status:

In progress

• • Support Advancement in testing and planned Q3 campaigns
  • Make IDEAL payment processor campaign (support for Q4 campaigns)
• March 31, 2020 status:

In progress

• • Make IDEAL payment processor campaign ready (support for Q4 campaigns)
  • Matching Gifts V2 (Fr-tech side)

 Done

• • Support Advancement in testing and planned Q3 campaigns
  • Recover from the holidays

Machine Learning / Scoring Platform

Team Manager: Aaron Halfaker

Content Integrity

JBuild/improve models in response to community demand (ongoing every quarter)

Machine Learning Infrastructure

Jade expansion/Iteration

Session-model use, maintenance, and user-research

Dependencies on:

Status

• January 24, 2020 status
  • In progress
    • Expansion of Topic Model to ar, ko, and cswiki
    • Support operations infrastructure improvements (prometheus)
    • Deploy Entity UI for Jade
    • Deployment of session-based models
    • Build/improve models in response to community demand (ongoing every quarter)
    • Improve topic models in based on initial use
• February 28, 2020 status:

In progress

• • Build/improve models in response to community demand (ongoing every quarter)

N Blocked

• • Improve topic models in based on initial use
• March 31, 2020 status:

In progress

• • Build/improve models in response to community demand (ongoing every quarter)
  • Improve topic models in based on initial use

Research

Team Manager: Leila Zia

Address Knowledge Gaps

Conduct a literature review, plan and set up collaborations for projects about understanding engagement with Wikimedia images around the world. (continued from Q2)

Build one formal collaborations in the disinformation space to start the research for building solutions starting Q3. (continued from Q2)

Dependencies on:

Status

• January 24, 2020 status
  • In progress
    • Finalize the research brief for crosslingual topical model laying out the work that will be done in this space starting Q3.
    • Build one formal collaborations in the disinformation space to start the research for building solutions starting Q3.
    • Plan for a challenge: come up with an initial format, put a committee together, choose a venue for presentations.
    • Finalize a proposal for changes in Research based on learnings about Research's audience, what they expect from the team, our positioning within WMF, Movement, and the Research community, and the opportunities for impact.
    • Document and communicate with the team: expectations of the Research Scientist role and trajectory in the IC track.
    • Measuring the consistency of information between Wikipedia articles and Wikidata items
    • Run office hours WMF/Research&Analytics
    • A list of meaningful Commons Categories whose images can be used to train image classifiers
    • Link recommendation
    • Submit work on section alignment
    • Research showcase take-over transition
    • Start formal collaboration around the project "Understanding Readers' Image Usage in Wikipedia"
    • Investigate Knowledge Gaps in Multimedia
    • Supervise Outreachy Internship on Releasing data dumps for Citation Needed Classifiers
    • Start research project on navigation paths "How we read wikipedia"
    • Build taxonomy of readership gaps
    • Define research agenda for external re-use
    • Submit paper on reader demographics surveys for peer-review
    • Research showcase improvements for 2020
    • Pilot social media traffic reports for English Wikipedia
    • Organize Wiki Workshop 2020
    • Submit Citation Usage Camera Ready Paper to the Web Conference 2020
  • N Blocked
    • Start the first version of the Research Internships program
  •  Done
    • Launch experimental API for Wikidata-based topic model
• February 28, 2020 status:

In progress

• • Measuring the consistency of information between Wikipedia articles and Wikidata items
  • Research support for cross-wiki content propagation
  • Run office hours WMF/Research&Analytics
  • Template recommendation exploratory research work
  • Submit work on section alignment
  • Start formal collaboration around the project "Understanding Readers' Image Usage in Wikipedia"
  • Investigate Knowledge Gaps in Multimedia
  • Develop metrics for quantifying knowledge gaps
  • Build taxonomy of readership gaps
  • Define research agenda for external re-use
  • Submit paper on reader demographics surveys for peer-review
  • Research showcase improvements for 2020
  • WMF Research portal/program page template for Meta
  • Pilot social media traffic reports for English Wikipedia
  • Organize Wiki Workshop 2020

 Done

• • A list of meaningful Commons Categories whose images can be used to train image classifiers
  • Link recommendation
  • Research showcase take-over transition
  • Supervise Outreachy Internship on Releasing data dumps for Citation Needed Classifiers
  • Start research project on navigation paths "How we read wikipedia"
  • Launch experimental API for Wikidata-based topic model
  • Submit Citation Usage Camera Ready Paper to the Web Conference 2020

N Blocked

• • Start the first version of the Research Internships program
• March 31, 2020 status:

In progress

• • Measuring the consistency of information between Wikipedia articles and Wikidata items
  • Research support for cross-wiki content propagation
  • Run office hours WMF/Research&Analytics
  • Template recommendation exploratory research work
  • Submit work on section alignment
  • Start formal collaboration around the project "Understanding Readers' Image Usage in Wikipedia"
  • Investigate Knowledge Gaps in Multimedia
  • Develop metrics for quantifying knowledge gaps
  • Build taxonomy of readership gaps
  • Define research agenda for external re-use
  • Submit paper on reader demographics surveys for peer-review
  • Research showcase improvements for 2020
  • WMF Research portal/program page template for Meta
  • Pilot social media traffic reports for English Wikipedia
  • Organize Wiki Workshop 2020

 Done

• • A list of meaningful Commons Categories whose images can be used to train image classifiers
  • Link recommendation
  • Research showcase take-over transition
  • Supervise Outreachy Internship on Releasing data dumps for Citation Needed Classifiers
  • Start research project on navigation paths "How we read wikipedia"
  • Launch experimental API for Wikidata-based topic model
  • Submit Citation Usage Camera Ready Paper to the Web Conference 2020

N Blocked

• • Start the first version of the Research Internships program

Search Platform

Team Manager: Guillaume Lederrey

Core Work

1.1 New query parser is used in production by the end of Q3

2.2 WDQS storage expansion (continued from Q2)

7.1. Increase understanding of our work outside our team, and outside the Foundation

8.1. Improve search quality, especially for non-English wikis by prioritizing community requests - Positive feedback from speakers/community on changes made

10.1 Newcomer task

Wikidata

Improve WDQS updater performance

Machine Learning Infrastructure

3.1. Glent method 1 (comparison to other users' queries) offline tested, tuned, A/B tested and possibly deployed end of Q3 (continued from Q2)

Address Knowledge Gaps

6.1. Increase of training data retention (>90 days) is validated with Legal / Privacy (continued from Q2)

6.2. Any new data retention requirements are implemented (validate with Legal) (continued from Q2)

Structured Data

9.1. Proof of Concept SPARQL endpoint for SDoC is available on WMCS and updated weekly. (stretch) (continued from Q2)

Dependencies on: SDC, Legal

Status

• January 24, 020 status
  • In progress
    • 8.1. Improve search quality, especially for non-English wikis by prioritizing community requests - Positive feedback from speakers/community on changes made
    • 10.1 Newcomer task
    • Import RDF data to answer questions on partioning graph databases - a better understanding of the data
    • 3.2. Glent method 1 (comparison to other users' queries) offline tested, tuned, A/B tested and possibly deployed end of Q3
    • Improve WDQS updater performance
    • 1.1. New query parser is used in production by the end of Q3
    • 2.2. WDQS storage expansion
  • Blocked
    • 6.2. Any new data retention requirements are implemented (validate with Legal)
    • 6.1. Increase of training data retention (>90 days) is validated with Legal / Privacy
    • 7.1. Increase understanding of our work outside our team, and outside the Foundation
    • 9.1. Proof of Concept SPARQL endpoint for SDoC is available on WMCS and updated weekly. (stretch)
• February 28, 2020 status:

In progress

• • Improve WDQS updater performance - quick fixes
  • 3.2. Glent method 1 (comparison to other users' queries) offline tested, tuned, A/B tested and possibly deployed end of Q3
  • 10.1 Newcomer task
  • 8.1. Improve search quality, especially for non-English wikis by prioritizing community requests - Positive feedback from speakers/community on changes made
  • 3.3. Glent method 2 (offline testing)
  • Rework the WDQS updater as an event driven application

 Done

• • Import RDF data to answer questions on partioning graph databases - a better understanding of the data

N Blocked

• • 6.2. Any new data retention requirements are implemented (validate with Legal)
  • 6.1. Increase of training data retention (>90 days) is validated with Legal / Privacy
  • 7.1. Increase understanding of our work outside our team, and outside the Foundation
  • 9.1. Proof of Concept SPARQL endpoint for SDoC is available on WMCS and updated weekly. (stretch)
• March 31, 2020 status:

In progress

• • 3.2. Glent method 1 (comparison to other users' queries) offline tested, tuned, A/B tested and possibly deployed end of Q3
  • 7.1. Increase understanding of our work outside our team, and outside the Foundation
  • 8.1. Improve search quality, especially for non-English wikis by prioritizing community requests - Positive feedback from speakers/community on changes made
  • 3.3. Glent method 2 (offline testing - Chinese)
  • Rework the WDQS updater as an event driven application
  • Import RDF data to answer questions on partioning graph databases - a better understanding of the data

 Done

• • Improve WDQS updater performance - quick fixes
  • 10.1 Newcomer task

N Blocked or N Postponed

• • 6.2. Any new data retention requirements are implemented (validate with Legal)
  • 6.1. Increase of training data retention (>90 days) is validated with Legal / Privacy
  • 9.1. Proof of Concept SPARQL endpoint for SDoC is available on WMCS and updated weekly. (stretch)
  • 1.1. New query parser is used in production by the end of Q3
  • 2.2. WDQS storage expansion

Security

Team Manager: John Bennett

Core Work

Incident response Table Top and updates to security after action reports and improvement plans (continued from Q2)

Create design document for DAST implementation and development tools pen testing

Threat Intel/Hunt

NIST Assessments

Create or improve language-based best security practices documentation (continued from Q2)

Dependencies on:

Status

• January 24, 2020 status
  • In progress
    • Create privacy engineering charter
    • Assess / Refine Phab Usage and Workflows
    • Create design document for DAST implementation and development tools pen testing
    • Enhance and Deploy (to beta) StopForumSpam
    • Evaluate providing a Tier 1 (Clinic-like) function for the Security team.
    • Develop organization-wide privacy training.
  • N Postponed
    • Incident response Table Top and updates to security after action reports and improvement plans
• February 28m, 2020 status:

In progress

• • Enhance and Deploy (to beta) StopForumSpam
  • Evaluate providing a Tier 1 (Clinic-like) function for the Security team.
  • Get started with ERM product
  • Develop organization-wide privacy training.
  • Plan & Operationalize ERM

N Postponed

• • Create design document for DAST implementation and development tools pen testing
  • Understand and evaluate threats to the WMF
  • NIST Assessments
• March 31, 2020 status:

In progress

• • Enhance and Deploy (to beta) StopForumSpam
  • Develop organization-wide privacy training

 Done

• • Evaluate providing a Tier 1 (Clinic-like) function for the Security team.
  • Plan ERM
  • Q3 Security Release

N Postponed or N Blocked

• • Create design document for DAST implementation and development tools pen testing
  • Understand and evaluate threats to the WMF
  • NIST Assessments

Site Reliability Engineering

Directors: Mark Bergsma and Faidon Liambotis

Cross-cutting

Service Operations

Team Manager: Mark Bergsma

Core Work

Data Persistence

Team Manager: Mark Bergsma

Core Work

Traffic

Team Manager: Brandon Black

Core Work

Infrastructure Foundations

Team Manager: Faidon Liambotis

Core Work

Observability

Team Manager: Faidon Liambotis

Core Work

Data Center Operations

Team Manager: Willy Pao

Core Work

Modify existing dc-ops processes to be able to measure SLAs effectively

Create landing page, that directs end users to the various types of data center requests, its appropriate template, and expected turnaround time

Partner with Joel and Automation team to establish reports that can measure SLAs via Phabricator and/or Netbox

Define SLAs for each type of dc-ops tasks

Order and receive all Q3 hardware procurement orders by end of quarter

Clean out eqiad storage room and send all decommissioned equipment and unneeded parts for recycling by end of January

Partner with Julianne and Automation team to revamp decommission process of manually entering information into spreadsheet

Reduce total number of open data center tasks by 30%

Dependencies on:

Status

• January 2020 status - To do
• February 28, 2020 status:

In progress

• • Modify existing dc-ops processes to be able to measure SLOs effectively
  • Create landing page, that directs end users to the various types of data center requests, its appropriate template, and expected turnaround time
  • Partner with Joel and Automation team to establish reports that can measure SLOs via Phabricator and/or Netbox
  • Define SLOs for each type of dc-ops tasks
  • Order and receive all Q3 hardware procurement orders by end of quarter
  • Clean out eqiad storage room and send all decommissioned equipment and unneeded parts for recycling by end of January
  • Partner with Julianne and Automation team to revamp decommission process of manually entering information into spreadsheet
  • Reduce total number of open data center tasks by 30%
  • Receive all incoming data center deliveries in Coupa within 2 business days
  • Eliminate all issues identified by the Coherence, Cables, PuppetDB and Accounting Netbox reports.
• March 31, 2020 status:

In progress

• • Modify existing dc-ops processes to be able to measure SLOs effectively
  • Partner with Joel and Automation team to establish reports that can measure SLOs via Phabricator and/or Netbox
  • Order and receive all Q3 hardware procurement orders by end of quarter

 Done

• • Create landing page, that directs end users to the various types of data center requests, its appropriate template, and expected turnaround time
  • Define SLOs for each type of dc-ops tasks
  • Clean out eqiad storage room and send all decommissioned equipment and unneeded parts for recycling by end of January
  • Partner with Julianne and Automation team to revamp decommission process of manually entering information into spreadsheet
  • Reduce total number of open data center tasks by 30%
  • Receive all incoming data center deliveries in Coupa within 2 business days
  • Eliminate all issues identified by the Coherence, Cables, PuppetDB and Accounting Netbox reports.

Technical Engagement

Team Manager: Birgit Müller

Developer Advocacy

Team Manager: Birgit Müller

Key Deliverables: Reduce Complexity of the Platform; Movement diversity

• Create a blog by and for technical audiences where members of the technical community can post about their technical work
• Publish 6 (min) technical blog posts
• Create regular cadence of content -- strive for 3 x per week -- @MediaWiki and @Wikimediatech
• Run Wikimedia Technical Talks -- increase views on talks by 10%
• Prepare release of 3rd edition of the Tech Community Newsletter (publishing date: April 2020)
• Make further improvements to the dashboard for Wikimedia Cloud Services edit data and announce it on targeted channels.
• Publish current numbers on technical contributions provided by Bitergia in the Quarterly Tech Community newsletter (by Jan 2020)
• Coordinate with Bitergia and get data on "Avg. Time Open (Days)" for Gerrit patchsets per affiliation and "time to first review" data for patches by end of Q4.
• Find out what is needed to get data on technical contributions/contributors (by Q3)
• Provide “showroom”, introducing newcomers to a variety of different tools to show what developers can do in Toolforge (by Q3, in collaboration with GCI students)
• In Q2/Q3, at least 700 task instances are completed in Google Code-in.
• At least five projects are successfully completed by Outreachy interns by end of Q3.
• At least 12 projects are promoted in GSOC and Outreachy programs.
• Kick off Friends of the Docs initiative (prep work in Q3; kick off in Q4)
• Develop workshop concept with partner community for technical workshops in Q3/Q4
• Provide continuous support for teams and individuals in Phabricator
• Conduct at least 4 workshops + introductions into Phabricator at movement events by end of Q4
• Establish Phabricator training for new staff members
• A starter kit for small wikis containing a recommended set of templates, Gadgets, bots etc. is available by Q4

Wikimedia Cloud Services

Team Manager: Bryan Davis

tbd

Dependencies on:

Status

• January 24, 2020 status
  • In progress
    • At least five projects are successfully completed by Outreachy interns by end of Q3.
    • Provide “showroom”, introducing newcomers to a variety of different tools to show what developers can do in Toolforge by Q3
    • Find out what is needed to get data on technical contributions/contributors (by Q3)
    • Coordinate with Bitergia and get data on "Avg. Time Open (Days)" for Gerrit patchsets per affiliation and "time to first review" data for patches (by end of Q4).
    • Create a blog by and for technical audiences where members of the technical community can post about their technical work.
    • Develop workshop concept with partner community for technical workshops in Q3
    • Advocate for better processes + practices to support developer productivity
    • Create regular cadence of content -- strive for 3 x per week -- @MediaWiki and @Wikimediatech
    • In Q2/Q3, at least 700 task instances are completed in Google Code-in.
    • Conduct at least 4 workshops + introductions into Phabricator at movement events by end of Q4
    • Provide “showroom”, introducing newcomers to a variety of different tools to show what developers can do in Toolforge (by Q3, in collaboration with GCI students)
    • Provide Bug Management support in Phabricator
    • Publish 6 (min) technical blog posts
    • Develop workshop concept with partner community for technical workshops in Q3/Q4
    • At least five projects are successfully completed by Outreachy interns by end of Q3.
    • A starter kit for small wikis containing a recommended set of templates, Gadgets, bots etc. is available by Q4
    • Create a blog by and for technical audiences where members of the technical community can post about their technical work
    • At least 14 projects are promoted in GSOC and Outreachy programs by Q4
    • Replace Debian Jessie with newer operating system release on all physical hardware managed by the Cloud Services team before Jessie EOL to keep all software under active security patch management
    • All Debian Jessie instances are removed/replaced in Cloud VPS hosted projects
    • Provide a forecast analysis of CEPH resource utilization used to identify the full impact of migrating virtual machines disks to shared storage.
    • Upgrade OpenStack components from Pike to Queens
    • Provide a more modern, secure, and performant PaaS experience for Toolforge tools
    • Improve readability, and content where applicable, of 5 - 10 pages of Toolforge and Cloud VPS "Help" documentation on Wikitech chosen based on data from 2019 WMCS user survey
  •  Done
    • Publish current numbers on technical contributions provided by Bitergia in the Quarterly Tech Community newsletter (by Jan 2020)
    • Upgrade OpenStack components from Ocata to Pike
• February 28, 2020 status:

In progress

• • Publish 6 (min) technical blog posts
  • Develop workshop concept with partner community for technical workshops in Q3/Q4
  • At least five projects are successfully completed by Outreachy interns by end of Q3.
  • A starter kit for small wikis containing a recommended set of templates, Gadgets, bots etc. is available by Q4
  • Create a blog by and for technical audiences where members of the technical community can post about their technical work
  • Create regular cadence of content -- strive for 3 x per week -- @MediaWiki and @Wikimediatech
  • Establish Phabricator training for new staff members
  • Conduct at least 4 workshops + introductions into Phabricator at movement events by end of Q4
  • Kick off Friends of the Docs initiative (prep work in Q3; kick off in Q4)
  • At least 14 projects are promoted in GSOC and Outreachy programs by Q4
  • Provide Bug Management support in Phabricator
  • Advocate for better processes + practices to support developer productivity
  • Replace Debian Jessie with newer operating system release on all physical hardware managed by the Cloud Services team before Jessie EOL to keep all software under active security patch management
  • All Debian Jessie instances are removed/replaced in Cloud VPS hosted projects
  • Ceph cluster capacity building: Finalize plan for physical OSD deployment in consultation with DCOps and Infrastructure Foundations teams
  • Upgrade OpenStack components from Pike to Queens
  • Research and design Neutron changes needed for multi-row and IPv6
  • Provide a more modern, secure, and performant PaaS experience for Toolforge tools
  • Improve readability, and content where applicable, of 5 - 10 pages of Toolforge and Cloud VPS "Help" documentation on Wikitech chosen based on data from 2019 WMCS user survey
  • Refine technical content review process by applying it to documentation created/updated during Q2

 Done

• • Publish current numbers on technical contributions provided by Bitergia in the Quarterly Tech Community newsletter (by Jan 2020)
  • Provide “showroom”, introducing newcomers to a variety of different tools to show what developers can do in Toolforge (by Q3, in collaboration with GCI students)
  • In Q2/Q3, at least 700 task instances are completed in Google Code-in.
  • Provide a forecast analysis of CEPH resource utilization used to identify the full impact of migrating virtual machines disks to shared storage.
  • Ceph cluster capacity building: Identify the pool of existing cloudvirts that can be used as SSD donors for Ceph OSD nodes
  • Upgrade OpenStack components from Ocata to Pike
• March 31, 2020 status:

In progress

• • Prepare release of 3rd edition of the Tech Community Newsletter (publishing date: April 2020)
  • Implement virtual workshop concept with partner community for technical workshops in Q3/Q4
  • A starter kit for small wikis containing a recommended set of templates, Gadgets, bots etc. is available by Q4
  • Conduct at least 3 workshops + introductions and 1 virtual into Phabricator at movement events by end of Q4
  • Replace Debian Jessie with newer operating system release on all physical hardware managed by the Cloud Services team before Jessie EOL to keep all software under active security patch management
  • All Debian Jessie instances are removed/replaced in Cloud VPS hosted projects
  • Ceph cluster capacity building: Finalize plan for physical OSD deployment in consultation with DCOps and Infrastructure Foundations teams
  • Upgrade OpenStack components from Queens to Rocky
  • Research and design Neutron changes needed for multi-row and IPv6
  • Improve readability, and content where applicable, of 5 - 10 pages of Toolforge and Cloud VPS "Help" documentation on Wikitech chosen based on data from 2019 WMCS user survey
  • Refine technical content review process by applying it to documentation created/updated during Q2

 Done

• • Publish 6 (min) technical blog posts
  • Publish current numbers on technical contributions provided by Bitergia in the Quarterly Tech Community newsletter (by Jan 2020)
  • At least five projects are successfully completed by Outreachy interns by end of Q3.
  • Create a blog by and for technical audiences where members of the technical community can post about their technical work
  • Create regular cadence of content -- strive for 3 x per week -- @MediaWiki and @Wikimediatech
  • Provide “showroom”, introducing newcomers to a variety of different tools to show what developers can do in Toolforge (by Q3, in collaboration with GCI students)
  • In Q2/Q3, at least 700 task instances are completed in Google Code-in.
  • At least 14 projects are promoted in GSOC and Outreachy programs by Q4
  • Provide Bug Management support in Phabricator
  • Advocate for better processes + practices to support developer productivity
  • Provide a forecast analysis of CEPH resource utilization used to identify the full impact of migrating virtual machines disks to shared storage.
  • Ceph cluster capacity building: Identify the pool of existing cloudvirts that can be used as SSD donors for Ceph OSD nodes
  • Upgrade OpenStack components from Ocata to Pike
  • Upgrade OpenStack components from Pike to Queens
  • Provide a more modern, secure, and performant PaaS experience for Toolforge tools

N Postponed

• • Establish Phabricator training for new staff members
  • Kick off Friends of the Docs initiative (prep work in Q3; kick off in Q4)