Wikimedia Technology/Goals/2019-20 Q3

Q2 Wikimedia Technology Goals, FY2019–20, Q3 (January - March 2020) Q4

Technology Department Team Goals and Status for Q3 FY19/20 in support of the Medium Term Plan (MTP) Priorities and Annual Plan for FY19/20


Team Manager: Nuria Ruiz

Modern Event Platform
Build a reliable, scalable, and comprehensive platform for creating services, tools and user facing features that produce and consume event data
Deploy a new event stream for analytics using the new Event Platform infrastructure task T241241 N Not done
Client side Error logging enabled for 1 wiki such errors from browser clients are surfaced to developers task T226986 Yes Done
Public schema endpoint for event schemas task T233630 Yes Done
Smart Tools for Better Data. Make easier to understand the history of all Wikimedia projects
Design (together with core platform team) an alternative architecture for historic data endpoints used by iOS application task T241184 In progress In progress
Move stats.wikimedia.org domain to point to Wikistats2 by default task T237752 Yes Done
Wikistats2 UI is localized task T186122 Yes Done
Project Newpyter: First Class Jupyter Notebook system. Deliver Project plan task T241186 Yes Done
Enable Presto access for shell users, much faster way to query hadoop task T243312 Yes Done
Smart Tools for Better Data. Increase Data Quality, Privacy and Security
Bots: Label high volume bot spikes in pageview data as automated traffic task T238357 In progress In progress
Core. Operational Excellence. Increase Resilience of Systems
Deploy Eventstreams with Kubernetes task T238658 Yes Done
Create a MySQL replica for backups for all MySQL instances we use MySQL on, like Oozie or Superset task T234826
Core. Operational Excellence. Reduce Operational Load by Phasing Out Legacy Systems/Technologies
Reimage at least one Hadoop worker to Debian Buster (to unlock the upgrade) - task T231067 Yes Done
Spark 2.4 encryption working in Hadoop task T240934 Yes Done
Refresh 1004 with new host and GPU task T241187 Yes Done
Airflow as an easier job scheduling alternative, PoC for refine workflow task T241246 In progress In progress

Status

  • January 21, 2020 status - updated above
  • February 28, 2020 status:

In progress In progress

    • Project newpiter. Spike about rework of current jupyter network environment
    • Client side Error logging enabled for 1 wiki such errors from browser clients are surfaced to developers
    • Design (together with core platform team) an alternative architecture for historic data endpoints used by iOS application
    • Wikistats2 UI is localized
    • Deploy Eventstreams with Kubernetes

Yes Done

    • Deploy Public schema endpoint for event schemas
    • Move stats.wikimedia.org domain to point to Wikistats2
    • Enable Presto access for shell users, much faster way to query hadoop
    • Spark 2.4 encryption working in Hadoop
  • March 31, 2020 status:

In progress In progress

    • Project newpiter. Spike about rework of current jupyter network environment
    • Replacing oozie. PoC of Airflow for refine workflow
    • Design (together with core platform team) an alternative architecture for historic data endpoints used by iOS application

Yes Done

    • Deploy Public schema endpoint for event schemas
    • Client side Error logging enabled for 1 wiki such errors from browser clients are surfaced to developers
    • Move stats.wikimedia.org domain to point to Wikistats2
    • Wikistats2 UI is localized
    • Enable Presto access for shell users, much faster way to query hadoop
    • Deploy Eventstreams with Kubernetes
    • Create a MySQL replica for backups for all MySQL instances we use MySQL on, like Oozie or Superse
    • Reimage at least one Hadoop worker to Debian Buster (to unlock the upgrade)
    • Spark 2.4 encryption working in Hadoop
    • Refresh 1004 with new host and GPU


Team Manager: Corey Floyd

Core Work
Enable MultiDC Reads
IP Masking
IP Masking Implementation
Platform Evolution / Modern Event Platform
Initial modularization of MediaWiki (planning) (continued from Q2)
Push notification service
Migrate Service - changeprop (continued from Q2)
Initial modularization of MediaWiki (one component)
FAWG (desktop refresh work)
API Gateway
Tech & Product Partnerships
Developer Portal Implementation
Integrate OAuth 2.0 into API, API Keys, rate limiting
Paid API project
Safe and Secure Spaces
Image Hash Checking (in beta)

Dependencies on:

Status

  • January 24, 2020 status
    • In review:
      • Migrate Service - changeprop
    • N Blocked
      • Enable MultiDC Reads
    • In progress In progress
      • Developer Portal Implementation
    • N Postponed
      • IP Masking Implementation
  • February 28, 2020 status:

In progress In progress

    • Developer Portal Implementation
    • Integrate OAuth 2.0 into API, API Keys, rate limiting
    • Image Hash Checking (in beta)
    • FAWG (desktop refresh work)
    • API Gateway

N Postponed

    • IP Masking Implementation
    • Enable MultiDC Reads
  • March 31, 2020 status:

In progress In progress

    • API Portal Implementation
    • Integrate OAuth 2.0 into API, API Keys, rate limiting
    • FAWG (desktop refresh work)
    • API Gateway (routing layer)

Yes Done

    • Image Hash Checking (in beta)

N Blocked

    • IP Masking Implementation
    • Enable MultiDC Reads
    • Initial modularization of MediaWiki (one component)


Team Manager: Gilles Dubuc

Core Work
Figure out the right store to use for the main stash (continued from Q1)
Publish 8 blog posts about performance (continued from Q1)
Support and maintenance of MediaWiki's object caching and data access components (continued from Q1)
Support and maintenance of WebPageTest and synthetic testing infrastructure (continued from Q1)
Support and maintenance of MediaWiki's ResourceLoader (continued from Q1)
Support and maintenance of Fresnel (continued from Q1)
Add operational monitoring for 100% of the performance-team services
Have at least 2 years of retention for ArcLamp flame graphs
Organise and run the Web Performance devroom at FOSDEM 2020 (continued from Q2)
Make seen/unseen state of watched pages reliable (all affected users who reported the issue are satisfied)
DC-shared object caches are available and replicate even if 20% of the servers fail
DC-shared temporary data is written via queues or replicated stores (continued from Q2)
Shared caching and temporary data storage use established/documented patterns
A memory-only lightweight store for temporary data exists and supports global (cross-wiki) keys (continued from Q2)
A disk-backed lightweight store for temporary data exists and supports global (cross-wiki) keys (continued from Q2)
Document how to add your own User Timing and how to see it in RUM and synthetic testing
Collect and graph First Input Delay
Document how to add your own Element Timing and how to see it in synthetic testing
Document how to add your own user journey for synthetic testing
Document the search user journeys in synthetic testing
Platform Evolution / Modern Event Platform
Provide performance expertise to FAWG outcome (continued from Q1)


Team Manager: JR Branaa

Core Work
Team inception, formalization, and assessment of current organizational practices (continued from Q1)
Add all deployed to production repos to the Code Health pipeline (Code Health Metrics).
Solicit feedback from current users of CHM POC and define phase 2 enhancements (continued from Q2)
Interview engineering teams to understand their current code review practices (continued from Q2)
Relaunch the Code Review Office Hours (continued from Q2)
Put in place Code Review performance metrics (continued from Q2)
Develop Test Strategy for CPT
Platform Evolution / Modern Event Platform
Make CI warn about slow tests, and publish a collated list of slow tests
Transfer maintainership/ownership of API Test Tooling from CPT


Team Manager: Tyler Cipriani

Core Work
Set up an experimental elastic search instance to store and analyze CI logs and metrics
Continuation of Phabricator and Gerrit improvement (in conjunction with SRE) (continued from Q2)
Migrate from Gerrit version 2.15 to 2.16
Platform Evolution / Modern Event Platform
Other service deployment pipeline migrations as prioritized between SRE/RelEng and relevant teams (continued from Q2)
A demonstration MediaWiki development environment hosts the full TimedMediaHandler front-end and back-end workflow

Dependencies on:

Status

  • January 24, 2020 status
    • In progress In progress
      • Reduce weekly rate of RDBMs commit/shutdown errors to less than 10
      • DC-shared temporary data is written via queues or replicated stores
      • A memory-only lightweight store for temporary data exists and supports global (cross-wiki) keys
      • A disk-backed lightweight store for temporary data exists and supports global (cross-wiki) keys
      • Migrate prod localisation cache to faster static-array distribution
      • Improve MediaWiki PHP startup time and recover from PHP7 regression
      • Add all deployed to production repos to the Code Health pipeline (Code Health Metrics)
      • Develop Test Strategy for CPT
      • Develop Test Strategy for Editing Team
      • Transfer maintainership/ownership of API Test Tooling from CPT
      • (Stretch): MobileContentService
      • Continuation of Phabricator and Gerrit improvement (in conjunction with SRE)
      • Other service deployment pipeline migrations as prioritized between SRE/RelEng and relevant teams.
      • Migrate from Gerrit version 2.15 to 2.16
      • Scap static PHP array for l10n cache
      • A demonstration MediaWiki development environment hosts the full TimedMediaHandler front-end and back-end workflow
      • Create Basic Local Development Environment for MediaWiki/Core
      • Create PipelineLib documentation to support deployment pipeline migrations
    • N Blocked
      • Reduce weekly rate of lost post-send updates to less than 10
  • February 28, 2020 status:

In progress In progress

    • Add operational monitoring for 100% of the performance-team services
    • Have at least 2 years of retention for ArcLamp flame graphs
    • Collect and graph First Input Delay
    • Migrate prod localisation cache to faster static-array distribution
    • Improve MediaWiki PHP startup time and recover from PHP7 regression
    • Performance review of the DiscussionTools extension
    • Organise and oversee org-wide frontend web performance training
    • Organise and oversee implementation of First Paint on Safari
    • Add all deployed to production repos to the Code Health pipeline (Code Health Metrics).
    • Transfer maintainership/ownership of API Test Tooling from CPT
    • Develop Test Strategy for CPT
    • Develop Test Strategy for Editing Team
    • A demonstration MediaWiki development environment hosts the full TimedMediaHandler front-end and back-end workflow
    • Migrate from Gerrit version 2.15 to 2.16
    • Migrate "integration" Cloud VPS project to stretch
    • Developer satisfaction survey

Yes Done

    • Document how to add your own User Timing and how to see it in RUM and synthetic testing
    • Document how to add your own Element Timing and how to see it in synthetic testing
    • Document how to add your own user journey for synthetic testing
    • Document the search user journeys in synthetic testing
    • Create Basic Local Development Environment for MediaWiki/Core
    • Create PipelineLib documentation to support deployment pipeline migrations
    • Scap static PHP array for l10n cache
  • March 31, 2020 status:

In progress In progress

    • Add operational monitoring for 100% of the performance-team services
    • Have at least 2 years of retention for ArcLamp flame graphs
    • DC-shared object caches are available and replicate even if 20% of the servers fail
    • Collect and graph First Input Delay
    • Improve MediaWiki PHP startup time and recover from PHP7 regression
    • Performance review of the KaiOS app
    • Performance review of the GrowthExperiments extension
    • Organise and oversee org-wide frontend web performance training
    • Organise and oversee implementation of First Paint on Safari
    • Propose and (if approved) help establish a stakeholder forum around TechCom
    • Add all deployed to production repos to the Code Health pipeline (Code Health Metrics).
    • Develop Test Strategy for CPT
    • Put in place Code Stewardship Review Committee
    • A demonstration MediaWiki development environment hosts the full TimedMediaHandler front-end and back-end workflow
    • Migrate zuul deployment from .deb to scap3
    • Migrate "integration" Cloud VPS project to stretch
    • Developer satisfaction survey

Yes Done

    • Document how to add your own User Timing and how to see it in RUM and synthetic testing
    • Document how to add your own Element Timing and how to see it in synthetic testing
    • Document how to add your own user journey for synthetic testing
    • Document the search user journeys in synthetic testing
    • Performance review of the DiscussionTools extension
    • Develop Test Strategy for Editing Team
    • Create Basic Local Development Environment for MediaWiki/Core
    • Create PipelineLib documentation to support deployment pipeline migrations
    • Scap static PHP array for l10n cache

N Blocked or N Postponed

    • Migrate prod localisation cache to faster static-array distribution
    • Shared caching and temporary data storage use established/documented patterns
    • Make seen/unseen state of watched pages reliable (all affected users who reported the issue are satisfied)
    • Transfer maintainership/ownership of API Test Tooling from CPT
    • Set up an experimental elastic search instance to store and analyze CI logs and metrics
    • Migrate from Gerrit version 2.15 to 2.16


Team Manager: Erika Bjune

Core Work
Support Advancement in testing and planned Q3 campaigns
Make IDEAL payment processor campaign (support for Q4 campaigns)
Start Matching Gifts V2

Dependencies on:

Status

  • January 24, 2020 status
    • In progress In progress
      • Support Advancement in testing and planned Q3 campaigns
      • Make IDEAL payment processor campaign (support for Q4 campaigns)
      • Recover from the holidays
  • February 28, 2020 status:

In progress In progress

    • Support Advancement in testing and planned Q3 campaigns
    • Make IDEAL payment processor campaign (support for Q4 campaigns)
  • March 31, 2020 status:

In progress In progress

    • Make IDEAL payment processor campaign ready (support for Q4 campaigns)
    • Matching Gifts V2 (Fr-tech side)

Yes Done

    • Support Advancement in testing and planned Q3 campaigns
    • Recover from the holidays


Team Manager: Aaron Halfaker

Content Integrity
JBuild/improve models in response to community demand (ongoing every quarter)
Machine Learning Infrastructure
Jade expansion/Iteration
Session-model use, maintenance, and user-research

Dependencies on:

Status

  • January 24, 2020 status
    • In progress In progress
      • Expansion of Topic Model to ar, ko, and cswiki
      • Support operations infrastructure improvements (prometheus)
      • Deploy Entity UI for Jade
      • Deployment of session-based models
      • Build/improve models in response to community demand (ongoing every quarter)
      • Improve topic models in based on initial use
  • February 28, 2020 status:

In progress In progress

    • Build/improve models in response to community demand (ongoing every quarter)

N Blocked

    • Improve topic models in based on initial use
  • March 31, 2020 status:

In progress In progress

    • Build/improve models in response to community demand (ongoing every quarter)
    • Improve topic models in based on initial use


Team Manager: Leila Zia

Address Knowledge Gaps
Conduct a literature review, plan and set up collaborations for projects about understanding engagement with Wikimedia images around the world. (continued from Q2)
Build one formal collaborations in the disinformation space to start the research for building solutions starting Q3. (continued from Q2)

Dependencies on:

Status

  • January 24, 2020 status
    • In progress In progress
      • Finalize the research brief for crosslingual topical model laying out the work that will be done in this space starting Q3.
      • Build one formal collaborations in the disinformation space to start the research for building solutions starting Q3.
      • Plan for a challenge: come up with an initial format, put a committee together, choose a venue for presentations.
      • Finalize a proposal for changes in Research based on learnings about Research's audience, what they expect from the team, our positioning within WMF, Movement, and the Research community, and the opportunities for impact.
      • Document and communicate with the team: expectations of the Research Scientist role and trajectory in the IC track.
      • Measuring the consistency of information between Wikipedia articles and Wikidata items
      • Run office hours WMF/Research&Analytics
      • A list of meaningful Commons Categories whose images can be used to train image classifiers
      • Link recommendation
      • Submit work on section alignment
      • Research showcase take-over transition
      • Start formal collaboration around the project "Understanding Readers' Image Usage in Wikipedia"
      • Investigate Knowledge Gaps in Multimedia
      • Supervise Outreachy Internship on Releasing data dumps for Citation Needed Classifiers
      • Start research project on navigation paths "How we read wikipedia"
      • Build taxonomy of readership gaps
      • Define research agenda for external re-use
      • Submit paper on reader demographics surveys for peer-review
      • Research showcase improvements for 2020
      • Pilot social media traffic reports for English Wikipedia
      • Organize Wiki Workshop 2020
      • Submit Citation Usage Camera Ready Paper to the Web Conference 2020
    • N Blocked
      • Start the first version of the Research Internships program
    • Yes Done
      • Launch experimental API for Wikidata-based topic model
  • February 28, 2020 status:

In progress In progress

    • Measuring the consistency of information between Wikipedia articles and Wikidata items
    • Research support for cross-wiki content propagation
    • Run office hours WMF/Research&Analytics
    • Template recommendation exploratory research work
    • Submit work on section alignment
    • Start formal collaboration around the project "Understanding Readers' Image Usage in Wikipedia"
    • Investigate Knowledge Gaps in Multimedia
    • Develop metrics for quantifying knowledge gaps
    • Build taxonomy of readership gaps
    • Define research agenda for external re-use
    • Submit paper on reader demographics surveys for peer-review
    • Research showcase improvements for 2020
    • WMF Research portal/program page template for Meta
    • Pilot social media traffic reports for English Wikipedia
    • Organize Wiki Workshop 2020

Yes Done

    • A list of meaningful Commons Categories whose images can be used to train image classifiers
    • Link recommendation
    • Research showcase take-over transition
    • Supervise Outreachy Internship on Releasing data dumps for Citation Needed Classifiers
    • Start research project on navigation paths "How we read wikipedia"
    • Launch experimental API for Wikidata-based topic model
    • Submit Citation Usage Camera Ready Paper to the Web Conference 2020

N Blocked

    • Start the first version of the Research Internships program
  • March 31, 2020 status:

In progress In progress

    • Measuring the consistency of information between Wikipedia articles and Wikidata items
    • Research support for cross-wiki content propagation
    • Run office hours WMF/Research&Analytics
    • Template recommendation exploratory research work
    • Submit work on section alignment
    • Start formal collaboration around the project "Understanding Readers' Image Usage in Wikipedia"
    • Investigate Knowledge Gaps in Multimedia
    • Develop metrics for quantifying knowledge gaps
    • Build taxonomy of readership gaps
    • Define research agenda for external re-use
    • Submit paper on reader demographics surveys for peer-review
    • Research showcase improvements for 2020
    • WMF Research portal/program page template for Meta
    • Pilot social media traffic reports for English Wikipedia
    • Organize Wiki Workshop 2020

Yes Done

    • A list of meaningful Commons Categories whose images can be used to train image classifiers
    • Link recommendation
    • Research showcase take-over transition
    • Supervise Outreachy Internship on Releasing data dumps for Citation Needed Classifiers
    • Start research project on navigation paths "How we read wikipedia"
    • Launch experimental API for Wikidata-based topic model
    • Submit Citation Usage Camera Ready Paper to the Web Conference 2020

N Blocked

    • Start the first version of the Research Internships program


Team Manager: Guillaume Lederrey

Core Work
1.1 New query parser is used in production by the end of Q3
2.2 WDQS storage expansion (continued from Q2)
7.1. Increase understanding of our work outside our team, and outside the Foundation
8.1. Improve search quality, especially for non-English wikis by prioritizing community requests - Positive feedback from speakers/community on changes made
10.1 Newcomer task
Wikidata
Improve WDQS updater performance
Machine Learning Infrastructure
3.1. Glent method 1 (comparison to other users' queries) offline tested, tuned, A/B tested and possibly deployed end of Q3 (continued from Q2)
Address Knowledge Gaps
6.1. Increase of training data retention (>90 days) is validated with Legal / Privacy (continued from Q2)
6.2. Any new data retention requirements are implemented (validate with Legal) (continued from Q2)
Structured Data
9.1. Proof of Concept SPARQL endpoint for SDoC is available on WMCS and updated weekly. (stretch) (continued from Q2)

Dependencies on: SDC, Legal

Status

  • January 24, 020 status
    • In progress In progress
      • 8.1. Improve search quality, especially for non-English wikis by prioritizing community requests - Positive feedback from speakers/community on changes made
      • 10.1 Newcomer task
      • Import RDF data to answer questions on partioning graph databases - a better understanding of the data
      • 3.2. Glent method 1 (comparison to other users' queries) offline tested, tuned, A/B tested and possibly deployed end of Q3
      • Improve WDQS updater performance
      • 1.1. New query parser is used in production by the end of Q3
      • 2.2. WDQS storage expansion
    • Blocked Blocked
      • 6.2. Any new data retention requirements are implemented (validate with Legal)
      • 6.1. Increase of training data retention (>90 days) is validated with Legal / Privacy
      • 7.1. Increase understanding of our work outside our team, and outside the Foundation
      • 9.1. Proof of Concept SPARQL endpoint for SDoC is available on WMCS and updated weekly. (stretch)
  • February 28, 2020 status:

In progress In progress

    • Improve WDQS updater performance - quick fixes
    • 3.2. Glent method 1 (comparison to other users' queries) offline tested, tuned, A/B tested and possibly deployed end of Q3
    • 10.1 Newcomer task
    • 8.1. Improve search quality, especially for non-English wikis by prioritizing community requests - Positive feedback from speakers/community on changes made
    • 3.3. Glent method 2 (offline testing)
    • Rework the WDQS updater as an event driven application

Yes Done

    • Import RDF data to answer questions on partioning graph databases - a better understanding of the data

N Blocked

    • 6.2. Any new data retention requirements are implemented (validate with Legal)
    • 6.1. Increase of training data retention (>90 days) is validated with Legal / Privacy
    • 7.1. Increase understanding of our work outside our team, and outside the Foundation
    • 9.1. Proof of Concept SPARQL endpoint for SDoC is available on WMCS and updated weekly. (stretch)
  • March 31, 2020 status:

In progress In progress

    • 3.2. Glent method 1 (comparison to other users' queries) offline tested, tuned, A/B tested and possibly deployed end of Q3
    • 7.1. Increase understanding of our work outside our team, and outside the Foundation
    • 8.1. Improve search quality, especially for non-English wikis by prioritizing community requests - Positive feedback from speakers/community on changes made
    • 3.3. Glent method 2 (offline testing - Chinese)
    • Rework the WDQS updater as an event driven application
    • Import RDF data to answer questions on partioning graph databases - a better understanding of the data

Yes Done

    • Improve WDQS updater performance - quick fixes
    • 10.1 Newcomer task

N Blocked or N Postponed

    • 6.2. Any new data retention requirements are implemented (validate with Legal)
    • 6.1. Increase of training data retention (>90 days) is validated with Legal / Privacy
    • 9.1. Proof of Concept SPARQL endpoint for SDoC is available on WMCS and updated weekly. (stretch)
    • 1.1. New query parser is used in production by the end of Q3
    • 2.2. WDQS storage expansion


Team Manager: John Bennett

Core Work
Incident response Table Top and updates to security after action reports and improvement plans (continued from Q2)
Create design document for DAST implementation and development tools pen testing
Threat Intel/Hunt
NIST Assessments
Create or improve language-based best security practices documentation (continued from Q2)

Dependencies on:

Status

  • January 24, 2020 status
    • In progress In progress
      • Create privacy engineering charter
      • Assess / Refine Phab Usage and Workflows
      • Create design document for DAST implementation and development tools pen testing
      • Enhance and Deploy (to beta) StopForumSpam
      • Evaluate providing a Tier 1 (Clinic-like) function for the Security team.
      • Develop organization-wide privacy training.
    • N Postponed
      • Incident response Table Top and updates to security after action reports and improvement plans
  • February 28m, 2020 status:

In progress In progress

    • Enhance and Deploy (to beta) StopForumSpam
    • Evaluate providing a Tier 1 (Clinic-like) function for the Security team.
    • Get started with ERM product
    • Develop organization-wide privacy training.
    • Plan & Operationalize ERM

N Postponed

    • Create design document for DAST implementation and development tools pen testing
    • Understand and evaluate threats to the WMF
    • NIST Assessments
  • March 31, 2020 status:

In progress In progress

    • Enhance and Deploy (to beta) StopForumSpam
    • Develop organization-wide privacy training

Yes Done

    • Evaluate providing a Tier 1 (Clinic-like) function for the Security team.
    • Plan ERM
    • Q3 Security Release

N Postponed or N Blocked

    • Create design document for DAST implementation and development tools pen testing
    • Understand and evaluate threats to the WMF
    • NIST Assessments


Directors: Mark Bergsma and Faidon Liambotis

Cross-cutting

Service Operations

edit

Team Manager: Mark Bergsma

Core Work

Data Persistence

edit

Team Manager: Mark Bergsma

Core Work

Traffic

edit

Team Manager: Brandon Black

Core Work

Infrastructure Foundations

edit

Team Manager: Faidon Liambotis

Core Work

Observability

edit

Team Manager: Faidon Liambotis

Core Work

Data Center Operations

edit

Team Manager: Willy Pao

Core Work
Modify existing dc-ops processes to be able to measure SLAs effectively
Create landing page, that directs end users to the various types of data center requests, its appropriate template, and expected turnaround time
Partner with Joel and Automation team to establish reports that can measure SLAs via Phabricator and/or Netbox
Define SLAs for each type of dc-ops tasks
Order and receive all Q3 hardware procurement orders by end of quarter
Clean out eqiad storage room and send all decommissioned equipment and unneeded parts for recycling by end of January
Partner with Julianne and Automation team to revamp decommission process of manually entering information into spreadsheet
Reduce total number of open data center tasks by 30%

Dependencies on:

Status

  • January 2020 status - To do To do
  • February 28, 2020 status:

In progress In progress

    • Modify existing dc-ops processes to be able to measure SLOs effectively
    • Create landing page, that directs end users to the various types of data center requests, its appropriate template, and expected turnaround time
    • Partner with Joel and Automation team to establish reports that can measure SLOs via Phabricator and/or Netbox
    • Define SLOs for each type of dc-ops tasks
    • Order and receive all Q3 hardware procurement orders by end of quarter
    • Clean out eqiad storage room and send all decommissioned equipment and unneeded parts for recycling by end of January
    • Partner with Julianne and Automation team to revamp decommission process of manually entering information into spreadsheet
    • Reduce total number of open data center tasks by 30%
    • Receive all incoming data center deliveries in Coupa within 2 business days
    • Eliminate all issues identified by the Coherence, Cables, PuppetDB and Accounting Netbox reports.
  • March 31, 2020 status:

In progress In progress

    • Modify existing dc-ops processes to be able to measure SLOs effectively
    • Partner with Joel and Automation team to establish reports that can measure SLOs via Phabricator and/or Netbox
    • Order and receive all Q3 hardware procurement orders by end of quarter

Yes Done

    • Create landing page, that directs end users to the various types of data center requests, its appropriate template, and expected turnaround time
    • Define SLOs for each type of dc-ops tasks
    • Clean out eqiad storage room and send all decommissioned equipment and unneeded parts for recycling by end of January
    • Partner with Julianne and Automation team to revamp decommission process of manually entering information into spreadsheet
    • Reduce total number of open data center tasks by 30%
    • Receive all incoming data center deliveries in Coupa within 2 business days
    • Eliminate all issues identified by the Coherence, Cables, PuppetDB and Accounting Netbox reports.


Team Manager: Birgit Müller

Developer Advocacy

edit

Team Manager: Birgit Müller

Key Deliverables: Reduce Complexity of the Platform; Movement diversity

  • Create a blog by and for technical audiences where members of the technical community can post about their technical work
  • Publish 6 (min) technical blog posts
  • Create regular cadence of content -- strive for 3 x per week -- @MediaWiki and @Wikimediatech
  • Run Wikimedia Technical Talks -- increase views on talks by 10%
  • Prepare release of 3rd edition of the Tech Community Newsletter (publishing date: April 2020)
  • Make further improvements to the dashboard for Wikimedia Cloud Services edit data and announce it on targeted channels.
  • Publish current numbers on technical contributions provided by Bitergia in the Quarterly Tech Community newsletter (by Jan 2020)
  • Coordinate with Bitergia and get data on "Avg. Time Open (Days)" for Gerrit patchsets per affiliation and "time to first review" data for patches by end of Q4.
  • Find out what is needed to get data on technical contributions/contributors (by Q3)
  • Provide “showroom”, introducing newcomers to a variety of different tools to show what developers can do in Toolforge (by Q3, in collaboration with GCI students)
  • In Q2/Q3, at least 700 task instances are completed in Google Code-in.
  • At least five projects are successfully completed by Outreachy interns by end of Q3.
  • At least 12 projects are promoted in GSOC and Outreachy programs.
  • Kick off Friends of the Docs initiative (prep work in Q3; kick off in Q4)
  • Develop workshop concept with partner community for technical workshops in Q3/Q4
  • Provide continuous support for teams and individuals in Phabricator
  • Conduct at least 4 workshops + introductions into Phabricator at movement events by end of Q4
  • Establish Phabricator training for new staff members
  • A starter kit for small wikis containing a recommended set of templates, Gadgets, bots etc. is available by Q4

Wikimedia Cloud Services

edit

Team Manager: Bryan Davis

tbd

Dependencies on:

Status

  • January 24, 2020 status
    • In progress In progress
      • At least five projects are successfully completed by Outreachy interns by end of Q3.
      • Provide “showroom”, introducing newcomers to a variety of different tools to show what developers can do in Toolforge by Q3
      • Find out what is needed to get data on technical contributions/contributors (by Q3)
      • Coordinate with Bitergia and get data on "Avg. Time Open (Days)" for Gerrit patchsets per affiliation and "time to first review" data for patches (by end of Q4).
      • Create a blog by and for technical audiences where members of the technical community can post about their technical work.
      • Develop workshop concept with partner community for technical workshops in Q3
      • Advocate for better processes + practices to support developer productivity
      • Create regular cadence of content -- strive for 3 x per week -- @MediaWiki and @Wikimediatech
      • In Q2/Q3, at least 700 task instances are completed in Google Code-in.
      • Conduct at least 4 workshops + introductions into Phabricator at movement events by end of Q4
      • Provide “showroom”, introducing newcomers to a variety of different tools to show what developers can do in Toolforge (by Q3, in collaboration with GCI students)
      • Provide Bug Management support in Phabricator
      • Publish 6 (min) technical blog posts
      • Develop workshop concept with partner community for technical workshops in Q3/Q4
      • At least five projects are successfully completed by Outreachy interns by end of Q3.
      • A starter kit for small wikis containing a recommended set of templates, Gadgets, bots etc. is available by Q4
      • Create a blog by and for technical audiences where members of the technical community can post about their technical work
      • At least 14 projects are promoted in GSOC and Outreachy programs by Q4
      • Replace Debian Jessie with newer operating system release on all physical hardware managed by the Cloud Services team before Jessie EOL to keep all software under active security patch management
      • All Debian Jessie instances are removed/replaced in Cloud VPS hosted projects
      • Provide a forecast analysis of CEPH resource utilization used to identify the full impact of migrating virtual machines disks to shared storage.
      • Upgrade OpenStack components from Pike to Queens
      • Provide a more modern, secure, and performant PaaS experience for Toolforge tools
      • Improve readability, and content where applicable, of 5 - 10 pages of Toolforge and Cloud VPS "Help" documentation on Wikitech chosen based on data from 2019 WMCS user survey
    • Yes Done
      • Publish current numbers on technical contributions provided by Bitergia in the Quarterly Tech Community newsletter (by Jan 2020)
      • Upgrade OpenStack components from Ocata to Pike
  • February 28, 2020 status:

In progress In progress

    • Publish 6 (min) technical blog posts
    • Develop workshop concept with partner community for technical workshops in Q3/Q4
    • At least five projects are successfully completed by Outreachy interns by end of Q3.
    • A starter kit for small wikis containing a recommended set of templates, Gadgets, bots etc. is available by Q4
    • Create a blog by and for technical audiences where members of the technical community can post about their technical work
    • Create regular cadence of content -- strive for 3 x per week -- @MediaWiki and @Wikimediatech
    • Establish Phabricator training for new staff members
    • Conduct at least 4 workshops + introductions into Phabricator at movement events by end of Q4
    • Kick off Friends of the Docs initiative (prep work in Q3; kick off in Q4)
    • At least 14 projects are promoted in GSOC and Outreachy programs by Q4
    • Provide Bug Management support in Phabricator
    • Advocate for better processes + practices to support developer productivity
    • Replace Debian Jessie with newer operating system release on all physical hardware managed by the Cloud Services team before Jessie EOL to keep all software under active security patch management
    • All Debian Jessie instances are removed/replaced in Cloud VPS hosted projects
    • Ceph cluster capacity building: Finalize plan for physical OSD deployment in consultation with DCOps and Infrastructure Foundations teams
    • Upgrade OpenStack components from Pike to Queens
    • Research and design Neutron changes needed for multi-row and IPv6
    • Provide a more modern, secure, and performant PaaS experience for Toolforge tools
    • Improve readability, and content where applicable, of 5 - 10 pages of Toolforge and Cloud VPS "Help" documentation on Wikitech chosen based on data from 2019 WMCS user survey
    • Refine technical content review process by applying it to documentation created/updated during Q2

Yes Done

    • Publish current numbers on technical contributions provided by Bitergia in the Quarterly Tech Community newsletter (by Jan 2020)
    • Provide “showroom”, introducing newcomers to a variety of different tools to show what developers can do in Toolforge (by Q3, in collaboration with GCI students)
    • In Q2/Q3, at least 700 task instances are completed in Google Code-in.
    • Provide a forecast analysis of CEPH resource utilization used to identify the full impact of migrating virtual machines disks to shared storage.
    • Ceph cluster capacity building: Identify the pool of existing cloudvirts that can be used as SSD donors for Ceph OSD nodes
    • Upgrade OpenStack components from Ocata to Pike
  • March 31, 2020 status:

In progress In progress

    • Prepare release of 3rd edition of the Tech Community Newsletter (publishing date: April 2020)
    • Implement virtual workshop concept with partner community for technical workshops in Q3/Q4
    • A starter kit for small wikis containing a recommended set of templates, Gadgets, bots etc. is available by Q4
    • Conduct at least 3 workshops + introductions and 1 virtual into Phabricator at movement events by end of Q4
    • Replace Debian Jessie with newer operating system release on all physical hardware managed by the Cloud Services team before Jessie EOL to keep all software under active security patch management
    • All Debian Jessie instances are removed/replaced in Cloud VPS hosted projects
    • Ceph cluster capacity building: Finalize plan for physical OSD deployment in consultation with DCOps and Infrastructure Foundations teams
    • Upgrade OpenStack components from Queens to Rocky
    • Research and design Neutron changes needed for multi-row and IPv6
    • Improve readability, and content where applicable, of 5 - 10 pages of Toolforge and Cloud VPS "Help" documentation on Wikitech chosen based on data from 2019 WMCS user survey
    • Refine technical content review process by applying it to documentation created/updated during Q2

Yes Done

    • Publish 6 (min) technical blog posts
    • Publish current numbers on technical contributions provided by Bitergia in the Quarterly Tech Community newsletter (by Jan 2020)
    • At least five projects are successfully completed by Outreachy interns by end of Q3.
    • Create a blog by and for technical audiences where members of the technical community can post about their technical work
    • Create regular cadence of content -- strive for 3 x per week -- @MediaWiki and @Wikimediatech
    • Provide “showroom”, introducing newcomers to a variety of different tools to show what developers can do in Toolforge (by Q3, in collaboration with GCI students)
    • In Q2/Q3, at least 700 task instances are completed in Google Code-in.
    • At least 14 projects are promoted in GSOC and Outreachy programs by Q4
    • Provide Bug Management support in Phabricator
    • Advocate for better processes + practices to support developer productivity
    • Provide a forecast analysis of CEPH resource utilization used to identify the full impact of migrating virtual machines disks to shared storage.
    • Ceph cluster capacity building: Identify the pool of existing cloudvirts that can be used as SSD donors for Ceph OSD nodes
    • Upgrade OpenStack components from Ocata to Pike
    • Upgrade OpenStack components from Pike to Queens
    • Provide a more modern, secure, and performant PaaS experience for Toolforge tools

N Postponed

    • Establish Phabricator training for new staff members
    • Kick off Friends of the Docs initiative (prep work in Q3; kick off in Q4)