User:KSmith (WMF)/Technology planned program work Q1

The contents of this page is automatically generated by a script managed by KSmith (WMF)

Technology Program 1: Availability, performance, and maintenanceEdit

Outcome Objective Teams Work
1 1 Release_Engineering with Operations, Services, Discovery, Cloud Services

Deprecate use of Trebuchet across production - task T129290

1 2 Performance with Operations

Follow-up bugfixes and improvements after Thumbor deployment to production - T121388

1 n/a Technical_Operations with Finance, Legal, Partnerships Continue Asia Cache PoP procurement, installation, and configuration tasks

Finish up trailing purchasing tasks from previous quarter (DC, hardware, network links, etc). Procure at least one transit or peering link to help advance address space issue Physically install all hardware Acquire address space & communicate it to Wikipedia Zero partners (via the Zero team) Turn up network links (stretch) Configure network devices and hosts (stretch)

1 n/a Technical_Operations with Release Engineering, WMCS Remove Salt from our infrastructure

Port debdeploy to Cumin Migrate the reimage script to Cumin Remove support for the Trebuchet deployment system Remove Salt from production & WMCS

1 n/a Technical_Operations with Release Engineering Prepare for Puppet 4

Support directory environments in our Puppet infrastructure and add an environment that uses Puppet's future parser Switch at least 3 node groups to the future parser environment Force both current and future parser for every test in the puppet-compiler Integrate puppet-compiler with the Continuous Integration infrastructure (task T166066) (stretch) Speed up CI for operations/puppet (task T166888) and add future parser validation

1 n/a Technical_Operations Improve database backups' coverage, monitoring and data recovery time (part 1)

Adjust configuration management manifests to support MariaDB multi-instances Migrate at least 2 instances on 1 dbstore host to the new multi-instance setup Research backup storage options and prepare a design document Investigate and experiment with replacements of mysqldump

2 2 Performance with Operations

Test user performance from Asia to validate changes when the Asia Cache goes live - T169180

2 3 Performance

Rework NavigationTiming metrics to make them stackable - T104902 Add metrics for master queries on HTTP GET/HEAD - T166199

2 n/a Performance with Release Engineering

Preload base module requests T164299 Upgrade to jQuery 3 T124742

3 1 Analytics_Engineering with Ops

Bootstrap the new Kafka cluster (hardware refresh) with an upgraded version and TLS encryption and access control lists support. task T152015 STRETCH: switch the Varnishkafka cache misc configuration to the new cluster and enable TLS authentication for it.

3 3 Analytics_Engineering with Ops

Hadoop health monitoring (add newer ones like default rack, corrupted blocks, etc..): task T166140 Prep work for Eventlogging databases refresh (OS install, racking..) task T156844 STRETCH: Stats for Vanishkafka errors: task T164259

4 1 Cloud_Services Define a metric to track OpenStack system availability
4 2 Cloud_Services Create a detailed migration plan for implementing Neutron as our OpenStack SDN layer
5 m1 Release_Engineering with All developers, Notably: Wikidata, CirrusSearch

Migrate majority of developers to JavaScript based browser test framework (

Port Selenium tests from Ruby to Node.js - task T139740

Technology Program 2: MediaWikiEdit

Outcome Objective Teams Work
1 n/a MediaWiki_Platform

Hire product manager Develop MediaWiki roadmap

2 n/a MediaWiki_Platform

MimeAnalyzer improvements (T155320) Namespaceization (T166010)

Technology Program 4: Technical community buildingEdit

Outcome Objective Teams Work
1 2 Cloud_Services Plan contract documentation work
5 1 CTO with Technical Collaboration Decide on Dev Summit event location, dates, theme, deadlines, etc. and publicize the information

Technology Program 5. Scoring Platform (ORES)Edit

Outcome Objective Teams Work
1 + 2 1 + 1 Scoring_platform with Operations, Community Engagement

Deploy thresholds selection system (T162217) Deploy advanced support for Albanian and Romanian Wikipedias and basic support for Greek & Tamil Wikipedia Design schema for meta ORES (T153152) Report on implementations of meta ORES (T166053)

Technology Program 6. Streamlined service deliveryEdit

Outcome Objective Teams Work
1 1 + 2 Technical_Operations with Release Engineering, Services

Implement a pod networking policy approach

Upgrade to Kubernetes >= 1.5

Standardize on a "default" pod setup

Experiment with ingress solutions (stretch)

2 2 Release_Engineering with Operations, Services

Define functional tests for Mathoid running on the staging Kubernetes cluster for use in future gating decisions - task T170482 Define method for monitoring and reacting to the above functional tests - task T170483

Technology Program 7. Smart tools for better dataEdit

Outcome Objective Teams Work
1 1 Analytics_Engineering with Cloud Services

(carry on from last quarter) Initial deployment of wikistats 2.0 UI task T160370 Start development of a new backend and API on top of the Data Lake Edit data in Hadoop. task T156384

1 1 Analytics_Engineering with Cloud Services

Add Edit count to Data Lake. task T156384 STRETCH: Add meta data about data lake calculations task T155507

1 2 Analytics_Engineering Spike: load data in superset, test and productionize if pertainstask T166689
1 3 Analytics_Engineering with Ops Deprecate socket.IO RC feed and help clients migrate (by July 7th) task T156919
2 1 Services with Operations Start gradual roll-out of Cassandra 3 & new schema to resolve storage scaling issues and OOM errors.
3 1 Cloud_Services Begin migrating customer-facing Dumps endpoints to Cloud Services

Technology Program 8. Multi-datacenter supportEdit

Outcome Objective Teams Work
1 1 Performance with Operations

Enable HTTPS for swift clients (MediaWiki) - T160616 Enable HTTPS for mariadb clients (MediaWiki) - T134809 Install and use mcrouter in deployment-prep - T151466

1 3 MediaWiki_Platform with Ops

etcd MW config (T156924)

2 1 Services with Operations, Analytics Begin migrating job queue processing to multi-DC enabled eventbus infrastructure.

Implement ChangeProp deduplication and rate limiting. Start migrating job queue use cases.

Technology Program 9. Growing Wikipedia across languages via recommendationsEdit

Outcome Objective Teams Work
1 1 Research with External collaborators (Bob, Michele, Tiziano)

Clean up the category system for machine consumption (this has been the focus for the past few months and we are very close to have a solution) Start surfacing recommendations to collect feedback from editathon organizers

Technology Program 10. Public cloud services & supportEdit

Outcome Objective Teams Work
1 2 Cloud_Services Manage shared tool accounts via Striker
2 1 Cloud_Services Perform initial Cloud Services rebranding
3 1 Cloud_Services Attend Wikimania to support and promote Cloud Services products with the Wikimedia communities
4 1 Cloud_Services Hire first line technical support contractor

Technology Program 12: Grow contributor diversityEdit

Outcome Objective Teams Work
1 1 + 2 Research with External collaborators (Bob and Jerome)

Create one or more formal collaborations for this research (T166085) Perform a literature review and potentially run survey(s) to identify the self-reported causes of imbalanced representation in contribution Initiate the design of a framework to address one of the identified/hypothesized causes of imbalance in contributor demographics, if time permits. (Stretch) Run a couple of quick surveys to get a better sense of where in the pipeline we start loosing diversity. (Stretch)

Product Program 1: Make knowledge more easily discoverableEdit

Outcome Objective Teams Work
1 1 + 2 Search_platform with Analytics, Operations, Community Engagement Quarterly Objective 1:

Perform load and A/B tests on new models to make sure they can be safely deployed to production When ready, deploy newly automated models which match (at a minimum) current performance of manually-configured search result relevance

Quarterly Objective 2:

Perform research spikes to find new analyzers for different languages

Test new analyzers to see if they are improvements (Japanese and Vietnamese) Deploy new / updated analyzers

Deploy analyzers in progress from last quarter (Hebrew)

1 n/a Search_platform with Analysis team, Operations, WMDE Quarterly Objective 3:

Work on expanding category search in the Wikidata Query Service, while also collecting SPARQL statistics.

Product Program 7: Payment processor investigation and long-term strategyEdit

Outcome Objective Teams Work
1 n/a Fundraising_Tech with Ingenico (external vendor) Continue Ingenico intergration

Product Program 8: Donor retentionEdit

Outcome Objective Teams Work
1 n/a Fundraising_Tech with Major Gifts feedback Time box work on Civi duplicate records

Cross-Departmental Program: Privacy, Security, and Data ManagementEdit

Outcome Objective Teams Work
1 2 Security with TBD Update MediaWiki security release process and tooling

Implement continuous integration of security patches Transfer knowledge from Release Engineering Develop regular release regimen

2 2 & 3 Analytics_Engineering with Ops

Audit existing stat/analytics shell accounts and incorporate expiration dates, make all accounts compliant with "time restricted access" stat1002/3 replacement task T152712 To do Revamp docs for access to data (carry on from last quarter) Data purging for Eventlogging task T156933

Cross-Departmental Program: Structured Data on CommonsEdit

Outcome Objective Teams Work
1 + 2 n/a MediaWiki_Platform with Ops (DBA)

Actor table (T167246) development substantially complete, ready for deployment Comment table (T166732) development substantially complete, ready for deployment De-globalize EditPage.php (T144366) Narrow AbuseFilter interface EditPage backend (T157658)

2 n/a Research

Develop interview protocol interview 6-8 GLAM stakeholders Share initial user stories and research themes at Structured Data offsite

n/a n/a Search_platform with Operations, WMDE Quarterly Objective 1:

Commons search extended to support search via structured data for media

Quarterly Objective 2:

Advanced search will be updated to support more specific media search filters