MediaWiki

Core Platform Team/Initiatives/REST JobExecutor

< Core Platform Team‎ | Initiatives
REST JobExecutor
Product Owner: Petr Pchelko
Technical Contact: Petr Pchelko
Status: In Progress
Initiative Vision: Read Edit
Initiative Description: Read Edit
Epics, User Stories, and Requirements: Read Edit
Time and Resource Estimates: Read Edit
Open Questions:
Documentation Links:

Initiative Vision

< Initiatives

Vision:
  • WMF JobQueue calls a REST endpoint for job execution
  • JobQueue and AppServer MW cluster setup is unified
Stakeholder(s):
  • PETeam
  • SRE
  • Language team
 Problem:
  • Currently there’s 4 ways of executing a job in MW, which is 3 too many
  • WMF jobqueue uses an RPC endpoint living in mw-config repo, it’s not covered by tests and is hard to maintain
  • Jobrunner and Appserver has different apache config which will be an obstacle to MW on k8s
 Solution:
  • Introduce a REST job execution endpoint (done)
  • Unify Jobrunner and Appserver apache config
  • Drop RPC endpoint from mw-config
 Aligned Goals:
  • Reduce tech debt
  • Make MW on k8s easier

Initiative Description

< Initiatives

Summary

Replace calls to mediawiki-config/rpc hierarchy with a REST API call for job execution

Significance and Motivation

Tech debt cleanup

Outcomes

rpc endpoints removed from mediawiki-config

Baseline Metrics

100% of job executions go via rpc

Target Metrics

0% of job executions go via rpc

Stakeholders

Platform Engineering Team

Known Dependencies/Blockers

SRE support

Epics, User Stories, and Requirements

< Initiatives

Time and Resource Estimates

< Initiatives

Estimated Start Date

None given

Actual Start Date

None given

Estimated Completion Date

None given

Actual Completion Date

None given

Resource Estimates

1 eng, 1 sre, 1 sprint

Collaborators

SRE


Subpages


50% done

Poll

Team members are invited to voice support or opposition, and give their reasoning. Please use the polling templates, for example: 

* {{support}} I like this example!  ~~~~
* {{oppose}} This is not a great example example... ~~~~
  •   Strong support This work is vital for reducing complexity in advance of k8s migration work. A few attempts have been made at unifying this config and there are WIPs that we can leverage to make this happen more easily. --HNowlan (WMF) (talk) 13:32, 4 September 2020 (UTC)Reply[reply]
  •   Support This doesn't only "just make sense", we also already started it. So we should go ahead and finish it. -- DKinzler (WMF) (talk) 14:35, 4 September 2020 (UTC)Reply[reply]
  •   Support This would reduce tech debt and since ~50% of it is already done we should finish it. CAndrew (WMF) (talk) 14:51, 4 September 2020 (UTC)Reply[reply]
  •   Support Its ~50% done and it would make MW on K8's easier AMooney (WMF) (talk), 9 September 2020
  •   Support If it will clear the path for moving k8s forward and its already half done i think it will be difficult to find a counter argument to this work :) NNikkhoui (WMF)
  •   Support Support for reasons above. WDoran (WMF)
  •   Support Important benefit for MW on k8s and should be completed NNzali (WMF)
  •   Support Supported for all the reasons already mentioned. BPirkle (WMF)
  •   Support Support for reasons above. CCicalese (WMF) (talk) 19:34, 10 September 2020 (UTC)Reply[reply]
Retrieved from "https://www.mediawiki.org/w/index.php?title=Core_Platform_Team/Initiatives/REST_JobExecutor&oldid=4328750"