Wikimedia Technology/Annual Plans/FY2019/TEC13: Code Health/Goals

Program Goals and Status for FY18/19 edit

  • Goal Owner: Greg Grossmeier
  • Program Goals for FY18/19: The program shares its goals/vision with the Code Health Group, which is: work on efforts that universally improve the lives of engineers and their ability to write products with shorter iteration time, decreased development effort, greater stability, and improved performance. Working closely with the Code Health Group, the Code Health Program will focus on the software engineering processes and practices that enable both experienced and junior technical contributors to deliver high value/quality features to the various Wikimedia projects in the most productive way possible
  • Annual Plan: TEC13: Code Health

edit

Outcome 1 / Output 1.1 edit

Increase software stewardship levels of our deployed code

Assess deployed code and prioritize stewardship gaps.

Dependencies on: Code Health Group

Goal(s) edit

  • Investigate and propose record of origin (ROO) for deployed code (currently Developers/Maintainers page)

Status edit

  Note: July 2018

  To do

  Note: August 10, 2018

  To do should get started on this in the next week or so (trying to get to a single record of origin (automated if possible))

  Note: September 14, 2018

This is now   In progress and will need to do a few conversations with SRE folks for clarification and information for ROO.

  Note: October 2,, 2018

This is now   Partially done and will wrap up in the next couple of days.


Outcome 1 / Output 1.3 edit

Increase software stewardship levels of our deployed code

Using stewardship review process, create plan of action for top priority items each quarter.

Dependencies on: Development Teams

Goal(s) edit

  • Perform existing review process for Q1 cycle.

Status edit

  Note: July 2018

  To do

  Note: August 10, 2018

  To do this work is scheduled to start in the next week.

  Note: September 14, 2018

There wasn't anything submitted so far this quarter, so it's still in a   To do status until there is work to be done.


Outcome 2 / Output 2.1 edit

We reduce the number of testable regressions from hitting our users

Integrate regression testability evaluation into our on-going post-mortem process.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s) edit

  • Add test evaluation to post mortem review process.

Status edit

  Note: July 2018

  To do

  Note: August 10, 2018

  In progress

  Note: September 14, 2018

This is now   Done, we might need a slightly different way to collect this information, as right now the document is a bit difficult to use after an incident. We'll take a look at this in the future.


Outcome 2 / Output 2.2 edit

We reduce the number of testable regressions from hitting our users

Jointly create smoke tests addressing high priority needs for 15 projects over the year.

Dependencies on: Development Teams

Goal(s) edit

  • Review existing e2e test coverage.
  • Define prioritization scheme.
  • Prioritize e2e testing gaps.

Status edit

  Note: July 2018

  To do

  Note: August 10, 2018

  To do Need to find time to work on this, but it's a high priority task

  Note: September 14, 2018

This has been   In progress for the last couple of weeks — we're looking at the repos in production and noting the activity in the repos over time and will figure out which ones need the end to end coverage.

  Note: October 2, 2018

This has nearly finished, and is   Partially done


Outcome 2 / Output 2.3 edit

We reduce the number of testable regressions from hitting our users

Pro-actively add unit tests to MediaWiki core and deployed extensions.

Dependencies on: Development Teams, Code Health Group

Goal(s) edit

  • Make current unit testing coverage more visible by reporting out to Engineering Management.
  • (maybe, TBC) set coverage goals

Status edit

  Note: July 2018

  To do

  Note: August 10, 2018

  In progress The unit testing coverage reporting is in progress but the actual report hasn't yet gone out quite yet.

  Note: September 14, 2018

This is still   In progress and will have the first report out by end of Sept.

  Note: October 2, 2018

This is now   Done


Outcome 3 / Output 3.2 edit

Reduce Technical Debt

Tech Debt Management process rolled out.

Dependencies on: Development Teams

Goal(s) edit

  • Platform and Search Platform teams are using TDM PoC

Status edit

  Note: July 2018

  To do

  Note: August 10, 2018

  In progress Worked with both Platform teams to talk about their tech debt and how to manage them. Might want to revise / change up the existing tech debt tags in Phabricator.

  Note: September 14, 2018

This is currently  N Stalled due to the other teams being super swamped. Will roll over to Q2's work.


Outcome 3 / Output 3.4 edit

Reduce Technical Debt

Reduce technical debt in the MediaWiki core, by refactoring and improving internal interfaces and policies.

Dependencies on: Platform Team

Goal(s) edit

  • Identify key Tech Debt areas
  • Put in place Tech Debt management process for PEP

Status edit

  Note: July 2018

  To do

  Note: August 10, 2018

  In progress Part of output 3.2's work, going slowly because PE Platform is still getting organized.

  Note: September 14, 2018

This is currently  N Stalled due to the other teams being super swamped. Will roll over to Q2's work.


Outcome 4 / Output 4.1 edit

Increase visibility into Code Health

Define Code Health Metrics

Dependencies on: Code Health Group

Goal(s) edit

  • Define base Code Health metric set.

Status edit

  Note: July 2018

  To do

  Note: August 10, 2018

  In progress Gotten skeleton crew up and running, getting base set of metrics identified.

  Note: September 14, 2018

The working group has met for the first time, and have a follow-up meeting in another week or so. Everything is   In progress at this point and we might want to update the goal based on the working group's conversations to figure out 1 or 2 meaningful metrics.

  Note: October 2, 2018

This is now   Done

edit

Outcome 1 / Output 1.1 edit

Increase software stewardship levels of our deployed code

Assess deployed code and prioritize stewardship gaps.

Goal(s) edit

  • Update/refresh review queue (review process for initial code deployment)

Status edit

  Note: October 2, 2018

This work has not yet started   To do

  Note: November 8, 2018

Initial proposal was created and now   In progress and waiting to discuss with stakeholders.

  Note: December 6, 2018

This is still   In progress and we will meet up with SRE to go over initial proposal thoughts. This will continue in to Q3.


Outcome 2 / Output 2.1 edit

We reduce the number of testable regressions from hitting our users

Integrate regression testability evaluation into our on-going post-mortem process.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s) edit

  • none

Status edit

  Note: October 2, 2018

This work has not yet started   To do - will work on it when there is work to do.

  Note: November 8, 2018

No PMs thus far.

  Note: December 6, 2018

This goal was adding to post mortem reports, and can be considered   Done


Outcome 2 / Output 2.2 edit

We reduce the number of testable regressions from hitting our users

Jointly create smoke tests addressing high priority needs for 15 projects over the year.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s) edit

  • 5 of the 15 prioritized repositories have at least 1 end-to-end test.

Status edit

  Note: October 2, 2018

This work has not yet started   To do

  Note: November 8, 2018

Prioritization data has been collected and starting to reach out to top 5 candidate teams.

  Note: December 6, 2018

We will contact stakeholders about creating tests for the prioritized repos, this goal is now   In progress.


Outcome 2 / Output 2.3 edit

We reduce the number of testable regressions from hitting our users

Pro-actively add unit tests to MediaWiki core and deployed extensions.

Dependencies on: Development Teams, Core Platform Team, Code Health Group

Goal(s) edit

  • Assess Platform unit test practices and define improvement plan

Status edit

  Note: October 2, 2018

This work has not yet started   To do

  Note: November 8, 2018

This work has not yet started   To do

  Note: December 6, 2018

We have now begun working with the Core Platform team on this   In progress


Outcome 3 / Output 3.2 edit

Reduce Technical Debt

Tech Debt Management process rolled out.

Dependencies on: Development Teams, Core Platform Team, Search Team, Code Health Group

Goal(s) edit

  • Core Platform and Search Platform teams are using TDM PoC

Status edit

  Note: October 2, 2018

This work has not yet started   To do

  Note: November 8, 2018

Had initial discussions with Editor group to understand their current approach to managing tech debt.

  Note: December 6, 2018

Met with the Core Platform team last week, much of the work currently   In progress with the Platform Evolution program, and is already oriented around this goal of tech debt.


Outcome 3 / Output 3.4 edit

Reduce Technical Debt

Reduce technical debt in the MediaWiki core, by refactoring and improving internal interfaces and policies.

Dependencies on: Core Platform Team, Code Health Group

Goal(s) edit

  • Identify key Tech Debt areas
  • Put in place Tech Debt management process for PEP

Status edit

  Note: October 2, 2018

This work has not yet started   To do

  Note: November 8, 2018

This work has not yet started   To do

  Note: December 6, 2018

Met with the Core Platform team last week, much of the work currently   In progress with the Platform Evolution program, and is already oriented around this goal of tech debt.


Outcome 4 / Output 4.1 edit

Increase visibility into Code Health

   Define Code Health Metrics

Dependencies on: Code Health Group

Goal(s) edit

  • Metrics defined and deployed for all 4 Code Health areas. 

Status edit

  Note: October 2, 2018

This work has not yet started   To do

  Note: November 8, 2018

Code Health Metrics working group has been working steadily to review/assess code metrics tools. Current focus is on SonarQube. PHPMetrics has also been deployed for review purposes.

  Note: December 6, 2018

We did an update in TEC3 for risk assessment; this goal has changed a bit this quarter, as we are now doing a proof of concept with SonarQube and is   Partially done

edit

Outcome 1 / Output 1.1 edit

Increase software stewardship levels of our deployed code

Assess deployed code and prioritize stewardship gaps.

Goal(s) edit

  • Develop and communicate guidelines and best practices for successful Code Stewardship.
  • (Continued from Q2) Update/refresh review queue (review process for initial code deployment)

Status edit

  Note: January 10, 2019

  • Discussed that as we've just gotten back from our vacations, this work is ramping up and is   In progress

  Note: February 5, 2019

  • Have a basis for guidelines that should be published / reviewed in the coming weeks. Review queue work will start in another week or so.

  Note: March 12, 2019

  • We are   In progress and will wrap this up by end of this quarter (develop guidelines). Review queue work has gotten a bit bigger than expected and still   In progress, we do hope to be done by end of quarter.

  Note: April 12, 2019

  • The Code Stewardship Best Practices wiki page has been created with a skeleton of topics. This will be a living resource, but the work for Q3 is considered   Done


Outcome 2 / Output 2.2 edit

We reduce the number of testable regressions from hitting our users

Jointly create smoke tests addressing high priority needs for 15 projects over the year.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s) edit

  • 5 of the 15 prioritized repositories have at least 1 end-to-end simple sample test.

Status edit

  Note: January 10, 2019

  • Discussed that as we've just gotten back from our vacations, this work is ramping up and is   In progress

  Note: February 2019

  • We've been reaching out to the top 5 emails/users for this goal, we're waiting on feedback from folks now that All Hands is over.

  Note: March 12, 2019

  • This goal has now been indefinitely  N Declined for a variety of reasons (possible death march)


Outcome 2 / Output 2.3 edit

We reduce the number of testable regressions from hitting our users

Pro-actively add unit tests to MediaWiki core and deployed extensions.

Dependencies on: Development Teams, Core Platform Team, Code Health Group

Goal(s) edit

  • Evolve/develop tools and processes to support the PE refactoring effort to improve code health.
  • Develop common test strategy that enable teams to engage in more effective and efficient testing practices. (maybe should be output 2.4?)

Status edit

  Note: January 10, 2019

  • Discussed that as we've just gotten back from our vacations, this work is ramping up and is   In progress

  Note: February 5, 2019

Discussed this with the CPT team and will meet again this week for figuring out next action items. We've had additional talks with test strategy but will continue next week as we're getting over wikiflu.

  Note: March 12, 2019

  • Met last week with Core Platform team, to talk about addressing gaps and work is   In progress - we're making good progress.
  • The rough draft outline is   Done for the test strategy but we need to finalize a few things, so still overall   In progress with expectations that we'll finish by end of quarter.

  Note: April 12, 2019

  • Groundwork is set for this, we worked with CPT to define gaps and priorities and a course of action as they progress through their refactoring efforts. This will be a continued effort throughout the refactoring activities.   Done
  • We'll need to do more discussion to develop a common test strategy, so this work will continue in Q4   In progress


Outcome 3 / Output 3.2 edit

Reduce Technical Debt

Tech Debt Management process rolled out.

Goal(s) edit

  • Speak at All Hands on the status of Technical Debt
  • Engage and coach development teams on their approach to managing technical debt.

Status edit

  Note: January 10, 2019

  • Discussed that as we've just gotten back from our vacations, this work is ramping up and is   In progress

  Note: February 5, 2019

  • The talk did not happen at All Hands, will look for another forum for this for a technical audience to review with.

  Note: March 12, 2019

  • This goal has not been able to be completed and will probably be wrapped into the broader discussion of code health with various teams  N Postponed. We hope to have a session at the Hackathon to talk about tech debt and code health.

Outcome 4 / Output 4.1 edit

Increase visibility into Code Health

   Define Code Health Metrics

Dependencies on: Code Health Group, Code Health Metric Working Group

Goal(s) edit

  • Code Health Dashboard with 50% of repositories covered.

Status edit

  Note: January 10, 2019

  • Discussed that as we've just gotten back from our vacations, this work is ramping up and is   In progress

  Note: February 5, 2019

  • Made great progress on this during All Hands and will continue on it, should be on track for finishing it soon.

  Note: March 12, 2019

  • We are on track to finish this in the quarter,   In progress with the sonarqube integration.

  Note: April 12, 2019

  • SonarQube is our tool of choice for our current set of code metric analysis and reporting. It's been integrated into the experimental pipeline and reporting for all extensions have been enabled.   Done

edit

Outcome 1+3 edit

  • Outcome 1: Increase software stewardship levels of our deployed code
  • Outcome 3: Reduce Technical Debt

Goal(s) edit

  • Presentation/session(s) at the Wikimedia Hackathon on the current state of Code Health projects (technical debt and code stewardship)

Status edit

  Note: April 8, 2019

  • This will happen in May 2019

  Note: May 7, 2019

  • This will happen next week.

  Note: June 4, 2019

  • Presentations were   Done at the Hackathon and generated a lot of excitement


Outcome 1 / Output 1.1 edit

Increase software stewardship levels of our deployed code

Assess deployed code and prioritize stewardship gaps.

Dependancies on: ___________

Goal(s) edit

  • Publish a re-imagination of the Review Queue process.
  • Develop and implement metrics around task and code-review responsiveness

Status edit

  Note: April 8, 2019

  • This is also   In progress

  Note: May 7, 2019

  • Publishing a re-imagination of the Review Queue is  N On hold right now
  • Metrics is still   In progress

  Note: June 4, 2019

  • Review Queue is still  N On hold right now, blocked on more urgent work right now.
  • Metrics are still   In progress - more work to come from the code review group and cross-team conversations. Work will continue in to next FY.


Outcome 4 / Output 4.2 edit

Increase visibility into Code Health

Code Health Dashboard with 50% of repositories covered.

Dependancies on: _CI__________

Goal(s) edit

  • Expand SonarQube reporting into CI infrastructure
  • Perform SonarQube analysis on all extensions
  • Engage user communities in direct feedback solicitation

Status edit

  Note: April 8, 2019

  • Expanding and performing the SonarQube is   In progress and will engage the community later on in the quarter.

  Note: May 7, 2019

  • Most of the patches are ready, just need to be reviewed, still   In progress

  Note: June 4, 2019

  • First pass was to create a pipeline and we then added in a lot of extensions (more than 50%); created code health pipeline - currently at 10 extensions and probably will add about 80 more. We think that the rest of the work will be done by the end of the quarter.