Code review management/status

Last update on: 2012-09-monthly

2011-03-31

After the 1.17 code review sprint, the number of unreviewed new revisions started to increase again (see the automatically generated chart). Mark Hershberger started to assign name tags to revisions, to help developers track reviews that are requested from them.

2011-04-30

Tim Starling, Sam Reed, Chad Horohoe and Roan Kattouw devoted part of their time to code review. Despite their efforts, the backlog of unreviewed new commits is still increasing. A new feature in the CodeReview tool since the deployment of MediaWiki 1.17 is the ability to "sign off" on commits. Developers are encouraged to test and sign off on commits, in order to help the team prioritize what is ready for review.

2011-06-02

Thanks to concerted efforts by code reviewers, the backlog of unreviewed commits is now decreasing.

Until a few weeks ago, the amount of unreviewed commits was increasing at the same rate as they were before the 1.17 code review sprint. The group of reviewers (Brion Vibber, Tim Starling, Chad Horohoe, Trevor Parscal, Roan Kattouw and Sam Reed) was expanded to include Timo Tijhof, Bryan Tong Minh, Alexandre Emsenhuber, Aryeh Gregor, Neil Kandalgaonkar and Andrew Garrett. Thanks to a conscious effort made by code reviewers, the trend was reversed, and the backlog of unreviewed commits slated for inclusion in the next 1.18 release is now decreasing.

2011-06-30

Many Wikimedia Foundation engineers have committed to spend 20% of their time helping out with code review and other community service-related activities. We're currently planning a training session to bring staff members up-to-speed.

2011-07-01

The backlog of unreviewed commits continued to decrease in June. A long but productive discussion between developers happened on the wikitech-l list about how to further improve the code review process. It led to a proposal of a "20% policy", according to which every eligible Wikimedia engineer would spend 20% of their time doing service work that directly benefits the rest of the community.

2011-07-24

Work continued to review commits (see chart); the re-branching of MediaWiki 1.18 aims to reduce the backlog faster. In July, Wikimedia Foundation engineering staff and contractors also attended a Code review workshop; the goal was to share experience and practices on the general review process, as well as security and performance. The accompanying documentation is now being organized.

2011-08-31

Code review efforts largely focused on MediaWiki 1.18 in August. Still, code review of trunk also remained under control (see chart), which is encouraging, since it's likely to lead to a shorter release cycle. Revisions are now tagged more systematically where specific expertise is needed, e.g. with "front-end", "database" or "i18n"; this also makes it easier to involve volunteers and to hold focused sign-off triages. Also, Wikimania's developers days included a code review training.

2011-09-30

Even though engineering and code review efforts were focused on MediaWiki 1.18 in September, the backlog of unreviewed commits in trunk still continued to decrease (see chart), which means we will be able to release MediaWiki 1.19 fairly rapidly, possibly as soon as December 2011. Work on MediaWiki 1.19 is scheduled to start as soon as MediaWiki 1.18 is officially released.

2011-10-31

The focus on 1.18, as well as events and conferences, have allowed the backlog of unreviewed commits to increase in October (see chart). There are currently about 350 unreviewed revisions in trunk/phase3/, which means the deployment of MediaWiki 1.19 might not happen until January 2012.

2011-11-30

Foundation engineers signed up in greater numbers for weekly slots to devote to the development community (including code review), and discussed how to speed the code review that precludes deploying MediaWiki 1.19. Rob Lanphier also set code review goals and made projections for 1.19.

2012-04-monthly

The Wikimedia engineering 20% policy is the current approach of the Wikimedia Foundation to improving the code review situation. With the move to Git, we no longer have a code review backlog in trunk, but we are still facing a backlog of patches to review (in Gerrit and Bugzilla), RFCs to comment on, and extensions to review.

2012-05-monthly

We continue to handle the bulk of code review via cross-review among team members plus the Wikimedia engineering 20% policy for reviewing volunteer code. Diederik van Liere is working on getting Gerrit stats published so that we can establish a trendline on our backlog. In addition to code review in Gerrit, we continue to keep an eye on Bugzilla, RFCs and extensions to review.

2012-05-31

We continue to handle the bulk of code review via cross-review among team members plus the Wikimedia engineering 20% policy for reviewing volunteer code. Diederik van Liere is working on getting Gerrit stats published so that we can establish a trendline on our backlog.

Current statistics on all MediaWiki (core and extensions):

• 30 that have received a positive tentative review (+1) but have not been merged (+2)
• 216 that received neither -2, -1, +1, or +2 reviews (but might have textual comments)
• 38 received a negative tentative review (-1) with issue to be addressed by the original contributor
• 6 that have been rejected (-2) but not yet abandoned by their original authors

In addition to code review in Gerrit, we continue to keep an eye on Bugzilla, RFCs and extensions to review.

2012-06-28

Diederik van Liere is gathering Gerrit stats now, and is planning to publish the first batch soon. In the meantime, current statistics on all MediaWiki (core and extensions): 49 that have received a positive tentative review (+1) but have not been merged (+2) 203 that received neither -2, -1, +1, or +2 reviews (but might have textual comments) 61 received a negative tentative review (-1) with issue to be addressed by the original contributor 15 that have been rejected (-2) but not yet abandoned by their original authors

2012-06-monthly

Diederik van Liere is gathering Gerrit stats now, and is planning to publish the first batch soon. In the meantime, current statistics on all MediaWiki (core and extensions):

• 49 that have received a positive tentative review (+1) but have not been merged (+2)
• 203 that received neither -2, -1, +1, nor +2 reviews (but might have textual comments)
• 61 received a negative tentative review (-1) with issue to be addressed by the original contributor
• 15 that have been rejected (-2) but not yet abandoned by their original authors

2012-07-monthly

These are the numbers as of 2012-08-01

• +1 but not merged: 41
• 0 but not merged: 210
• -1 but not merged: 87
• -2 and not merged: 15

2012-08-monthly

The analytics team released code review graphs, and Brian Wolff created a tool showing a view of unmerged patchsets and a "Wall of Shame" for authors with several patchsets requiring improvement. Both tools helped inform the discussion about the code review situation. Sumana Harihareswara encouraged authors to take steps to get their code reviewed faster, and actively requested reviews for many submissions.

2012-09-24

Volunteers have been of a lot of help in managing the wiki configuration (operations/mediawiki-config).

2012-09-monthly

Volunteers have been of a lot of help in managing the wiki configuration (operations/mediawiki-config) repository. The WMF analytics code review graphs

show an uptick in patchsets awaiting review at the end of September, and a Signpost analysis showed (among other statistics) that WMF staff provide 86% of first reviews for core patchsets, and just five staffers collectively account for about 55% of that total.