Developer Satisfaction Survey/2023/Code review
📖 Developer Satisfaction 2023
The developer satisfaction survey is an annual survey of the Wikimedia developer community.
💯 Code review
editWe asked survey takers if, in the past year, they have had their code reviewed by someone else and/or reviewed someone else's code.
🔗 Hours spent on code review per week
We asked survey takers, "In the past year, on average, about how many hours per week do you spend reviewing others’ code as part of your role as a member of the Wikimedia Developer Community?"
- 45% of respondents said they spend 0-4 hours per week reviewing others' code
- 32% said they spend 4-8 hours per week
- 19% said they spend 8 or more hours per week
- 4% were unsure
One person noted in the open answers…
💬 This is probably a really misleading question, because many people do not do wikimedia developer activity full time.
🔗 Review tools used by respondents
💬 I'd love to use GitLab's CD/CI pipeline for more projects
💬 GitLab seems to be missing several features I depend on, including stacked patch sets […] and the ability to ask reviews from multiple people.
We asked survey takers, “Thinking about your work as a member of the Wikimedia Developer Community, have you used any of the following tools used to host code and conduct code review in the past year?"
Of the 116 survey takers who indicated they had participated in code review in the last year,
- A large majority (91%) said they used Wikimedia's Gerrit
- A smaller majority (66%) said they used GitHub.com
- 45% said they used Wikimedia's GitLab
- 10% said they used GitLab.com
- 7% said they used another review tool
🔗 Review tools used by respondents (by role)
Most roles used Wikimedia's Gerrit, GitHub, and Wikimedia's GitLab in descending order of use over the past year.
The exceptions are the following, who were more likely to use GitHub and Wikimedia's GitLab for their work.
- Data engineers
- Service/API developers
- Test engineers
🔗 Review tools used by respondents (by tenure)
Respondents with 0-3 years tenure and respondents with 4+ years tenure showed similar distributions of review tools used.
🔗 Review tools used by respondents (by hours spent on code review)
Respondents who spent 0-8 hours per week reviewing code and respondents who spent 8+ hours per week reviewing code showed similar distributions of review tools used.
🔗 Code review tools to keep in 2023
We asked survey takers, “You indicated that you used the following tools used to host code and conduct code review in your role as a member of the Wikimedia Developer Community in the past year (2022). Given your experience, do you want to continue using that tool in the current year (2023)?”
If a respondent used a tool in 2022, we asked them if they wanted to keep using it in 2023:
| Tool used in 2022 | Keep using % | Discontinuing/unsure % |
|---|---|---|
| Wikimedia's GitLab | 66% | 34% |
| Wikimedia's Gerrit | 60% | 40% |
| GitHub | 55% | 45% |
| GitLab | 33% | 66% |
🔗 Code review tools to try in 2023
We asked survey takers, “You indicated that you did not use the following tools used to host code and conduct code review in your role as a member of the Wikimedia Developer Community in the past year (2022). Given your experience, do you want to start using that tool in the current year (2023)?”
If a respondent avoided a tool in 2022, we asked them if they wanted to use it in 2023:
| Tool avoided in 2022 | Keep avoiding/unsure | Ready to use |
|---|---|---|
| Wikimedia's Gerrit | 100% | 0% |
| GitLab.com | 91% | 9% |
| GitHub.com | 84% | 16% |
| Wikimedia's GitLab | 66% | 34% |
🔗🕸️ Web of review tools (2022 vs 2023)
We examined the relationships between users' review tool use in 2022 vs. their desired review tool use in 2023.
In the diagram below,
- Colors show the tool used in 2022
- Numbers around the edge show number of respondents
- Ribbons flow from tools respondents used in 2022 to tools they want to use in 2023
- No ribbon is exclusive
- Example: Green represents survey takers who used Wikimedia's GitLab in 2022. Green ribbons flow to the tools those respondents want to use in 2023. The green ribbon furthest to the left shows that approximately 30 users who used Wikimedia’s GitLab in 2022 would be happy to use Wikimedia’s Gerrit in 2023. The green ribbon furthest to the right shows that approximately 40 users who used Wikimedia’s GitLab in 2022 would be happy to continue using it in 2023.
*Excludes “Other”/write-in review tools used in 2022 (N=9)
🔗 Code review quality
We asked survey takers how often the feedback on their code is kind, helpful, respectful, and/or provided in a reasonable amount of time (as a member of the Wikimedia Developer Community during the past year).
- The majority of respondents indicated that feedback is respectful, helpful, and kind
- 92% said that feedback is always or often respectful
- 87% said that feedback is always or often helpful
- 84% said that feedback is always or often kind
- Less than half (45%) said that feedback is always or often timely
🔗 Balancing code review
💬 Finding people to review my patches […] ends up taking more time than actually developing new features/fixing bugs for it.
We asked survey takers how easy or difficult it has been to balancing code review with other Wikimedia developer tasks in the past year as a member of the Wikimedia Developer Community.
- 34% of respondents said that balancing code review with their other Wikimedia developer tasks is easy or somewhat easy
- 32% said balancing was neither easy nor difficult
- 28% said balancing was difficult or somewhat difficult
- 6% were unsure
🔗 Code review and hosting features ranked by priority
We asked survey takers, “How would you rank the following code hosting and code review features by importance to your existing workflows over the past year? Rank the following features from 1 (most important) to 5 (least important)”
Respondents were asked to rank the following features:
- Assigning multiple reviewers
- Stacked patch sets
- Cross-project dependent changes
- Setting up or reconfiguring CI / CD for your repo
- Onboarding new team members to your code-review workflow
- Another priority not listed above
The feature with the highest average rank was "stacked patch sets", and "setting up or reconfiguring CI/CD for your repo" was the feature with the most #1 rankings.
Repeated responses in "Another priority not listed above" were:
- Cross-project search
- Syntax highlighting on diffs/patchsets/merge-requests
- Attention sets