MediaWiki

New Developers/Quarterly/2017-10

< New Developers | Quarterly
(Redirected from User:SSethi (WMF)/New Developers Quarterly Report (July - September 2017))

This is the first edition of the New Developers Quarterly, a report covering activities, metrics, surveys and lessons learned in the Onboarding New Developers program. This report covers July to September 2017.

Your questions and feedback are welcome in the Discussion page.

Timeline

edit

Newcomer-focused events, programs, and activities between July–September 2017:

Key findings

edit

1. Regular trends in attraction, and stagnating in retention of new developers

On an average, we are attracting around 54 developers per quarter and retaining 8% of them. Two noticings around this: 1) number of new developers joining are more than usual around the time outreach programs, and events are taking place, 2) the retention percentages in 2016 were much higher than they were in the current year. We will check on this in the future to reach a conclusion.  

2. Extension:Example, Wikipedia Android app, and MediaWiki Core, top projects receiving most contributions from new developers in the previous quarter

After investigating further, Extension:Example receives a vast number of test changesets because it is mentioned in our Gerrit documentation. By uploading test changesets, new developers learn about the Wikimedia code contribution process, which they often struggle to understand. These changesets do not necessarily land in the codebase. As a next step to these key findings we will consider: 1) Reviewing the tutorials explaining the code contribution process and investigate what we can improve further. Such as: include a list of easy tasks requiring one-line fixes, etc. that might bring some value to the first contribution made by newcomers 2) Getting Wikipedia Android Apps project listed as a featured project on the New Developers page

3. Is code review less of a problem for new developers than we thought?

Wikimedia technical community seems to be active in reviewing changesets of new developers. One week after closing the quarter, maintainers had merged 50.70%, abandoned 26.06% and haven’t taken a decision yet on 23.24% of the changesets submitted at any point during the quarter. However, a few new developers mentioned in the survey that they struggle with Wikimedia conventions around Gerrit code review.

4. Dedicated spaces, platforms, and learning environments for new developers both online and offline empower them and contribute to an engaging experience

New developers who participated in the mentoring program at the Wikimedia/Wikimania Hackathons in Vienna/Montreal seemed to have a satisfying experience, have ideas for projects they could contribute to, and guidance to work on them. We leveraged communication platforms like Zulip for Outreach programs and Telegram for Hackathons that helped in keeping the new developers engaged and connected with community members.  

5. Continued guidance offered by the local community, support in easy onboarding of new developers

The Africa Wikimedia Developers project kicked off earlier this year in June with a goal to involve developers from Africa in the Wikimedia movement. So far, they have reached a community of over 100 people, conducted MediaWiki development workshops in Ghana and Cameroon, and their ongoing efforts of engaging students after these events by mentoring them on tasks has led to the onboarding of three active developers. 

6. Featuring newcomer-friendly projects helps in attracting developers

New developers in the past have reported being directed to Phabricator workboards for finding tasks or projects as challenging. We tried to change this approach by presenting projects with clear goals and expectations, and a listing of required skills (see New Developers). In Wikimedia's participation in the ongoing round of Outreachy, this approach helped draw 39 potential candidates to seven projects in one month, who expressed their interest in working on a project by commenting on the corresponding Phabricator task.

7. Better tracking information of new developers who participate in Wikimedia international developer events

We currently struggle to keep track of information about new developers such as project they worked on, their contact details (e.g., Phabricator, Gerrit, and Github username), etc. We’ve tried to collect this information during the events manually, but it's challenging. This information is crucial for: a) our follow-up planning of engaging new developers afterward b) to measure retention levels. We will investigate how we could do better.

8. Low response rates (~26%) for the New Developers Survey 

The results of this first survey, which will be repeated on a quarterly basis, must be taken with a grain of salt. The survey was lengthy, and we plan to shorten it in order to increase the response rate. The survey was sent to new contributors for the months of July and August only, leaving the newcomers from September out due to lack of time to process their responses. In the next iteration, we will survey new developers who contributed code between September and November 2017. We hope that higher percentage of responses will help solidify our understanding of new developers and plan for next steps.

9. New Developers tend to become involved in contributing code to Wikimedia software projects when they learn about it through their peers

The small sample of responses in our first survey showed that new developers heard about Wikimedia through a friend or someone who they met at an event. This fact holds true for a few of our developer outreach activities. For example, through the extended network of the Wikimedia outreach program participants, we draw more candidates from their geographic location for the next rounds. 

10. Do new developers tend to be working professionals with over five years of experience developing software?

The small sample of responses showed a higher percentage of experienced professional developers than we expected. We will keep watching for these answers in future surveys.

edit

See also Technical Collaboration Metrics - Onboarding New Developers.

Volunteers contributing patches for review

edit

180 volunteers contributed between July–September 2017. (Source)

QoQ: +11.11%. YoY: +20.81%.

New volunteers attracted

edit

49 new volunteers contributed between July–September 2017. (Source)

QoQ: -19.67%. YoY: -12.50%.

New volunteers retained

edit

Percentage of volunteers active one year (± 3 months) after their first contribution, out of all new volunteers attracted one year ago (between April–June 2016). (Source: Calculation on data)

QoQ: -26.5%. YoY: -60.0%.

Quarterly data (July–September 2017)

edit

Review of changesets by new volunteers

edit

142 changesets were contributed by new volunteers. (Source: Calculation on data[1]. Data as of October 06, 2017.)

Projects with most new volunteers

edit

49 new volunteers contributed to 24 repositories. (Source: "Repos by New Authors")

Outreach events

edit

Outreach programs

edit

Survey analysis

edit

View detailed analysis of survey or read through below for a quick summary.

We sent a survey to 35 new developers who submitted code for the first time between July–August 2017. The survey was designed using the Qualtrics platform, with a goal of understanding more clearly demographics and background, motivations, challenges and needs of our new developers.

Out of the 35 new developers to whom we reached out, 9 (25.7%) completed the survey. Four people started filling out the survey but didn’t finish.

Note: Since we got only a few responses, the analysis of this survey may not be entirely justified, and actions for next steps and recommendations indicated in this report (coming soon!) may not have many conclusions from the survey.

Demographics and background information

edit
  • Respondents of the survey were from United States (a third), Canada, Côte d'Ivoire, Germany, India, Sweden, and Turkey
  • Almost half said they belong to the age group between 25-34
  • Three quarters identified themselves as male

Contributions

edit

Types of contributions

Participants could select more than one option

Participants (n) = 9


Types of contributions (Count)

  1. Fixing bugs (6)
  2. Reporting bugs (5)
  3. Editing articles (5)
  4. Uploading media to Wikimedia Commons (4)
  5. Task discussion (3)
  6. Feature development (2)
  7. Documentation (2)
  8. Organizing meetups (1)
  9. Planning technical implementation (1)
  10. Answering developer queries (1)
  11. Other (1)
Total (31)

Motivations

edit

A few new developers indicated that they first heard about Wikimedia as a place to contributing code through a friend, Wikimedia member or at an event. New developers decided to work on a project as they:

  • Found it interesting
  • Project was a good fit for their graduate studies
  • Could share their research by working on the project
  • Wanted to improve the project
  • Wanted to get started with fixing bugs, submitting patches, etc.
  • Wanted to learn new things and to minimize burden on Wikimedia developers
  • Heard about it at a hackathon

Challenges

edit
  • A few respondents of this survey said they are developing their understanding of Wikimedia's code contribution process and a few others said they struggle with it.
  • Some challenges that new developers face are: finding time, connectivity issues, documentation, git-review process and understanding conventions when working with Gerrit, ignorance to patch for a bug fix, not being able to access Phabricator due to IP ban, poor communication, lack of agile practices, poor requirements, project vision, etc.
  • Learning resources and channels that new developers refer to when they get stuck: Help pages on Wikitech, IRC, Google, Github, Gerrit, other developers, video tutorials, books, Stack overflow, documentation on MediaWiki, extension documentation page, etc.

Experience contributing to Wikimedia

edit

Half of the respondents to this survey said they are extremely satisfied with their experience contributing to Wikimedia. A few others said that they are extremely likely to recommend Wikimedia for code contribution to friends and colleagues.

Suggestions for improvement

edit

Here are some suggestions made by new developers (responses below have been only slightly edited):

  • Standard toolbox over in-house or unusual systems (Gerrit...)
  • Breaking focused services out of the monolithic MediaWiki core.
  • Be bold with code changes. Approve more patches in a timely fashion. Enlarge the group of developers who are authorized to approve patches.
  • Surveying and documenting conventions on how to interact with other developers through Phabricator and Gerrit.
  • Git commit message guidelines suggests "To express that a commit resolves a bug, add Bug: Txxx in the footer at the end of the commit message." What do you do when the commit doesn't resolve the bug because the bug has multiple items? The "Bug: Txxx" footer should _always_ be there, because that automatically adds a change notice to Phabricator. That convention isn't made particularly clear from the guidelines.
  • Fix the IP ban that blocks access to Phabricator. Side note: this issue refers to specific telecom providers in specific areas (learn more).
  • Be better on follow up issues
  • Better guides. Make more documentation on Wikitech. So that we can contribute more.

Footnotes

edit
  1. ↑ Steps: After extracting the list of "New Authors", use that list to filter on authors on Gerrit and its "Status" section.
Retrieved from "https://www.mediawiki.org/w/index.php?title=New_Developers/Quarterly/2017-10&oldid=4169620"