Season of Docs/2024/Case study
The Wikimedia Foundation: 2024 Google Season of Docs Case Study
editOrganization or Project Name: The Wikimedia Foundation
Season of Docs link: Season of Docs/2024
Organization Description: The nonprofit Wikimedia Foundation provides the essential infrastructure for free knowledge. We host Wikipedia, the free online encyclopedia, as well as many other free knowledge projects that are created, edited, and verified by volunteers around the world.
Author: Alexandra Paskulin, Staff Technical Writer, The Wikimedia Foundation
Summary
edit| # Tech Writers | TW Project Hours | Budget | % Project Completed |
| 1 | 848 | $12,650 | 100% |
What problem were you trying to solve? And how did you try to solve it?
User documentation for MediaWiki was split across two different sites, making the information difficult to find and maintain. We solved this problem by de-duplicating content where overlap existed and creating and organizing new content to fill gaps.
What were the outcomes of your project?
We were able to move all the documentation content to the correct site and improve the usability and organization of more than 80 pages, making it easier for users to find the information they need and contribute to the wikis they care about. This work completed a project that was first started in 2006.[1]
What went well? And what would you do differently?
Overall, the project went extremely well, and there's no major aspect I would do differently. Communication during the project was effective, and our technical writer, Charles Uneze, was a privilege to work with. His excellent technical, writing, organizational, and communication skills enabled the project to be completed on time and on budget. There was great collaboration between Charles and our community, and I'm very proud of the work he was able to accomplish.
What advice would you give to other projects trying to solve a similar problem with documentation?
Track your progress as you go, so you have a clear picture of what has been completed and what remains to be done. We were lucky to have a great community to help review content, but if you don't have one in place, try to recruit volunteers.
Project Description
editProject Proposal
editMediaWiki.org is the documentation wiki for MediaWiki. Before MediaWiki.org existed, MediaWiki documentation was stored on a different wiki: Meta-Wiki (also called Meta). Since MediaWiki.org was established as the official wiki for MediaWiki documentation, most MediaWiki documentation pages on Meta have been moved to MediaWiki.org. However, there were still some pages on Meta that contain MediaWiki documentation. The goal of this project was to review the remaining MediaWiki documentation on Meta and migrate it to MediaWiki.org, reviewing the content for accuracy, organizing it in relation to other docs, and re-writing the content where necessary to comply with licensing restrictions. Completing this project has enabled users of MediaWiki to find the information they need and more effectively contribute to the wikis they care about. For more information, see Season of Docs/2024/Proposal.
Proposal Creation Process
editMediaWiki.org is a large, long-running documentation wiki with over 63,000 content pages.[2] This means that there are a lot of different areas of the documentation that can be improved. To choose a proposal for this year's Season of Docs, we looked for opportunities to improve users' information experience that had a clear scope. Because our ecosystem requires significant onboarding, our project needed to budget time for the technical writer to learn our toolset and processes. Because of this, it was important that the project have a clear scope and approach, so that the technical writer would have enough time to complete the project. Because the Wikimedia Foundation has several active development teams, the project also needed to be in a subject area that wouldn't conflict with any upcoming development work. For larger projects, Season of Docs can be an opportunity to choose projects that are a high priority solely from a documentation perspective, giving attention to areas of the docs that are often overlooked.
Our intention was to launch our project page with one or two project ideas pre-populated, then offer a period of time during which community members could suggest additional project ideas. We would then use community feedback received during this period and interest shown by prospective technical writers to choose one project idea to move forward with. Here's a snapshot of our project page when we announced the project in February 2024.
We wanted to make sure that the project ideas were described in a consistent way that would demonstrate that they were well-scoped and ready to work on, so we created a template based on Google's proposal template. The template included the problem, scope, success metrics, and skills the writer should have.
We populated the project page with two project ideas:
- A project to complete an essential content migration that had begun in 2006 but hadn't been fully completed. Completing this work would solve a content fragmentation problem in the MediaWiki user docs.
- A project to improve the documentation coverage for a new feature within a set of pages. This feature is one of the main extension points for MediaWiki, so improving this documentation would make it easier for developers to build integrations with MediaWiki core.
Once the project page was ready, we announced our participation in Season of Docs on our community mailing list[3], our internal Wikimedia Foundation Slack, and the Write the Docs Slack. We officially launched the program on February 26, 2024 and set a deadline of March 22, 2024 for submitting project ideas. Community members were encouraged to add project ideas by editing the wiki page and leaving comments on the on-wiki talk page, although they could also respond to the mailing list thread or Slack threads.
When the deadline arrived, we had received no additional project ideas and very little community feedback. This may have been due to the fact that the template asked for a lot of information that can be difficult to gather, but I think this generally speaks to the fact that planning and scoping documentation projects is hard. The community feedback we did receive was from long-time documentation and technical contributors and was in the form of simple up-vote-style +1s. Next time, it might be useful to provide more lightweight ways to +1 a project idea. By this time, we had received six statements of interest from technical writers. Of these, five were for the migration project, and one was for the feature project. Based on this and the feedback from community members, we decided to move forward with the migration project.
Budget
edit| How much money did you ask for? | $12,650 USD |
| How did you come up with this estimate? | In Google's Season of Docs documentation, they ask that statements of interest from technical writers include a proposed budget. To decide on our final budget, we looked at the early statements of interests that we received and the budgets that those technical writers proposed. These budgets ranged from $8,000 to $12,000 with most between $10,000 and $12,000. We believed that this project would be a lot of work, so we chose $12,000. We felt confident that this was a fair value for the work and would allow us to hire the best applicant. |
| How many hours of work did you budget for the project? | Google's documentation recommends a budget for the entire project instead of an hourly rate, so we did not budget a specific amount of hours for the project. Different writers work at different rates, so it would be difficult to estimate this with any accuracy. Instead, we asked technical writers to include their weekly availability in their statement of interest. We felt that at least 15 hours per week would be necessary to complete the project within the proposed timeline. When we discussed this with our selected technical writer, he expected the project to take 832 hours. |
| How many hours of work were actually needed for the project? | Our technical writer reported that the project needed 848 hours in total. |
| What other expenses did you include in your budget? | We added $150 for swag from the Wikimedia store (which we saw in a budget from a previous Season of Docs project and thought was a great idea) and $500 for a support volunteer to answer questions, review work, and help guide the technical writer. |
| Did you run into any budget surprises during the project (e.g. misestimates)? If so, please explain. | There was a slight increase in the number of hours required to complete the project. This was due to not including writing a final project report in the original estimate. This did not affect the budget, and the project was completed on budget. |
Tech Writer Recruitment
editTo find interested technical writers, we shared our project page in the Google Season of Docs repo in the list of interested organizations, in the Write the Docs Slack, and in our Wikimedia technical community mailing list. On our project page, we asked technical writers to submit a statement of interest using our public task tracking system by April 28, 2024.[4]
Originally, we linked to Google's statement of interest template, but later we added a callout reminding writers to include their approach, timeline, and budget. We also asked writers to leave a comment on the project talk page introducing themselves. This was an additional step to help introduce them to working on the wiki and with our communities. We chose not to conduct synchronous interviews with applicants. Interviews carry a heavy overhead for both the interviewer and the interviewee, and we felt that the application process provided enough information for us to make an informed decision.
Once the application period was over, we created a spreadsheet and evaluated each writer based on:
- Available working hours per week
- Technical writing experience
- Open-source experience
- Wikimedia experience
- Community endorsements
- Writing sample
In retrospect, these qualifications were sometimes hard to find and involved a lot of digging through GitHub profiles. Next time, I would ask applicants to list these qualifications more explicitly in a statement of interest template.
Our first consideration was whether the applicant had any experience with Wikimedia documentation. This is because it can be challenging to onboard into our ecosystem, so having prior experience would make it easier to complete the project on time. Of the 25 applicants, six had prior experience with Wikimedia documentation. Of the six applicants with Wikimedia experience, three applicants had good quality writing samples. We then ranked these three based on the other qualifications.
Of these top applicants, two gained Wikimedia experience by completing tasks related to the project alongside their statement of interest. This was an invaluable way to see their approach, skills, and the quality of their work. Next time, I would consider asking applicants to complete a task as part of their statement of interest, although it would be challenging to find a sufficient number of tasks. It would also be necessary to limit the amount of tasks each applicant could complete since applicants naturally have varying levels of availability at this stage.
We eventually selected Charles Uneze. We reached out to him via email to let him know that he had been selected. We notified the other applicants by commenting on their statement of interest tasks. I invited each applicant to email me personally if they wanted additional feedback on how to improve their application for next time. This was time consuming, but I felt that it was worth doing because of the effort applicants put into submitting their statements of interest.
Charles successfully completed the project and worked with us for the duration of the project. Charles' work has been truly exemplary. He was able to onboard onto our tooling and ecosystem quickly, manage and report on his work independently, and coordinate with community members to receive and respond to feedback. We were very fortunate to be able to work with Charles on this project.
Other Participants
editGoogle offers a stipend of $500 USD to a community member who helps answer questions, review work, and generally takes on a considerable mentorship or guidance role in the project. On our project page, we included a section offering this role, and, during our application period, a volunteer reached out to offer his support. This volunteer, Okereke Chinweotito, was a helpful resource for Charles during the project, answering questions and providing support, and I'm grateful that we were able to work with him. I especially appreciated his support while I was on vacation this summer. Okereke participated for the duration of the project.
In an unofficial capacity, one of our community members, Pppery, was very active during the project, helping review content and providing advice. Having this active community participation was invaluable to the project.
Timeline
editHere's a detailed view of the project schedule vs. the actual completion dates for each phase. Phase 1 went faster than expected due to the fact that Charles had done some work on that phase prior to the project kickoff. Charles also completed the onboarding phase prior to starting the project. This isn't something I would expect of a technical writer, but in this case, it allowed him to begin work on the project more quickly. Because of this, we were able to extend phase 2 to include enough time to complete all the work, which needed more time than we initially planned for. Charles also took a week of vacation during phase 2; I highly recommend encouraging writers to do this to avoid burnout.
| Phase | Description | Schedule | Actual completion date |
| Project kickoff and onboarding |
|
2 weeks (May 20 - June 1) | May 13, 2024 |
| Phase 1: De-duplication (20 pages) |
|
4 weeks (June 2 - June 29) | June 3, 2024 |
| Phase 2: New content (62 pages) |
|
9 weeks (June 30 - August 31) | September 9 2024 |
| Phase 3: Organize |
|
5 weeks (September 1 - October 5) | October 10, 2024 |
| Project wrap-up |
|
3 weeks (October 6 - October 26) | November 4, 2024 |
Deliverables
editPlanned Deliverables
editUnplanned Deliverables
editThere were no unplanned deliverables.
Metrics
editWe were able to achieve our targets for both of our chosen project success metrics:
- Number of pages on Meta that need to be reviewed and consolidated into MediaWiki.org[5]
- Baseline: 82 (April 2024)
- Target: 0
- Current: 0 (November 2024)
- Percentage of help pages with relevant content linked from meta:Help:Contents
- Baseline: 8% (April 2024)
- Target: 100%
- Current: 100% (November 2024)
We'll continue to monitor new pages on Meta and edits to Help:Contents to ensure that these metrics remain at their target values.
I also want to include a qualitative improvement that resulted from this project. In our original project proposal, we called out the fact that a Google search for “MediaWiki edit summary”, one of the most commonly used features of the software, returned the top result as a page on Meta-Wiki instead of a page on MediaWiki.org. I'm happy to report that, as of November 2024, the same search now returns the top result as the correct page on MediaWiki.org.
Analysis
editOur project was extremely successful. Charles was able to complete all the planned deliverables and hit the target values for both of our success metrics. He was able to onboard quickly into our ecosystem and managed his work efficiently and independently. Throughout the project, Charles worked closely with members of our community, receiving feedback and reviews of his work. I was very impressed by Charles' skills as a technical writer and his ability to thrive in our community.
Although I credit most of the project's success to Charles' excellent work, I also think that having a well-scoped project helped make the project successful. Because the goals, timeline, and requirements of the project were clear, Charles was able to work independently with confidence.
At the beginning of the project, we discussed providing reviews of each page before Charles published it. However, it became clear that this would be a bottleneck to Charles' progress. In keeping with the Wikimedia wiki way, I encouraged Charles to be bold, publish his drafts, and work with community members to make any corrections. This ended up working really well and enabled Charles to get feedback early and often.
Another strong point of the project was our communication. We created a chat channel in Wikimedia's Zulip instance where Charles, Okereke, and I stayed in regular contact. This provided a near-synchronous way for Charles to ask questions and get support as things came up. For more formal status reports, Charles posted a status update to the project task in our task tracking system.[6] We decided to keep this process lightweight and chose not to create tasks for individual steps in the project. We also attended short video calls every two weeks to check in, see how things were going, and provide any synchronous support. These calls were usually between 15 and 30 minutes.
The biggest hurdle we faced during the project was getting payments out in a timely fashion. We had initially planned to use a service called Payoneer to receive and disburse the funds, but we ran into an issue and Payoneer refused to accept the transfer from Open Collective. In the end, we were able to use a different approach that worked, but it resulted in payments being delayed by four weeks. I don't think there was any way to avoid this beforehand, but now that we know, we can improve our processes for next time.
Lessons Learned
editWhat went well?
edit| Topic | What we did | Lesson Learned |
| Budget | We included $150 in our budget for our technical writer to get swag from our swag store. | Set aside some budget for swag for your technical writer. This helps them feel like a part of your community. |
| Communication | We met with our technical writer once every two weeks to check in. | +1 Determine a regular meeting cadence with your tech writer.[7] |
| Communication | We created a dedicated channel for the project in a community chat platform. | +1 Use Slack (or similar) for regular communication.[8] |
| Communication | We agreed that our technical writer would post written updates every two weeks to the task in our public task tracker. | Agree with your technical writer that they will post regular, written project updates in a place where community members can follow along. |
| Mentorship | We met with our technical writer once every two weeks via video call to check in. | Provide synchronous time to chat with your technical writer (for example, via a video call). This creates space for questions and conversations that may not happen over text. |
| Metrics | As an organization, we're still working to develop standardized documentation metrics. For this project, we used metrics that would help us define success in terms of this project specifically, even though those metrics aren't necessarily widely applicable to the rest of our documentation. | Documentation metrics can be challenging to define and measure. Work with what you have, and focus on incremental improvement. |
| Project Management | We used a detailed template to propose project ideas that included the problem, scope, success metrics, and skills the writer should have. | Choose a project with a well-defined scope and definition of success. This can be hard, but doing this work up front will help your project run smoothly. |
| Project Management | We created our own timeline based on Google's program timeline with our own deadlines for different steps in the project. This allowed us to communicate clearly with our participants throughout the process. | Create your own timeline with deadlines for important milestones, such as the deadline to submit a statement of interest. Make sure to budget enough time to complete the admin work needed for each step. |
| Project Management | We kept a running document where we documented each decision we made and how we made it. This helped provide a record of the project that we can use to inform our processes for next time. | Keep a record of your decisions throughout the project and how you made them. |
| Project Management | Our technical writer took a week of vacation during the project. | Encourage your technical writer to take a vacation during the project to help prevent burnout. You can account for this in your timeline if necessary. |
| Recruiting & Hiring | We used our project page to recruit a volunteer to act in an official support capacity during the project. | On your project page, ask for a volunteer to help support the project in exchange for a small stipend, as offered by Google. This role can be a helpful resource for your technical writer. |
| Recruiting & Hiring | We ensured that our selected technical writer had sufficient availability per week to complete the project. | Ask applicants to include their hourly availability per week in their statement of interest. |
What could be improved?
edit| Topic | What we did | What we would do differently | Lesson Learned |
| Recruiting & Hiring | We did not ask applicants to complete any sample tasks as part of the application process, but our top applicants ended up being those who had done this on their own initiative. This ended up being a great way to see their skills and approach to the work. | We would find a way to provide a limited amount of sample tasks for applicants to complete as part of the application process. | If possible, ask applicants to complete a sample task, but limit the amount of sample tasks each applicant can do. This is a great way to get a sense of how they work. |
| Recruiting & Hiring | We asked applicants to follow Google's statement of interest template when submitting their application. This resulted in a wide range of formats and some applicants leaving out some information. | We would create our own template that specifically asks for the things we want to see. We would plan out what qualifications we were looking for and include those in the template. | Decide beforehand which qualifications you'll use to evaluate applicants and ask for that information directly. For example, if you want to review a writing sample, ask them to provide a link to one. |
| Payments | We planned to use a third-party service to pay participants, but this didn't end up working and resulted in a delay in payments. | We would choose a simpler solution for processing payments. | Dealing with financial institutions can be complex. Sometimes the simplest option is the best. |
| Payments | We assumed our participants' banks could accept wire transfers. | We would check beforehand to make sure their banks accepted the type of transfer we were planning to send. | Check with your technical writer beforehand to see what kind of transfers their bank accepts (wire transfer, ACH, etc.) |