Technical communications/Fall 2012 consultation

Context

Communication between Wikimedia contributors and "tech people" (primarily MediaWiki developers, but also designers and other engineers) hasn't always been ideal. In recent years, Wikimedia employees have made efforts to become more transparent, for example by writing monthly activity reports, by providing hubs listing current activities, and by maintaining "activity pages" for each significant activity. Furthermore, the yearly engineering goals for the WMF were developed publicly, and the more granular Roadmap is updated weekly.

Now, that's all well and such, but what's more interesting to discuss is how we can better engage in true collaboration and 2-way discussion, not just reports and announcements. It's easy to post a link to a new feature that's already been implemented, and tell users "Please provide feedback!". It's much more difficult to truly collaborate every step of the way, from the early planning to deployment.

Some "big" tech projects sponsored by the Wikimedia Foundation are lucky enough to have Oliver Keyes who can spend a lot of time discussing with editors, basically incarnating this 2-way communication channel between users and engineering staff. But Oliver can only do so much: he has to focus on a handful of features, and primarily discusses with the English Wikipedia community. We want to be able to do this for dozens of engineering projects with hundreds of wikis, in many languages, and truly collaborate to build new features together. Hiring hundreds of Community Liaisons isn't really a viable option.

There are probably things in the way we do tech stuff (e.g. new software features and deployments) that drive editors insane. You probably have lots of ideas about what the ideal situation should be, and how to get there: What can the developer community (staff and volunteers) do to get there? (in the short term, medium term, long term?) What can users do to get there?

Instead of just postulating that "The problem is X" and "The solution is obviously Y", we've started an extensive consultation process to learn from users, to hear them, to listen to their complaints and their ideas on how to fix the issues. We're hoping that this open and collaborative thinking process will yield better results than a one-sided analysis.

Phase 1: en and fr wikis

• Goal: Start the consultation on a few select wikis (to give us enough time to follow up appropriately) in languages we know (so we can participate in the discussion) to give us a first overview of the issues and possible solutions
• Timeline: Week 43 (2012)

Phase 2: Wider outreach and Summary

• Goal: Widen the circle of consultation by reaching out to more wikis, after summarizing the initial findings from Phase 1 in order to limit duplication, and to mitigate the fact that we'll be less involved and reactive in a wider and more multilingual consultation. Consolidate and summarize findings, and identify solutions to try out (in the short, medium and long term) to fix the issues discovered.
• Timeline: Weeks 48–51 (2012)

Summary

Volume and scatteredness of information

Issues:

• There needs to be a human filter and conduit between users and developers, to bubble up serious issues and good ideas with consensus.
• Pointing out things that don't work well and suggesting improvements on Village Pump (or other on-wiki forum) is often a waste of time.
• Developers can't cope with the huge amounts of feedback (often duplicate) of the community.
• Users can't cope with the huge amounts of technical information which is also scattered and difficult to find. Technical documentation, announcements and discussions are on other sites (mediawiki.org, meta-wiki, bugzilla, tech blog, etc.) or mailing lists
• There's too much and too little information at the same time. Messages are missed and communication breakdowns happen because people aren't talking and listening in the same venues.
• There are no excuses for developers not announcing changes.
• Many users are unfamiliar / intimidated by bugzilla. and prefer local wiki pages .
• Users rarely read the commit messages.
• Developers don't advertise their changes enough; now that deployments have become routine, they aren't as advertised as before. Developers don't talk as much with users as when there were fewer of both.

Possible solutions:

• (short) Communicate more widely, but on less stuff. Only select activities / changes likely to interest the audience. For example, select deployment changes likely to impact editors, and have them translated and delivered across all wikis. Augment with specific notices for particularly important/urgent/attention-worthy messages.
• (short/medium) Set up a network of local tech teams/ambassadors to act as liaison, to communicate feedback about the announcements, create bug reports or surface requests for tech assistance (examples: w:en:Wikipedia:MediaWiki/DeveloperMemo, s:fr:Wikisource:Dialogue avec les développeurs) with a dedicated on-wiki coordination space that can be a technical village pump or a dedicated page for more persistent discussions
• (short) Having a dedicated team that works on bugs each day (pairing): A community person and a developer. The daily conversation between the two worlds helps each other understand our craft languages.
• (short/medium) Add a link to Special:Version to a list of recent changes / curated changelog with UI changes, API changes, etc.
• (medium) Tie in Bugzilla with CentralAuth and/or allow bug reporting from a wiki page (bug 27001).
• (medium) Make w:Template:Tracked an extension and use it to link back from bugzilla to relevant on-wiki discussions. Make it possible to 'watch' an issue trough it.
• (medium/long) Archive mailing lists posts on-wiki, allow posting to the mailing list from the wiki.
• (medium/long) Set up a dedicated venue (wiki, forum) to report and discuss issues, with sections by language.
• (medium/long) Hire more Community Liaisons for engineering. Being able to predict what will or will not be a problem is dependent on having the institutional knowledge to do so. A CL aids in this: their entire job and experience is having that domain knowledge. He/she can also be understood as a force multiplier, allowing for deep rather than wide efforts in transparency and feedback when necessary.
• (long) Set up a tech search engine that has some understanding about latests changes and is able to find relevant blog posts, bugzilla tickets and projects based on that context.
• (long) Cross-wiki watchlist/notifications and subscriptions by topic (see Echo).

Language and cultural issues

Issues:

• Engineering teams prefer to discuss with editors in English; it's much more difficult for communities of wikis who don't use English to report issues or guide product development.
• One must be able to read and write in English to report bugs in bugzilla.

Possible solutions:

• Tech ambassadors (see above)
• (short) More translation of tech documentation and updates
• (short) Make the technical roadmap translatable and advertise it

Difficulty to find help

Issues:

• It's often unclear who is responsible for what and who to contact about what problem.
• Some users are lost when they encounter technical issues, and don't know who to turn to or how to find help.

Possible solutions:

• (short/medium) Set up a wizard to guide people through the appropriate process and venue depending on what they want to achieve (on wiki? Bugzilla? i18n may be an issue)
• (short/medium) Encourage local tech support in collaboration with users who have enough technical knowledge and experience to help in that local language, and/or report issues in bugzilla (similar to the Ambassadors idea)

Difficulty to get involved or have impact

Issues:

• There isn't a way for the community to prioritise development projects. Part of the clash is the dissonance between the community empowerment ethos which is the norm for most community activities, and the disempowerment that characterises community involvement in engineering development.

Possible solutions:

• (medium/long) Set aside a million dollars of the annual engineering budget for projects that the community can suggest and prioritise via a page on meta.

Perception issues

Issues:

• Feature requests and other changes to MediaWiki are perceived as hard to get implemented.
• Upcoming features and changes are perceived as "going to happen anyway" whether the local community weighs in or not.
• Developers don't do enough testing or evaluation as to whether a change actually accomplishes it s objective before unleashing it on users. When this fails, they lose the trust of users.
• Overall, the majority of feedback provided by users to developers is negative; users aren't as vocal when they like or aren't bothered by a change. Developers need a healthy mix of positive and negative feedback; it will make it more likely that negative feedback doesn't get ignored due to numbness.
• Not following up on new bug reports and feature requests is worse than rejecting them.
• Having layers of empowered community members between users and developers can help a lot to build a common understanding and more constructive discussions.

Possible solutions:

• (medium/long) Developers should treat editors as colleagues rather than customers.
• (medium) Developers should discuss big projects and disruptive/big features with users, and get some sort of consensus, before starting to code. This would increase both awareness and acceptance, and would also help design better features.

Other

Issues:

• Hub pages like Wikimedia Features engineering don't allow users to watch all projects easily.

Possible solutions:

• (short) Extend the StatusHelper gadget to include a button on hub pages to "Watch all projects on this page".
• (long) Build this into Echo.

Phase 3: Experiments

• Goal: Prioritize and try out solutions identified in Phase 3
• Timeline: Week 52 (2012) and Q1–Q2 (2013)

The prioritization phase consists of passing the list of possible solutions above through filters based on those criteria:

• Solutions involving any coding work are discarded.
• Long-term solutions are discarded.
• Translation of documents that change very rapidly isn't welcome.

What remains of the list after this pruning is the following:

• Communicate more widely, but on less stuff. Only select activities / changes likely to interest the audience. For example, select deployment changes likely to impact editors, and have them translated and delivered across all wikis. Augment with specific notices for particularly important/urgent/attention-worthy messages.
• Set up a network of local tech teams/ambassadors to act as liaison, to communicate feedback about the announcements, create bug reports or surface requests for tech assistance (examples: w:en:Wikipedia:MediaWiki/DeveloperMemo, s:fr:Wikisource:Dialogue avec les développeurs) with a dedicated on-wiki coordination space that can be a technical village pump or a dedicated page for more persistent discussions
• More translation of tech documentation and updates
• Encourage local tech support in collaboration with users who have enough technical knowledge and experience to help in that local language, and/or report issues in bugzilla (similar to the Ambassadors idea)

List of places and tools containing technical-related discussions and information

• technical village pumps on local wikis
• mediawiki.org
• wikitech.wikimedia.org
• bugzilla.wikimedia.org
• gerrit
• RT
• technical mailing lists

Out of scope ideas

Stuff that came up during the consultation process but isn't directly relevant to improving communication between technical and user communities:

Bugzilla-specific suggestions:

• Try to find a way to do more with 'assigned'. (unassign automatically)
• For WMF issues, perhaps add a state 'scheduled' + date
• Make a stricter separation between WMF and non-WMF items in bugzilla.

Other:

• Have a central page that summarizes requests for the inclusion of extensions and the reasons for not including these extensions. Checking periodically whether the reasons still apply.