User:KBach-WMF/Collections/PywikibotCollection/Analysis

This page is related to a project that has now concluded (phab:T320625) and is only retained for archival purposes. Information on this page might be inaccurate.

This page describes my conclusions from the Pywikibot collection assessment (task T312992) and proposes improvements that we could make.

Please note that these are not final ideas that I'm certain will succeed. Rather, I think of them as experiments that we can try, iterate over, gather feedback on, and reevaluate as we go.

I am planning to work on proposals outlined in conclusions #1 and #2 (or points 1-3 in Tl;dr below) myself or with volunteers during this quarter. I also welcome any feedback and support the community can provide, starting with these proposals, and continuing later on in the process. If you have any comments, please share them through the talk page or contact me directly (see contact details on my user page). If you want to help, please let me know in task T320625. You can also subscribe to that task to follow its progress.

Too long, didn’t read

Proposal:

1. Use documentation on mediawiki.org (Manual:Pywikibot and sub-pages) as the main entry point for the documentation. Add new, or fix existing newcomer material and links to different resources on these wiki pages.
2. Link to doc.wikimedia.org/pywikibot where applicable to emphasize it’s the most up-to-date and accurate source of detailed information.
3. Clean up mediawiki.org, minimize duplication between doc.wikimedia.org/pywikibot and mediawiki.org, archive/deprecate pages that are outdated on wiki, and link/redirect to the doc.wikimedia.org/pywikibot resources instead.
4. Consider localization of doc.mediawiki.org/pywikibot in the long term.

Conclusion #1: Documentation structure

Documentation is spread around different wikis, sites, and services (see this page for details). The primary sources of information are mediawiki.org (with the Manual:Pywikibot landing page and its sub-pages), and https://doc.wikimedia.org/pywikibot/ (a Sphinx-based static site that's updated alongside Pywikibot itself). There are also valuable Pywikibot resources on Wikidata, Wikitech, on mediawiki.org but outside the Manual:Pywikibot structure, on Meta, and on PAWS.

That's a lot of useful information, but a distributed structure like this makes it difficult to maintain and navigate.

Guiding questions

Which location should be maintained as the main source of truth? What is the purpose of Manual:Pywikibot on mediawiki.org, doc.wikimedia.org/pywikibot, and others?

Proposal

Use both doc.wikimedia.org/pywikibot and Manual:Pywikibot on mediawiki.org as valid documentation hubs:

1. Consider doc.wikimedia.org/pywikibot (dwmo/pywikibot) as the main source of accurate, up-to-date, technical information about Pywikibot
   • Try to emphasize that dwmo/pywikibot is where the most accurate information can be found.
   • Keep reference materials and highly technical information on dwmo/pywikibot.
   • Resist the urge to cloak dwmo/pywikibot by making content from that site available on mediawiki.org (mw.org). Instead, make it very clear that these are two separate sites, but also emphasize that both contain valuable information.
2. Use Manual:Pywikibot and mediawiki.org as a starting point for newcomers, and a place with more general and newcomer-friendly materials.
   • Think of Pywikibot documentation on mediawiki.org as a user-friendly wrapper around dwmo/pywikibot. This is where we’ll focus on additional, more descriptive material that links extensively to dwmo/pywikibot and other resources.
3. Deprecate or archive mw.org content that duplicates information already featured on doc.wikimedia.org/pywikibot, especially if it’s outdated. Link to and improve that site instead if possible. Note that it might not be possible or advisable to immediately archive or deprecate certain content on mw.org. For example, Manual:Pywikibot/Scripts is a set of sub-pages that might require further investigation to ensure we do not lose any valuable information.
4. Make any necessary changes to dwmo/pywikibot to improve its scannability and readability.

Conclusion #2: Newcomer material

While the resources on different sites cover a broad range of topics that will be useful to experienced and inexperienced Pywikibot users and developers alike, the documentation is lacking more approachable materials that could help onboard newcomers more effectively.

Guiding questions

Is it possible to create and link to materials that make it easier to understand and use Pywikibot in simple scenarios? Is it possible to provide ready-to-use recipes that users with less familiarity with Python can make use of?

Proposal

1. Consolidate and highlight some of the more newcomer-friendly materials that already exist. For example, link to the Python 3 tutorial on Wikidata (and similar material) from the Manual:Pywikibot landing page or one of the direct sub-pages.
2. Improve and finalize existing newcomer-oriented materials (e.g. https://phabricator.wikimedia.org/T134495). Make sure these are featured on the landing page as well.
3. Create new tutorials and overviews of Pywikibot, e.g. based on the Small Wiki Toolkits workshops. Make them easily discoverable.

Conclusion #3: Localization

Documentation on dwmo/pywikibot is not localized. In the light of the earlier proposal to use it as the core of Pywikibot documentation that we want to emphasize, we need to consider the implications related to localization.

Documentation on mediawiki.org is relatively easy to localize using mechanisms known broadly within the community. To my knowledge, Sphinx-based documentation cannot make use of these mechanisms as they are right now.

Guiding questions

Is there a way to introduce translations to dwmo/pywikibot? How can we source these translations and build infrastructure that would automate translations deployment?

Proposal

1. Define the requirements for a project that introduces translations in Sphinx-based documentation.

2. Investigate the architectural and technical possibilities and needs.

3. Prototype an implementation of an automated localization pipeline.

4. Produce a ready-to-use recipe that can be applied to other projects on doc.wikimedia.org (initially those based on Sphinx, other ones in the future if needed)

Additional comments

Note that, at the moment, solving this problem has a lower priority compared to the problems listed above. This is because of two reasons:

• Pywikibot documentation has existed in this form for a while. Based on current Phabricator tasks related to Pywikibot, there isn't a strong need for localized technical documentation.
• It is broadly unclear if there is ever (not only for Pywikibot) a general expectation of localization in highly specific, technical materials.

That said, I believe we should look into this problem in the medium to long term anyway, and I’m happy to assist anyone interested in this.

Conclusion #4: Outdated materials

Due to the structure problems outlined above, some documentation related to Pywikibot on wikis ended up outdated. This is mainly because in such a distributed structure, it is difficult to track which resources require an update after Pywikibot itself has changed.

Guiding questions

In addition to the structural changes in Pywikibot's documentation, is it possible to implement automatic mechanisms (e.g. a bot) that will copy some content from the dwmo/pywikibot to the relevant wikis and other resources?

Proposal

Overall, I don't think duplicating content via automation is a good solution to this problem. This is because such changes introduce additional complexity. It would essentially amount to turning a single source of information - dwmo/pywikibot, into three places we'd need to keep in good shape - the original source, the target documentation on wikis, and the mechanism of automation itself.

I suggest using simple links between wikis and dwmo/pywikibot to directly encourage readers to explore that particular source of information. However, if the community decides automated duplication is the way to go, the work in this area would probably consist of the following steps:

1. Identify which resources we want to duplicate. Be very deliberate and select only the materials that are absolutely necessary. Resist the urge to duplicate all content from dwmo/pywikibot.

2. Create a bot that copies the relevant information to very specific wiki pages, making sure translation mechanisms (on source - if applicable; and target) are not broken.

3. Make sure the bot is easily available for configuration changes and is well documented (i.e. maintainable).