Wikimedia Technical Documentation Team/Doc metrics/Sources

This page provides an overview of the data and research sources TBurmeister (WMF) used to design v1 of the proposed metrics for MediaWiki core technical documentation. Where applicable, the page links to the sources and/or to additional information about how these sources were used.

	Metrics criteria	Doc characteristics and metrics categories	MediaWiki core doc collections
Tech Docs team	☑	☑	☑
SME interviews (focused on MediaWiki doc metrics): Timo Tijhof (WMF) / Krinkle Amusso (WMF) / Hashar MSantos (WMF)		☑	☑
Existing wiki metrics tools		☑
Documentation style guides		☑
Wikimedia metrics projects	☑	☑
Wikimedia technical documentation projects		☑	☑
Community discussions in event notes, Phabricator tasks, and Talk pages		☑	☑
Industry and scholarly literature	☑	☑

Wiki metrics tools

These tools offer data at the page or wiki (aggregate) level. For this project, this data is considered "raw": we can't directly correlate a single piece of data with a meaningful metric, like "accuracy of technical documentation". Some well-known wiki statistics tools and data sources may not appear here because they aren't implemented to include mediawiki.org.

Built-in MediaWiki metrics

• Page info (example data for Gerrit page)
• Aggregate wiki info: Special:Statistics
• Special:FewestRevisions
• Special:AncientPages
• Special:LongPages
• Special:DeadendPages
• Special:LonelyPages
• Special:MostLinkedPages
• API:Links#Example_1:_Fetch_all_the_links_in_a_page
• Special:MostRevisions

Dashboards

• https://techdoc-dashboard.toolforge.org/ - this tool is in alpha mode, and is intended to provide aggregate metrics for curated collections of technical documentation on wikis
• XTools (example data for New_Developers page)
• WikiStats (example data for mediawiki.org
• Pageviews tool (example data for mediawiki.org

Extensions

• Extension:RelatedArticles and Help:CirrusSearch#Morelike
• Extension:Linter

Style guides and policies

Documentation style guides

• Documentation/Style guide
• Manual:Coding conventions/Documentation
  • Manual:Coding conventions/PHP#Comments and documentation
  • Manual:Coding conventions/JavaScript#Documentation

• Documentation/Tool docs

en-wiki policies

• wikipedia:Wikipedia:Article_size

Wikimedia metrics projects

WMF Essential / Standard Metrics

• Essential_metrics#Essential_metrics_evaluation_dimensions
• Research_and_Decision_Science/Data_glossary
• https://meta.wikimedia.org/wiki/Research:Standard_metrics
• https://meta.wikimedia.org/wiki/Category:Standardized_metric
• https://meta.wikimedia.org/wiki/Research:Page_view

Knowledge Gaps Index

• meta:Research:Developing_Metrics_for_Content_Gaps_(Knowledge_Gaps_Taxonomy)
  • Especially: Choosing_a_Set_of_Metrics and Standard_Quality_Criteria
• meta:Research:Content_gaps_on_Wikipedia

NLP-based tools

For assessing readability:

• https://meta.wikimedia.org/wiki/Research:Multilingual_Readability_Research
• API endpoint: https://api.wikimedia.org/wiki/Lift_Wing_API/Reference/Get_readability_prediction
• functional user interface on toolforge: https://readability.toolforge.org/

(Also consulted non-wiki-specific readability tools like https://www.expresso-app.org/about, and some tech docs industry discussions of readability metrics.)

Developer surveys

• Developer Satisfaction Survey 2024
• Developer Satisfaction Survey 2023
• 2020 Cloud Services Annual Survey
• 2021 Cloud Services Annual Survey
• 2022 Cloud Services Annual Survey

Wikimedia technical documentation and docs projects

MediaWiki technical documentation

Just a sampling of sources consulted, not a comprehensive index of all MediaWiki tech docs!

• https://doc.wikimedia.org/mediawiki-core/master/php/index.html and https://doc.wikimedia.org/mediawiki-core/master/php/topics.html
• Category:MediaWiki technical documentation
• Project:Categories#Category organization
• Entry point files (Manual:Code, such as Manual:load.php)
• maintenance scripts
• Manual:Hooks
• Manual:Global object variables, Manual:Configuration settings
• API:Main page / https://www.mediawiki.org/w/api.php

Other technical documentation projects

Sources consulted that are specific to MediaWiki docs:

• Requests_for_comment/Documentation_overhaul
• User:Waldyrious/Docs#Proposed_structure
• User:Zakgreant/MediaWiki Technical Documentation Plan and WMF_Projects/Technical_Documentation(historical)
• Manual:Contents/To do - may be useful to inform workflows / developer tasks for that facet of doc collections

Sources consulted that relate to the larger ecosystem of tech docs, not just to MediaWiki docs.

• Documentation/Toolkit/Collection audit/Assess

• Developer Portal/History#Key themes from conversations
• User:TBurmeister (WMF)/Sandbox/Collections
• User:KBach-WMF/Collections/Conclusions
• User:APaskulin (WMF)/Sandbox/Metrics
• User:Pavithraes/Sandbox/Technical documentation prioritization#Prioritizing

Records of past community discussions

• Wikimedia Technical Conference 2019 Session
• Wikimedia_Hackathon_2017/Lessons_Learned (search for "technical documentation")
• Documentation/Hackathon 2022 docs discussions
• phab:T156301 - "Create a developer documentation special interest group"
• Talk:Requests_for_comment/Documentation_overhaul
• Wikimedia_technical_documentation:_Friends_of_the_Docs and related notes
• https://phabricator.wikimedia.org/T93026
• https://phabricator.wikimedia.org/T113694
• https://phabricator.wikimedia.org/T48526
• https://phabricator.wikimedia.org/T1287

Industry and scholarly literature

See the references on the Research page.