Wikimedia Technical Documentation Team/Doc metrics/v0

Tracked in Phabricator
Task T372102

This page summarizes the first set of metrics (v0) the Tech Docs team plans to test as part of the Doc metrics project.

Proposed metrics

Based on the research and analysis completed during Q1 of FY24-25, TBurmeister proposed the following for a first round (v0) of technical documentation metrics:

Doc characteristic	Relevant	Accurate	Usable	Findable
Are succinct; avoid walls of text #Succinct			✅✅
Use consistent organization and structure #ConsistentStructure			✅	✅✅
Are readable on mobile devices, with all information visible #MobileDevice			✅✅
Use consistent format and style #ConsistentFormat			✅✅
Orient the reader within a collection; Are organized alongside related pages #CollectionOrientation			✅	✅✅
Freshness #Freshness	✅	✅✅	✅
Are translatable / translated #Translation			✅
Align with real developer tasks and problems #Developers	✅✅			✅
Are connected to / findable from / automated by code #CodeConnection	✅	✅		✅

Data elements to measure

Data element	Doc characteristics	Metrics categories
Cross-references between code and on-wiki docs	CodeConnection, CollectionOrientation	Findable, Relevant
Page headings	Succinct, ConsistentStructure, ConsistentFormat	Usable, Findable (see Discussion)
Page title	Succinct, ConsistentStructure, ConsistentFormat	Usable, Findable
Page sections Intro Next steps See also	Succinct, ConsistentStructure, CollectionOrientation	Usable, Findable
Navigation Layout grid Navigation template Links from navigation template	ConsistentStructure, CollectionOrientation, MobileDevice	Findable, Usable
Revisions # unique / repeat editors Frequency of edits Time since original publishing vs. last edit Most revisions	Freshness; Developers	Accurate, Relevant
Tables	Succinct (+); MobileDevice (-)	Usable (+Inclusive)
CSS, HTML	ConsistentFormat (-); MobileDevice (?)	Usable (+Inclusive)
Lists Page length # of paragraphs	Succinct	Usable
Translate markup Pages w/o language links	Translation	Usable (+Inclusive)
Code samples	Developers; CodeConnection	Relevant
Incoming links	Developers	Relevant
Pageviews	Developers	Relevant

Next steps

We will pick 3-5 of these metrics to test on a sample set of documentation collections. See the project timeline and milestones for more info.