Templates that display warnings or notices about page content are currently not visible on Wikimedia’s mobile sites. As a result, readers may be unaware of issues affecting the reliability of the pages they’re viewing. This lack of transparency can be problematic, especially in cases where pages flagged for serious concerns, like hoax articles or pages for deletion, are not visible.
The Readers web team aims to improve how these page issue templates are handled on the mobile version of the site, starting with the ambox templates and their cross-wiki equivalents. The goal is to include clear descriptions of the issue or notice, along with an indication of its severity. These changes will help users make more informed judgments about the reliability of the content they are reading Additionally, displaying these warnings could spark readers’ interest in Wikimedia projects, potentially increasing their engagement as contributors.
Below are some examples of how these changes might appear. Feedback is welcome.
-
New treatment for page notice styling - generic treatment -
New treatment for page notice styling - issue with medium severity -
New treatment for page notice styling - issue with high severity -
New treatment for page notice styling - multiple issues
Updates
We have published a draft of our full report with the results of our A/B tests. Overall, we saw a large increase in awareness with the new treatment for page issues. We also noticed that users tend to have higher engagement with issues of higher severity than those of medium or low severity.
We will be continuing with the last series of deployments on English wikipedia later this week.
- We have deployed the feature to all Wikipedias, except English Wikipedia
- We have finished the final stages of the analysis of the results from our A/B tests: https://phabricator.wikimedia.org/T200794
We will be launching an A/B test of the feature on the following wikis:
- Persian, Russian, Japanese, English
- The test will run for two weeks
- We will be tracking the performance of the feature quantitatively but also hope to receive some feedback from readers and editors as well.
We have launched an A/B test of the feature on Latvian Wikipedia
Introduction
When articles have issues, they often feature a large coloured box above the article notifying readers and editors of the issues at hand.
These notices are actually special templates inserted into the content of the article.
English Wikipedia
Each notice in the article has its own template, and each template uses a meta-template called {{Ambox}} (Article message box). Ambox uses a Lua module called Module:Message box. Other MediaWiki namespaces like categories and talk pages have notices that are specific to their namespace as well.
These notices have existed for over a decade and have many conventions around their usage. Contributors maintain these notices. Every language community is free to adopt or invent their own notices, specific to the needs of their project. For example, Commons has a wide array of licensing notices while Wiktionary uses entirely different notices.
The broad range and diversity of these notices makes them hard to standardize. One successful effort happened in 2007 on the English Wikipedia.
-
Before 2007 standardization -
After 2007 standardization
This refresh occurred before the modern mobile web even existed. Despite later efforts by the WMF to bring these notices into the mobile web, they are still not very mobile friendly. As a starting point, this proposal will focus on improving notices in the article namespace that based on the {{Ambox}} template.
Current mobile treatment
-
The link "page issues" below the article title -
After tapping the link, the user is presented with the page issues overlay.
On the current mobile Wikimedia sites we don't render the entire {{Ambox}} template. Instead we display a small grey link with the text "page issues" below the title of an article. When clicked, an overlay with a list of the issues appears. Some notices have text for a "compact" version of the template. When available, only the text for the compact version is displayed.
Limitations
The current implementation depends on modifying the HTML generated by the {{Ambox}} templates. Slight changes to the template HTML can break this feature on mobile. The current implementation doesn't work on all wikis. Templates are different across languages. For example, French Wikipedia refreshed their message boxes in 2016. They now use different templates than English Wikipedia. Here is a list of their maintenance modules. Because of this change, French alerts are not visible on mobile Wikipedia.
English Wikipedia Village Pump proposal
There was a proposal on the English Wikipedia in 2016. The proposal argued that the small "page issues" link doesn't convey the importance of certain notices.
In September 2016, the community approved this proposal to expose page issues on the mobile website. There was clear consensus that some warning templates should be visible on mobile.
"If an article is up for AfD and flagged as a possible hoax with insufficient medical sourcing, any reader visiting that article on a computer is greeted by three large red and orange boxes at the top of the page, one with a cautionary stop sign... If a reader instead visits the same hoax medical article on their phone... they just get two tiny grey words "Page issues" under the article title..."—McGeddon (talk) 10:03, 21 September 2016 (UTC)
Objective
The goal of this project is to improve awareness of particular issues within an article on the mobile web. We will do this by changing the visual styling of page issues.
The team will work with communities to provide guidelines for for styling templates. This work will result in the optimal formatting of issues on mobile site without changing the formatting on desktop.
This goal maps to Program 2, Objective 1 of the annual plan for Readers. "Enable readers to gauge the quality and reliability of an article during their reading experience".
Proposed changes
Page issues definition
For the extent of this project (as well as in the documentation below), we will define a page as having page issues if it is within the main article namespace (NS_MAIN, 0) and contains HTML table with the CSS class ambox etc.
In English Wikipedia this means transcluding the following template (also via nested transclusion):
- w:Template:Ambox (article message box)
Interwiki equivalents with localized names will not be affected yet.
The following templates are not classed as "page issues", as they are only used in the non-article-namespace (NS_MAIN, 0) but are related to this project, in that they are collapsed for readability on mobile:
- w:Template:Tmbox (talk page message box) when in the "Talk" namespace
- w:Template:Cmbox (category message box) when in the "Category" namespace
- w:Template:Fmbox (file message box) when in the "File" namespace.
Note, any template that does not identify itself as an "ambox" is not an issue, this explicitly includes:
- w:Template:Asbox (article stub box) - these are suggestions for improvement (not issues)
A note on "type" and "issue" parameters
We will only be displaying issue severity (and associated icons) for templates that contain the "type" parameter. More about the type parameter can be seen on the ambox template pages on each project and wiki (here is the page for enwiki). Issues that do not contain the type parameter within the template will still appear on the page using the default styling.
Similarly, short descriptions will only be available if the "issue" parameter is available within the template itself. An overview of the templates that already have this text can be found on this phabricator task.
Workflow
- A user visits a page with a page issue.
- The user interacts with a "Page issue" element.
- The user is directed to details on the page issue.
General
Mobile page issues will display the following:
- Issue description
- Short descriptions for the issue will be used where available.
- If no short description is available, we will display the beginning of the long-form issue description.
- Issues length for both short and long descriptions will be restricted based on the space constraints.
- Issue severity
- Severity will be set according to ambox type as follows (this means they will inherit the color of the template type):
- Severe: type=speedy, type=delete
- Medium: type=content
- Low: type = style
- Notice: type = notice, type = move, type = protection
- All other templates will display with normal severity.
- Each severity level will have a custom appearance and position within the page.
- Severity will be set according to ambox type as follows (this means they will inherit the color of the template type):
- If an issue contains links to other pages, the links will be removed. The links will appear on the page issue detail modal.
Examples:
.png)
.png)
.png)
Single issue
- For pages that have a single issue, severity will be derived from the template as defined above.
- An example. If template {{no footnotes}} has severity level "Medium", the article will have a single page issue appear at the Medium level. (example).
Multiple issues
- Issue severity levels will be derived from the severity levels of the individual issues.
- If an article contains the template {{Multiple issues}}, the issue level of the article will be the highest issue level available.
- For example, if template {{Disputed title}} has level "High", the article will have issues appear at High level.(example).
- Each multiple issue level will receive unique copy, appearance, and position within the page.
Example:
.png)
Multiple issues without Template:Multiple issues
We do not support the case where the multiple issues template is not used. If several issues are adjacent to one another but are not wrapped in the multiple issues template, then there are potential problems with the display - in particular with icon display. See phab:T202349 for further discussion.
Section issues
The initial implementation of this project will display issues within individual sections.
Potential future improvements
Once we put these styles in place, we would like to look at being able to provide them in a more centralized way by delivering mobile-friendly CSS using the TemplateStyles extension. This will allow us to cover a larger percentage of templates with specialized treatment and give more power to template editors to select the specialized treatment for each template.
Initial updates to the page issue modal
To start, the Readers team made the following changes to the page issue modal. This change happened in late March 2018. The goal of this change is to expose more detail about the page issue itself. It also provides information to readers on how to fix these issues.
-
Page issues modal before changes -
Page issues modal after changes
Feature evaluation
The Readers team wants to understand the effect increasing the awareness and prominence of page issues on the mobile website has, particularly on the perceived quality and reliability of pages by readers. Our goal is to help readers better gauge the reliability and quality of the article they are reading. Our hypothesis is that making them more aware of page issues will contribute to this goal.
We'd like to answer the following questions:
Does the new treatment for page issue notices increase the awareness among readers of page issues?
- Qualitative research:
- Do readers notice the new page issue treatment more than the current treatment?
- Do readers notice version B (with titles) of the new treatment more than version A (without titles)?
- Do readers notice page issue notifications located after the lead paragraph more than if they are located at the top of the page?
- Quantitative research:
- Is there an increase in click-through based on the new issue treatments (from the article page to the issues modal)?
- Is there any correlation between severity of the issue and click-through rate?
How do users feel about being informed of page issues? How does awareness of page issues affect their perception of Wikipedia?
- Qualitative research:
- RQ1 - Do page issues make sense to readers?
- RQ2 - Do readers care about page issues? Do they find them useful? Important?
- RQ3 - Are readers familiar with page issues already? Have they seen them on other articles?
- RQ4 - Do readers understand how page issues work? In other words, how they appear on a page?
- RQ5 - Does becoming aware of page issues change readers’ perception of Wikipedia?
Additional research questions:
- How do readers form opinions about the quality and reliability of Wikipedia pages in general?
- Do readers care more about issues considered by Wikipedia to be of higher severity than issues considered to be of lower severity?
- What feedback loops (if any) get activated as a result of increased awareness of page issues? For example, do mobile edits increase with page issues as referrer? Does the new issue treatment changes affect issue removal rates?
Technical challenges
The technical challenges of improving Ambox templates have a few areas of concern. To summarize, templates are inconsistent.
The output of {{Ambox}} is inconsistent in many ways.
- across languages
- across different types of templates
- location inside the article
There is no consistent machine-readable way of extracting the template content.
Areas of concern
- Text length
- The length of many Ambox messages are very long and not suitable for mobile devices.
- Desktop specific HTML
- Templates using {{Ambox}} make heavy use of tables for layout, which doesn't work on mobile.
- Language specific HTML
- The {{Ambox}} HTML output is different across languages, making it nearly impossible to consistently extract the message content and repurpose it for mobile.
-
Very long text on mobile article notice -
"More citations needed" template on English Wikipedia -
"More sources needed" template on French Wikipedia -
Notice template on Portuguese Wikipedia -
Generic article notice on Italian Wikipedia
Proposed technical solutions
Use the TemplateStyles extension to create mobile friendly template CSS
To be discussed for future iteration.
Use Page Content Service to display page issues
https://phabricator.wikimedia.org/T172002
Modify MobileFrontend/Minerva CSS to accomodate existing template markup
This solution would work within the existing template structure and probably only on English Wikipedia.
Many {{Ambox}} templates contain a CSS class that marks text as hide-when-compact. Using CSS, we could hide this extra text and only present users with the "summary" text. This is the same text that is currently presented in the page issues overlay. Many templates, notably deletion templates, do not have this CSS class. The issue of text length would have to be addressed on a per template basis.
Here is a demo of what modifying the CSS might look like.
Add structured HTML attributes to Ambox templates
This solution requires modifying templates.
As mentioned before, the HTML produced by the {{Ambox}} templates varies. It can change at any time and in any language. We shouldn't hinder these changes, but make sure that contributors can adapt them to new contexts, such as mobile.
Adding semantic attributes to the {{Ambox}} HTML could provide us with the necessary "hooks" to extract the Ambox content. We could then present it in an appropriate way on mobile. We could provide a standard set of attributes that template editors can insert into their templates. This would work regardless of the template used or language. These attributes could be "machine readable" which means they can be parsed or extracted. For example like the Page Content Service for consumption by the mobile apps. These semantic attributes would be invisible to readers. They would not effect the appearance of existing templates.
Semantic HTML attributes fall into the category of "microformats". The idea is to extend HTML with the ability to express very specific types of data, in a way that is machine readable. We could extend the Ambox HTML with attributes to identify the templates as "message boxes" with various properties. These properties could include things like priority, date, and actionable steps. This information could then be parsed. Consumers could include the mobile site and apps. Also presented to end-users in a mobile friendly fashion.
We could create our own standard, but there already exists a W3C recommended specification called RDFa. RDFa standardizes on how to use attributes to add rich meta-data to HTML.
RDFa defines rich data structures as "vocabularies" or sets of properties that define an data structure. We can define a custom vocabulary that describes the properties of a "message box", and use those properties in the HTML attributes.
As an example, RDFa uses HTML attributes such as vocab, typeof, property, and content. The following example shows how to add them to templates.
<div vocab="https://meta.wikimedia.org/ContentSchemas" typeof="MessageBox" name="Citations needs">
<span property="summary">
This article needs additional citations for verification.
</span>
Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.
<small property="date" content="2017-06-01">
(June 2017)
</small>
<a property="action" href="/wiki/Help:Maintenance_template_removal" title="Help:Maintenance template removal">
Learn how and when to remove this template message
</a>
</div>
Inventory of mobile-friendly page issues
The changes proposed above will improve most page issue templates. Two kinds of templates will receive specialized treatment. One are templates with defined short descriptions (those that contain text marked as "hide when compact"). The other kind are templates with defined ambox type. These templates will display the shorter more mobile-friendly form of the text. They will include visual indicators for the severity of the issue.
Hide when compact
To determine coverage for these changes we performed an inventory of templates. We looked at templates that contain text marked "hide when compact". The following were our results. Further documented in phab:T189132.
Method
An extensive inventory of "page issue" templates was taken across a variety of languages using the MediaWiki API and this script. The script parses templates that belong to a specific category. For example, on English Wikipedia it parses all the templates that are members of the category Category:Article message templates. The templates are then rendered in an HTML table with CSS added to see which can be made compact or not. A sum of the compact templates is also generated.
Disclaimer
This method works for languages that actually have a category for all "page issues". Not all languages do, some only have more specific categories. Only the top-ten Wikipedias were used in this report.
Project Implementation
Design goals and cross-wiki compatibility
High level design goals:
- Compact: page issues come in a wide variety of lengths, but screen space on phones is limited and we want to respect the reader's desire to access the content, so we decided the banners needed to be compact (two lines of text, or roughly 62px tall max). We hope that in turn template editors will further refine page issue descriptions to (more) succinctly describe the issue.
- Easy to understand: there are over 300 unique page issues on English Wikipedia, many with beautiful, custom icons. While editors may be familiar with each individual issue we wanted a way to simplify them for readers. In a Vector-like manner we chose severity (i.e. type) as the main concept to communicate visually using 5 generic, globally understandable icons.
- Easy to engage with: we think that it's important for readers to have the opportunity to easily access additional information about a page issue, and perhaps even be intrigued enough to try fixing it. While many page issues have various links embedded in their descriptions, we wanted to focus the reader's attention on a single action: Learn more. The consistent placement of the blue Learn more link makes it easy for anyone to further engage with an issue they discover.
Here is an example of an ideal page issue banner:
Compatibility across Wikis:
Since page issues are not structured in any way across the hundreds of Wiki projects we knew there would be a large compatibility challenge. In other words, there was no simple way for us to render all page issues from all projects in a consistent manner on mobile devices that met our design goals listed above. Since we are most familiar English Wikipedia we decided to anchor the rendering code around how English page issues are formatted. Page issues on some other Wikis work quite well out of the box (e.g. Japanese, Chinese), however for other projects reformatting of page issue templates will be necessary in order to get them to play nicely with how we render them on mobile.
Here are a few examples from various Wikis that explain what kinds of fixes we recommend:
| Italian | Polish | Spanish | Russian |
|---|---|---|---|
|
|
|
|
| We recommend removing the colored border to the left of the banner, and instead using the provided icons. | We recommend differing the icon to the one provided, and constraining the width to two lines of text. | We recommend differing the icon to the one provided, and constraining the width to two lines of text. | This is almost ideal, however the text need to be constrained to two lines. |
| link | link | link | link |