Wikimedia Apps/iOS Suggested edits project

Overview

Alt Text Experiment

Background

Suggested edits presents opportunities for small but vital contributions to Wikipedia. The aim is to raise awareness of the various ways people can edit Wikipedia while making quality contributions easier and more accessible.

Suggested edits was initially introduced on Android , and was later incorporated into the Growth experiments for the newcomer homepage . The iOS app was initially created as a reading focus app. Over the last year we focused on adding communication features. This year's annual plan created the perfect opportunity to bring Suggested edits to the iOS app.

Objective

By July 2024 we aim to release a Suggested edits task that:

• Increase unreverted mobile contributions from iOS by 10%
• 2,000 articles enhanced using Suggested Edits

Hypothesis 1 alt text suggested edit proof of concept

We believe we can achieve the objective above by releasing a suggested edit task focused on adding alt text to images.

Presently, 50% of images on Wikipedia have captions, 10% of images have alt text, and only 3% have effective alt text. Users of the Wikipedia apps can go into settings when they are in low bandwidth environments and choose to not load images. If an image has alt text, the user in the low bandwidth environment will be able to read the alt text and get an idea of what the image is about. The iOS Wikipedia app was Apple's editor's choice in 2017 as a result of the app's user accessibility features. Accessibility is an important factor to our design and development process on the apps teams, so a task to fill the gap in images with quality alt text is fitting for our team.

Due to this concept being a new suggested edit type, it is important to start with a proof of concept that will allow us to evaluate if the task will be effective.

Fundamental requirements for proof of concept

• Entry point in Settings
• Prominent guidance for writing good alt-text
• Users ability to get context about the image from the article
• Users ability to access relevant metadata (can take user to Web)
• Detection of which images do not have alt-text
• Ability to store the responses we get to evaluate if they are good alt-text
• Users should be able to give feedback about the feature over all

Nice to haves

• Positive Reinforcement
• Exposing to users how many tasks or edits they've completed
• Ability to publish alt-text to Wikipedias
• Limits on number of task can be completed in a given day or session if it is being published to Wikipedia
• Entry point is easily discoverable
• Do not allow users to copy and paste in image caption
• Users should be prompted provide feedback about the feature
• Suggest Alt-text (think Machine Assisted Article Descriptions)
• Can playback what was written in preview

User stories

• As a participant at the GLAM Conference in Uruguay, I want to test out an alt-text Suggested Edits Task, to get a concrete idea of the concept and provide meaningful feedback.
• As the Director of Product, I want concrete proof an alt-text Suggested Edits would increase edits on iOS, without creating a burden for patrollers.
• As a accessibility specialist, I want to evaluate the quality of alt text submitted through Suggested edits tool, so that I can advise if it is a feature that would be helpful or harmful to low vision users

Consultation Strategy

• November 2023 - During the 2023 GLAM conference we will have attendees test the proof of concept
• December 2023 - Partner with accessibility specialists to evaluate submissions through the proof of concept
• January 2024 - Decide whether to build the full featured version of an alt text suggested edit task or pivot to another suggested edit task type

Prototype

We created a prototype of an Alt Text suggested edit, and editors were invited to test the feature at GLAM Wiki 2023 and provide feedback. The prototype is no longer available for use, but we plan to continue working on an Alt Text experiment, and you can follow along on the project page.

We collected feedback through this survey. You can read the guidance for adding alt-text via this feature on the Suggested Edits FAQ page.

• 
  Suggested edits can be accessed via 'Settings' in the 'Explore' tab.
• 
  On the 'Settings' screen, tap on your 'Account.' Note: The list item is only visible when you’re logged in.
• 
  'Suggested edits' can be found in your 'Account' view. Tap it to get started!
• 
  Please read the onboarding screen for the 'Alt text' task carefully.
• 
  Tap 'Suggest alt text for image' to start. Options include 'Learn more' for the FAQ page, 'Send feedback' for a dialog, 'View image details' for the Commons file page, 'Read the full article' for the article page, and 'Skip suggestion' for the next image.
• 
  Commons file page: Check the image details before adding alt text.
• 
  Article page: To get full context when suggesting alt text to images.
• 
  Tap 'Describe this image' to enable the edit mode/keyboard on this screen.
• 
  The preview screen lets you see your alt text suggestion below the image.
• 
  After suggesting alt text for an image, it’s confirmed with a message at the bottom of the screen and the next suggestion is displayed.

Decision Matrix

• If less than 45% of edits are scored as a 3 or higher then we will pivot to a different suggested edit.
• If 46%-70% of edits are scored a 3 or higher we will improve guidance or use AI to better assist users. If 71% or higher of edits are scored a 3 or higher we will scale feature.
• If average edit per unique user is under 3 we will pivot to a different suggested edit. If it is 3-6, we will consider interventions to reduce friction. If it is 7 or higher we can scale
• If user edit through feature a second day we should proceed with improvements and scaling
• If less than 55% or less of users are satisfied with feature we will not scale without making changes
• If we do not have at least 50 people try the feature we will do direct outreach to gain more edits
• If more there are more than 30% of users find the task too difficult we will create an intervention to reduce difficulty before scaling. If 80% or more users find the task too difficult we will consider abandoning depending on supplementary responses
• If the skip rate is 20% higher than image captions on Android we will consider pivoting to a different suggested edit unless evidence points to an intervention that could reduce this rate

Key Qualitative Insights from GLAM Participants

• It would be good to add a bit more context: show information about the specific context of the image in the article (e.g. I had the article 'wheat' with an image of rust on wheat leaves, while I was unsure if I needed to mention the rust stains in the alt text or not). It would also be good to just mention the name of the article itself in the alt text editing screen - a few times I forgot the article and needed to click back.
• Avoid including paintings and other artwork
• Add a field that suggests things via depicts statements
• I'm still unsure whether I provide alt text that is actually useful for the target audiences. "keep it simple" is a very generic tip, maybe a short video with a bit more detailed guidelines (how much do you take the context into account?) would be welcome
• I would add categories that help me place the element in the image in case I don't recognize it. Although it is a random suggestion, it may not separate exactly what is what you are going to describe.

Quantitative Insights from GLAM Participants

• Only 9% of participants found the task difficult
• All participants found the onboarding and guidance somewhat to very helpful
• 54% of participants stated they would try the feature again if no other changes were made to the prototype
• All participants stated they would try the feature again if changes were made to the prototype
• We did not reach our goal of having at least 50 people try the feature, even with follow up with affiliate groups in Latin America.

Next Steps

We believe we did not have at least 50 people try the feature because it required people to download a separate app. Of the people that tried the feature, there were positive indicators that the tool could be successful with improvements. As a result we will run a scaled experiment in the production version of the app.

More details on the scaled experiment can be found on the Alt Text Experiment project page.

Hypothesis 2 Add an Image Suggested Edit

We have seen positive results after releasing the Image Recommendations (Add an image) suggested edits task in the Android App , where more than 2,000 articles were improved in a 30 day period.

Our hypothesis is that adding Image Recommendations as the first suggested edit available in the iOS app will increase the amount of mobile contributions made in the iOS app by 10%, improving over 2,000 articles.

For guidance on using the feature, please see our Suggested Edits FAQ page.

Feature Requirements

• Entry point should reside in the Explore feed (not the first card- under top read) and a permanent home based on navigation work
• There should be a tooltip, notification or some sort of intervention to draw attention to the task for qualifying editors
• Users should have an option to add a caption and alt-text on the screen but it should not be prompted
• There must be a way for users to report issues with the feature, it should take them to the support email (pre-populated)
• Users should have an entry point in the task to the FAQ page (via overflow)
• There should be some sort of way to ensure users aren’t continuously hitting yes in a short (less than 5s) span of time to guard against bad faith edits
• There must be onboarding to the feature
• Users should be able to access full article (via read more) and image metadata
• Do not allow users to skip without providing some feedback (Yes, No, Not Sure)
• Preview the edit and provide an edit summary before publishing
• Selecting No triggers a survey (reasons should have parity with Android)
• Allow users to zoom/pinch on image
• Considerations for images running out or task not being available in certain languages
• Non-intrusive guidance for adding captions and alt-text (can take user out of the app)
• Accessibility in accordance with WCAG
• Interface must be translated in target languages prior to release
• Feature should not be available to users with less than 50 edits in the language they are editing

Nice to Haves

• Users able to filter by topics
• Subtle positive reinforcement

Target wikis

While we welcome feedback from everyone, we are especially interested in hearing from:

• Spanish, French and Portuguese speakers in the Americas (North, South and Central) and Caribbean
• Chinese speakers in North America

In our user testing, we are committed to fostering a balanced and diverse group of testers.

User Stories

• As a Wikipedia iOS app user with a small screen and inconsistent internet, I would like to evaluate images and determine if they should go into an article, to contribute to Wikipedia articles that are in need of more content.
• When I am using the app, I want to be able to add images with one hand to many articles on my mobile device, so that I can be productive while riding the bus in Bogotà, and listening to music.

How will we know we are successful?

Our leading indicators will be captured after 15 and 30 days:

• Image rejection rate does not exceed 29%
• Edit over-acceptance rate (never skip or reject recommended images) does not exceed 35%
• Task completion rate is not below 30%
• Revert rate does not exceed 18%

Validation

• KR 1.1: 1,500 articles have images in a 30 day period
• KR 1.2: Average at least 6 edits per day per unique user
• KR 1.3: 10% of eligible Suggested Editors try image recommendations task
• KR 1.4: 30% of those users complete the task again on a separate day in a 15 day period
• KR 1.5: Accept rate does not deviate from Mobile Web or Android by more than 10 percentage points
• KR 1.6: 10% increase in unreverted edits from iOS in the main namespace

Guardrails

• KR 1.1: Less than 5% of users report NSFW or offensive content
• KR 1.3: All users spend at least 10s evaluating a task before publishing it
• KR 1.4: Bounce rate does not exceed 30%
• Bounce rate defined as users that click Yes then abandon the flow before publishing
• KR 1.5: At least a 50% task completion rate
• Defined as users that click on Add an image as a task, and actually clicks Yes, No or Not sure (interact with the feature)
• KR 1.6: Revert rate does not exceed 5%

Curiosities (nice to have)

• KR 1.1: Do these numbers differ by user tenure?
• KR 1.2: Are we seeing differences in these metrics when focusing on our target audiences as compared to the general population?
• KR 1.3: At what point in workflow are most frequent dropoff events?
• KR 1.4: How often are users adding captions and alt-text (distinguish between the two)?
• KR 1.5: How often are reverted edits captioned?

How to Follow Along

We have created T355270 as our Phabricator Epic to track this work. We invite you to collaborate with us there or on our Talk Page. We will provide periodic updates on this page as we make progress.

We plan to release the Add an Image suggested edit to all Wikipedias by April 2024, and it will be available to users who have more than 50 edits.

Updates

August 2024

• We began work on the Alternative Text experiment. We added a new capability in the app: developer settings. This will allow us to move quickly and deploy things behind feature flags more in the future. Tasks can be viewed in the sub-epic: T357437
• We completed analysis for iOS’s “Add an image” feature 30 days after deployment. T362835 & T371906
• We calculated the total images added through Apps Image Recommendations on both Android and iOS (more than 20,000!) T372954

July 2024

• We added a warning message if users accept an image recommendation very quickly T364049
• We added an end date to our new feature announcement for Image Recommendations. T364397
• We investigated the difficulty of adding image recommendations edits automatically into common infoboxes for English Wikipedia T359134

July 2024 - Results after 30 days

Analysis has been completed for the first 30 days after Add an Image was released into the iOS app. 4/6 Key Indicators were met or almost met, and 3/5 Guardrails were met.

Validation:

• KR 1.1: 600 articles have images in a 30 day period
  • Met: We saw a total of 1707 unreverted edits that improved articles with images in a 30-day period, across 24 different wikis. We were almost on par with the 2118 edits that the Android Add an Image task saw in a 30-day period.

Top 5 Wikis by Number of Add an Image Edits

English Wikipedia	1091
Sindhi Wikipedia	256
German Wikipedia	104
French Wikipedia	94
Italian Wikipedia	65

• KR 1.2: Average at least 6 edits per day per unique user with 50+ edits
  • Not met: The average edits/day users with 50+ edits was 2.6 edits/day, significantly lower than Android’s average edits per editor per day of 13.8. When we zoom out to average edits/editor over 30 Day period, we see an average of 7.39 average editors per editor. This indicates that users are not using the Add an Image feature repeatedly on the same day, but instead using it briefly and then returning another day to complete more edits. Stickiness of the feature could be improved by adding positive reinforcement.
• KR 1.3: 10% of eligible Suggested Editors try image recommendations task
  • Almost met: 9.7% of eligible editors (those with more than 50 edits) tried the image recommendations task
• KR 1.4: 30% of those users complete the task again on a separate day in a 30 day period
  • Almost met: We saw 24.8% of users return to complete the task again on a separate day. This metric may have been affected if users saw repeat suggestions in the feed.*
• KR 1.5: Accept rate does not deviate from Mobile Web or Android by more than 10 percentage points
  • Somewhat met: The accept rate, the proportion of users who opened the task, and clicked “Yes” to accept the image suggestion, was 19.2%. This was more than 10 percentage points lower than the comparable rate from Mobile Web (38.8%). We also observed a higher bounce rate of 26.4% at the point of image accept/reject/skip for users on iOS compared to Android’s 19.7%. This discrepancy could be due to the technical issues with the image suggestion algorithm*, iOS users in this period may have been more likely to reject an image because the image had already been added to the article, or bounce away from the task due to seeing images suggested that were not necessary. Once iOS users accept the image, they had a similar level of success to android in publishing, at 72.9%.

Event	iOS	Android	Mobile Web
Image Accept -> Edit Success Rate	72.90%	68.2%	23.50%
Image Suggestion Accept Rate	19.20%	15.9%	38.80%
Reject Rate	7.00%	4.2%	28.10%
Skip/Not Sure Rate	47.40%	60.2%	12.70%
Bounce Rate	26.40%	19.7%	34.40%

• KR 1.6: 10% increase in unreverted edits from iOS in the main namespace
  • Met: Edits made in the iOS app main namespace increased by 8% compared to a 3-month average, and 19.2% compared to the previous month. The smaller effect when averaged over 3 months may have been due to the release of Native Editor in mid-February, which caused an uptick of edits in March.
• KR 1.7: 10% increase in unreverted edits from iOS in the main namespace
  • Met: Edits made in the iOS app main namespace increased by 8% compared to a 3-month average, and 19.2% compared to the previous month. The smaller effect when averaged over 3 months may have been due to the release of Native Editor in mid-February, which caused an uptick of edits in March.

Guardrails:

• Less than 5% of users report NSFW or offensive content
  • Met: Out of 796 rejected images, only 4, 0.5% of the total were reported as offensive,
• All users spend at least 10s evaluating a task before publishing it
  • Not met: 15.3% of edits were made in under 10s (3.02% of unique editors made edits in under 10 seconds). The time was measured from the point a user accepts the image suggestion, to the point of publishing. 22.3% of the edits had a caption, and 36.6% of these edits had an edit summary. We are adding an warning message to prevent users from moving too quickly through the flow, to ensure quality edits from this feature.
• Bounce rate does not exceed 30%. Bounce rate defined as users that click Yes then abandon the flow before publishing
  • Met: The bounce rate was 29.8%. This was comparable to Android’s bounce rate of Bounce rate of 28.7%.
• At least a 80% task completion rate. Defined as users that click on Add an image as a task, and actually clicks Yes, No or Not sure (interact with the feature)
  • Not met: The completion rate was 73.9%, lower than expected. This may have been due to suggestions being repeated.*
• Revert rate does not exceed 5%
  • Met: Revert rate was low, at 2.6%.

Curiosities

• Do these numbers differ by user tenure?
  • We saw that average edit counts increase with editors who have more edit experience. Users with 50-100 edits averaged 3.1 edits/editor, vs users with 501-1000 edits averaged 9.9 edits/editor.
• Are we seeing differences in these metrics when focusing on our target audiences as compared to general population? Average edits over 30 Day period.
  • We saw a slightly lower average edit/editor over a 30 day period of 6 in our target audiences (es, fr, pt, zh) than in our general population, were it was 7.4.
• At what point in workflow are most frequent dropoff events?
  • The “Add image details” (adding Caption and Alt text) step had the highest bounce rate at 40.5%. This was the percent of users who reached this step and then abandoned.
• How often are users adding captions and alt-text?
  • 80% of all edits had captions added, and 16.4% of all edits had alt text added. An estimated 15% of alt text entered was exactly the same as the caption
• How often are reverted edits captioned/not captioned?
  • We were curious if edits were being reverted due to not being captioned. We did not see an obvious relationship between caption & likelihood to be reverted. Of all reverted edits, 40% were uncaptioned, and 60% were captioned.
• How many reverted edits on English Wikipedia were due to the image not being inserted automatically into the infobox?
  • None of the English Wikipedia reverted edits had any indication that they were reverted due to not being inserted in infobox.
• Why were image suggestions rejected?
  • If someone rejected the suggestion, we had a survey asking them to provide a reason, and allowed for open text.
  • The most popular reason for rejecting a suggestion was "Image is not relevant".

Rejection reason	% of Respondents
Image is low quality	9%
Not enough information to decide	10%
Image is not relevant	61%
Image is offensive	1%
Other	13%

• Within "Other", themes among answers included:
  • 14 respondents who answered “Other” noted that either the image, or the image caption and information was in the wrong language
  • 19 people wrote in reasons related to infoboxes and an image already being in the article ("Already in article", "Added info box", "Infobox").

June 2024

• Thanks to volunteer developer support, the search function within “Add an image” has been improved. T363942

• After the release of “Add an image” suggested edit, we have results for our leading indicators after 15 days. There were 891 edits completed in 15 days using the feature.
  • LI 1: Image rejection rate does not exceed 29%
    • Met: We saw an image reject rate of 8%
  • LI 2: Edit over-acceptance rate (never skip or reject recommended images) does not exceed 35%
    • Met: Over-accept editors were only 3.1% of all editors
  • LI 3: Task completion rate is not below 30% (percent of editors who enter the tool and complete an action such as accepting, skipping, or rejecting an image)
    • Met: Task Completion rate was 72.3%
  • LI 4: Revert rate does not exceed 18%
    • Met: Revert rate of 1.5%

May 2024

• We released “Add an image” suggested edit to production! It is available to all users with more than 50 edits. Our work was coordinated in this phabricator epic: T355271

• We improved the edit summary screen. You can now easily add the page to your watchlist, or see if the page is already on your watchlist. The “Publish” option has been de-emphasized until the Edit summary is filled out to encourage users to add edit summaries. For those completing edits within “Add an image”, they have suggested edit summaries such as “Added image” and “Added image and caption” T354219
  • Left: Old Edit Summary page / “Save changes” screen
  • Right: New Edit Summary page / “Save changes” screen (example shown is view from “Add an image”)

Compare between the old and new edit summary page

• We learned that we can improve our default syntax for inserting images, and worked with Android to identify and implement a solution.
• We’re scoping the difficulty to implement automatic infobox insertion where possible for English Wikipedia.
• Solved the related bugs of:
  • Fixed issues with the explore feed toggle: T363601, T364038
  • Ensured users could scroll through all information on Add an Image halfsheet. T363929
  • Addressed landscape orientation issues T360037, T363742
  • Ensured that the users theme remained consistent throughout entire “Add an image” flow T363220

April 2024

• We continued development work on Image Recommendations and released to Beta testers at the end of April. We expect it to be available in production early May.
• We launched usability testing of our Alt-text suggested edit prototypes in English and Chinese.
• The Alt Text scaled experiment will be a part of the FY24245 Annual plan’s Key Result, Wiki experiences 1.2 for Constructive activation. Our hypothesis is: If we conduct an A/B/C test with the alt-text suggested edits prototype in the production version of the iOS app we can determine if adding alt-text to images is a good task for newcomers or should be reserved for experienced users.

April 2024, Special Mid Month Update

• The “Add an image” suggested edit has been released to beta testers! You can go to the Apple App store and download the Testflight app and join Beta Testing for the Wikipedia iOS app. The feature is available if you have more than 50 edits in your primary app language. You can open the feature from the Explore feed. A card called “Suggested Edits” should appear as the 2nd card in your feed.

Add an image demonstration

March 2024

• The “Add an image” suggested edit is in development, with a planned release of early May.
• We investigated different options when inserting images in wikitext, and decided to support inserting image wikitext after initial article templates. T356819
• We are reviewing the latest designs for the Alt-text experiment before starting usability testing.
• We heard feedback surrounding the alt-text experiment
  • A screen reader user expressed the importance of having quality controls and ensuring folks that are adding alt-text are properly educated. They also expressed faith in us partnering with accessibility experts for the project, which is being done with an organization via the GLAM team.
  • A volunteer expressed concern about not having proper alt-text guidance in their language.
• The team is investigating using the new hidden LINT rules created by the Content Transform Team for creating a feed of images that need Alt-text.

February 2024

• Designs are complete for the Add an Image suggested edit task Image iOS (T355271), and development will begin. The flow is similar to the existing Add an Image suggested edit in the Android App.
• We will now shift to designing an alt-text experiment, which will invite users to add Alt-text to an article’s images after adding an image, or editing that page.

January 2024

• We are planning to pivot how we approach our Suggested Edit hypothesis. Instead of an alt-text task as the first suggested edit, we are going to release Image Recommendations, which will serve as the first suggested edit across all platforms. Then we will run an experiment for alt-text that will help us evaluate that suggested edit type at a larger scale.
• Designs are in review for image recommendations. T355271

• 
  1. Announcement of image recommendation feature
• 
  2. Onboarding screen for image recommendations
• 
  3. Example onboarding tool tip for recommended image
• 
  4. Choosing no requests more information
• 
  5. Choosing yes brings up image details
• 
  6 Preview of image in article
• 
  7. Add edit summary and publish

December 2023

We connected with Editoras Lx and joined their editathon December 16th to gather feedback about an alt-text Suggested Edit task on iOS. In preparation we made our feedback form automatically appear after a first edit to increase the feedback we receive. Attendees were generally positive about the prospect of the feature. Experienced editors thought it could be a good task for onboarding new editors, but also a more casual way of contributing for new editors. Attendees of the edit-a-thon expressed that editing alt-text on the Web is currently difficult because it takes you out of context of the article. There was also an expression of interest in the feature being brought to Android.

November 2023

The team released and presented a working prototype at the GLAM Wiki Conference in Montevideo. There was generally positive feedback with requests for machine suggestions and encouragement to partner with an expert accessibility organization. To test the prototype that attendees saw:

1. Visit https://testflight.apple.com/join/ETjXlUyi to download the test build
2. Once the test build of the app is downloaded, tap the gear icon in the top right part of the screen
3. Click Login and sign in or create an account
4. Tap Account once logged in
5. Tap Suggested Edits
6. Review onboarding
7. Start Editing!

The prototype is currently only available in English, Spanish and Portuguese.

In addition to having attendees test the prototype, the team shared screens of a possible ideal version of the feature.

• 
  Entry point in the Explore feed
• 
  Log in screen for non-logged in users
• 
  Topics selection
• 
  Home for Suggested edits on iOS
• 
  Onboarding to alt text adding
• 
  Tooltips to onboard people 1
• 
  Tooltips to onboard people 2
• 
  Tooltips to onboard people 3
• 
  Main UI to add alt text
• 
  Sheet can be dragged down to get more context
• 
  Article can be explored in its entirety to get more context
• 
  Tapping the image or file name...
• 
  ... leads to the Commons detail page
• 
  Tapping "Add alt text"...
• 
  ... leads to the "Add alt text" interface
• 
  Tapping "Show alt text suggestions" reveals machine-generated alt text
• 
  Tapping one of the machine-generated suggestions fills the input field
• 
  Audio previews help to evaluate if the text has been written properly for visually impaired users
• 
  Tapping the play button reads the previously typed text out loud and highlights it word by word
• 
  After tapping "Next" on the previous screen, the edit is published.
• 
  Tapping view on the toast message before takes users to the diff view of their edit
• 
  Tapping back from the diff view takes users back to the main task UI

Our next steps are to evaluate the alt-text written by attendees that tried the feature, review the feedback through our surveys (349149) and use our decision matrix to decide our next steps for this feature.

Suggested edits- add alt text

Suggested edits- Participating in the user survey

October 2023 – Usability Testing

The unmoderated usability tests have been performed with this prototype on userlytics.com with 5 participants.

• 
• 
• 
• 
• 
• 

Protocol

Background

The apps team discussed the feature's flow and initial design concepts. In addition, a review within the design team has been completed. The next step is to create a clickable prototype with a protocol to get feedback from outside our team.

Introduction

We are excited to announce that the ‘Suggested edits’ feature is debuting in the Wikipedia iOS app. Suggested edits are a new way to edit Wikipedia on iOS. It presents opportunities for small but vital contributions, so-called micro contributions. The first ‘Suggested edits’ task will be adding alt text to images in Wikipedia articles. As we strive to create the best user experience, your feedback will shape the initial version of the feature.

Questionnaire and Testing Steps

• Where would you go to access ‘Suggested edits’?
• Please navigate to ‘Suggested edits’. If you have not done so already, please navigate to ‘Suggested edits’ by tapping ‘Settings’ (cogwheel icon at the top right), then ‘Account’, then ‘Suggested edits’.
• Please narrate your understanding of the ‘How to write alt text for images’ screen and this feature.
• Where would you go to see more information about ‘Suggested edits’?
• If you have explained your understanding of the onboarding screen, tap the blue ‘Continue’ button
• Please narrate your understanding of this screen. (the screen after onboarding)
• Where would you go to see more information about the image?
• Where would you go to see more information about the article?
• Please add alt text to this image.
• If you have not done so already, please tap the blue Suggest alt text for image’ button.
• Please narrate your understanding of this screen.
• Where would you look for guidance on how to write good alt text for images?
• Where would you tap to write alt text for this image? Please go ahead and do so. (The prototype does not have typing capabilities that’s why the input field is filled automatically once users tap it)
• If you have not done so already, please Continue to the next screen.
• Please narrate your understanding of this screen, then go ahead and publish your alt text.
• Please narrate your understanding of this screen, then dismiss ‘Published’ information at the bottom.
• Where would you tap to learn more about Suggested edits on this screen?
• Can you narrate in your own words what the ‘Suggested edits’ feature is and how you think it is used? Thank you for your time!

Participant Demographics

• Age: 21-37
• Countries: Philippines, United Kingdom, Brazil, United States, Mexico
• Devices: iPhone 6s, iPhone 8, iPhone 11, iPhone SE (2nd gen), iPhone 13 Pro Max
• Sex: 1x female, 4x male

Findings

• 1 user read through the onboarding screens but still was surprised that the task was around describing images.
• 1 user did not notice the image information and had a hard time differentiating between article and image information
• 1 user wanted to edit the alt text on the image info page
• 1 user did not initially see the CTA to add alt text due to its position on the page (under the scroll)
• ‘View examples’ is not tappable, but there is a task around this. This wasn't very clear for 1 user.
• 3 of 5 users were confused about the preview screen – they believed they had published after the initial submission on the editing screen
  • 1 user suggested being able to type the text directly on the first screen and then publish vs. going to a secondary screen, seeing a preview, and then having the text be published

September 2023

The team is exploring bring suggested edits to the iOS app. Below are initial designs of a suggested edit task that adds alt text to images.

The designs were based on a comparative review of the Growth team and Android team’s implementation of Suggested Edits:

Our team then iterated on those designs to develop a design prototype that we plan to use for user testing, which will be launched in October through userlytics.