Topic on Talk:Growth/Personalized first day/Structured tasks/Add an image

The Growth team has started thinking through a potential new project: "add an image". I'm pinging some community members who participated in a discussion this past May around the general idea of "structured tasks". That was a really productive discussion, and resulted in the Growth team deciding to build the "add a link" structured task workflow. As of now (December 2020), the Growth team is in the middle of engineering that feature, with plans to release in February. But we are also looking ahead to what might be other future structured tasks that would be more valuable to our wikis than simply adding wikilinks.

I've put up a project page around this "add an image" idea, writing out the thinking we've done so far. We know this is more ambitious, and with more pitfalls, than just adding wikilinks -- but we think there is potential here, to make use of the valuable media in Commons. That's why we definitely need to discuss with community members, who know more than we do about Commons and about illustrating Wikipedia. On the project page, you can see why we want to pursue this idea, and you see how we're thinking about the algorithm and trying to validate the idea with newcomers. There is an interactive prototype to try out, mostly geared toward showing the algorithm rather than the user experience.

The page also has a list of open questions, which is what I'm hoping community members can speak to. If you can weigh in, please add a comment or thread about any of the open questions -- or bring up your own open questions and ideas. Especially for community members who are active in non-English wikis, we want to hear how you think this idea might or might not work, given that most metadata about images in English.

And if you know of others who we should ask to weigh in, who have expertise that we need, please feel free to ping them! We'll be conducting similar conversations on Arabic, Czech, Vietnamese, and Bengali Wikipedia to gather perspectives from other communities.

Pinging @John Broughton @Sdkb @Zoozaz1@NickK @Nick Moyes @Galendalia @Barkeep49 @Pelagic @Czar @LittlePuppers @HLHJ

Catching up on pings and had some initial thoughts:

• In case you haven't already, have you tried asking WP editor communities directly what common tasks would (1) benefit from being semi-automated, (2) be friendly for new editors? I would be surprised if adding images was one of the initial suggestions. I would personally rate adding existing images to be an intermediate+ activity since the easy images have already been attached to articles, leaving harder or unclear matches, and we wouldn't want new editors automatically adding the wrong image to an article, leading to one of the press's favorite rags on Wikipedia: the wrong image showing in Google's Knowledge Graph.
• Relatedly, I would be curious if developing wikis would rather have newcomers add images or instead just set their infoboxes to import the default image from Wikidata. In my experience, I've seen a fair amount of the latter on smaller wikis and it is a more elegant solution than manually entering the file name in wikicode. Also relatedly, adding metadata to Wikidata à la meta:Mix'n'match could be interesting for newcomers... especially if it would automatically show up in multiple Wikipedias at once too.
• "50% of articles have no images" Just want to point out that this is often by design. I wouldn't expect a new editor to understand that we're not looking for any related decorative element, only for visualizations that illustrate something we want the reader to understand about the topic itself.
• "'To add an image' is a common response newcomers give on the welcome survey" I'd be curious if there is follow-up data on this. In my experience, new users generally are looking to add non-free images and don't yet realize that it's against WP ethos. I'd be surprised if they just wanted to add images in general.
• re: the open questions,
  • "Will newcomers think this task is interesting?" I think I said something similar last time, but do new users want to associate random images? It requires a lot of cognitive load. I'd be curious if this was instead posed as a suggestion when the user browses to an article without an image. I.e., "we think you can contribute to this article—would you have time to review images?" That way they'd be entering having read the article, knowing the context, and would only need to review the image's metadata.
  • I tried your prototype and felt like I couldn't confidently associate images and articles by the single view alone. Most of the details could be hidden. I mainly needed to see the target article and the file name (not even the description). The categories should have been useful but they're hard to parse in paragraph form. Another major difficulty was translation. If the title or category is in another language, usually I'd translate it on desktop and play around with it to make sure my translation was correct—that would just add complexity in this prototype. My two takeaways: (1) from being a Commons admin, I don't really trust Commons metadata for these associations, (2) it doesn't feel like a good fit for the format. I want to see so much more context from both the article side and the image side before committing an image to the infobox. Perhaps smaller wikis might be less stringent but I'm not sure what they'd gain from doing so if it creates more cleanup/inaccuracy work for their cohort of active editors.
  • Adding images to a section can be just as rewarding as adding images to an infobox
  • If I were doing this as a new editor, I'd only want high-quality matches. I'd be frustrated if I had to think hard/compare multiple pages for each image. Or more likely, I'd probably move on to something else with less friction unless I'm committed to completing a task.
  • For judging quality, if you're able to detect whether an image has artefacting/pixelation or is low-res and exclude those, I wouldn't pass the quality test to a new user
  • I wouldn't trust new users to write captions in WP house style en masse. More of an intermediate user thing.
  • The question of accessibility for non-English readers is a really good one. I wonder how non-English editors currently fare on Commons for desktop web, even before it's reduced to a mobile view for someone with less experience.
  • re: bias, it's going to reflect Commons bias, the images that were most accessible for upload and the images that have the best metadata (i.e., maintained by a user or a GLAM) are more likely to be paired as ready for linking. I think that's beyond your scope, though it could be interesting to run your prototype on a single data set, e.g., a certain type of media (photos, paintings, etc.) released from a specific region as another alternative to random image suggestions
• I went to follow along with the image trials but the Google Sheets do not appear to have public access, e.g.,

Okay—hope that's helpful! :)

Hi @Czar -- thank you for these thoughtful comments. It's really great to hear from a Commons admin. Here are my replies and follow-up questions:

In case you haven't already, have you tried asking WP editor communities...

Yes, this was something we talked about in the original discussion about structured tasks, particularly in this thread. A type of task that several communities mentioned as being particular desirable would be some sort of copyediting/spellchecking/grammar task. We talked a lot about how one of the challenges is to find spellchecking or grammar-checking engines in all languages. This is an idea that I'm still learning more about and figuring out what's possible.

In choosing which structured tasks to build, we're trying to take several things into account, including (a) whether the task is useful, (b) whether newcomers can do it successfully, and (c) whether we have a technical approach to building it. The technical approach part is what helped us decide to prioritize "add a link" and "add an image". For both of those, the WMF Research team already had ideas for algorithms that could get us there. But for a task like copyediting, the path is not as clear.

Relatedly, I would be curious if developing wikis would rather have newcomers add images or instead...

Thanks for bringing this up. We've heard about ideas and concerns around Wikidata infoboxes from @Mike Peel and others. It would be super helpful if you could check out the questions I posted for Mike, and add any of your own thoughts.

"50% of articles have no images"...

Good point -- we already saw in the user tests we've done so far that users are unclear on what makes an image appropriate for an article. They're not sure whether they are strictly judging on relevance, or also on quality, or on other things. We would need the feature to include a little bit of onboarding to give them guidelines around how to decide.

"Will newcomers think this task is interesting?"...

In terms of whether users will want to go through a feed of articles missing images, we think there are some users who will really like it. In the fifteen user tests we've run so far (we'll post the summarized results in the next couple weeks), several users have said things like, "This is fun" or "I could do this all day." And in building this task, we would also allow users to narrow the articles to their topics of interest, which we do for existing newcomer tasks, and we think that would make the work more engaging.

But the idea you're talking about is different. We call it the "entry point in reading experience": if someone is just reading Wikipedia articles, how about something tells them something like, "This article has 4 suggested edits you can make." Then they could discover the link suggestions or image suggestions, or any other kinds of tasks in the future. And presumably, they would be interested in the content, because they navigated to that article on their own interest. Yes, we think that would be a great thing to build, and we definitely want to in the future. This workflow actually exists in the Wikipedia Android app for the "article description" and "image caption" tasks. But we're starting out with the "feed" approach for two reasons: (a) this gives us more control over how many tasks we make available and which users can do them, and (b) it is technically less ambitious to build the feed because we only need to generate and store suggestions on a sufficient number of articles to populate the feed, rather than on all articles to prepare them to be discovered.

I tried your prototype and felt like I couldn't confidently associate...

Regarding the format, I definitely agree that the context of the article is really important to have. Although our prototype shows one image match at a time on one screen with a small amount of information, it may make more sense to adopt a design in which the user actually navigates to the full article in order to complete the image match. Our team calls this a "Concept A" design, as opposed to the "Concept B" design that you saw in the prototype. We're actually building "Concept A" now for "add a link" (in which the user goes to the full article to complete the task. To read more about Concept A vs. Concept B, see this section.

Could you tell me more what you mean when you say, "I don't really trust Commons metadata for these associations"? What goes wrong?

If I were doing this as a new editor, I'd only want high-quality matches.

I think this could be a good idea. Perhaps we could show only the highest confidence matches for a user's first tasks, so that they can quickly grasp the objective of the task and be productive. Then they could advance to lower confidence matches as they gain skills.

For judging quality, if you're able to detect whether an image has artefacting/pixelation...

The WMF researcher who works on the image algorithm does plan to add some computer vision elements in the future to filter out low quality images, or perhaps NSFW images.

I wouldn't trust new users to write captions...

I agree that writing captions might be the most challenging part of the task for newcomers. But I think, at least on English Wikipedia, that images in articles should almost always have captions, unless they are an obvious image in an infobox. Is that right? If so, I'm trying to think of how else the images could be appropriately captioned, if not by the newcomers who make the match.

I went to follow along with the image trials...

I don't think I can share the spreadsheets publicly, but the prototype you were playing with actually does contain 60 actual matches from the algorithm. I hope that can give you a sense of the sorts of matches that it makes. What do you think of them?

In reverse order

prototype ... contain 60 actual matches from the algorithm. ... What do you think of them?

I couldn't confidently associate many of those images and articles. I often couldn't ascertain exactly what connection the image had, whether its details were correct, and whether it fit in the article (before viewing it).

I think, at least on English Wikipedia, that images in articles should almost always have captions, unless they are an obvious image in an infobox. Is that right?

Ideally, yes, but not as a rule. Editors reviewing edits look at every edit and ask, "Is this an improvement?" Most editors would see an uncaptioned, relevant image as an improvement the same way that the most minor of typo fixes is an improvement. But there's nothing editors despise more than automated edits that do not have wide consensus as being an improvement (i.e., irrelevant images or irrelevant captions added en masse). As for how else they'll be captioned, at least on enwp, it's fine to leave that to another editor. :) If the tool was restricted to infobox images, captions would appropriately be completely optional.

Could you tell me more what you mean when you say, "I don't really trust Commons metadata for these associations"? What goes wrong?

(1) It's hard to tell the provenance of the image. Did a new user just upload this from Google Image Search and claim it as their own (fairly common problem with understanding copyright)? If it's public domain (PD), what was the evidence of prior publication? (If it just "looks old", it's reasonable that it was created recently to look that way. I had this question with the Josias image in the prototype—depends whence it came.) Copyright is difficult! With the possible exception that trusted uploaders and uploaders with photo EXIF data of what claims to be their own work is likely okay. (2) Categories (and even descriptions) tend to be added piecemeal over time. Is this the right Josias? Could this have been some other historical Josias with a similar name? Who added the description and when? Are the "facts" in the description right, or did someone just infer about the image what I'm about to infer myself? Usually I just need a lot more info to make a concerted decision if the image is appropriate. One category in the prototype is "Promptuarii Iconum Insigniorum"—do I need to know what that is before I decide its connection to the proposed article? Etc. (3) The prototype's fourth and fifth images have similar connection issues. This Apollo and Artemis plate thing is somehow connected to "Polemos" (a lesser known deity) but for me to know how, I'd have to visit the article that the app says is linked to see its context, because what if the Commons description just listed the wrong deities? And once I view its usage on the Lithuanian WP, it's used in a transcluded navigation template, not the article itself and now I'm down a rabbit hole far more complicated for mobile user than simply clicking yes/no/unsure. Similarly, on #5, knowing so little about whether this monument is representative of Aizpute Parish, there's no information on whether it represents the area so even if it is related, is it good for the page? Even as I do the same queue on desktop web (i.e., added context), image confidence tends not to be absolute and more like good enough, but on mobile without that context, doubt makes it even harder to take confident action on this queue.

With Concept A vs. Concept B, it's hard for me to visualize how that would work with images. In-line image placement within the article I can see with Concept A, potentially, but to compare an image's context, I would need to see more metadata, its placement within the recommended comparison, possibly refer between the two, etc.

A type of task that several communities mentioned as being particular desirable would be some sort of copyediting/spellchecking/grammar task.

I absolutely love this. Yes, yes, a thousand times yes. I would suggest, in addition to your a/b/c criteria, (d) the community wants or will tolerate it, because it could be a beautiful tool but if the community has no will to adopt it, they will just vote to ignore it, and unfortunately this happens with a number of well-meaning WMF-developed tools. Typo correction is the sweet spot of what new users already do as their first edits (I see new IP editors change typos as their first edits all the time), what is actually helpful (no one will say no to this, unless the new user just doesn't understand the difference between British and American English, but that's an edge case), and what suits the mobile context. I see how it misses the technical approach—your (c)—but something like typo correction with "entry point in reading experience" is :chefskiss: To most editors, I'd wager, the editorial value of a newcomer who fixes even just one typo while wikiwalking down a rabbit hole is far more valuable than an editor who adds 15 or more questionable images to articles, potentially creating more work for someone to track down and clean up.

Indulging my fantasy for one more minute, would spell check be necessary? MVP could just be asking users if they'd like to know how to correct typos, single-screen onboard them to double tap on the word, edit and click save, and all good! In my experience, newcomers don't necessarily want or need a queue—they just need an easy opportunity offered within context of something they're already doing. The editors we want (or at least one type) are those who read on the app, are bothered by a typo, and are given an easy recourse for doing something about it. Then it feels good, you get immediate reinforcement, might do it again, and have an entry point into more advanced edits. And hell, even if they fall out of the funnel, that one typo edit has a far better likelihood at helpfulness and stoking curiosity than pushing users to a queue, no? My take, at least :)

Hi @Czar -- thanks for getting back to me.

About the prototype: when I went through the 60 matches, I ended up considering about half of them to be "Yes" matches, with maybe another 10 being borderline (even without digging deeper into the article or Commons or anything.) I generally consider that rate of accuracy to be appropriate for a task like this. For example, some of the matches that I considered "Yes" were for these articles: "Alexander Nikolayevich Engelhardt", "Kollel", "Raymond J. Barry", "Original Vampires (The Vampire Diaries)", and "Gloria (given name)". I'm saying this because I want to make sure that I am on base here with my consideration of the strength of this algorithm, which is the lynchpin of this whole feature idea. You've already invested a bunch of time into helping us with this idea, so please feel free to say no -- but if you are interested and have time to go through the 60 images in the prototype, and mark down your judgments on them, it would be an invaluable data point for us to judge the accuracy of this algorithm.

About the Commons metadata: yours is a really interesting perspective of not trusting the Commons metadata. That makes sense, since you have deep involvement with Commons and you're aware of all that can go wrong with the metadata. Newcomers are probably not equipped to think critically about provenance. I was more thinking that if an image has survived in Commons for a while, and especially if it is used as a P18 or is used on another Wikipedia, then it is likely trustworthy, and therefore other users could just trust the metadata to decide what to do with the image. Sort of like how a scholar cites another's work and builds on it. I also think our design and the text in the feature would encourage the newcomer to be conservative, saying something like, "Only add images when you have high confidence -- when in doubt, skip!" What do you think of all this?

About Concept A vs. Concept B: we'll likely make a mockup of the Concept A version soon, and I'll post it and we can discuss.

About copyediting/spellchecking/grammar: I'm glad to hear you think this is a good idea. We actually talked to User:Beland about this, because he's built a bot to spellcheck all of English Wikipedia for Typo Team. The notes from that talk are here. For your idea of an MVP, what would initiate the workflow for the newcomer? When someone is reading an article, and they notice a typo, they already have the Edit button, but nothing explicitly invites them to tap it at that moment. I think that's where having some sort of spellchecker comes in -- it allows us to highlight or otherwise draw their attention to the typo with a call-to-action, making them realize they can join in. What do you think?

Sorry for the late reply, somehow missed the notification...

I would say that as a user mainly of non-English wikis, I would appreciate this task. In many cases it is not needed to speak any language to identify if the image is appropriate. It is a relatively easy task where a user needs more of a general knowledge than of Wikipedia experience.

I think there would be two major obstacles:

• Images that are misidentified. For instance, someone uploaded a picture saying it depicts some particular species. An expert later came and corrected that is misidentified, but fixed only a part of description. That expert would be annoyed if this image appears in the article again.
• Items which lack good quality images on Commons and where we have illustrations that really need context (e.g. a person on a group photo or a building on a large panorama). In the example of Nora Strømstad, it would be hard to understand whether Nora is on the left or on the right.

Other than these two (quite rare) cases the workflow should work pretty well.

To suggest articles we can look for some popular articles (pageviews, interwikis, incoming links) to avoid some obscure topics (and yes, this would likely mean bias, as an alternative we might at some point ask a user to suggest a region they want to work with). To select images, I think we need to define the concept of 'main image' (I would say infobox or top left / top right) and suggest images used in other wikis in this position, or images used on Wikidata. It is is very important to use this and not any other image: for instance, for a person an infobox image is most likely their portrait, but somewhere below in the text we can find something else, e.g. pictures of their works or of a house where they lived.

Overall I don't think it will be dull, some people would like it more than others (pretty much like not everyone enjoys copyediting) but I think it would be useful.

Thank for replying, @NickK; I'm glad to hear from you, especially with your perspective from a non-English Wikipedia.

About misidentified images: the point that metadata on Commons can be wrong has been brought up by some other community members. It sounds like the specific point you're bringing up is what if an image is removed from an article for a good reason, but then our feature suggests that a newcomer put it right back on? In that same vein, the problem might occur if we suggest an image in our feature, then a newcomer adds it, but then it gets reverted...and suggested again to some other newcomer. I think this is a very good point. Maybe there's a way for us to suppress suggestions like those by looking through the edit history.

About context: maybe our best way to address this is to make sure the users know they are supposed to be conservative in their evaluations, like, "If you aren't sure if this is the right image, or if it's unclear which part of the image is relevant to the article, do not push yes." Something like that?

I like your idea about asking the region the user wants to work in. I've heard from other communities as well, that they want to make sure users can focus their efforts on improving locally relevant content. Right now, the topics we use allow users to select regions like "Europe", "Africa", etc. They can be more specific like "Eastern Europe" and "Western Africa". But we don't have country-specific topics right now, like "Ukraine" or "Benin". Is that something that could make a big difference for your community?

Thanks for your feedback @Miller (WMF):!

Yes, my first point was on cases when good-faith editors keep adding a misidentified image to an article and experts keep removing it. Filter that the image was not removed sounds like a good idea, but please try to add a vandalism check.

I think there is actually a huge added value of a human review that humans can read. I would rather suggest adding a good caption if possible (maybe with a wording like 'if this image depicts two or more objects, use caption to identify which one illustrates the article').

On the last point, for example, I know user of Ukrainian Wikipedia who is an expert on Argentina. This is an unusual interest and I think we can use it. Not a high priority I think, 'South America' should also work in this case.

Thanks, @NickK. This all makes sense. What do you recommend we do for a "vandalism check"?

@MMiller (WMF): Sorry for the typo in your username (perhaps autocorrect on mobile). Perhaps checking whether an edit was reverted and/or if a user was blocked (either the one who added or the one who removed).

The ability to add "tens of thousands" of appropriate images to a language version of Wikipedia is certainly worth a fair amount of effort. And while this is only still in an early stage, you might think about a related structured task for editors who reach a certain point - for example, someone who has added 50 images. That add-on would be a structured task to add an image to the Wikimedia Commons. Because, obviously, at that point the editor knows the value of adding existing images; certainly there is a lot of motivation to take a photo, add it to Commons, and then add it to an article.

Other comments:

• For logic, I don't know if you have data as to what order you follow to decide which of three methods to try first, to get an image. It seems to me that the third listed - "Look at the articles about the same topic in other language Wikipedias. Choose a lead image from those articles" - is the best of all.
• It's not clear what happens if the user rejects an image as a match - do you keep offering images, or do you switch to another article? I hope that you give the user multiple tries - maybe three (four, at most), with different images.
• If an image comes from an existing article, and has a caption, why not try a translation as a starting point?
• Articles with no "File" wikilink have no images, yes?
• If the image isn't going to go into an infobox, then it's probably best in the second section, rather than in the lead section. But if there is an infobox in the article (is this easy to determine), you could offer the user the option of putting the image there or elsewhere.
• I don't really see the problem of "bias" in recommendations - I think you're going to run out of available images (per the logic you've set out) long before the issue will arise of the lack of images for topics unrelated to Europe and North America. And it's really the job of the community (and the WMF board) to address the lack of images related to topics in developing countries.
• If the images are being fed by the algorithm in the workflow, there really is little opportunity to vandalism, if by that you mean adding inappropriate images. (I hope you tag these image additions; if so, it's easier to see if they are being reverted to any great extent.) If you're worried that a user may inappropriately reject a lot of images, you might put a limit on the number of negative responses that a user can do (in a day?); when that is hit, the user doesn't see this option any more.

Hi @John Broughton -- thank you for the quick reply! More than one person brought up your first point (helping users to upload their own images), so I'll make a separate thread for that. I'll also make a thread about what priority to give the different image sources (Wikidata, Commons category, other wikis). For your other comments:

It's not clear what happens if the user rejects an image as a match - do you keep offering images, or do you switch to another article? I hope that you give the user multiple tries - maybe three (four, at most), with different images.

This is a question we were thinking about -- whether to offer multiple image options for the article. For some articles, the algorithm can only offer one option, but there are articles where there are many options. One concern we had about it is if we give the user multiple choices, they might be inclined to choose the "best" one, even if the "best" one is actually not a good match. We would want them to feel free to reject all of them. This is also a design challenge, especially for mobile: how to let the user see multiple choices? It's good to know you think we should keep trying to figure this one out.

If an image comes from an existing article, and has a caption, why not try a translation as a starting point?

Are you talking here about the Commons caption, or the local wiki's caption in the wikitext for the article where the image is placed? It sounds like your idea is to use something like Google Translate to offer users something in their own language if nothing is available -- is that right?

Articles with no "File" wikilink have no images, yes?

The challenge of identifying unillustrated articles turns out to be harder than it sounds! Lots of images end up in articles through templates. Sometimes, we would consider those images "illustrations", like in an infobox. But a lot of the time, the only images in an article are little icons, flags, or symbols, such that we would not consider the article illustrated. Here's an example of an article in Arabic Wikipedia that has a bunch of tiny flags. Those images end up in the article through templates. It's looking like a smart approach would know how to filter out those sorts of little images, perhaps by seeing whether they are used dozens of times in a wiki. If it's used dozens of times, it probably isn't the sort of image that we would consider an illustration for an article.

If the image isn't going to go into an infobox, then it's probably best in the second section, rather than in the lead section. But if there is an infobox in the article (is this easy to determine), you could offer the user the option of putting the image there or elsewhere.

The logic of where to place the image is definitely an important question, and your comment caused me to do some more exploration of how this might work. I was thinking it would be something like:

1. If there is an infobox with an image slot, put the image there, with the caption written by the user in the caption slot. 2. If no infobox with image slot, put the image in the wikitext under all the templates at the top of the article, but above the body of the article.

Do you think that would work? Do you know of other cases that would make this more complicated? That second step would put the image in the lead section, not the second section as you recommend. But I was looking at this article, for instance. Do you think that if the article had an image, it would be better in the lead or in the "History" section? I think the problem is that the user would be choosing a single image to represent the article as a whole, but the first section might not be appropriate for the image, i.e. it might not be a "history" image.

If the images are being fed by the algorithm in the workflow, there really is little opportunity to vandalism, if by that you mean adding inappropriate images. (I hope you tag these image additions; if so, it's easier to see if they are being reverted to any great extent.) If you're worried that a user may inappropriately reject a lot of images, you might put a limit on the number of negative responses that a user can do (in a day?); when that is hit, the user doesn't see this option any more.

Yes, I think this sounds right. And part of the idea with structured tasks is that we're giving newcomers guardrails so that they can make productive edits in a way that can't be too damaging to the wiki. I think the worst damage someone could do with the "add an image" workflow that we're talking about here would be to choose "Yes" on a bunch of images that are bad matches, or maybe writing bad captions for them. We would definitely be able to tag these edits to see their revert rate. For the sorts of edits we suggest to newcomers so far, the revert rate is actually slightly below the revert rate for newcomer edits overall.

• Regarding "whether to offer multiple image options for the article", I wasn't clear - I think your programming work would be much simplified - as would the task in front of the new editor - if you presented images (where more than one is possible) sequentially. If the editor says "Yes, good match" to the first, that's the end - they never see any additional images. If the editor says "No, not a good match", then the application says "Here's another, what do you think?" [And no, don't tell the editor this is "1 of 3"; if you do that, you have to allow the editor to go back, or say "Unsure"; that just unduly complicates things.]
• On second thought, I don't think it's worth overthinking where the image goes. One of the good things about Wikipedia is there are lots of editors who like to do cleanup, which in this case would probably be to create an infobox and put the image into that. In other words, getting a good image into the article is 90% of the value; placement is only 10%. [But you do have to think about sizing.]
• Regarding "does this article have an image", I would have thought (perhaps naively) that looking at the article wikitext (and specifically looking for the textstring [[File: ) would be straightforward. Apparently not.

Thanks for the quick response, @John Broughton.

• Yes, I see how your idea for multiple images could be easy for the user. I suppose my concern would be that someone might pick the first image in the queue, but the second one (which they wouldn't end up seeing) is much better. I'll talk about this idea with our team's designer.
• I like your point about how getting the right image somewhere into the article is 90% of the work. Could you tell me more about sizing? Would it not be sufficient to add all these images as "thumb"? What do you recommend?

Sizing as a thumb is a good default. I'd be really reluctant to introduce the concept of cropping, which involves alternative versions of images. There are lots of editors who care about images and can do cropping in their sleep, I'm sure (I'm not one of them).

I can see that "best of the bunch" seems tempting, but it's certainly going to complicate the interface, and if most of the time it makes little difference ... ? More to the point, this is really about (a) teaching new editors that they can add images, and (b) getting good images into articles; it's not a project whose goal is to get as many images as possible into Wikipedia articles.

Re sizing, the main times I adjust away from the default size are very wide images, which appear small as default (since thumb width is fixed) and often need to be made bigger, and very tall images, which appear big as default and often need to be made smaller.

Images aren't really my area of expertise, but there is definitely value to the idea.

Looking through the suggestions, there are a decent number of good ones, as well as some questionable ones; I think it would be beneficial to give some examples of what pictures are and are not good matches, and emphasize that it's fine to say no - if the system were perfect, there wouldn't be humans in the loop. I would like to note that in your prototype I came across one for a "given name" article, which per your guidelines should be excluded. As a result and some thinking I'm not sure exactly which articles should and should not have pictures - in looking through some given name articles, for example, I came across some without images, some with images of what the name comes from (eg. flowers), and some with images as maps of where the name is common. As such I do think it is difficult to make a call for what the best image for an article should be, or even if there should be one (I've been here for a bit, although I'm by no means an overly prolific editor, and I'm unsure on some of these - how are we to expect new editors to make the call? On the other hand, there's always the simple guideline of what you would want as a reader, which everyone has experience with.), but I'm not coming across anywhere where an image would be obviously harmful.

That's a somewhat unstructured stream of consciousness, but hopefully it's somewhat helpful. LittlePuppers (talk) 03:10, 16 December 2020 (UTC)

Hi @LittlePuppers -- thanks for these thoughts. I think you're right on with showing the users some examples. We hear something similar whenever we user test any feature: people say, "it would help me to see an example". That led us create an onboarding screen for the "add a link" feature that shows an example sentence about the moon (see image).

With respect to the question of whether an article (like a list of names) should have an image, I agree it can be a tough call. I think that something that could help is showing the user a preview of what the article will look like with their added image. Then perhaps they might have a moment where they say, "Oh, hmmm, this article is actually a list of a dozens of people named Jane, and so this image of one of the people at the top doesn't seem right."

Mockup of first of three onboarding screens for the "add a link" workflow, showing an example sentence.

This is definitely task that I think has a clear benefit. I do have some suggestions and questions. As an example, how would it deal with adding an image to the article Transport in the Republic of the Congo (assuming no image exists)? The wikidata item linked to in the article does not contain the commons category, though going through P910 gets you there, which might be a solution. As a suggestion in cases like these or other cases where the wikidata item is not complete, it might be a good idea to check if there are any categories on commons with the same name of the article.

I also think a clear distinction should be made between adding an image and uploading it; I could imagine some newcomers getting confused at the distinction. If an article has an infobox with an image parameter, the image should automatically be added to the infobox; otherwise it should be added to the top. A lot of the other open questions are best solved through testing. Looking at the phabricator task, it might be a good idea to limit the task to specific types of articles where the task is most accurate.

Another suggestion I have is to give images on foreign language articles on the same topic before ones from the commons category. Linking to commons categories has a higher potential for error (such as if it is populated only by subcategories, or the images are fairly unhelpful or miscategorized, or even if it contains hundreds of possible images that someone would have to sort through) than getting it from foreign language articles, where another human looked over it and decided one particular image was suitable for the article.

Hi @Zoozaz1 -- thanks for quickly joining the conversation. Regarding the "Transport in the Republic of the Congo" article, I think you're using that as an example of an article for which it's hard to find an image via the algorithm I described, right? That's right -- for most articles, there is no clear way to grab an image for them. They don't have a Wikidata image, they don't have a Commons category on their Wikidata item, and no other languages have images either. So for now, it just means that we'll only be able to offer a minority of articles for users to address. And to be clear, it's not that the users would navigate on their own to articles of their choosing and try to add images, rather they would be provided with a feed of articles for which we know we have recommendations, and they have to review them.

That's a good point about making the distinction between "adding" and "uploading". We'll need to think about that when we write the copy: if we encourage newcomers to do this task, and we call "add an image", they might think they're about to take a photo. Perhaps it needs to be something like, "add images from repository" or "add suggested images".

About where in the article the image should go: I wonder if there are some wrinkles in the idea to "add it to the top". For instance, the top of an article frequently has a bunch of templates, like maintenance templates, before the body starts. So perhaps it would be more like "add it below the templates and above the body. Can you think of other pitfalls or wrinkles around that kind of logic?

I'll start a different thread for the thoughts about the order in which to prefer the logic in the algorithm.

With all the markup and additions to many Wikipedia article, certainly adding it to the top can be challenging. I would say as a general rule of thumb doing it below the short description and any templates, but above the first word of the article, would work well enough.

Talking more about which articles the should be recommended, first of all an obvious approach would be not to recommend disambiguation pages. You've mentioned this, but there has to be a balance between recommending the articles with the highest pageviews without images (likely without images for a reason) and the lowest pageviews (with the least impact). It might be helpful to take a random sample of articles without images by pageviews to determine how much of a problem this is, and from that determine which pageview numbers newcomers should generally receive.

Thanks, @Zoozaz1. I agree about not recommending disambiguation pages. Other questionable types of articles might include:

• List articles: for instance, for an article like Julius (name), one might add an image of a bust of Julius Caesar -- or is it bad form to include an image of just one item in the list?
• Year articles: for instance, in an article like AD 112, one might add an image of Trajan, who became a Roman consul that year. I see that some year articles have illustrations, and some don't.
• BLPs: perhaps because of the heightened sensitivity around BLPs, we should exclude them just to make sure the wrong images don't end up in them.
• Featured and Good articles: people have illustrated these articles with care, and so we wouldn't want to casually change them (though most of them likely already have images).

What do you think of these? Can you think of other special cases?

There's a bit of a spectrum between disambiguation pages and list articles, and name lists are sort of in the middle. Disambiguation pages certainly shouldn't have illustrations as we want to get people off of them to their intended destination as quickly as possible without distraction, but nearly all featured lists do, and for Julius (name), it mentions Julius Caesar, so adding an image of him would probably be fine.

For year articles, there's certainly a potential to add images, but context is important—we'd want the Trajan image to have a caption that explains he became consul.

For BLPs, yeah, they're more sensitive, but they're also perhaps among the most straightforward, since if you find an image of the person, it should generally be added.

Excluding GAs and FAs definitely sounds like a good idea—if those are unillustrated, it's highly likely it's for good reason.

Thanks for the ping, Marshall! I have a bunch of thoughts on this which I'll throw out here.

First, looking at the broad picture of what's needed to improve the image situation on Wikipedia, I'd say the #1 thing is making it easier for people to upload images they've taken themselves. Despite the huge collection of images on Commons, it often happens that there's just no freely licensed image available. For instance, we've been seeking for months a decent image of the Huanan Seafood Wholesale Market (the Wuhan wet market), but the only image available is an uncompelling overhead shot of the street outside it. The Commons category has some images that look like probably copyvios (they're tagged as "own work" of a new Ukrainian user), and at one point someone uploaded a map that on second glance is promoting a conspiracy (that survived on en-WP for a few days and still survives on several wikis, from the looks of it), but there's just nothing else available. I'd guess that some Wikipedia reader somewhere probably has a photo of the market on their camera roll, and if they'd been prompted to upload it as aggressively as Google prompts me to add my photos to Google Maps we might have something on Commons. The point is that no amount of clever searching can find an image where one doesn't exist on Commons, and searching harder will only turn up problematic options. I'm not sure to what extent uploading one's own work could be made into a structured task, but that's where I see the greatest need, so perhaps it could get passed on to another team if it's not a fit here. Concretely, one thing needed is help revamping the local upload wizard for fair use files (until I recently implemented a stopgap, its workflow for anyone uploading their own work was to make them fill out the entire form and then say "actually, please restart at Commons", which is obviously terrible).

Zooming in a little, the main way that newcomers tend to interact with images is to come across an article on something they care about that's lacking one, say "I can find an image for that on Google!", and then try to upload something non-free (obviously not ideal, but also probably inevitable to some degree). I'm not sure it's safe to assume that this enthusiasm would translate to other pages that are just surfaced randomly. Emphasizing licensing needs to be up front, if only so that people understand why they're selecting from a limited pool rather than allowed to roam free on Google Images. And allowing people to choose a type of article rather than all articles would help, since I'm not going to find it that satisfying to upload images of subjects I don't care about.

Regarding the algorithm, it seems like it'll do a good job spreading images from one language to others, although from the long-term perspective that's just a stopgap to building a multilingual encyclopedia via Abstract Wikipedia. On the topic of globalization, I disagree on a wikiphilosophy level with classification -1. We're trying to document global knowledge, not just knowledge in our own language, so ko:나비 shouldn't prioritize an image of a Korean butterfly just because most Korean-language readers are from Korea. Maybe small wikis see this differently, and certainly de facto it tends to happen, but on en-WP we at least aspire to not give additional weight to English-speaking countries at pages on global topics (see w:Template:Globalize).

I agree with Zoozazz1 that Wikidata/other languages should be prioritized before Commons categories. As with the Ukrainian example above, though, there's some risk to this, as I suspect some of the smaller Wikipedias struggle with enforcing copyright, and we don't want to spread their errors into other languages. When there's desire for an image that isn't available, that creates a kind of pressure that tends to get filled with improperly licensed media, and that's hard enough to counteract even when it doesn't have an algorithmic boost.

Regarding the difficulty of the task, I think it could actually end up being fairly tricky depending on the quality of the metadata available. For instance, if it's a group photo that includes the subject of a page, then the editor will need to identify which person the subject is, something that's not always in the caption, and then crop to them, which is a whole additional component. And if the title/description/caption is in a different language, that massively magnifies the potential for error; I'd suggest avoiding that until this is up and running smoothly with non-translated suggestions.

Regarding determination of articles to surface, I'd suggest taking into account pageviews, as we want this feature to have impact for readers, not just be wasted on essentially unread pages. However, that gets tricky, since the most-viewed unimaged pages are presumably that way for a reason (this is again the problem of a lack of images creating pressure to add inappropriate ones).

I'll stop there as I've already written a lot, but I hope the above is useful, and feel free to ask any follow-ups.

Hi @Sdkb -- thanks for quickly posting your thoughts! It's all very useful. I'll start a separate thread to talk about the idea of how to help users upload images themselves, and another to talk about the algorithm. About your other points:

Emphasizing licensing needs to be up front

This is a good reminder that an important part of structured tasks is teaching newcomers about how the wiki works, so that they can graduate to more ambitious tasks.

And allowing people to choose a type of article rather than all articles would help

Yes! I think that we would want to use the same topic modeling for this task as we've used for other kinds of suggested edits, in which a user chooses from a set of 39 topic areas (Performing arts, Physics, Transportation, etc.) and then only sees suggestions in that area. 75% of newcomers choose some topics, giving us some signal that people like to do this narrowing to topical areas.

just a stopgap to building a multilingual encyclopedia via Abstract Wikipedia

I'm also excited about Abstract Wikipedia, and glad to hear that you're following along with that project. For what it's worth, we're in touch with the Abstract Wikipedia team, and hopefully that will help us see opportunities in the farther future.

On the topic of globalization, I disagree on a wikiphilosophy level with classification -1

That's good to know. I haven't heard that perspective as much. I think the globalization issue is one that we would think more about down the line, and so I want to defer figuring that out until a bit later. We also don't really know how often these "-1s" would come up in practice. In our tests, they were very rare.

For instance, if it's a group photo that includes the subject of a page, then the editor will need to identify which person the subject is, something that's not always in the caption, and then crop to them, which is a whole additional component.

I think you're bringing up an important larger point here: we need to be mindful of image policies on the various wikis. It sounds like you're saying that on English Wikipedia, images are expected to be cropped to the specific person? It we look at the example I posted in this section on the project page, the article is about an individual, but the suggested image contains two individuals. Would you say that needs to be cropped? Or might it still be a productive contribution without the cropping?

Regarding determination of articles to surface, I'd suggest taking into account pageviews

Yes, this is a question that has come up for our team a lot. On the one hand, we want to help newcomers to be impactful. On the other hand, they're new, and so maybe we should not be asking them to try their first edits on the highest-visibility articles. Perhaps there's an idea in here about preferring higher-visibility articles for users who have more experience.

One other thing I'd like to ask you -- I think that the people who usually follow along with Growth team conversations are engaged because they care about newcomers. I was thinking it might be good to hear thoughts from people who are image experts on the wikis. Would you recommend anyone I can ping to ask if they want to weigh in?

Thanks for those replies!

Regarding cropping, the w:MOS:IMAGES guideline includes A biography should lead with a portrait photograph of the subject alone, not with other people. This isn't strictly enforced (particularly outside of infoboxes), though, and a group photo would be better than nothing, especially if it's clear from genders who the subject is or if the caption identifies their position. Regarding the example, yes, given that there's enough space in that photo to crop it naturally, I would do so if I were adding it to a page.

Regarding image experts, Rhododendrites comes to mind as someone I respect who is experienced with images. They might have thoughts or know someone else who would. Many image experts hang out at Commons, so you might want to post to the Village pump there.

Got it -- thanks, @Sdkb. I sent a message to the user you recommended.

About cropping: perhaps rather than ask users to do the cropping, there could be a way to flag the image as "needs cropping", so that they could add it to the article, but someone with more expertise could come along later to crop it.

Yeah, that might be a good happy medium. Cropping isn't too difficult if one has the CropTool extension enabled, but since it creates a new file (see this wishlist item), it can get a little messy if anyone messes up.

Strong second to that, Sdkb, MMiller (WMF). See also Commons:Commons talk:CropTool#PDF quasi-extract for a detailed step-by-step description of what the tool should do. Wikisource has... thousands? of scanned books with transcribed texts but without extracted images. Where the images should be, it says "An image should appear at this position in the text". Scroll through Wikisource:Ten Books on Architecture/Book V for a varied example; the missing images are all implicitly tagged as needing cropping. Unextracted, still embedded in a scanned page of text, these images are unusable on other projects, and often unfindable.

I've spend many tens of hours manually extracting images, not only from scanned book pages, but from PDFs where the image files are already machine-extractable, and just need a human to check that the autogenerated name is sane. The latter are often academic articles. It's very frustrating.

So:

1. Cropping images out of scanned book pages

2. Guiding an automated extract of images in machine-readable PDFs

Both could readily be done by newbies. Both would be genuinely useful.

@HLHJ -- thanks for pointing out these ideas, which sound like good structured tasks for Wikisource content (but possible could be surfaced in the Wikipedia context). I'm going to add them to the full list I keep of structured task ideas.

MMiller (WMF), thanks. To some extent this is also a Commons task (and readily translatable). For instance, at Commons:Category:Japanese Homes and Their Surroundings (1885 book), you will find some of the line drawings from that book, which I began uploading because I wanted to use a few on Wikipedia. You won't find all of them, because I got too bored uploading them. I filed a request somewhere for automation, but as you can see, that was nearly a year ago. HLHJ (talk) 15:44, 6 March 2021 (UTC)

Thanks for the ping. I haven't read this whole thread yet, but did look at the page Growth/Personalized first day/Structured tasks/Add an image. My first thought on seeing the prototype screenshot was "oh, is that how you plan to do it?" I'd been thinking that add an image would involve showing 2–4 options, and the user chooses a preferred one.

I realise the final flow is still to be decided, but how would it affect the suggestion algorithm if you were aiming to pick four suggestions instead of one?

Hi @Pelagic -- thanks for joining in. @John Broughton also brought up the question of whether we would show just a single image or multiple options. It's true that the suggestion algorithm can sometimes suggest more than one option. Our first inclination was to only show one, because (a) it would be simpler to design/engineer/understand, and (b) we would worry that if the user has multiple options, they would be inclined to choose the best of the options, even if none of them are actually good enough. What do you think about that balance?

I just posted mockups of three potential designs along these lines. The first just shows a single image. The second shows multiple images all at once. The third shows multiple images in serial, and the user decides on a best one if they liked more than one along the way. Does one of these designs seem best to you?

It's a little hard for me to anticipate how likely people might be to choose an image even if none fit if the "multiple" option is chosen. You might have to do some user testing to figure that out. I do like the disclaimer text shown with the "single" option, but otherwise I lean toward multiple since image quality on Commons can vary a ton and it's a big plus to have a good image rather than a bad one. (In some cases, it could even be worse to have a subpar image than no image, since even if it's an improvement in the short term, long-term the existing presence of an image might discourage people from going to Commons and finding the better one, which might have happened otherwise.) I'm not really a fan of the "serial" option, since that seems like a bit clunky and would require more taps (when the goal is just to find one good image, asking people to rate multiple images is just extra work).

Linking Articles to Commons categories with a "commonscat" template might be more useful and automatable. I think we need to be clear that we are asking the user to judge whether the suggestion bot is out to lunch, or they will tend to assume that the suggested edit must be right because that's what Wikipedia's program says. So more "does this image category match this article?". HLHJ (talk) 02:50, 25 January 2021 (UTC)

@HLHJ -- thanks for joining in. Can you explain what you mean with the commonscat template? I'm not familiar with what it is and what it does.

I think you're right, though, about the user potentially trusting the algorithm too much. Some good news is that trusting this algorithm a lot is pretty reasonable, since it's only suggesting connections that have been made by other Wikimedians already via Wikidata or other Wikipedias. But we did notice in our user tests that some users would tend to give the algorithm the benefit of the doubt. For instance, they might have an article about a church, and then the suggested image might be of a church, but it might be hard to tell if it is the church from the Commons metadata. In a situation like that, the user might say, "Well, the image is of a church, so I'm assuming it's the right church." We definitely want them to be skeptical, and err on the side of "no" instead of "yes". This is something I think we'll be able to influence in the design and onboarding to the feature, by making it clear that that the user is not just supposed to rubber-stamp the algorithm; they're supposed to be quite critical. Does that make sense?

Template:@MMiller (WMF) en:Wikipedia:Template:Commons category. It makes the little box that says "Wikimedia Commons has media related to X" in the article. The Names of the article and the Commons category are often but not always identical.

Yes, that makes sense. "Is this correct?", rather than "Do this".

Why not make it a CAPTCHA? The CAPTCHA user-interface format makes it clear that you are being asked to make a judgement. The time is ripe; Google just started charging for their "reCAPTCHA" service. Now way back when dinosaurs roamed, reCAPTCHA was an academic project used to digitize public-domain books (it showed two non-computer-decipherable words, one of which had already been translated by a few humans, one novel, and judged correctness by consensus). When Google bought it in 2009 they gave the usual reassurances, but they used it to train proprietary algorithms; identifying address numbers on Google Street Views, and now training self-driving cars. Now they have also started charging larger websites to use it. A lot of the websites are not really keen on paying millions to Google. So if you can make an embeddable wikimediaCAPTCHA, you can get a LOT of people involved in editing. And once the CAPTCHA has concluded that they are human, it could thank them for contributing to WP and invite them to come edit an article related to the site they are on when they have time. Or invite them to make an account so they get their CAPTCHA work credited to it. This would get pretty much everyone on the internet saying "I contribute to Wikipedia sometimes".

Hi @HLHJ -- I think that the CAPTCHA idea is interesting, and I think it's come up a few times in our history. I found this example of when the idea came up a few years ago, and I think there are others. I think the broader idea you're getting at is whether the sort of "structured tasks" we're building now could be portable off Wikimedia properties. I think that's a potentially exciting future idea, and I'm going to bring it up with some of my colleagues.

Thank you; that Phab link is exactly my idea, much more succinctly expressed. Carpe diem! HLHJ (talk) 15:46, 6 March 2021 (UTC)

Hi @Marshall, sorry for the long delay in returning to this. I’m really impressed how you managed to fit the multiple thumbs into a single small screen for the Multiple mock-up. I seems more intuitive to me than Single and Serial options. The extra grey tile (x) tells one that no-picture is an option.

But if I was doing that in real life, I may have wrongly added the millefeuille picture to the vanilla slice article (similar shape and icing, different filling)! * The only thing that tipped me off was the file name. Would I have been more likely to notice that in the Single version and tap “not sure”? I don’t know.

If a user gets six cases in a row that are all skip, skip, ... skip, will they become discouraged? Could there be an option to “load more suggestions” before “skip”? The chance of surfacing something good with an alternate search strategy might be low, though.

( * Argh, there's a merge discussion about that right now. )

Hi @Pelagic -- thanks for the response! We've actually learned a lot since posting those different mockup ideas. We conducted user tests using a simple "single" image prototype, and one of the most important things we learned is that it is absolutely crucial to see the image metadata (filename, description, caption, etc) in order to make a confident decision. Just as you say, something might look like a vanilla slice, but not actually be one, and you need the metadata to tell if it really is a vanilla slice, or some other kind of cake. Going forward, I think we're going to be favoring designs that convey the metadata as a "first-class" citizen of the experience, as opposed to a design that implies that one could make a strong decision without it. When we do our next set of tests, I expect that we'll try the "serial" design as opposed to the "multiple" design for this reason.

We're also going to learn a lot in the next couple months from the Android MVP, which uses a "single" design, but will generate lots of data to inform our future designs.