This page used the Structured Discussions extension to give structured discussions. It has since been converted to wikitext, so the content and history here are only an approximation of what was actually displayed at the time these comments were made.
Latest comment: 8 years ago8 comments6 people in discussion
I'll go ahead and throw this out there—we should consider sorting alphabetically instead of or in addition to sorting by language count. If you aren't familiar with the rough size of the Wikipedia for the language you are looking for, you have no idea where to look. Ctrl+F works on desktop, but searching within a page is awkward on mobile. Basic navigation features like jumping to/near a particular letter in the sorted list would be very helpful. How to sort non-Latin characters needs to be thought through, but there's precedent in the way things are sorted in the Languages list on the left of every wiki page or the sorted list in the search box on the portal page. I suppose another option would be typing in the language name and getting a type-ahead match. Anyway, sorting by Wikipedia size doesn't seem user friendly to me. TJones (WMF) (talk) 14:33, 5 July 2016 (UTC)Reply
Very good points on the usability issues of the current grouping & hierarchy.
First off I'd like to point out that the language links are actually already sorted in alphabetical order -- within their current grouping.
The sorting of the language links can therefor be described as: grouped by article-count, and then sorted alphabetically within that group (I think everyone can agree that this sorting scheme is rather unintuitive).
Since the language links are already sorted alphabetically within their groupings, I think it's worth looking at the groupings themselves.
An alternative to grouping the links by article-count (as I think TJones suggests) would be to group languages by their first letter, like in an index, from A-Z. This approach poses some technical challenges with non-latin scripts, but it is feasible. However, out of the 300-ish languages on the page, about 95 are non-latin. For these languages, grouping them under a foreign alphabet character doesn't seem like the most intuitive solution either.
I personally like how the Universal Language Selector widget groups languages by region, so that languages from a similar geographic area are grouped together. It makes more sense for me to see, for example, east asian languages grouped together than it does to see Japanese (Nihongo) under an 'N' heading and Chinese (Zhōngwén) under 'Z'. JDrewniak (WMF) (talk) 12:43, 12 July 2016 (UTC)Reply
Hello, I think that selecting the compact list of languages by article count is a terrible idea. Cebuano is 3rd and Waray is 10th, but these Wikipedia versions have few readers and editors.
The list should consider the number of active editors and the number of visitors. Also, as I said in other threads here, it should also consider the web browser's list of languages sent by the URL query. NaBUru38 (talk) 22:02, 21 July 2016 (UTC)Reply
I mean the dropdown list. It has about 50 languages, including Waray (2.6 million speakers, 75 active users) and Volapük (artificial language, 28 active users), but not Greek (13 million speakers, 800 active users) or Bengali (200 million speakers, 500 active users) NaBUru38 (talk) 16:15, 23 July 2016 (UTC)Reply
Hi,
Are you refering to the new dropdown we want to show on the wikipedia.org portal page? It contains all the same language links we currently have on that page. DTankersley (WMF) (talk) 16:04, 25 July 2016 (UTC)Reply
Latest comment: 8 years ago14 comments7 people in discussion
We found that the test group had a 3.64% higher probability of interacting with the page and was 1.07 times more likely to engage with the page than the control group. Furthermore, although both groups received the dynamic primary link feature, users in the test group had a 6.82% higher probability (1.09 times more likely) of visiting a Wikipedia in their most preferred language.
When phrases such as "1.07 times more likely" and "1.09 times more likely" are used, does this mean ".07 times" and ".09 times"? This sounds pretty insignificant. MZMcBride (talk) 18:17, 8 July 2016 (UTC)Reply
I interpreted them as "7% more likely to engage" and "9% more likely" to visit. And if that is the correct interpretation, it seems like a clearer wording. KSmith (WMF) (talk) 20:29, 8 July 2016 (UTC)Reply
The "times more likely" reported are what is called the relative risk, probability of outcome in group 1 vs probability of outcome in group 2. The probability of the test group engaging was 1.07 times the probability of the control group. Specifically, it was 3.64% higher in the sense that if probability of a control group user engaging is 75%, then the probability of a test group user engaging is 78.64%. Hope that helps clear up any confusion! MPopov (WMF) (talk) 21:43, 8 July 2016 (UTC)Reply
Is there a reason to say "was 1.07 times more likely to engage" as opposed to "was 7% more likely to engage"? I think non-math folks would find the latter to be much easier to understand. Or do those not have the same meaning? KSmith (WMF) (talk) 18:29, 12 July 2016 (UTC)Reply
Yes, because I think that might be misunderstood as a difference because there are multiple ways to say "probability" such as "chance" and, specifically, "likelihood". So "was 7% more likely to engage" might be misunderstood as "if the probability of a user in the control group engaging is X%, then the probability of a user in the test group engaging is (X+7)%" (wrong) instead of "1.07 * X%" (right).
I would be comfortable saying that the test group was 3.64% more likely to engage and their probability of engaging was 1.07 times the probability of the control group to engage. It's an unfortunate side-effect of there being so many ways to express probabilities and uncertainty. I'll think about ways to address this. Maybe the solution would be to include a thorough explanation of these phrases in every report going forward. MPopov (WMF) (talk) 21:29, 12 July 2016 (UTC)Reply
I have problem to understand and translate " dynamic primary link feature". Maybe it's me (I read very superficially becausee I am in lunch pause) but can someone explain a little bit more? Thanks. Alexmar983 (talk) 04:35, 13 July 2016 (UTC)Reply
Although both groups received the dynamic primary link feature, users in the test group had a 6.82% higher probability (1.09 times more likely) of visiting a Wikipedia in their preferred language.
The primary link feature (in this context) was referencing rolling up the long list of languages by article count into a dropdown. As a side note, we typically call the language links that encircle the globe the 'primary language' links.
Were you referring to the new language by article count dropdown that is the subject of this talk page or the other links around the globe?
I still don't get what did they receive... sorry. I though someone got "A" and someone got "B".... so are you describing something that both "A" and "B" show? Alexmar983 (talk) 02:12, 14 July 2016 (UTC)Reply
@Alexmar983 "dynamic primary links feature" refers to the links around the globe. These links are 'dynamic' because they change depending on your browsers language preference, for example: if your browser is set to Hungarian, the link to Hungarian wikipedia should be visible around the globe.
The point regarding this feature was meant to explain that in this A/B test, both the test group and the control group received this feature, but the test group (which was given the new layout) was more likely to use it. JDrewniak (WMF) (talk) 12:44, 14 July 2016 (UTC)Reply
If some of the native speakers are having problems with these stats, the rest of us non-natives are going to have a much harder time following. 3.64% rise in engagement probability doesn't seem significant. Theo10011 (talk) 11:05, 13 July 2016 (UTC)Reply
The portal gets 10-25 million views per day, so a 3.64% rise in engagement could mean an extra half-million visits into wikimedia projects. Perhaps we should include those kinds of numbers in the report, so people realize that even a fraction of a percent can make a pretty big difference. KSmith (WMF) (talk) 20:54, 13 July 2016 (UTC)Reply
Hi, thanks for the comment - I had placed the mocked image of what the mobile view of the language dropdown would look like, not what we actually tested with.
Please refresh the page or go here to see a screenshot (in Commons) of what we actually tested with and want to release into production, as the new layout of the languages by article count has the links in columns to make them easier to read. DTankersley (WMF) (talk) 20:13, 8 July 2016 (UTC)Reply
Latest comment: 8 years ago3 comments1 person in discussion
If you really want to promote adoption, then this home page should display first using the user's prefered language (if he's logged on: use the user's preferences to set the default language, otherwise use the prefered language of the browser).
Then use that language to include it first in the list (you may also add a secondary language from the browser settings. Keep the two columns but fill then a few ones below (not more than 8 languages). Use also that language to translate the links to sister projects.
Users will then see their prefered language immediately without looking up into the list, as there always be one or two languages from the browser's settings, and always the user's prefered language. However I don't know if the portal itelf has preferences (may be he uses the user preferences from the English Wikipedia to determine the UI language, or it could use the language preference from Commons, or from Meta, which are international); for now the language preference is not stored globally but for each wiki.
In my opinion the language should be settable globally and each wiki should have the option to use either the global language, or the language set specifically for a given wiki. If a global language is defined, then this should be the language used by this interwiki portal.
With this addition, we could even reduce more the list of languages, to display at most 4 (the others will be in the dropdown menu).
Also the dropdown should just be used to select an alternate language (and set it for the session even if the user is not logged on, using session cookies). If the cookie is present, use it for the two first languages, add a couple of languages below. In fact that page could as well memoize in the cookie the last 8 languages selected on that page. The user preferences (if logged in) or otherwise browser defaults will only be used next, and then the major languages will complete the list.
The full list of languages in the dropdown may be dynamic (removing the languages already displayed in the top), or static (listing all languages independantly of preferences, sorted by articles count, including the major languages : English, German, French, even if it's displayed at top).
Because we have tnen a prefered language, the sister languages should display their name and description according to the preferences (the selected languages are ordered at top, if you need fallbacks). Verdy p (talk) 18:31, 11 July 2016 (UTC)Reply
Also I don't like much the fact that the dropdown is displayed on top of the existing page, it should just work as a language selector for the page itself (much like the ULS): clicking on it should close that list and update the page to reflect this choice.
Note also that there are TWO language selectors on this page: the search bar itself displays "EN", but cliking that button could also be used to select the language. No need of a second selector below: you'll save space for showing sister projectS.
Here also, the selection of a language in that search bar could be used to move that language to the top of list of 8 Wikipedia languages.
You can still save more by drippong the separation horizontal ruler. It's certainly possible top show the 12 sister projects (or at least 4 of them) without scrolling, even on the smallest smartphones.
Anyway I like the grid presentation of the list: for mobile users this should work like a tiled interface, easy to touch. Keep it flat, suppress unnecessary borders or lines. And possibly adapt that page for mobile users to use mobile's prefered colors (many mobile users like dark backgrounds as it preserves the battery), consider what is being done on Android preferences panel. In HTML/CSS accessibility, there are several UI colors that can be used by symbolic names for the background and foregound color.
The next step would be to allow the wikis themselves to use these UI preferences (and deprecate the use of colors in articles: most wikis abuse color patterns or font settings for the content; colors and font sizes may be set only on warning banners or notifications that should be closable or collapsible; and Wikimedia could adopt a generic notification bar, like on mobiles or on Windows 10, so that collapsed items may still be accessed from a central location near the screen border; when opening that notfication bar, it would reveal all notifications that can be deleted or archived, or it could be used for a few tools (that are for now spread on the left bar or top bar: all this should be a single collapsible bar and the colors and font sizes of that bar should be independant of the content of the page; that bar would also contain interwikis, the search tool, links to special pages... alf this is not needed when viewing or editing any page, it's just waste of space, which is also difficult to use when we need to scroll up/down to use them). Verdy p (talk) 18:51, 11 July 2016 (UTC)Reply
Latest comment: 8 years ago30 comments8 people in discussion
I could not find any evidence that people were more likely to use the drop-down menu than the old list. I suppose that the links hidden in the new menu are called "secondary", is it correct? In this case the figures show that users were less likely to use the new layout:
12 (or 1.1%) clicked on a link in the current list
3 (or 0.2%) clicked on a link in the drop-down menu
This means that the test shows that the new layout is 4-5 times worse than the older one, as people were less likely to find the language they were looking for.
Another flaw is that we do not know what language the user was looking for. It would be much more meaningful to do the following test:
pick users whose preferred language is not in the top-10 but whose accept language includes one of top-10
check how many of them would pick their preferred language in the old list and how many will pick the accept language in the top-10
same for the new drop-down menu.
As an example, many users from Ukraine would be looking for Ukrainian Wikipedia, but they would not mind reading Russian Wikipedia if there is no Ukrainian Wikipedia. Are they more likely to find Ukrainian Wikipedia in the new menu than in the old list or are they more likely not to find it and choose Russian Wikipedia which is more prominent? NickK (talk) 09:25, 13 July 2016 (UTC)Reply
The "top 10" ("primary') links were dynamic, using the user's preferred languages to populate the links around the globe, and filling the rest with the remaining of the default top 10. This was implemented as a result of a previous A/B test where we tested dynamic link population through language detection. So a user from Ukraine who has Ukrainian and Russian as preferences in their browser would see links to those respective Wikipedias as the first two links in the top 10, making the secondary links (which is what they were always called) unnecessary for their goal. I hope this addresses the confusion. MPopov (WMF) (talk) 17:25, 14 July 2016 (UTC)Reply
@MPopov (WMF): No, this does not address, as there is no confusion. I will try to explain the problem by presenting a use case.
Let us think of a user who is looking for a Wikipedia which is not in the top 10. This may happen for a variety of reasons:
User does not know how to set preferred languages. From my experience this is the main reason: experienced computer users who know how to set preferred languages will most likely find the wiki directly, without using wikipedia.org. In this case the user will have default languages, e.g. a user from Ukraine may have English and/or Russian by default.
User uses a public computer (e.g. school, library or internet café) and cannot change preferred languages
User is explicitly looking for a Wikipedia not in their preferred language, e.g. a user from Ukraine is looking for Belorussian Wikipedia
Now, we want to know what will this user do. There are three options:
User will find the wiki they are looking for, either in the new drop-down menu or in the standard list (in our case a user from Ukraine finds Ukrainian Wikipedia)
User will not find that wiki but will choose one from the top-10 (e.g. a user from Ukraine goes to Russian Wikipedia which is more prominent)
User will leave the page without clicking anywhere.
What I want to know is whether case 1 is more likely or less likely with the new layout. Unfortunately we cannot detect users whose preferred language is not set correctly, thus we can make a test on users who have their preferred language set to a language other than one in top-10. We can do the following test to check this:
Both Control and Test users get default (not dynamic) top-10.
Control users get the old list of Wikipedias
Test users get the new drop-down menu
We can split users into two subgroups:
Users whose most preferred language is in the top-10. For these users we want to know the % of users who click on a Wikipedia not in their most preferred language (i.e. they were explicitly looking for it); we can suppose the % of such users is the same in both subgroups.
Users whose most preferred language is in not the top-10. For these users we want to know the % of users who would click on their most preferred Wikipedia with the old or the new layout.
So far the figures are not satisfactory (by the way, it is very difficult to find the right figures in the report, for instance, I could not understand what 1.1% in "12 (1.1%)" refers to, as I could not find any figure in the range between 1043 and 1143 in the report):
Engaged with page. 60% (1802 out of 3019, Test) vs. 56% (1444 out of 2560, Control) is 99% statistically significant, thus indeed users were more likely to engage
Used secondary link. 0.1% (3 out of 3019, Test) vs. 0.5% (12 out of 2560, Control) is also 99% statistically significant, which means that users were much less likely to use secondary links.
This can mean that users who were looking for a Wikipedia among secondary links were 4.7 times less likely to find it. This would be a significant problem for Wikipedias not in top-10: they already receive less traffic by not being in the top-10, and they would receive even less traffic if users would be unable to find them (and this will more severely affect minority languages). However, this can also mean that users explicitly looking for a Wikipedia among secondary links were just underrepresented among the Test group. That's why I think that a different test is needed to measure this. NickK (talk) 13:33, 15 July 2016 (UTC)Reply
The point about the user not setting the preferred language is an especially hard issue for us. A thing that really helps in this regard is that the browser sets the language automatically based on the language the operating system has set. In my personal experience with Russian users (particularly the elderly who are new to computers), the OS has been in Russian. I guess what I'm trying to say is that inexperienced computer users may not know how to manually change the settings of the browser, but the browser installers have also been made to account for this.
I apologize for the lack of clarity in the report, and going forward I will try to be better at referencing figures when I reference the numbers within them.
@MPopov (WMF): One of Ukrainian users made an experiment in a rather small Ukrainian-speaking village, and he found out that most computers had their OS and browser in Russian, much less frequently in English. Now, these users will not see Ukrainian Wikipedia neither in the top-10 nor in the list, thus they may think that Ukrainian Wikipedia does not exist. In addition, users with the same problem may also not find Ukrainian Wikipedia in the interwiki list due to compact interwikis and their wrong language settings, effectively thinking that there is no Wikipedia in Ukrainian.
This is just an example, but the problem is much wider: we have minorities like Catalan speakers in Spain or Welsh speakers in the UK whose default language is usually wrong (Spanish and English respectively), most of India who have English as a default language, most of Sub-Saharan Africa who have English or French as a default language etc.
So far I find a very disturbing statistics that users are more than 4 times less likely to select these wikis (i.e. a Ukrainian speaking with Russian as a default language is 4 times less likely to find Ukrainian Wikipedia with a new layout than with an old one), which is in my opinion something we cannot proceed with.
Thus I would like to ask you to measure impact on these users before implementing these changes: we are not that in a hurry but we have a risk of seriously harming a lot of users. Thank you NickK (talk) 12:09, 16 July 2016 (UTC)Reply
Hi,
Please take a look at this link with your own browser settings to see how various browsers will always display the user's preferred browser language settings in the top 10 links displayed around the globe. This recent improvement has gone a long way to solve the issue (that has always existed) of the user having to search for their preferred language wiki link in the very long list that is currently still displayed on the Wikipedia.org portal page.
Also, please realize that the statistics that we talk about in our research may seem small (for example, research might find that 1% of users are more likely to click into the search box or a particular set of links) but compared to the average page views on the portal, it's actually a quite large number of users that are finding their links / information faster and easier.
Let's do the math for a moment. For instance, on June 1, 2016, we recorded an average of just under 14 Million pageviews to the wikipedia.org site. If we take 1% of that 14 Million visits, that means that in one day, 140,000 users were more likely to take an action on the page. We feel that those numbers are completely statistically important and positive.
I realize that this new language link drop down (that is the subject of this particular talk page) won't fix everything for everyone but it's a step in the right direction to help out many people that use the site to find information. DTankersley (WMF) (talk) 14:52, 18 July 2016 (UTC)Reply
14 Million pageviews are going to wikipedia.org (source: your figures).
0.3% of users of Wikipedia visit Ukrainian Wikipedia (source)
As a result, 42 thousand pageviews from wikipedia.org are likely to come to Ukrainian Wikipedia.
Many Ukrainian users do not know how to set preferred languages and have only default language (Russian and/or English), and not Ukrainian (software in Ukrainian is not that widespread). According to statistics, only 21.3% of Ukrainian users have Ukrainian is a primary language in their browser.
As a result, 33.1 thousand pageviews are coming from Ukrainian users who do not have Ukrainian as a preferred language. For these users, Ukrainian will be in a secondary language and will now be hidden in a drop-down menu.
With the new layout users are 4.72 times less likely to choose a secondary language (source + my comment above), and this figure is statistically significant.
As a result, instead of 42 thousand pageviews only 15.9 thousand will come to Ukrainian Wikipedia. Other users will most likely choose another Wikipedia, such as English or Russian.
This will represent -26.1 thousand pageviews per day for Ukrainian Wikipedia. Or -0.8M pageviews per month. This is a lot, and it is statistically significant. While it might be a move in the right direction for some, this will definitely be a move in the wrong direction for Ukrainian Wikipedia.
As I have mentioned above, this concerns not only Ukrainian Wikipedia but also other languages in which software is not that widely available, like Catalan or Hindi, for example. NickK (talk) 15:27, 18 July 2016 (UTC)Reply
Your theoretical potential (42 thousand as the maximum potential visitors from the portal to the UK wiki) sets an artificial limit, decreasing any percentages derived from that potential. You're statistics for primary language being set to Ukrainian also depends on visitors to that particular web page (refreshing the page a few times increased the counter.). That makes those statistics less reliable.
I understand your valid concerns. That users who do not have their preferred language set in their browser will have to find their language in a dropdown menu as opposed to a list of languages by size.
Language detection work (for the primary search box on the portal and elsewhere across Wikimedia), a potential update to the design of the dropdown, and a discussion on sorting are all things that may improve the interface - and potentially outright negate the need for an interface - for users whose language we are unable to detect.
The portal team measured the impact of this change in an A/B test. The results were an incremental improvement to visitor action. We have a task to investigate tracking portal to wiki traffic, but are do not have it in our plans to investigate this quarter. We'd much rather implement this incremental change now than to wait for an unknown point in the future. CKoerner (WMF) (talk) 21:09, 18 July 2016 (UTC)Reply
I could have provided more figures for Ukrainian (for instance, this study gave similar results), but this is not just about Ukrainian. This is about all countries where most software is not in national languages (actually most of Global South), this is about languages not supported by any browser (and there are over 100 of them, including, say, Cebuano, which despite having 3rd biggest Wikipedia is not supported by IE, Firefox or Chrome).
What is disappointing for me is that we have two proven facts.
With this change users are 1.07 times more likely to use primary links or search. This is good news for large wikis already in top-10 that will get a bit more additional visitors.
With this change users are 4.7 times less likely to use secondary links. This is very bad news for small and medium-sized wikis not in top-10 as they are very likely to lose visitors.
Still, getting some 140K page views per day seems to become more important than multilingualism. Wikipedia was always proud to have more languages than any other leading website, now these languages are hidden and we rely on browsers that are much less multilingual.
I agree that tidying up the main page is a good idea, but is it possible to come up with a solution where people are at least as likely to use secondary links? We need a win-win solution, now we have only a win-lose, unfortunately. NickK (talk) 06:35, 19 July 2016 (UTC)Reply
I'm not sure that there is a way to teach everyone how to change their browser settings, unfortunately.
Yes, you are not taking them away, but your study shows that users are 4.7 times less likely to use them. It is a lot, and it might mean much less pageviews for Wikipedias outside top-10.
Wikipedia is available in 291 languages, while browsers do not offer that much: Firefox offers to choose among ~190 languages, IE and Chrome have just 120. For instance, Crimean Tatar (crh) is not available in any of these three browsers, thus all Crimean Tatar speakers (even if they make their best effort to change their browser settings) will have to use the dropdown menu. Multilingualism has always been an advantage of Wikipedia, and it should not be hidden from readers. NickK (talk) 17:23, 18 July 2016 (UTC)Reply
I completely agree with @NickK. One may think of the decrease in view not essential, but for small wikis, struggling for their readers, it is significant. Ата(talk)08:40, 19 July 2016 (UTC)Reply
Do you think this particular treatment would be better - with the 'more languages' dropdown directly underneath the globe and in the same area as the top ten links?
@DTankersley (WMF): I don't know. I have suggested a way to test this above in this thread. You can even try an A/B/C test (with "A" for current layout, "B" with "more languages" as a button and "C" with more languages as a globe) to find out whether it will have an effect or not. Personally I do not have a necessary level of UI expertise to confirm if it will have any impact. NickK (talk) 14:24, 19 July 2016 (UTC)Reply
I think @NickK lays a strong argument for location-based language detection on the portal.
This idea has been suggested before, but not with the breadth of languages mentioned above (Ukrainian, Catalan, Welsh, Crimean Tatar etc). This 'minority language' use-case, where users browse the web in a more popular language than their own, might be quite high. Off the top of my head, I can image that Irish, Scottish & Welsh user might just click on English Wikipedia and Swiss German users might just click on German wikipedia.
Surfacing smaller languages was the original goal of localizing the top-ten links, and further supplementing this effort with location-based languages might help smaller wikis gain more readers.
There is a precedence for location-based language detection as well. The ULS on translatewiki.net offers 'suggested languages' based on location. Because I'm in Poland, I see languages like Lithuanian, Ukrainian & Belarusian, even though my browser is set to just english:
Indeed, can we not simply ensure that a Ukrainian IP would be guaranteed to see the Ukrainian Wikipedia prominently, no matter what their OS/browser settings?
From my own travels, I can confirm that many users, including experienced editors(!), use computers or devices without setting the preferences to the language they actually contribute in. And this is the case, as User:NickK pointed out, not just in Ukraine, but in other eastern Europe countries, in India, and in southeast Asia. Ijon (talk) 08:17, 22 August 2016 (UTC)Reply
It might be easy for Ukraine, as Ukraine has only one national language. Unfortunately it will not work in, say, South Africa which has 11 languages, 10 of which have Wikipedias, effectively leaving no space for anything else beyond national languages. This might be very disappointing for, say, a German speaker living there: they might have set German as their first language and German is in top-10 by any approach, but they will fail to see it.
Thus can we either expand the circle beyond 10 languages or add something like "languages popular in your area" as a separate line? NickK (talk) 11:52, 22 August 2016 (UTC)Reply
Hi @NickK and @Ijon, thanks again for your feedback. We probably won't expand the listing of languages (by browser setting or by top viewed) simply because there really isn't that much space around the globe to display more than 10 language links. However, we can have some sort of widget or link or line of regional languages in addition to the language links around the globe. This would enable us to detect what the visitor's preferred language is -and- be able to provide them with language links that are typical to where they physically are in the world. Hopefully it would be the best of both worlds!
We haven't fully vetted / flushed out the idea of using the translatewiki type of detecting which country your browser is accessing the internet in and then displaying the top X amount of languages that are used in that country/region. I say this because even though we've talked about it a lot and created some rough mocks, we haven't done any A/B testing on how well it would work and if visitors would like it.
This will be in our future work - to take a closer look and to run some tests on the interaction and we hope that you'll be able to provide us guidance at that time! DTankersley (WMF) (talk) 16:58, 22 August 2016 (UTC)Reply
Thanks, @DTankersley (WMF). Could you offer something more concrete than "future work" and "at that time"? When would the team do that A/B testing, to introduce more data into this decision-making? Ijon (talk) 17:15, 22 August 2016 (UTC)Reply
Hi @Ijon - I don't have anything more concrete that 'future work' at this time, unfortunately.
We are taking a bit of a hiatus from working on the portal to focus on other priorities within the Discovery department. We are planning to do maintenance to the wikipedia.org portal page such as stats updates and additional of translations as well as bug fixes for the next couple of quarters, but no larger scale work is planned to be done right now. Please view the page here that describes the work we've done so far and the work we want to continue to do in the future, mostly likely restarting in March 2017. DTankersley (WMF) (talk) 19:08, 22 August 2016 (UTC)Reply
I don't think that March 2017 is really a good date for this. Wikipedias in smaller languages were very visible, now they are hardly visible (unless user has already pre-selected these languages: A/B test shows that users are much less likely to use the drop-down menu). This has negative impact on smaller Wikipedias, in particular in Global South, which are less likely to get visitors. Having this last for 7 months is, in my opinon, a really bad idea. NickK (talk) 19:18, 22 August 2016 (UTC)Reply
@DTankersley (WMF): could you clarify who made this decision at WMF? It is not immediately obvious that this should be a decision made by the engineers implementing it.
And it seems to me that @NickK brought up some concrete arguments (and numbers) arguing the new state is decidedly inferior to previous status quo, from the small and medium Wikipedia point of view, and that these argument have not yet been refuted, and yet there does not seem to be any change planned by the team. Ijon (talk) 19:01, 23 August 2016 (UTC)Reply
@Ijon If you mean the decision to halt new work on wikipedia.org for a few quarters, that was Tomasz Finc and myself, and later Katie Horn. This was announced on the Discovery mailing list. In order to deliver on the annual plan, it's necessary for us to halt some work in order to pick up new work such as improving the search page. If you have a question about that, I'd be happy to talk to you about it, but we shouldn't do that on this page as it's specifically about the portal; feel free to email me, reply on that mailing list thread, or contact me elsewhere on-wiki. Dan Garry, Wikimedia Foundation (talk) 22:37, 23 August 2016 (UTC)Reply
Thank you, @Deskana (WMF), for this clear response. That is almost the decision I meant, although I was specifically asking who made the decision (if an explicit decision was made) to continue to consider this featureDoneand better than the previous status of the page, in the face of the concerns raised by @NickK. I do understand that the team's original plan dictates moving on, having spent the time originally allocated for this feature's implementation; but I think there is reason to consider the feature flawed, and therefore to at least consider altering timelines.
As I wrote in the second paragraph of my previous comment, I don't see that NickK's arguments have been fully engaged with, and considering that he makes a (to my mind) convincing argument that the new status is worse than the status it replaces, it seems to me to warrant a(n additional) PM/manager decision, that would result in one of the following: 1. NickK is wrong, and here's why.; 2. NickK is right, but here's what we can do to mitigate the damage to smaller languages (e.g. use geo-location to suggest languages, as suggested upthread); 3. NickK is right, and we'll roll back this new layout and go back to the drawing board to find a design that does not cause this collateral damage to smaller languages; or conceivably 4. NickK is right, but due to This Even More Crucial Thing, we will neither roll it back nor fix it until March 2017.
What I would like to hear is that that further decision has indeed been made, and that it gave due consideration to these concerns.
@Ijon Fair questions. The short answer is that it is inconclusive whether @NickK is correct or not that a decrease in page views to the Ukrainian Wikipedia is observed as a result of the changes. Discovery Analysis will do some further analysis to make a proper determination. If it is determined that there is a decrease in page views, we can consider reverting the changes to wikipedia.org.
Now, the longer answer. Regarding the quantitative analysis, NickK's reasoning regarding decreasing page views to the Ukrainian Wikipedia sounds compelling. However, it is not possible to reason about nonlinear systems in the manner he did, especially if one is combining statistics and data points from different sources and contexts. Experimental controls generally need to be identical in order for the data to be cross-comparable. We'd need to do a more scientific analysis to know if the change we made to wikipedia.org has negatively impacted page views to the Ukrainian Wikipedia (or other projects). Our previous statistical analysis during the A/B test suggests it is unlikely that this is the case, and the page views dashboard does not seem to show a decrease, but we do not know definitively. I've filed T143853 for us to perform that analysis. If the analysis shows that there was a statistically significant decrease in page views to the Ukrainian Wikipedia, we can consider potential solutions to that problem, one of which is reversion of the changes in question to wikipedia.org.
Thank you, @Deskana (WMF): , this answers my question.
I agree that more scientific analysis is indeed needed. I suggested a way to do A/B tests, now that this new version is live one can study clickthrough rate from the page:
One can compare % of users who clicked on wikis on the large list to % of users who clicked on wikis from the drop-down menu.
One can take as an example wikis that cannot be set as the main language (not supported by any browser), e.g. Cebuano or Crimean Tatar.
Based on our conversation earlier in the year, we created a new dashboard page that can be used to monitor the usage (clickthroughs) to any of the individual language wikipedia's from the wikipedia.org portal page.
You can get to the new page by clicking here and then you'll need to click on 'languages' and then 'languages visited' to see the data I'm referring to. (Note: we just released this new page into production this morning and the phabricator ticket for this work is here.)
Once you're on the new languages visited page, you can select a particular sorting mechanism from the 'sort languages' dropdown. For instance, in the dashboard image attached to this comment, I used Tatar, Catalan, Croatian, Cebuano, Ukrainian and Turkish.
By hovering with your mouse, you can view any date that we have collected in the dashboard data to view how many clicks occured on those particular language linked wiki's from the portal or you can also view how many users went to those language portals from the wikipedia.org portal page. Please be aware that these are numbers gained from our eventlogging schema and aren't the actual total clicks (or users) but are a representative view of the actions taken on the portal.
We created this new page specifically to address the concerns that you and others have had about the new portal page layout with respect to adding the languages into a dropdown (other than the browser preferred language and the top ten viewed languages). We wanted the community to know that we do take the portal page layout change very seriously and we've created tools in which to monitor any drops in usage or any other weirdness.
The other image attached to this comment is from this page that shows the amount of pageviews to a particular language wikipedia - you can see that pageviews to specific language portals are fairly seasonal (I used the same languages as my other screenshot).
We plan on watching the community's actions via these two dashboards and - as @Deskana (WMF) already pointed out - we have an analysis task to review the clicks to various languages from the wikipedia.org portal and if they changed after the new layout was pushed into production. DTankersley (WMF) (talk) 22:40, 25 August 2016 (UTC)Reply
Latest comment: 8 years ago6 comments3 people in discussion
I have a few questions regarding the drop-down menu implementation:
Will the message "200+ more languages" be English-only? If the user speaks, say, only Farsi and is looking for Farsi Wikipedia, they may be unable to understand that they should click on "200+ more languages"
Is this drop-down menu compatible with all browsers, including users with JavaScript disabled? We have people who use Wikipedia on old (e.g. school or library) computers, and it would be disappointing if people would not be able to open this menu (as users with older browsers are also less likely to be good at using search)
Will Ctrl+F search inside the menu? For instance, if I am want to check if there is a Wikipedia in Hindi, will I get a result if I click Ctrl+F and type "हिन्दी"?
Currently, the wording in the box will be in English, but we'll ask for translations. We've also been thinking of a way to display the dropdown with a more intuitive description so that visitors will realize that they can click into any of those additional language wiki's. It'd be great if a icon or some type of image can be created to convey this information without having to have it translated into specific languages. We're open to suggestions!
If the visitor has their browser settings to view pages in Farsi, then our code will detect those browser settings and display that language wiki link in the primary links around the globe.
Regarding the wording: is it possible to make it translatable? Or even make it bilingual (English + primary language), e.g. "200 more languages / 200 langues de plus"? We already use this message for compact interwikis, why can't we reuse it here?
Does this mean that there will be an alternative CSS version for IE8 and older?
Will it work before dropdown menu is open? (I suppose not, which may be an obstacle, but perhaps a minor one)
Yup, we use AcceptLanguage (as shown in these images) to capture the visitor's preferred language and display it around the globe with the other top 10 languages.
Latest comment: 8 years ago8 comments6 people in discussion
What about having a link to the local language wikipedia near the English Wikipedia link by detecting IP address. For example, with this feature if I go to wikipedia.org from Kerala, I will see a link to വിക്കിപീഡിയ near "English 5 188 000+ articles" Libperry (talk) 11:42, 13 July 2016 (UTC)Reply
@ഏത്തപഴം Currently, the languages around the globe do change, but they change is based on your browser language, and not your IP address. The reason for this is that a user's browser language is better representation of a users preferred language than their location. If your browser is set to Spanish, you can be sure that the user knows how to read Spanish, but just because I'm in Spain doesn't necessarily mean I can read Spanish. JDrewniak (WMF) (talk) 13:05, 14 July 2016 (UTC)Reply
It's certainly a possibility but we'd need to do some testing to be sure we get the results that we're expecting; and there is a bit of a privacy concern as well. But, if we can generalize the IP address to a region/state/country, etc that might work. DTankersley (WMF) (talk) 22:46, 14 July 2016 (UTC)Reply
User:DTankersley (WMF), geo-location needn't come at the expense of the user's settings. Indeed, we *must* respect what the OS/browser tells us, but, knowing as we do that for large sections of the population outside the west, those settings *aren't* representative of the user's actual wishes, we should also try to Do The Right Thing using geo-location.
As you say "we'd need to do some testing", can you offer a timeline for when such testing could be done? The current feature should certainly be considered incomplete without geolocation. Ijon (talk) 08:21, 22 August 2016 (UTC)Reply
We did not change the font-size in the 'floating box'. Currently on wikipedia.org, the font-size is smaller for the wikis with less than 10,000 articles. The font-size for these wikis gets even smaller on mobile devices, but this is something we're considering changing.
Latest comment: 8 years ago3 comments3 people in discussion
There's one thing I don't get about the reactions of the two group.
I read "The test group consisted of 3019 users who received the new design while the control group consisted of 2560 users who received the current design". So they're users, like registered users?. Now, if they were random people I may understand the rationale, but as a user who has been interacting with the current type of interface wouldn't I be more surprised with a new one in any case? If it were me, I would definitely interact with a new one on the first time a little bit more, but that's because I'm just more curious. Alexmar983 (talk) 13:53, 14 July 2016 (UTC)Reply
Yeah, by user we really mean "sessions", because on the portal we simply don't know who has an account and who doesn't. You bring up a good point about about the 'novelty value' of a new interface though. It's true that many people will try it out just out of curiosity, so measuring behaviour over a longer term would certainly be valuable. JDrewniak (WMF) (talk) 17:34, 14 July 2016 (UTC)Reply
Latest comment: 8 years ago6 comments4 people in discussion
There have been a few comments about how the proposed design introduces two language pickers onto the page. Semantically, I wouldn't consider the "200+ more languages" button a true 'language picker' because it doesn't really change the language of anything, it simply reveals or hides a list of links.
A few people we've spoken too however, find it confusing that there is a language picker near the search box and another 'language picker' below the search bar. I think this confusion might be in part because the '200+ more languages' button is shaped... like a button..., so it might be associated with the search form.
New Idea
The idea below is to treat the '200+ more languages' not as a button, but more like a menu item, and also, to associate it more with the top-ten language links around the globe, because its function would be to reveal more languages. Similar to how a 'read more...' button on a blog post would work.
I like the concept, but I think the "more languages" would have to be more visually distinct. Right now, it just looks like an additional language at the bottom of the globe. Even though I was looking for it, I had to look a second time to actually find it. KSmith (WMF) (talk) 19:30, 22 July 2016 (UTC)Reply
But the real question is, when will WMF finally provide multilingual search for Wikipedia? German and english WP search with 1 click? Or arabic+frenchWP, or russian+kazakhWP etc etc.?
PS: I really hate Flow. You/WMF force me to deal with Flow on mediawiki, just to give feedback on betas. i often don't give feedback because Flow is so awful.
Not to mention the incessant Flow-notification-spam that even follows me home to any Wikipedia.
Hi @Atlasowa, we're glad you like the new placement.
We're hard at work on improving search across languages and across wiki's, please check out our sprint board for all the various tasks that we have in progress and planned; here's one such epic ticket. Unfortunately, we just don't have the resources that Google has to get search into a more perfect state.
Latest comment: 8 years ago6 comments3 people in discussion
I have found a link to the recent survey and I have one question: what are these users' languages? Do they speak English? The result is rather meaningless if all 5 users are English speakers, as they obviously will look for English Wikipedia only and do not need any secondary links, but it would be much more interesting to get a point of view of someone who is not an English speaker and who can be looking for a secondary link. NickK (talk) 20:33, 16 July 2016 (UTC)Reply
The participants in this survey research were ones that offered us their names and email addresses as part of a survey that was conducted on the Wikipedia.org portal site. They were all willing participants and we did not make any determination on who would participate in the research, only that they have freely given us their contact information and were ok with signing the release form, so that we can publish their comments and concerns.
I can guess from the fact that all comments are in English, screenshots are from English Wikipedia and only wiki mentioned is enwiki that all 5 users where English speakers. If this is the case, checking links to non-English Wikipedias on users who don't use them is not really meaningful.
What I am referring to is how one of the participants commented on this change: "I wouldn't use this very often. Before I'd click on random languages... how is it now?". This means that this person is less likely to use secondary links even for fun. At the same time other people use them for finding wikis in languages they speak, and I don't think we want them to be less likely to find these wikis. NickK (talk) 15:59, 18 July 2016 (UTC)Reply
@NickK, you might be interested in reading this analysis of the A/B test for the discussed change (Edit: oops, I see in a topic below you may have already read this!). While the survey contained English-speaking participants (I'm not sure we collected information on first/second language or nationality) the analysis used a random sampling of visitors. The analysis echos the survey - that the change resulted in positive interaction with the portal. CKoerner (WMF) (talk) 17:02, 18 July 2016 (UTC)Reply
@CKoerner (WMF): I already commented on this survey below (or above, or in parallel... can't properly refer to another thread in Flow, sorry) in #Use of the drop-down menu. Basically the result seems to be that users who don't use these links (in particular those most interested in English Wikipedia) indeed get a more positive experience, while users interested in these links are significantly (4.7 times) less likely to use them with a new layout. As an editor in a language which is likely to be in a dropdown menu for many readers I cannot qualify this result as a positive one. NickK (talk) 17:29, 18 July 2016 (UTC)Reply
All language links will still be accessible, for fun or otherwise; some will be in the top ten links around the globe in the browser's preferred languages and some will be in the language dropdown.
I apologize for not having more participants that used languages other than English, but that was not primary focus of the research we were doing at that time, with those participants. DTankersley (WMF) (talk) 16:41, 18 July 2016 (UTC)Reply
Latest comment: 8 years ago5 comments5 people in discussion
Hello, Wikipedia.org search bar detects surnames very poorly.
With Spanish configuration, if you type "Schumac", you get a disambiguation page (ok), Henrich Schumacher (rather ok) and several odd plants (???), instead of famous sports personalities Michael Schumacher or Toni Schumacher.
If you type "Ferre", you get Ferrero Rocher (ok), a random Spanish noble (???) and several tiny villages (???), instead of actor Will Ferrell, or even worse, ferretería!. NaBUru38 (talk) 00:35, 17 July 2016 (UTC)Reply
I checked the search function in RU, and I must say it is brilliant. It finds exactly what I was looking for, alternative suggestions are useful, and it even ignores minor typos when searchingю E.g. "Пшкин" returns "Пушкин" as intended and then "Пекин", which is a very good guess, and then "Пушкин (город)", which is also a good alternative. Well done! Hope you can replicate this for ES. SSneg (talk) 07:28, 29 July 2016 (UTC)Reply
Russian Wikipedia makes it a lot easier to search by surnames, since articles seem to generally be named "Surname, FirstName", so the prefix matching the suggestions use works really well on surnames. As CKoerner (WMF) mentioned, it looks like the DEFAULTSORT task could help on wikis where "FirstName Surname" is the usual standard. (Also, on the ticket, I talk more about how actually searching makes up for a lot of the shortcomings of the autocomplete suggestions—though in general they are really wonderful.) TJones (WMF) (talk) 17:54, 29 July 2016 (UTC)Reply
Latest comment: 8 years ago2 comments2 people in discussion
I think that the "200+ languages" text on the big button is just Wikipedia showing off. It does not serve much purpose to a normal reader to know that there is content in 196-199+ languages that they cannot understand.
I propose the message is changed to something actually useful to the user, a call to action like you see in places that actually care about making visitors do something (Register! Get a free trial! Donate now! etc) - and in Wikipedia's case this should be "Read in your language".
I am sure that "Read in your language" call to action (or similar) would prompt more users to discover Wiki content in their language than a passive statement like "200+ languages" (which is also not really true, if you think of it, it is just a marketing message that is well received by journalists.., because a huge part of those small language Wikipedias are not developed enough to actually be useful).
SSneg thanks for this comment. I agree that the wording on the button should be revised to have a more active voice, "Read in your language" sounds pretty good to me. JDrewniak (WMF) (talk) 10:19, 2 August 2016 (UTC)Reply
The post suggests that a more traditional button style with a dark background has a higher clickthrough rate than the 'ghost button' style we're currently using. Granted, this post is only describes a single test, and aesthetically I prefer the 'ghost' button, but I'd be curious to see if this style change would effect clickthrough rate.
The “button with the dark background” is also referred to as “primary button”. In all MediaWiki core interfaces we've been agreeing from UX standpoint on featuring at maximum one primary button per view for user guidance. In this case we would break that pattern. I strongly oppose to do this in the above proposed way. Volker E. (WMF) (talk) 15:33, 8 August 2016 (UTC)Reply
I appreciate the concept of having only one primary button. For me, the discussion would then turn to: How to make the other buttons look like (non-primary) buttons and not just empty rectangles. Perhaps a lighter shading, which seems to be fairly standard. (I'm looking at a google calendar popup right now which does that). KSmith (WMF) (talk) 19:56, 8 August 2016 (UTC)Reply
Latest comment: 8 years ago3 comments3 people in discussion
Hi @JDrewniak (WMF) - as discussed offline earlier, I agree with concept but did have concerns with the open 'drawer' of languages being disconnected from the "more languages" button (the search box comes up in between the two).
It is unexpected behaviour (initially when trying the demo I thought it was actually just scrolling browser window down to the extra languages) and also looks strange with the callout arrow appearing over the search bar.
Secondly, while styling the more languages looking like the other main languages around the globe fits well visually, it seems misrepresentative because it is actually a toggle button and not like the other language links around it.
I created a quick animated gif showing the button placement below the search instead, updated the visual style as well and added an animation sliding open of the more languages which makes the transition more overt.
Thanks for this feedback. I agree that the button in this prototype, although visually fits, seems disconnected from the 'drawer' that it operates.
Like in your gif, placing the button below the search-box offers the potential for animation, which could give the page some extra refinement.
I went ahead and created a new prototype with this idea, link below. I think the button still needs to be more prominent, but I like where this is going.
I agree that it looks better, but I don't like disconnecting the drawer from the globe links, since both do basically the same thing. Having the search bar between them just feels awkward to me. KSmith (WMF) (talk) 17:00, 3 August 2016 (UTC)Reply
Done and better than the previous status of the page, in the face of the concerns raised by @NickK. I do understand that the team's original plan dictates moving on, having spent the time originally allocated for this feature's implementation; but I think there is reason to consider the feature flawed, and therefore to at least consider altering timelines.
SSneg (talk) 20:43, 28 July 2016 (UTC)
