Sorry, your intentions are good, but this article feedback tool is, to put it simply, a bad idea. Please remove it.
Topic on Talk:Article feedback/Flow
Could you elaborate on why do you think it is a bad idea? I believe a lot of people like the tool, and some people are also requesting it to be enabled for Wikipedias in other languages.
I will try, but English is not my native language, sorry. I hope I do not come across rude, that is not my intention.
The system is based on the false assumption that people's opinions are actually valuable, even if that opinion is simplified into a rating of 1-5 stars. Their opinion would be more valuable if they just wrote the pro's and con's on the talkpage instead of using the pro's and con's to determine a score via an undisclosed process and then give an appropriate amount of stars.
The system discourages users from using the talkpage. Giving a bad article a 1-star rating is easier than fixing it yourself, tagging it or explaining what should be improved on the talkpage, but far less useful. People who see a bad wiki article have two options: fight or flee. This box is just another way for people to complain and leave, and even gives them the impression they did something useful.
Basically asking for a rating is asking the wrong question. Compare the answers that you might receive with these two questions:
- How many stars would you rate this article on a scale of 1-5?
- What made you decide to rate this article in this way?
The answers to the second question are much more interesting.
Usability/interface problem: the location of this box is wrong. It should not be shown in the articles themself (do not trust me, ask an usability expert!). The box is much bigger than it needs to be, and very ugly.
You made some good points about the hit'n'run. Not to mention that only IMDb has a working rating system (calculating the average technically, ignoring beginners, etc). I left some thoughts on a proper rating extension here. --Subfader 21:13, 7 June 2011 (UTC)
"You made some good points about the hit'n'run. Not to mention that only IMDb has a working rating system (calculating the average technically, ignoring beginners, etc). I left some thoughts on a proper rating extension here. --Subfader 21:13, 7 June 2011 (UTC)"
I'm one of the people who finds the tool (and its results) completely useless. As I've said in another thread - I've seen well written, well-sourced articles rated very low because 10-year-old girls don't like a character an actor plays on TV, and I've seen mere stub articles consistently given high ratings because people love and remember their work from 50 years ago. However, I wanted to chime in on the discussion I see emerging here about the IMDb rating system. This is something I've thought about before. If people absolutely INSIST on keeping this useless feature, then there should at LEAST be a 10 vote minimum before the votes show up (like there is on IMDb). This would at least provide something of fighting chance of there being a somewhat impartial vote result before the votes show up on the page. As it is now, the ratings tool is just another (and now sanctioned) way for vandals to operate anonymously on Wikipedia.
Since the excessive size of the rating box was mentioned also on English Wikipedia (see this topic), I opened the following request on Bugzilla:
- Bug 29303 - Improve layout of ArticleFeedback tool
@Subfader: Thanks! I agree; IMDb has had a long time to figure out which rating system would work best, and instead of inventing the wheel again we should learn from the mistakes they have made in the past.
@Helder: Good work! Because you included a link to this discussion in the bugreport we can try to give suggestions for a 'compact' version of the design here.
Maybe it is a good idea to move the ArticleFeedbackTool to the talkpages. The "normal" article feedback should also be on talkpages, not on the articles themselves...
People who give 4 stars only send the message: "I like the article this much". Instead we should ask them to tell us why they decided to choose that rating: what did they like about the article, what did they dislike, what should we improve to turn the 4-star rating into 5 stars? If we know that most people rate an article 4/5 for "completeness" that fact alone does not help us. We want to know what they felt was missing from the article, but we force them to use a 1-5 star system without a textbox where they can leave a comment.
I think we lose valuable information if we ask the people who rate the article to turn the pro's and con's list they have in their head into a rating of 1-5 stars: an actual textbox where they could express their opinion would be far more flexible and give them the opportunity to be a lot more specific and helpful. But we already have that, its a talkpage. So I suggest moving all the boxes to the talkpage. Ofcourse, this means there will be a bit less input and a lot less eyeballs on the article feedback tool. On the other hand I think you might get a bit less vandalism and more people who take the time to make a serious and informed rating.
Another problem is that it slows down page loading quite a bit. Maybe we can test it with software like Yslow.
The sentence "Your ratings have not been submitted yet" appears in green, a color that tells me the exact opposite of the message.
That sentence and the arrow icon in front of it should be removed imho (if we can not move all the article feedback boxes to the talkpage) because they make the box way too big and I think they do not have the positive effect they are supposed to have: the people who fail to see the huge blue button will also be unable to see the small green sentence, probably because they need to scroll down.
Its better to put the stars and the submit button next to eachother if that fits so that people see them at the same time when scrolling down.
This user will likely not submit the ratings, and the warning will fail as well.
Comparison of current design (bottom) with redesign
The redesigned interface is less cluttered because I removed the trashcan icons (they should be added back in) and the sentence that warns people they need to submit their rating with the green arrow (because you need to have scrolled down enough to see the submitbutton in order to see the warning and the warning is smaller than the submitbutton) and the icon behind "View page ratings" (no longer necessary because I changed the position of the text).
After I made the redesign I realised its probably better to swap "Objective" with "Trustworthy". Ofcourse you can move the "expert" checkbox to the bottom and the submitbutton to the top if you prefer that. Is it possible to replace "What's this?" with "Help" or "Info" or "How to rate?" or "Info about rating"?
Ofcourse, this is a very ugly and messy image, but you get the idea.
The endresult looks something like this. The fontsize and styling of the links on the lefthandside has changed slightly.
I think we can use this as an example:
The image contains a mistake, the text on the right side should be black instead of gray.
At the bottom I made two extra versions of the "expanded" version for "experts", with or without a solid line in between, I prefer the one on the bottom.
Design is an iterative and fluid process and we'll continue to take a look at it moving forward.
Of course. What do you think of http://i.imgur.com/k5lQ2.jpg compared to the current design?
I believe that design is a bit too cluttered. It is trying to jam everything into as little space as possible, which radically reduces its usability. The design requires that the user's eyes constantly be roving the entire set of elements in order to complete the task at hand, rather than flowing in a predictable manner.
Further, the overall color scheme appears to have been muted, which is not desirable.
I personally had little to do with the current design and may choose to revisit some elements in the future but for now we are going to continue with the current design path.
Why are you going to continue with the current design path? Is it set in stone? If so, please tell me, I will stop giving feedback.
If it is not set in stone: adapt. Things don't always work out the way you imagined them to be, we have to accept that. In a community this large its normal some people will oppose changes like this.
The current article feedback tool is not yet developed enough to implement on a large number of articles. If people want to continue development on it they should try and reach consensus, even if that means they will have to adjust their goals a bit so the people who dislike the articlefeedback tool think it is acceptable.
We are not going to apply significant resources towards the design path at this time because:
a) We're not convinced that this is a poor design path b) We do not have the resources available to iterate deeply on the design at this time
I'm going to disagree with you that it is not yet developed enough; many man hours have gone into ensuring that it is quite capable of handling a full-scale roll out.
And no, nothing is set in stone, and we do adapt. However, as I've said, there has not been sufficient data to indicate that a direction change is required.
Do you want me to provide that indication? I may be able to do that if you want me to. Does "we" mean the WMF?
"We" means the WMF in this case, yes.
OK, do you want me to provide that indication? I can rally the troops if you want me to.
I'm sorry, but vague threats along this line are not constructive. If you wish to continue talking in a constructive manner, I will do so.
That was not a threat, I probably should have added a smiley (sorry I am not a native speaker), it was an indication that I am not the only person with this opinion. If I would be the only person that opposes the article feedback boxes as they currently are then I would have no troops to rally.
You wrote: "...there has not been sufficient data to indicate that a direction change is required.". If you want me to I can help by asking the community for input. If, for example, < 33% agrees that we should change direction then you have the indication that a direction change is required. But if, for example, < 66% agrees that we are on the right track then I will focus my attention on something else.
I think there is no problem in starting a pool to ask this to the community, since it is a valid question. There is no need of WMF approval before you ask other wikipedians about something you think it is important to have more opinions about.
I would suggest you to just be bold and start a topic somewhere on en.wp asking requesting comments about the tool, if people like/dislike it, if they want it or not, and so one...
I agree, the design proposal is too cluttered. But the current one is indeed waaay too high. What about a toggle link? Furthermore I think it doesn't matter how many people dislike this feature since it will improve things. No? --Subfader 21:50, 8 June 2011 (UTC)
A toggle link is problematic for several reasons (many of which surround anonymous readers).
For registered users, there is now a preference that can be set to hide the tool from view.
Toggle links are NOT a problematic. All navboxes use the show/hide link.
If an anonymous user clicks the "Hide" link on the item, and then later wishes to see them again, how do they go about re-enabling the tool?
Answer: they can't. They don't have preferences pages. So they have to delete the cookies that hide the boxes, or wait for those cookies to expire.
That's only one of the design problems inherent in this.
I'm not sure if Subfader was talking about a toggle link to hide the whole box (which would be problematic for the reasons you've mentioned), but just to hide show part of it. I added a similar suggestion on this comment
Right. So, the reason we aren't exploring a "toggle down to a 'closed' state" feature is that we have found that when users want to have the tool removed, they want it gone forever. It's not a case of "oh, I'll get back to this eventually". Since it's at the bottom of the page, it doesn't cause interface clutter unless you're specifically looking at the bottom of the page.
With that logic in place, having a "toggle to closed" control doesn't make a lot of sense - in fact, it actually makes the tool more complicated, because now we have yet another control that the user has to understand and be able to interact with.
Uops... I was thinking in the opposite action when I wrote that: I meant a toggle button to show part of the interface. As I mentioned in the other comment, the bottom half of the box could stay invisible until the user interacts with the tool clicking in some of stars (note: this saves ~50% of its current height). This wouldn't require any extra control (the first click made would serve both to define a rating and to display the remaining of the interface).
@Jorm: You wrote: "Since it's at the bottom of the page, it doesn't cause interface clutter unless you're specifically looking at the bottom of the page."
That is not true, unfortunately. Especially short pages suffer a lot of interface clutter because of the AFT.
Well. You know. If you're on a short page, and you can see the tool, then you're looking at the bottom of the page.
I understand. Part of the interface is in the sidebar, part of it is at the top of the page, and the AFT is a huge box that you want at the bottom of every page. I hope no one suggests putting a part of the interface, or even worse, a skyscraper ad, on the right side of the screen (slippery slope).
Please move the AFT outside of the div that contains the content. Separating the interface from the actual articles is very important.
Is it a good idea to put the AFT directly beneath div#content, so that the top of the AFT touches the bottom of the border around div#content?
To make sure this is noticed by the developers, I've copied your request to the bugtracker: see Bug 29704.
Feel free to follow this link and request specific changes/report other bugs directly on bugzilla, so that it goes to the current list of open requests.
Thanks Helder! I love you!
FYI: you can use the following JavaScript code in your vector.js as a workaround while the AFT code isn't fixed:
$(function() {
$('#mw-articlefeedback')
.prependTo('#footer')
.css('font-size', '0.8em');
});
On monobook skin it would require more formatting, I believe.
If its too cluttered: please make your own suggestion with paint or photoshop. The screenshot is just an example that shows how we can position the elements, the amount of padding is a different subject but I agree it should be increased.
We can use toggle links, we can add a link called "rate this page" in the sidebar, if we ask for one rating per page instead of 4 we can put 5 stars in the sidebar, we can use a lightbox effect, there are many options and only popups is explicitly disallowed. I do not know why they chose to put this huge box on lots of articles, it is far from necessary, and some of the other possibilities seem much more userfriendly.
Unfortunately the article feedback tool will not improve things. I think it wastes a lot of time devs could spend on things that actually matter.
Afaik there is no consensus to implement the article feedback boxes, and I have yet to see consensus for the important designchoices (but I am still reading).
I think the definition of what are "things that actually matter" is very subjective and during the Strategic Planning the community helped to determine what are those things. Since the assessment of article quality is among them, I don't think it is a waste of developers' time to put some effort in creating/developing a system which may help all Wikipedias to get that. Whether the tool will or not be able to improve things, and how the tool could be changed in order to achieve that, are questions which I believe are being considered by the researchers. But I don't think we can deduce the answers without having the data which is being collected during the research. It would be just a guessing game.
As for the "lack of consensus for implementing the feedback boxes", this is not generally true: some wikis reached consensus for implementing it.
I am unable to speak Spanish. When I wrote: "Afaik there is no consensus to implement the article feedback boxes, and I have yet to see consensus for the important designchoices (but I am still reading)." I was referring to EN wiki.
I think we can (at least try to) give meaningful answer to questions like "Will this tool be able to improve things?" and "How could this tool be changed in order to achieve that?" without having the data which is being collected during the research.
"But the current one is indeed waaay too high"
I agree. Is it really necessary to display the submit button even if the user didn't click in any of the rating stars? Wouldn't be better to hide it and also the text "I am highly knowledgeable about this topic (optional)" (since we can exclude that hiding or displaying the expertise checkbox may significantly affect the number of ratings submitted by readers) until the user interacts with the tool?
I think this proposal may not fit well with some other languages, since the text may be longer than it is in the English version (e.g in de, the text "View page ratings" was translated to "Einschätzungen zu dieser Seite ansehen").
Good point. In this case it can probably be fixed by using "Einschätzungen ansehen", which means "view ratings" instead of "Einschätzungen zu dieser Seite ansehen" which means "View ratings for this page".
Disclaimer: I am unable to speak German, sorry.
/me neither ;-)
After I made the redesign I realised its probably better to swap "Objective" with "Trustworthy".
Why? Because of the lenght of the words? (If so, consider this comment about the translation to other languages).
Nope, I swapped them because I think the question if something is trustworthy makes people wonder if it actually is. I think the proverb "let a sleeping dog lie" applies. ;-)
I completely agree with Wasbeer's assessment, and reiterate that the tool provides readers a lame excuse for not posting an actual feedback comment on the talk page. The box could possibly be made much more useful if an extra comment field were available; at least it would provide encouragement to justify a rating.
I want to add that I feel that most casual readers do not care for citations at all, so if you are asking whether the page is trustworthy but by that mean to ask whether the article is well sourced you will most definitely get the wrong or a random answer. It also does not consider that outside readers are usually not familiar with our strict sourcing policy, and what they might find well sourced may not be so for us – in fact, we as editors will know best whether an article is sufficiently sourced or not.
The box could possibly be made much more useful if an extra comment field were available;
- This is already being considered, as you can check on these pages:
Thank you Wasbeer - you echo my thoughts exactly on article rating, but if this is going to be pushed on us at least it's better than the thumbs up/thumbs down style "assessment" that other websites provide. This being said, it was a great idea to plug the "did you know you can edit this page" message in at the end of the review. Hopefully this will encourage new editors. Themfromspace 20:12, 31 August 2011 (UTC)
It would also be fair to say that an equal amount of people dislike the tool.
Nope, the people who dislike this tool outnumber the people who like this tool afaik.
There is no data to support either of those conclusions, guys. Sorry to put a damper on this, but we're still collecting and collating the information.
While I appreciate and hear your opinions, we are beholden to the will of the entire community - who has charged the Foundation with increasing the overall quality of the articles on Wikipedia. Part of completing that task is to develop a mechanism whereby quality can be measured - and this is one of our experiments.
True, I just made that up, hence the "afaik". But as long as you have not proven < 50% of Wikipedians like this tool we can assume that the actual percentage is a lot lower than 50%.
The intention of developing a mechanism whereby quality can be measured is an excellent idea, but this articlefeedbacktool on every article is not the way to move forward.
That is why I try to change the direction now, before everything is set in stone.
Is it possible to reach consensus about the location of these articlefeedbacktool boxes? Can we please move them to the talkpages?
If we move all of them to the talkpages the majority of Wikipedians who does not want to use them has a lot less to complain about and the people who want to use them will still be able to.
The tool will not be moved to the talk pages. Most users of Wikipedia are not even aware that Talk pages exist. Since the goal of the tool is to attract impressions from as wide a range of users as possible, hiding the tool on the talk page is not a viable solution.
The Strategy plan - which all Wiki(p|m)edians were invited to join - includes a provision for this tool and a directive from the community to increase quality. This is part of that plan.
So the decision to NOT move the article feedback tool to the talkpages has already been made?
By who? Does he/she have the authority to make that decision? Even if the community disagrees?
The fact that most users of Wikipedia do not know talkpages exist is one of the main reasons why I want to move the article feedback tool there.
To be honest, I think we have two options here:
- Moving it to the talkpage and developing it into something useful
- Deleting it
The second option means there is a lot of time wasted, so the first option would be the best way forward imho.
Yes, that decision has already been made. It was actually made during the first iteration of the tool.
If you require a name to blame for "whose idea was this?", then you may use mine. I was the principle designer of the tool in its first iteration.
Burying the tool on the talk pages will have the exact opposite effect of what we are looking to do. Moving the tool there will not improve article quality, nor will it help educate users that talk pages exist.
Well, Jorm, lets agree to disagree and have a happy wiki! :-)
We agree on the fact that moving the tool to the talkpage will not educate users that talkpages exist. I think that moving the tool to the talkpage will not improve the quality of the article itself.
But I think it would be a huge improvement to the article en:Salt_Creek_(White_River) if the article feedback tool was moved to the talkpage.
I think that article is almost a perfect candidate for the tool. The ratings, as given, clearly reflect the article as a whole.
It's lowest score is in "Complete", and it's barely more than a stub (with several red links). I think it's working just fine on this one.
Is your primary objection to its being there based around the fact that you are the primary editor of the article and that you feel that low ratings reflect poorly on you? Editors do not "own" articles, and users do not attribute an article's state to specific editors.
I am in no way form or shape the primary editor of that article, user Nyttend wrote it. I never wrote an article on EN wiki with this username.
Lets talk about the article feedbacktool, not about me and my intentions, that is more important. I am not trying to attack you personally, my criticism is focussed on the AFT and not on you as a person.
I agree: the score fits the article. But does the Article Feedback Tool improve that article? How?
I think the addition of the AFT has a very negative effect on this article, lots of screen real estate is used to display interface, not content.
As I understand, the tool doesn't need to improve the article in order to be considered adequate. The project has multiple goals.
If a reader goes to that article, notice it doesn't has much content and then rate it as say 2/5 complete, the extension may show a message after the the user submits the data, inviting him to edit and improve the article. I think this possibility by itself justify the presence of the tool in that specific article. Besides, if after the page receives ratings from various readers its Trustworthy became something near 5/5 (even if the article is not lengthy), the readers can deduce that they can trust in the content which is available. The opposite situation would be a lengthy article with almost no references (if any), whose Trustworthy tends to 0: the readers could use the "View page ratings" link to discover that it is probably better not to trust in the content of the article.
Why not show the ratings on the article, with a "rate this article" link next to it instead of a box asking for a rating? That would save a huge amount of screen real estate, and it would be a lot easier to put into the existing interface.
Do you want to switch to this method after a certain period of time after an article has received enough ratings? Extremely popular articles might get lots of ratings, up to a point where its no longer useful to ask for them.
Is it possible to turn the AFT off for users that are not logged in?
https://secure.wikimedia.org/wikipedia/mediawiki/wiki/Article_feedback/Research#Overview
I am not very optimistic about the value of the ratings in general but especially those of users who did not log in, I fear people will base their ratings on the subject of the article and not on the article itself (e.g. people who like Bieber a lot will rate his article high and people who hate him give 1 star).
The ratings given by anon users are not very useful because they use the rating system like a like/dislike button system and prefer to give 1 or 5 stars. Some anon users are able to give high quality and reliable ratings, but the masses do not.
It is not possible for anonymous users to turn off the tool. At this point in time, if they wish to disable the tool, they will have to create user accounts and set preferences for it.
The entire purpose of this tool is to determine if the public's understanding of article quality can be measured at all. That's why the tool was made. We will need data to determine whether or not this actually works - that we can measure an article's quality over time.
I've used Ad block plus to hide the AFT, so even when I log out the AFT is hidden.
If we have the data, will it be publicly accessible?
Who gets to decide if the test was successful? What happens if others disagree?
I can guesstimate in advance that it is possible but totally useless to measure the public's understanding of the quality of an article,
We are currently working to get an anonymized dump of the data available to the public. I don't have a specific timeline for that to give you.
I don't know who gets to decide if the test was successful but I expect that it will be our strategy department. If others disagree, I suppose they can show their proofs as to how and why they think that the data doesn't support whatever conclusions we draw from it and discussions can be had from there.
I'm not sure how much more constructive this conversation can be moving forward. I believe you've made up your mind and no amount of data or conversation is going to change it.
Well, to be honest, I've spent some time thinking about this idea and indeed, the more I think about it the more convinced I become that this is a bad idea. The answers to my questions on this page are a good reason to oppose, so the data and conversation are confirming my suspicion that it is a bad idea.
But the intention of this discussion was not to change my opinion.
The intention was to give honest feedback, change your opinion a little bit, explain why some people dislike the AFT, and to try to find a compromise between AFT-advocates and people who dislike the AFT.
So, the strategy department will decide. Aren't they the ones who asked for the AFT in the first place? If so, they are unlikely to admit they made a mistake and decide to stop the AFT. Why not let the community decide?
I'm afraid you do not want me to ask the community to give feedback to see if we get an indication that a change of direction is required because the majority might have an opinion you disagree with. Is that why you see that as a threat?
Given that you have worked on this tool, and that you list it on you talk page as a feature you have worked on, I believe you should recuse yourself of any discussion of this feature. You have, I assume, invested much time in it, and thus this is something more emotional for you than logical. Please recuse yourself from discussion of this feature due to this conflict of interest. (Copying to talk page). --Timl2k4 06:11, 8 July 2011 (UTC)
Actually, I have very little time invested in this version of the tool. I am not speaking from a point of "emotion"; this is not "my baby."
There is no conflict of interest here. I believe this is the right thing to do, and there are several impeti that allow me to push for it. My interest is "to serve the whole of the vision of the mission"; the Article Feedback Tool does so. If it didn't, I'd oppose it.
If the basis of your argument is "oh, Jorm created it, so let's ignore his comments," you're off on a wild goose chase. I urge you to have fun with that adventure, though.
"My interest is "to serve the whole of the vision of the mission"; the Article Feedback Tool does so. If it didn't, I'd oppose it."
- Your discussion here seems to tell another story. I'm not here to have fun (well I am in a way, I wouldn't be on Wikipeida if it wasn't enjoyable). We allegedly have the same goals, but this AFT is a strange way IMO to try and meet them. --Timl2k4 06:47, 8 July 2011 (UTC)
Do you mean impetuses?
Jorm, I believe you when you say your interest is to serve the whole of the vission of the mission. And from what I understand you are passionate about it. So am I. And I feel confident that we both want the same result: Better and more articles.
But AFT is not necessarily the best means to reach that goal. Also, with the risk of getting drawn in to a philosophical discussion, I believe one can improve the quality of articles without measuring it.
"... beholden to the will of the entire community ... " ... that's interesting. Of course, you will never know the will of the entire community, you will only know the will of a very small sample -- which you already have. I suggest you go with it. Move that rating box as you "community" has suggested. It really, really does NOT belong on the article page button the talk pages.
Anyway, good for you, for trying!
"While I appreciate and hear your opinions, we are beholden to the will of the entire community"
Exactly. And given that other communication channels do not seem to be working very well, I am now using that other "feedback tool", the one that appears at the top of every page a couple times a year with Mr. Wales' mug on it.
I feel sorry to have donated actual money in the early years of this project, seeing how this is now going to financing the Wikibureaucracy's pet projects and other nonsense "the community" have neither asked for nor supported in any significant manner.
I will not be making further donations in the future. It will not make any impact on the path this project is going to take, but at least my conscience will be clear.
Just so you know.
I agree. This article rating things are starting to become annoying and they are cluttering up the encyclopedia. It's absolutely unclear if they actually help improve articles. Good in theory, well executed, but bad in practice. Get rid of them. 159.226.169.135 05:22, 17 June 2011 (UTC)
Anyone here? Please remove the AFT. Thanks in advance,
Earlier I wrote: "I am not very optimistic about the value of the ratings in general but especially those of users who did not log in, I fear people will base their ratings on the subject of the article and not on the article itself (e.g. people who like Bieber a lot will rate his article high and people who hate him give 1 star)."
If I look at the dashboard I see this:
"Born this way" is an album by Lady Gaga. Bieber and Gaga are probably the two most hated artists on this planet. Now look at the other article, the German guy called "Dirk". He was in the news because he won a championship-thingy in basketball (I am not a fan), which means some people hate him and some other people love him.
The articles are rated the lowest not because the articles are shitty but because people base their rating on the subject of the article and not the article itself. Go see for yourself.
Now look at the best articles. Among them is en:Lol@souffs, an article that is currently considered for deletion and obviously inferior to the 3 worst articles. Another one is United States, because in the U.S.A. nationalism and patriotism are considered to be positive things. The third is José Rizal, a Filipino national hero. This is no coincidence of course, today he will be commemorated.
UPDATE: It's 19:12, 19 June 2011 (UTC) now and suddenly Bieber is among the best articles!
Pay attention to the numbers.
Hi Wasbeer,
rating bias for/against the topic of the article is definitely a major issue with any open rating system. It's important to note that the dashboard is a very particular query on the data, i.e. the 24 hour time window with a 10 rating threshold, which is going to produce its own odd biases. I'm not convinced yet that the current dashboard queries are useful.
One could cut and weigh the rating data in different ways to correct for bias. For one thing, it may be the case that within the submitted rating data, certain rating behavior is more or less predictive of rater bias (e.g. all 1s vs. a more balanced rating; self-identified expertise vs. no self-identified expertise). It may also be useful to take into account the overall behavior of the rater (e.g. just one article vs. multiple articles).
Pretty much every community-driven rating system applies additional algorithms to produce the total rating, and so far we're taking a pretty simplistic approach in surfacing this data. For a project like Wikipedia, we think making the data as openly available as possible will be the best way to gain a collective understanding of what the strengths and limitations of the current approach are, and to continue to iterate on the overall system based on what we learn.
Hi Eloquence!
I wish I was eloquent in English! In my native language discussions like this one are much easier for me.
I think you are right about the current dashboard queries, they should be improved. To be honest, designing algorithms is not my expertise, but I will spent some time thinking about it and if there is any result worth mentioning I will post it on this page.
It is probably worthwhile to look at other high traffic websites that have had a rating system for a long time, we can learn from the mistakes they made in the past and steal their ideas.
Subfader wrote about this and used the IMDB as an example: Extension_talk:Rating_Bar#Adapt_IMDB_rating_system
I'm broadly supportive of the Article Feedback Tool, but I think we might be able to tweak it. Wasbeer raises a good point about the potential for gaming - whether it's the Justin Bieber "haters" downvoting the article because they don't like Bieber, or "Beliebers" upvoting the article because they like Bieber, the problem is that people are voting based on their preferences for or against the subject of the article, not on the quality of the article itself.
One simple fix for this would be to make the display of the voting panel much more random. Make it so that there's only a 1 in 20 chance of it turning up, so you don't get people trying to ballot stuff it for or against particular people or things that they like or dislike. Here, Slashdot is the model - on Slashdot, you only get a certain number of moderation points every so often. It is hard to use those randomly assigned moderation points to downvote the stuff you don't like because getting them is such a random thing. This seems like an easy fix for the more popular articles to prevent ballot-stuffing and other concerns that Wasbeer raised.
One of the most important things in designing an interface - any interface - is that it should behave consistently, every time. This falls into the Principle of Least Surprise.
While I see your point about randomizing the access to the voting panel (to avoid gaming situations), doing so would clearly make the tool unpredictable, which would be a rather bad user experience.
There are better ways to detect and handle gaming (mostly done on the server side, by tweaking the calculation maths). This is something that I and Dario are interested in a great deal, and if anyone has any suggestions as to how modify the calculations, we're all ears. I must confess that gamification problems are low on the list of issues, since (as with all things in Wikipedia), they end up solving themselves over time (like spates of vandalism or errors in the text).
An article may be gamed for a short period of time (say, a couple months, or even a year) but eventually people will stop caring about that and move on and the ratings will adjust back to normal.
Toms suggestion is a good one, we spoke about it on IRC and I asked him to write it here. He said he was broadly supportive, and he did get an answer within 30 minutes, while the people who are opposed to the AFT are being ignored. I see a pattern of dismissing or ignoring good feedback on this page.
The supporters of the AFT imposed a deadline upon the community. I used that very same deadline against the supporters of the AFT.
Do you really think ratings are valuable if people can game the system for a couple of months or even a year?
Wasbeer, your comments have ceased to be constructive in any way.
If you wish to continue to have a dialog about this, I am willing to do so but you must decide to be a useful part of the process.
Sorry, I do not understand you. Please remember this is not my native language.
Was I impolite to you in any way? If so, I apologize in advance, but please do tell me what the problem is.
I think my comments are highly constructive, and I even give a lot of useful suggestions that can be implemented if my advice to stop with the AFT is ignored.
You are right to question, that Wasbeer. The comments that you comment on appear to be a non sequitur. I wonder if this message board has malfunctioned somehow. That could explain the difficulty in following this...
Example of completely biased data is not helpful? Maybe because it is showing why it is bad idea? Is it possible for WMF to stop wasting time and money on wikilove extension, abandoned projects (LT) and useless surveys and provide WYSIWYG editor?
Development on a visual editor is and has been a priority for the Foundation. We've got. . . three people working on it full time, and another person working on it half-time.
The visual editor is being developed by a different team, however.
And finally I can say: good news!
The article feedback tool is a cancer on Wikipedia. At this point it ought to be abundantly obvious that is not a good idea but, like with Flash welcome pages of the 90's web, it takes so long so some people to recognize the obvious. A) These things are ugly and detract from the aesthetics of an article. B) They are huge and especially distracting on smaller pages. C) It's clear even from a quick glance here that there's a strong percentage opposed to them, so why are they still around? D) The proponents are using arguments against the opponents and are failing to apply those same standards to themselves. That's annoying. E) I do not think that the information collected will be properly used and interpreted by editors. Interpreting surveys is hard and naive attempts are more likely to misinterpret the data, F) Small sample size will lead to incorrect conclusions being drawn and acted upon. To all those involved in its creation, swallow your pride and abandon this blight on Wikipedia. We appreciate your effort but the solution is worse than the problem. Jason Quinn 02:04, 21 June 2011 (UTC)
- B) They are huge
For the record: This was reported on Bug 29303 (improve layout of ArticleFeedback tool).
Jason made a huge mistake by mentioning cancer, its like Godwins law, please refrain from using analogies that are incorrect and offensive. We, the people who oppose the AFT, should try to be way more reasonable and polite than the people who support the AFT at all times.
Remember: this is not a debate we are trying to win. We already have. Now we need to gently inform the supporters of the AFT that even though their efforts are appreciated we do not want the AFT. It is much easier for them to accept this if we are nice to them.
I acknowledge the fact that the people who support this idea are acting in good faith and I think certain parts of the AFT are certainly well thought-through. A lot of time and effort have gone into this idea, and even though Jorm and I are not always on the same page I am sure he has the same goal as we do (to improve Wikipedia), even though we disagree about the way to achieve this common goal, and is not responsible for spreading any deadly diseases.
I hope Jason will understand that I agree with the rest of his message, but that I think it is important to say something about that single word.
If we want to use analogies we need something that looked like a good idea when it was under development and turned out to be not so great in practice, but in general it is better to refrain from using analogies in situations like these.
When when did using the word "cancer" metaphorically become "incorrect and offensive"? I never got this memo. I consider these article feedback surveys a harmful addition that's spreading throughout the encyclopedia worsening its quality. According to you, "cancer" cannot be used to describe that? At worst I was being blunt, not offensive. For that matter, I don't even think "blunt" applies. I made a run-of-the-mill comment. Has PC'ness really degenerated to such a point? Quite frankly, I felt like I was reading an Onion-article when I read your comment.
Maybe your parents can explain this to you.
Incidentally, due to the annoyingly garbled nature of... I think this is LiquidThreads? Anyway, the point is I'm really not sure if anyone can see the message I posted yesterday. So... here's a fresh link.
If the WMF doesn't heed feedback, why would editors? DragonflySixtyseven 13:56, 7 July 2011 (UTC)
Yes, this is liquid threads, awful isn't it? I certainly find it hard to use, and hate having to keep clicking. Dougweller 06:21, 8 July 2011 (UTC)
This post was posted by Doug Weller, but signed as Dougweller.
Is there a way to disable this LiquidThreads thingy? Or to get the source of the full page? LiquidThreads is extremely annoying, especially in long discussions.
Update: I think we need to add {{#useliquidthreads:0}} to the sourcecode of this page.
There is not a "full page": at least in the current implementation, there is one page for each comment. The version 3.0 of the extension, to be deployed after its ongoing redesign is finished, will be a lot better (hopefully).
You can suggest specific enhancements to it on its talk page or filling a request on bugzilla.
Anyway, the new topics are displayed on Special:NewMessages to anyone who is watching the talk page.
"maybe your parents can explain this", That is a very nice retort isn't it.... >.< I have never known anyone to be offerended or disliked when they saw cancer being used to mean "unwanted", so perhaps you can expand on this.
(Oh, this is also coming from someone that has had a tumour removed their body)
I only now noticed this comment but I could not let it go. Let me get this straight. You made a big deal about me using the word "cancer" in a common metaphorical way and somehow found it offensive; yet, at the same time you can make a bad cancer joke (about your parents nonetheless!) and consider that kosher? That's messed up. Jason Quinn 14:17, 4 October 2011 (UTC)
Wasbeer appears to have quit participating in the middle of July, after discovering that his feedback would not result in the AFT being removed.
While the metaphorical use is common to the point of being unremarkable, in a sufficiently large audience, it's likely that someone will be dealing with the death of a loved one from cancer. Grieving people are not known for being rational, and people who have seen "dying from cancer" up close often forget that most cases of cancer don't kill people. For example, at least one-third of cancer cases are trivial non-melanoma skin cancer cases, which can be permanently cured in mere seconds: that's "cancer", but it almost never results in death.
I think it would be appropriate to just write off this whole exchange as someone having a bad day (bad month? The intense grief after losing a parent, spouse, or child commonly lasts for a median of four months).
That's why my mafia analogy fits much more!
@Jason Quinn: I expect you to apologize and retract that comment if you have any decency.
@WhatamIdoing: Bad year. The real reason that I quit commenting here was because it was useless. I am not a native speaker and I was unaware this metaphor is so commonly used in the English language.
I don't know the context behind this thing, and this is the only place I found where it is discussed besides some threads at the Village Pump.
First, and maybe I am not very well informed here, but why the hell is AFT forced on the English Wikipedia without a clear community consensus? Why is it discussed at this obscure wiki and not where it matters most?
Second, I echo Jason Quinn's comments above. Good effort and all due respect to those who thought it out and what not, but seriously, I cannot see a valid use for this thing besides being disruptive on the (seemingly random) pages where it's placed. At the very least, make it into a template or something, so that editors can remove it from pages and discuss on specific talk pages whether it's needed or not.
Please remove this ASAP until community consensus has emerged. CharlieEchoTango 03:48, 21 July 2011 (UTC)
See the section "You didn't ask my permission to do this!" of the FAQ.
As mentioned here, it is already possible to remove the tool from specific pages by adding those pages to a category which is used as a black list. On Wikimedia projects, the category is set as "Article Feedback Blacklist" by default.
I have to agree that this doesn't seem to be working. Whether there's a sudden spike in "pro-Bieber" or "anti-Bieber" ratings is just a symbol of a deeper problem: readers can't tell you if an article is of high quality. They can only tell you if they like it.
Trying to "quantify quality" is one of the most impossible things you will ever try to do as a scientist. You may as well ask people to rate the articles for how delicious they are, or how nice they smell.
Much better to make it easier to "qualify quality". If you gave readers a red pen, they'd tell you far more than a numerical rating.
I think it's too early to tell where the quantitative ratings are or aren't useful. There's definitely a fair amount of "like/dislike" based voting, as well as attempts to manipulate/game the ratings in various ways, which is entirely expected. A big question is whether, when looking at the trends in the overall rating data or on specific articles, it's possible to translate these ratings into meaningful action (e.g. articles that may need tagging/cleanup, undetected vandalism, articles that are candidates for promotion, etc.), or whether there's too much noise in the data to do that.
To answer that question, we'll have to let the ratings play out for a while, and look at the data from different angles.
But, I do agree that soliciting more specific feedback and action (other than direct edits) is likely to ultimately have more impact. What do you think of the ideas in Article feedback/Extended review?
How long is a "while"? How long do you estimate it would take to get enough data that would actually be sufficient to translate it into meaningful action? For the lesser frequented articles that may actually need the meaningful action I anticipate it would take years!
There is no way in which we can increase the volume of ratings for articles that receive a small number of visits unless we change the positioning of the AFT to make it much more prominent , which is not planned. The ramp-up of AFT to the English Wikipedia was completed yesterday so we are now able to collect and analyze data across a much larger number of articles than the previous 3% sample. We expect to have new results ready to be published in the coming weeks. If you have specific questions, please post them in the AFT research talk page or on specific research subpages: 1 - 2 - 3 - 4.
I don't like the feedback tool. It reeks of insecurity or hubris. Maybe both.
I don't read Wikipedia to pat or slap its authors on the back, really I just don't care that much about it. I understand that the authors/owners/whatevers really care a lot, but I don't.
I'm trying to articulate what I don't like about the feedback panel. Perhaps its because it's an advertisement/promotional style thing which is stylistical masquerading as content. Perphaps if it was flashing yellow/red "You're the 1,000th visitor to this page..." I might pay less attention.
In short it'd be great if the survey thang had an option [x] Please Don't Bother Me Again On This Computer With Surveys and Begging Notices.
If you are logged into an account, you can turn it off permanently. The directions are on the FAQ page.
This is another reason why the tool should be moved to outside of the content of the page (Bug 29704).
Yet another superflous feature that only makes the system more complex, makes the articles uglier, wastes screen space (lots), wastes work, consumes more bandwidth, and diverts the readers's attention. (Actually, the whole article evaluation machinery is a huge waste of resources.) Please junk it. Wikipedia must be made simpler, nor more complex.
I am user Cazort on Wikipedia. I gave a detailed explanation of why I think this tool goes against the spirit of Wikipedia's policies, and how I would like to see it removed, here: w:en:Wikipedia_talk:Article_Feedback_Tool#I_believe_this_tool_conflicts_with_the_spirit_of_Wikipedia_policy.2C_and_would_like_it_removed
Why is everyone's opinion being ignored?