Talk:Growth/Focus on help desk

This is a topic for community discussion around the rules, wording, and design of the help panel.

In User testing for help panel, the comment "Discoverability of the help panel call-to-action may be reduced in the case when the user is using the Visual Editor or wikitext2017 editor because those editors have a competing '?' icon", is certainly an issue. In VE [I'm less certain about the wikitext2017 editor], the dialog invoked by clicking on that "?" icon should be changed. It now begins "If you encounter any technical issues as you edit, please report them," which (a) is not likely to be why the user clicked on the icon, (b) is duplicated by "Leave feedback about this software", and (c) made sense back in the day when this was beta software, but it's not anymore.

As to what the VE dialog should look like, I suggest "Help:" as a heading, followed immediately by four items: "Open the help panel", "Read the user guide", "Keyboard shortcuts", and "Provide feedback on this software". That (obviously) gets the user to the help panel in just two clicks, while offering some other options.

Also, to quibble, a help icon is usually in the form of a question mark with a circle around it, not a naked question mark. And the icon for the user guide (and maybe the help panel), within the dialog, should be an "i" with a circle around it, for information.

Thanks for thinking about this, @John Broughton. We also were thinking about the idea of letting the help panel be an option in the "?" menu. But what do you think about the help panel potentially just superseding the "?" entirely? And removing it from the toolbar? We are going to be able to keep an eye on this, because we recently started tracking how often users open the "?" menu.

The only thing that I think is useful in the current ? dialog is the link to the user manual; I'd hate to lose that. Maybe split the difference - have a "circled i" (information) icon that links to the user manual, and a (circled) ? that offers help?

Alternatively, something that has been discussed at some length, but not implemented (for which I'm probably the most to blame, since WMF resources were offered to me a while back, to test the concept) would be to put links to various sections of the user manual in context-specific places (specifically, the various pull-down menus, and menu items of VE). That would obviate the need to put a main link somewhere.

Yeah, I was thinking that we could put the link to the user manual in the help panel, so that we would consolidate all the helpful links in one place. How does that sound?

The idea you described about put links to various sections in context-specific places sounds like the "In-context lessons" idea that you helped us discuss a few months ago. I'm hoping that after we get some learning through the help panel, we might turn attention to that idea.

Sure, a link in the help panel, to the user manual, is quite satisfactory.

Also, I'm guessing (I don't see an explicit statement on this) that you're relying on a volunteer editor's response to a user's question to actually include the user name, which will then trigger an email to that user [I think]. However, while including the user name in a response is common, it doesn't seem universal - see, for example, https://en.wikipedia.org/wiki/Wikipedia:Help_desk#Fixing_Odd_Formatting_on_Article as well as the question and answer immediately below that section. And it's not mentioned in best practices at Growth/Communities/How to work with newcomers on help desks . Are you planning to automate/ensure that an email is in fact generated for the new user when the user's question is posted?

Concerning the point about mentioning people, it is already in the best practices: "Make sure the newcomer is aware of the reply, by any method."

Thank you for pointing it out, though: I've clarified the sentence by having it as a separate point and added "For instance, on wikitext talk pages, mention the user when you reply."

@John Broughton -- an idea that @Martin Urbanec brought up is to use an abuse filter to remind responders to mention someone in their reply. Do you think that could be a helpful approach?

While using the abuse filter has potential, I worry about the details. Specifically, not all edits require a user name - the first in a section, obviously, but possibly the second, because sometimes the user with the questions posts and then edits his/her post, or adds a second post immediately. Or the pattern might be (1) question, (2) response, and (3) elaboration on the response by another (helping) editor; if so, I suspect that the editor of the third post would not appreciate getting a warning, since the second post should already be notifying the user who had the questions.

Regarding "Provide clearer times on both the review and confirmation screens for when a newcomer should expect a response from the help desk", that is obviously going to vary by community, and there isn't any way to pick a single number (or range) that is both accurate and helpful across all communities. (For example, "one day to two weeks" is probably accurate, but not helpful; "Within 24 hours" is helpful, and for the English Help desk is accurate, but may well not be for many other communities.

So, two options come to mind - either each community can specify that parameter (hidden comment on the Help desk page? central list that is updated by stewards upon request from an editor?), or an automated way of calculating the average time - say, a bot that checks once per day to compare the date/time stamps in each of the most recent ten sections that have both an initial posting and a posting that is [presumably] in response [different user name].

@John Broughton -- that's a good idea about letting each wiki specify their usual response time! We're going to hang on to that idea. For now, we know that Czech and Korean Wikipedias (the pilot wikis for this project) can commit to the "day or so" response time, and so we have it hard-coded that way. I hope that if this feature works, and if communities develop enthusiasm around it, they might be able to commit to shorter and more specific response times.

That response time may change over the year, since some people may be away, or some events may slow down he activity on wikis.

One way to deal with a response time promise that hasn't been kept is to provide a link/option for the new user to report that "I have not had a reply within the promised time". That report might go to OTRS, for example, or to someone specifically charged with maintaining the response table.

Interesting idea. It sounds like something we could include when we think about a way for users to give feedback on their experience with the help panel. We were thinking about that here on Phabricator, and I've added a note.

Some ideas are described here: https://www.mediawiki.org/w/index.php?title=Topic:Uq5p24kedymiw6pq&action=history.

I'd say the greatest failing of the current design is the fact that it includes an ability to immediately ask a question before even searching, and it makes it optional to add a page title. Neither should be visible in the first page, and it should not be possible to choose to include the page title as context, as that should always be included in the feedback.

In the short term it might be reasonable to include an extra item in the drop down for visualeditor, as suggested above. Long term, they should be combined into a button that routes the feedback to the appropriate place, either phabricator (for technical matters), VisualEditor/Feedback (for general feedback), a help desk (for general support), or a talk page (for page specific issues) depending on the nature of the problem or feedback.

Also, eventually there should probably be a non-javascript fallback, maybe Special:Help for when either editing without javascript or for users with temporary network glitches.

I think a non-JS fallback is essential; anyone editing on a low-bandwidth or expensive connection has a strong motivation to not use JS. Things that require JS, or lots of bandwidth, risk increasing systemic bias.

Thanks, @HLHJ. That's an important point. Right now, for non-JS users, the help panel icon shows up just like for other users. When they click it, however, instead of the dynamic experience of interacting with the help panel from the editing context, it opens up the help desk itself in a new tab. At that point, the user would be able to follow the directions on that page to ask their question. Do you think that is a good fallback? Can you think of other ways to do it?

Добрый день!

Хотелось внести правку в панель "Помощь по редактированию - Предложенные правки - Советы по быстрому старту".

В предложении "Корректура помогает статьям быть более качественными и надежной." заменить слово "надежной" на "надёжными".

Спасибо!

@Павел Витальевич Новиков, it has been fixed. The change will be visible soon!

Черновик готов! Как его опубликовать в Википедии? Чтобы все его видели при поиске например гугл? Моя статья не появляется??

Although we originally started referring to the feature we are building in 2018 Q2 as the "help pane", we received some feedback that the name would be clearer if we called it "help panel". "Pane" sounds too much like "pain", and could be confusing. I updated the name on the project page and on the related Phabricator tasks.

The panel says that it can be disabled but even when one unchecks the enable the help editor box and saves your preferences the panel still keeps showing up for me a day later. This is more annoying than the panel. Not sure if this is a bug or what but its very frustrating.

Hi, could you say a bit more about where exactly (main namespace; talk namespace; editing/reading?) you see the help panel after unchecking the box? Thanks!

Thank you for reaching at us. on which wiki do you see it? Are you logged-in?

I'm testing the add-link beta feature and I realized that all the sites I've edited are on my watchlist. Is it possible to disable adding them there because it makes a mess and annoys me terribly.

Thank you for testing the tool, Krzysiek.

Have you selected "Add pages and files I edit to my watchlist" 'Dodawaj do obserwowanych strony i pliki, które edytuję) in your preferences?

What do you think of the quality of suggested liks?

No, I have this option unchecked. Only when I make edits with this beta they are being added to my watchlist. I think suggestions are very good. Polish is very difficult language, and AI doing great and hardly ever misunderstoods the meaning. Most links I refuse are things that everybody knows.

Thank you for your feedback.

I'll ask about the watchlist but I think it is done purposefully. This way, newcomers can check what happen to the article after they added some links. What newcomers do is different than what experienced users do.

Good to know that the AI works well for Polish!

The auto-sliding is annoying. I do not succed to read the entire slide. The auto-sliding could be replaced with a round and blue button that says "Next".

Thank you for your feedback!

This is not something that showed up during our tests, since users were clicking back on the numbers to read the content of the tabs. However, I'm passing it to the development and design team. :)

May I ask where you checked on our tools?

I tested the help panel here, when editing pages. An alternative to button would be the adjustment of the speed for each slide. Also, each slide has more or less content to read, so (for example) for slide 1 you need 7 seconds to read it, for slide 2 you need 10 seconds to read it and for slide 5 you need 4 seconds to read it... (These numbers of seconds are randomly; only examples).

The goal of this auto-sliding is to show that tips are available. Then people can click back to see them, which stops the timer. We are considering to have arrows instead of tabs, to make it clearer. What do you think?

I think you need to update, I just don’t know which part of form is important to you.

You mean the prototype image in the infobox?

Yes, and the fact that no actual screenshot of the page looks a little strange.

@Iniquity -- thank you for pointing this out. I added three up-to-date screenshots to the top of the page under the infobox. Does this seem better?

@MMiller (WMF) Yes, much better, thanks :)

A note template with the text "The sections below may be changed significantly in the coming weeks, or they are too technical or less relevant for the understanding of the project. We have decided not to have them translated." is currently above the paragraphs that are marked for translation. May I ask that someone either moves it where it should be placed now or makes some other action, so that it was a bit clearer for translators? Thanks.

This note is above the paragraphs that are not mandatory for translations. However, some users prefer to have the whole page being translated. We, as the Growth team, put a limit on what is really important to be translated, but it is up to anybody to translate whatever they like.

Oh I see. Previously it wasn't even marked for translation down of the note. I guess I misunderstood it.

It increases edits during a newcomer's first day, but decreases them over the following two weeks.

I support improvements, exploration and experimentation. A project may of course require a few revisions before reaching positive results.

If the team doesn't find a way to reverse the longer term negative impact, can someone confirm that there is an explicit plan for full rollback? This is a general concern to avoid accumulating complexity, cruft, and inconsistency across our wikis when some projects don't work out.

@Alsee -- thanks for following along and reading our notes closely. The help panel indeed has ambiguous results. On the one hand, we see anecdotal stories of users who ask questions, get answers, and then continue to edit. On the other hand, we don't see an impact on activation and retention, and we see this ambiguous impact on edit volume. So one thing that you're pointing out that we don't know is whether the help panel seems to increase or decrease edits over the entire first two weeks. I think that so far our analysis looks at Day 1, finds a positive impact, then looks at Days 2 - 14 and finds a negative impact. We can look into what the overall impact is.

But beyond that clarification, our current thinking is that the help panel may work better as part of a broader "integrated newcomer experience", along with the welcome survey, homepage, and newcomer tasks. In other words, it may not be a specific feature that makes the difference for newcomers, but a general overall upgrading of their experience via several connected features. In fact, for newcomer tasks, we plan (in the January/February range) to evolve the help panel to be the feature that guides newcomers as they complete tasks.

All this is to say that we want to keep the help panel around for now and integrate it with the newcomer tasks experience, and see how that affects its usage. If it is not proving useful in that context, then I think we'll need to revisit the questions you're bringing up -- because I agree that we should not leave up features that are not working.

How does this sound?

Thanx. If you reach more positive results that's of course a good thing, and I apologize if it seemed I was jumping the gun about the initial results. I appreciate your confirmation that we should not leave up features if it turns out they're not working. To be honest, the topic caught my interest because there have been difficult times where teams have resisted their own data when it was unfavorable or counter-intuitive. It's easy for good intentions and big positive goals to feed into confirmation bias. The most crucial metric for any project is our inflow of new users as sustained-activity community members. Any negative data in that area should be taken very seriously.

"we also observe that large numbers of newcomers open the help panel (20%) and interact with it once open (50%)" -- how can the first percentage smaller than the second?

@Samat -- thanks for reading closely. It's because the 50% is 50% of the 20%. In other words, of the people who open the help panel, half of them interact with it. To get the overall percentage, you multiply them together (i.e. 10% of people who see the help panel button interact with something in the panel). Does this make sense?

Thank you, that's clear for me now. (And I made it clear in the translation as well.)