Section Editing – the feature – is intended to improve editors' experience editing with the visual editor on mobile. But, does it? This is a question this test is intended to help answer.
With the above in mind, what did you think of the prototype? In what ways does Section Editing function how you expected it to? In what ways does Section Editing not function how you expected it to? How do your past editing experiences with the visual editor on mobile compare to your editing experience with the prototypes?
The questions above are intended as starters. So please, do not feel limited – we value any comments, feedback or questions you feel relevant to share.
Feedback here will be summarized on the main project page.
Previously, we had stated contributors using Section Editing are 1% more likely to save the edits they start than contributors using full-page editing. The correct difference in the probability that a contributor using Section Editing will complete their edit compared to a contributor using full page editing is ~3%*.
We are sorry for any confusion this might have caused.
---
*2.71% to be more precise
Just saw the results of the A /B test, and it wasn't surprising at all. The mobile visualeditor is too limited to be used frequently.
Anyway, as far as section editing is concerned. I think that it ends up having limited impact because of the massive space that is being used by the site and the browser. Consider this, there is:
Meaning that one an average phone you're left with 2 sentences to write on. You can test this out with Chrome's mobile emulator, by setting it to use nexus 5, and triggering the keyboard, or just check out https://imgur.com/aSDRFup. Aside from that at least for anynomous users, one great hindrance of all A/B tests is the awful big message asking users to register. It is presented in such a way that the average user will think that they must register before editing, even if the message says otherwise, because it actively blocks attempts to go directly to the editor until you click one of the buttons. There are a few times that I also get an overlay saying something about "welcome to wikipedia", also requiring another click.
One suggestion might be making the main VE toolbar collapsible, and perhaps considering a sidebar toolbar with just simple semi-transparent buttons.
Anyway, I think that until such issues are sorted, the benefits will be greater on desktop devices, and it might be more interesting to evaluate editing patterns there (e.g. how many section edits vs full edits), once it is deployed.
First off, thank you for being as thoughtful as you have been in sharing your feedback.
I want to make sure I have understood all of your points, address each one and then ask you a few questions (if you're open to them!).
1. Section Editing is likely to have a limited impact because it does not resolve the fact that the mobile visual editor is difficult to use. It is difficult to use because there is limited space to edit.
We also think the limited space to edit in the mobile visual editor can make it difficult to use. In fact, we are actively (as in, at this very moment) working on a new feature (see: VisualEditor on mobile/Edit cards) that will ideally create more space in the editor.
One of questions we are asking with this feature is: Will contributors complete their edits more often when there is more space for them to do so?
This leads me to wonder: would you be up for trying out an early version of this new feature and sharing what you think about it? We have some testing instructions here if you are up for it: 📲 Test Edit Cards v1.0
2. The current mobile VE onboarding experience is disruptive: the account registration overlay (see: https://imgur.com/iM8DPts) deters contributors from editing anonymously.
We know about 40% of contributors drop off before the editing interface is ready to be used (see: VisualEditor on mobile report) and we have ideas for why this could be. But, to my knowledge, we do not know the extent to which the account registration overlay is a cause for this attrition.
This, and the other feedback you shared about onboarding is now now included in a Phabricator task. Do you have any more thoughts about mobile onboarding? If you do, we would value seeing them. Here is the task where that conversation is starting: task:T226262
3. The "Welcome to Wikipedia" overlay (see: https://imgur.com/a/m9rfDh6) is disruptive to someone who is used to editing Wikipedia.
For someone who regularly contributes anonymously, I can see how having to tap through an extra screen each time you are trying to make an edit would feel unnecessary/disruptive. I've asked our team about the logic that determines when the "Welcome to Wikipedia" overlay is shown in an effort to determine whether this message can be suppressed for returning IP contributors. I will reply to this comment when we have an answer.
4. Section Editing is likely to have a greater impact on desktop where editing is more comfortable.
This would be an interesting hypothesis to test. Although for right now, our team's priority is simplify editing for contributors on mobile, using the visual editor. Here is some more detail about that: Annual Plan/2018-2019/Audiences
With this said: we do have a task open for enabling section editing on desktop in. Here is that task in case you would like to track its progress: task:T221908
5. Maybe changing the editing toolbar could improve the mobile visual editor's usability.
While I think this suggestion was made as a possible way of creating more space for contributors to edit, I thought you would value knowing we are actively thinking about how the editing toolbar within the mobile visual editor can be improved to help contributors more start and save their edits. There is some more information about that work on this project page: VisualEditor on mobile/Toolbar refresh
Your comments have been helpful and clarifying. Thank you for taking the time to share them ^ _ ^
And by the way, my name is Peter. I work as the product manager on the Editing Team. 👋
It is difficult to use because there is limited space to edit.
Yes, that's definitely one of the prime reasons.
This leads me to wonder: would you be up for trying out an early version of this new feature and sharing what you think about it?
I went through it. The linking problems:
Also turning on the browser spelling checker might reduce fixing typos before fixing the links. The referencing problems are pretty much the same as above.
We know about 40% of contributors drop off before the editing interface is ready to be used
For desktop devices, Mediawiki's silly interface forces an Editing interface load whenever one clicks any redlink, sometimes even for cases like red category links that don't even need to be created (see https://phabricator.wikimedia.org/T29311). Mobile devices stop unnecessary loads, and when triggered a full edit dumps the user on an empty page. It is a bit like throwing a user into a lake and hoping they can swim. The average person will try their best to get out ASAP.
"Welcome to Wikipedia"
It appears for first time editing. What's worse is that the two dialogs / overlays sometimes are triggered consecutively, one after the other. This could really be a dismissible one line "tip" that doesn't obscure the editing interface.
The likely only feasible way of exposing the full extent of the tools is either sub-menus or context aware toolbars.
This feedback is wonderful - thank you for taking the time to experiment with the prototype and communicate your thoughts. I'm going to try to respond to each one...
1. Add a new link: the fullscreen link search overlay feels disorienting
This is understandable. You are now the second person that has mentioned issues with the search overlay. We have a few ideas about what could be contributing to this feedback:
- Whole screen: as you mentioned, the search overlay takes over the whole screen.
-View title: the search overlay doesn't explicitly communicate what action you should be taking inside of it. The text in the title bar just says "Link."
-Transition: the way the search overlay "transitions"/"overtakes" the article and editing tools does not feel right.
- Context: besides showing text within the search field, the search overlay does not include greater context about the content/article you are trying to edit.
We will be addressing the View title issue in the second version of the Edit Cards that we are working on now. Here is a peek into what they are likely to look like: Edit Card v2.0 designs. If you have feedback on these, please let me know...these designs will get posted to the project page later this week.
We are also beginning to think about Transitions. That work and the conversation around it is happening in this Phabricator task: T224277
We will see if these interventions have an impact. Although, it is possible the root of this "loss of context" issue could turn out to be, as you suggested, that the search overlay takes over the whole screen. We're going to keep experimenting so we can find out for certain :)
2. Change a link target to an external link: two tabs are unnecessary.
Oh, interesting. Are you suggesting having two tabs – one for wikilinks and one for external links – is unnecessary because it adds friction to adding external links? If so, this friction is partially by design. Adding external links is not something we are trying to encourage more of.
Spellchecking
I want to make sure I've understood you correctly. Are you suggesting the following?
a) Spellchecking is not working for you in mobile VE.
(If so, what device and browser are you using? Spellchecking should work.)
b) Without spellchecking, contributors might be compelled to fix typos instead of doing more high value edits, like adding and modifying links and citations.
Taps on redlinks do not likely represent an intent to edit
It's funny you mention how contributors who tap on redlinks are not likely intending to make an edit. Right now, we are having a conversation about this very issue, albeit in a different context. We are talking about whether it makes sense to include edit sessions started by taps on red links in our calculation for edit completion rate. If you are curious, that conversation is happening on Phabricator, in this thread: exclude edit attempts initiated by red links.
Welcome to Wikipedia/onboarding appears for first time editors
You are correct. I checked and the "Welcome to Wikipedia" message will appear when you edit for the first time. Where "first time" is determined either by the prefs database (for logged-in users) or cookies (for logged-out users). This explanation is courtesy of the always helpful, @Whatamidoing (WMF).
This aside, I share your perspective that the mobile onboarding could be improved :)
Last thing! I had a couple quick questions about your experience with visual editor...
> Edit Card v2.0 designs. If you have feedback on these, please let me know...these designs will get posted to the project page later this week.
They mostly seem sensible except for the link label taking up the whole screen, as well as the search dialog (once again). At most 2 lines should be sufficient for that.
Also, while it is easy to type on a desktop device, it is downright painful to write long sequences on a mobile device. I'd suggest possibly providing existing links on the article as possible link locations, in addition, it would be pretty good to collect Special:whatlinkshere or related articles entries from the main namespace, as possible link targets.
> is unnecessary because it adds friction to adding external links?
Yes, partly. But the main reason is that it is one more component that needs to be learnt. More elements = more confusion.
> Spellchecking
I tested it out again, and it seems to mostly work. The concern was that if spellchecking was disabled, people might have more difficulty linking, because words might have typos.
> Redlinks
A good number of redlinks are quite simply caused by random typos, or mistakes. Others just waste time because in some cases a page may be protected from edits, and the user can't ever edit it anyway. It might be interesting to evaluate how many times people click red links to pages that they can't edit.
> How familiar are you with the visual editor on desktop?
I'm very familiar with it.
> How familiar are you with the visual editor on mobile?
I'm moderately familiar with it.
>How frequently do you use the visual editor on a mobile device?
Not frequently. Due to size constraints it seems more suitable for minor edits like fixing typos, and small changes.
> How frequently do you use the visual editor on desktop?
I've used it quite frequently in the past, but sporadically nowadays because I don't edit as much.
197.235, is the "size constraints" problem primarily about the visual editor, or does the mobile wikitext editor have the same problem?
>197.235, is the "size constraints" problem primarily about the visual editor, or does the mobile wikitext editor have the same problem?
Visualeditor's makes size constraints worse because of an excessive amount of toolbars, dialogs, and menu icons, and full screen overlays. This is a generic problem in any mobile editor, but visualeditor makes it worse. It is built in to its design.
But it is possible to make it a bit more convenient, compare VE mobile editor, to froala (test it on a mobile too) See examples, www.froala.com/wysiwyg-editor/examples/predefined-link-list , www.froala.com/wysiwyg-editor/examples/quick-insert. That was one of the more prominent mobile compatible editors that I found in a quick search, there are probably more that do it even better.
It is not perfect in any way, but it is in many ways more intuitive than something that hides content from the user. I think there is a usability guideline about never hiding essential information from the user.
RE: Edit Cards v2.0 designs
1. The fullscreen link search and label editing overlays can feel disorienting
We hear you on this. As you've seen, v2.0 still contains a full screen overlay for the link search dialog (we have not yet implemented a full screen dialog for editing an existing link's label. Although, we are planning to experiment with this in a future iteration.).
The full screen overlay remains for the link search dialog primarily remains because despite some instances of people finding the full screen overlay confusing, the majority of test participants were able to complete the editing tasks successfully without any mention of the full screen overlays.
This is not to say this feedback is anyway less valid, but rather that we have prioritized revising other parts of the workflows that blocked test participants from completing an editing task. A list of these "blocking" issues can be found here: phab: T227894
2. Typing on mobile requires significant effort. Potential improvement: offer more suggestions of potential internal link locations within search results
This is an interesting idea. I wonder: what gave rise to this idea? For example: how often – if at all – do you find yourself not finding the Wikpedia article you are seeking to link to within the search results while editing on mobile?
In the meantime, this idea is being track in Phabricator here: phab:T228353
3. Links to Wikipedia pages being separate from links to external webpage could be confusing to newer contributors.
This is a good catch. In fact, confusion around external links came up in our most recent user tests. The solution we have in mind for right now is to provide better instructions for contributors when they are searching for internal and external links. If you'd like to read more about this, here is where it came up in Phabricator: phab: T221299#5330448
4. If spellchecking was not working, it is more likely for words within articles to be misspelled. This could increase the likelihood contributors would end up trying to link to a page that doesn't exist (because it was misspelled).
This makes sense. Thank you for clarifying. Although, I'm glad to hear spellchecking is now working as you expect it to.
5. Red links
Interesting. To your earlier point, it might also be interesting to do something on desktop – similar to what is done on mobile – to create an intermediate between a potential contributor tapping on a red link and the editing interface loading. This is being talked about in Phabricator here: phab: T223339#5297463
6. Usage of VE
This is helpful context to know – I appreciate you sharing this.
Also! We have a version of Edit Cards version (v2) you can actually use.
Would you be open to trying this new version and sharing your experience with it?
Here is a link to test this new version, provided you are interested [and have the time!] : 📲Edit Cards v2
...either way, thank you for all of the time and thought you continue to put into your comments.
> I wonder: what gave rise to this idea? For example: how often – if at all – do you find yourself not finding the Wikpedia article you are seeking to link to within the search results while editing on mobile?
It is simple, an essay at a school, or a research paper, or even a wiki page, needs cross links, even if they are made up in fiction based documents. This is similar in wikis, e.g. a well written document about aids will always include tuberculosis because the two are linked. It is wasteful and tiresome to require users to have to type and search for what are obvious links especially in mobile devices with limited space, and bandwidth. In a perfect world, the search results would always prioritize such links, in addition to any pages that come from the same category or in intersection of categories.
> Redlinks : create an intermediate between a potential contributor tapping on a red link
In many cases an editing interface should never load.
In their desire to promote editing, earlier mediawiki developers were too overzealous. For instance all redlinks to user pages (e.g. user:Bugabuga) should never trigger an editor (and user:pages should really be Special), especially when it is not the actual user or the user doesn't even exist. The same applies to category pages, and is exacerbated by bugs like phabricator.wikimedia.org/T14019 . In the case of a non-existing user, someone can pre-emptively create an attack / spam /insult page for a popular username, and if the user is ever created they automatically get that spam for "free".
You'll also note that Special:wantedpages on this wiki is full of pages that should never be created. It is order of magnitudes worse in wikis like wikipedia.
Finally, many users (including myself) just open up the editor to see how something looks like when previewed, never intending to save, because mediawiki and Visualeditor lack a proper sandbox interface like Special:graphsandbox .
>Would you be open to trying this new version and sharing your experience with it?
Done.
Both worked well for me with no preference either way. I did however compare to wikitext editing and it loaded in half the time.
Thank you for giving the prototype a try ^ _ ^
That's reassuring to hear about the loading times you encountered. Out of curiosity, in the times you've used visual editor on mobile in the past, how often do you remember feeling like you were "waiting" for the editing interface?
@PPelberg (WMF), check the title—VE's loading time was worse, not better :)
Oh, no – I'm a bit unclear...@Neil P. Quinn-WMF, what do you think is meant by this, "I did however compare to wikitext editing and it loaded in half the time." here?
I interpreted "it" in the second sentence of the post's body as referring to section editing given the thread's title, but perhaps "it" is actually a reference wikitext in which case, yes, I am mistaken...
213.57.79.144 if you're out there, somewhere, please know your help is needed 🔭
Issue: Unable to switch from wikitext section to visual editor section
Steps to reproduce 1. Edit section using wikitext editor 2. Click the dropdown to switch to visual editor
Expected Loads just a section in VE
Actual Loads the whole article in visual editor.
Note, this only happens after toggling the desktop mode, when clicked on the footer. So this might not be a 100% relevant to mobile devices.
A lot of established editors use the desktop system on their mobile devices, so it's possible that this will happen more than it might be expected.
I'm having trouble reproducing this issue because when I try to switch modes, I get the prompt to save. This is documented in phabricator.
197.235, if you see this, could you tell us what your web browser and OS is? For example, Safari on iOS or Chrome on Windows 10.
Extra white space is a known issue being tracked.
Thanks @CMadeo (WMF)
The white space is a known issue and can be tracked on phabricator.
Cool stuff.
General note: perhaps this was done intentionally, but I wasn't able to find any instructions regarding which parts of the prototypes to focus on/compare when giving feedback. I think it would be useful to get a bit of guidance regarding what is different about Version A and B, and perhaps even a sense of what decision you all are trying to make, i.e. why you're testing these two versions.
Thanks for the feedback @AHollender (WMF) - we were trying to leave the questions and test open-ended, but will keep this in mind for future tests.
Sorry, I hit ctrl-enter and didn't realize it was going to post
This is being tracked https://phabricator.wikimedia.org/T217783
Steps to reproduce:
Expected:
The page would still be there and editable.
Notes: The black loading screen appeared and it shut out of Mobile Web and returned to the GChat app. The phone may not have been able to handle the testing server, it may just be too much for it. However, from a user perspective this could be discouraging. Not sure what can do about it.
I don't know what we can do about it in the end (beyond "recover your edit", which would be handy there when it works). You're right, and there are all kinds of ways for that to happen: in addition to the case of the phone going to sleep while you're reading, you could lose your work because your battery dies, or the browser might drops everything in the editing window if it needs all the memory to open a large webpage. I'm sure you could think of more scenarios than I can right now. It is a worrying thing.
Do you happen to know whether Android behaves the same way?
Steps to reproduce
Expected
The user is in read mode
Actual
A blank page is shown with no information.
Note
This must be done directly by clicking the link.
Steps to reproduce 1. Click pencil to edit section 2. Make edit 3. Save page
Expected
Page scrolls back to section.
Actual
Page shown on top.
It seems reasonable that the user would like to see the saved section. Currently they'll need to scroll or try browser find.
Possible solution:
