Requests for comment/UploadWizard: scale to sister projects
This request for comment is about UploadWizard software which the Multimedia Team (WMF) is about to embrace and take over starting within next weeks and processing it during 2014, per the text on the software page.
| UploadWizard: scale to sister projects | |
|---|---|
| Component | General |
| Creation date | |
| Author(s) | Gryllida |
| Document status | in discussion |
Proposed change
editWhat I would like to propose is to restructure the UploadWizard software to make the following functionality available to contributors:
- Edit text of each wizard screen on-wiki.
- Edit the flow of the wizard (which buttons exist on a screen, where they take) on-wiki.
Context
editThis request is based on the following observations:
- English and Russian Wikipedia's article creation wizards appear to be entirely markup-based, with no JS. Contributors have the flexibilities mentioned above, but lack some nice flexible things JS would let them do, such as choosing a next wizard screen depending on multiple previous choices, or entering partial data to be carried forward from page to page.
- A product which has the flexibilities mentioned above would have the potential to be deployed across all sister projects.
- Another example use-case is the “Idea” submission at Meta which needs a wizard-like thing.
- Such a product would also have the potential to work a bit gadget-like, to address use-cases I call “interactive templates”, to aid in daily editing routine and make it more intuitive.
- A JS-based product of this kind is under development on English Wikinews.
- Multimedia Team mention they would start working “in the coming weeks”, making feedback pertinent earlier better than later.
- Editing on-wiki is more collaborative than querying editing files (like ones in an Extension) would be.
- Markup should also be less-prone to security risks.
Discussion
editAmount of complexity involved
editI'm not sure you fully appreciate the complexity involved in a project like this.
UploadWizard itself has thousands and thousands of lines of JavaScript code that deals with step navigation, different upload methods, error checking, interfaces for different choices along the upload path, and various API calls to save all of that information on the wiki. Putting all of that in a gadget *might* be possible, but putting most of it into wikitext is not.
And even if it's possible to shunt the backend UW code into core, and put all of the frontend code into a gadget, we would lose a lot of the benefits we get from having UploadWizard be an extension: It gets proper code review, it can be configured (and the configuration also gets code review), it can be distributed to third party sites, and so on.
I would be adamantly against pushing UploadWizard more on-wiki.
However, I would be very much in favour of sticking more client-side hooks into UploadWizard. You can already edit most of the text of the wizard through MediaWiki namespace message pages, so allowing gadgets to play with the workflow and the interface is IMO the only remaining task. Let's focus our attention on accomplishing a more extensible interface instead of totally rewriting it in a new language, and screwing with the established structure of the tool. --MarkTraceur (talk) 21:18, 10 February 2014 (UTC)
- Yeah I would say that it is important to slowly rework the wizard where possible in more subcomponents that can be reused and hooked into, instead of a monolithic and intertwined blob. That is much more important then making everything editable. TheDJ (talk) 10:27, 11 February 2014 (UTC)
- Ah, we appear to agree on the first step after all... Gryllida 02:01, 14 February 2014 (UTC)
- MarkTraceur, Mozilla has an extensive community-powered repository with extensions for its web browser, Firefox.
- They all go through code review by the AMO Editors — a community of volunteers as well as employees who review every submission before users can install it.
- At some points some of these extensions get merged into Firefox, like Personas_for_Firefox.
- It would seem that lack of code review is not a problem here, granted you can ship MediaWiki with some built-in gadgets which you support.
- --Gryllida 02:00, 16 February 2014 (UTC)
- Maybe I was unclear - I mean "code review before you're allowed to break the site with your modifications".
- While people are generally smart and work through things with collaboration, and test, and so on, it's not built into the system, and we need to be cognisant of the design of the system that can run code on every single one of our users' computers.
- In any case, the example of Mozilla extensions is a markedly different situation - extensions are heavily code-reviewed by Mozilla contributors before being merged into Firefox, and users must confirm multiple times that they want to install extensions that are not merged in, so again, this is not similar to gadgets that are run on users' computers without notification of changes or sometimes even any consent at all.
- "Shipping MediaWiki with built-in gadgets" is a bit of a weird way to put that. We call them modules, they don't use the gadgets infrastructure at all.
- In any case, you haven't really addressed the problem that shifting all of that really nasty, hard-to-maintain code into gadgets is going to be very difficult, nor have you addressed the issues with lack of backend gadget infrastructure, which UW would need. --MarkTraceur (talk) 18:13, 24 February 2014 (UTC)
UploadWizard isn't going to be as much wizard-like as it currently is
editI think the main issue with this RfC/discussion is that it's not distinguishing the current UploadWizard experience vs the one that's been collectively agreed on for the next version. They differ greatly, the new way of doing things doesn't rely on steps.
Having several steps to enter information is actually a weakness that we're about to get rid of. There is a big UX benefit to letting people enter the information while the content is being uploaded, as such most of the new experience will be happening on a single page. In order to make a compelling UI, this will need to be made in a traditional fashion, with JS running the show. As Mark described above, this is a level of complexity that goes way beyond what wikitext can do, there are too many moving parts. The history of the web is littered with attempts at making websites simple to write without knowing how to write code, but as soon as you delve into very technical sides like upload, it just gets too complex to do anything more than letting people bringing in a self-contained upload component that just handles everything.
Reuse is a big focus of the work we're about to do, at the javascript level. We intend to make UploadWizard very modular, which it currently isn't. So while the end-user experience will turn into a single dynamic page, the actual code that makes UploadWizard will be as modular as possible, to allow other extensions and projects to leverage upload ability with a different UI, to reuse components of the UploadWizard UI, etc. We always try to keep in mind the "amateur developer" experience when we organize the code like this, avoiding techniques that would be too complex and constantly improving the organization of the code, which benefits developers of all levels.
All of that being said, I can see the benefit in having wiki-specific preliminary wizard steps leading up to the new UploadWizard, similar to the current customizable "step 1" text. I'm unfamiliar with the current techniques that could be used for local wikis to achieve this, although I imagine that it's already possible.
Similarly, there could be benefit in building a general-purpose wizard system that could hook into anything, not just UploadWizard, that would display speech bubbles on top of specific elements, introducing how the UI works, step by step. Again, I'm unfamiliar with what's currently available (I think I recall something kind of like this when I created a wikipedia account recently), maybe there are similar things already out there that can do that. That I could imagine being fully definable in wikitext, by specifying pages to open, CSS selectors to target, etc. With options such as selecting a field for the user (keyboard focus), fading/hiding parts of the UI and so on. That would be a wizard as a layer on top of an actual extension/page, not a wizard that's part of the extension itself.
Bottom line is, if you want to customize the UI of the UploadWizard itself (where the fields are placed, which ones are required, etc.), CSS/JS seems unavoidable to me.--GDubuc (WMF) (talk) 10:50, 11 February 2014 (UTC)
- The bubble thing is probably the GuidedTour extension, which can even be abused into a wizard sort of thing (The Wikipedia Adventure does something similar), although there is a fair amount of JS involved.
- Anyway, if you want a wizard generator, UploadWizard is not the right place to start. Its code is structured to deal with things specific to uploads (files, licenses etc), it is not structured around dialogs in general. Creating an extension (or gadget) that can generate a sequence of dialogs from some sort of markup is certainly possible, but it will be easier to just write it from scratch. --Tgr (WMF) (talk) 19:24, 11 February 2014 (UTC)
- Tgr, Sister projects have their own upload scenarios, such as fair use thingie at Wikipedia. These might be quite useful at some stages of article creation, a thing to be integrated into article creation wizard (even if these bits just point to Commons for most uploads; even so, these people may as-well upload to Commons without leaving an article wizard at Wikipedia
). It would seem to be that sharing the upload-related bits of UploadWizard with sister projects could be a benefit likewise, not an obstacle. Gryllida 02:07, 12 February 2014 (UTC)
- Tgr, Sister projects have their own upload scenarios, such as fair use thingie at Wikipedia. These might be quite useful at some stages of article creation, a thing to be integrated into article creation wizard (even if these bits just point to Commons for most uploads; even so, these people may as-well upload to Commons without leaving an article wizard at Wikipedia
GDubuc, thanks for the thoughts. You'll notice that some wiki markup can call JS. The concepts are not mutually exclusive. Gryllida 01:47, 12 February 2014 (UTC)
GDubuc, MarkTraceur There are some workflow features
- next/previous screen (?),
- echoing user input from N and N-1 screen to N+1,
- typing a name of a page and checking that it doesn't exist already,
- previewing a box contents such as description,
- typing a category name and making sure whether it already exists
... which could be reusable by widgets/wizards at other sister projects, especially if they are accessible from wiki markup. --Gryllida 01:52, 12 February 2014 (UTC)
- Putting these things into wikitext is exactly the opposite of what the Parsoid project and others are trying to do. We want to move away from our markup language being turing-complete, not add UI creation libraries to it on top of everything. I like the idea of making some of these easily available via something like OOjs UI, but doing it in wikitext is a mistake. --MarkTraceur (talk) 18:16, 24 February 2014 (UTC)
- There's no reason for anything involved in this to be Turing-complete. Turing-power seems obviously undesirable. However, moving away from wiki markup is a completely separate issue from avoiding Turing-power, and indeed wiki markup is ultimately the technical wellspring from which the open character of wikis flows. To move away from it is to move away from wiki-ness toward a high-priesthood model of content production. --Pi zero (talk) 17:02, 26 February 2014 (UTC)
- So..."I agree that Wikitext shouldn't be Turing-complete, but let's add programming structure to it anyway"? Sounds like a solid plan.
- We should be seriously wary of pushing things on the wiki that don't belong there, and this is definitely one of those things - our software is exactly as "open" as a wiki is, it just has a higher standard of pre-commit review, and for good reason. We're definitely not moving towards some ivory-tower situation - we have loads of community members involved in our development process, and would like to have more - but we don't want to be stuck in a situation where we're chasing fires because an admin on some random project pushed a code change she didn't test.
- It's really not that hard to contribute to our codebase, and we accept patches from just about anyone. Maybe we should try working through that existing system before we throw this much power into wikitext. --MarkTraceur (talk) 19:39, 26 February 2014 (UTC)
- Wiki-markup-based wizard-buliding tools don't involve "add[ing] programming structure" (no more than having two wiki pages link to each other constitues a divergent computation). "It's really not that hard to contribute to our codebase"? Seriously? Do you really believe most wiki contributors would have a hope of doing so, or even of gradually working up to doing so? For the wikimedian movement to survive and flourish in the long run, the entire community needs to control the process of capturing and disseminating expertise —the fashioning of interative wizards— as expertise is the limiting factor on the ability of the community to produce ever-better content, and the community is where the expertise resides. Nor are wizard-bullding tools even all that difficult... except, of course, that the effort involved is almost entirely in avoiding design mistakes, which is very difficult to do with a group of people; generally, with a group, everyone starts moving toward some poor strategy that's traditional, and the more effort people put into it, the more its advocates can resist exploring better choices with the objection that 'we've already put so much work into doing it this way'. --Pi zero (talk) 21:50, 26 February 2014 (UTC)
- Yeah, I think anyone who wants to work on our codebase is totally welcome to do so, and if their patches fall under my purview I'll do my best to help review them, especially if they ask me specifically to review. Interactive wizards aren't just links between pages, they involve adding validation, adding AJAX calls that perform actions (API calls, so programming structure), adding hints in the interface, adding the ability to perform actions on a grander scale, etc. etc.. I think all of this is totally possible through gadgets that help people through difficult-to-understand pages, but I don't think it's necessary or even desirable to put this sort of thing into wikitext. Wikitext is already hard enough to work with as it is, and as I keep saying, dumping even more syntax into it won't help. JavaScript is already the way we do this sort of thing, so let's just focus on making sure people can access the bits of the MediaWiki workflow they need to. --MarkTraceur (talk) 22:04, 26 February 2014 (UTC)
- I think you're absolutely kidding yourself about anyone contributing to the codebase. I see no reason wiki-based wizard-building tools should involve more syntax. And I think the stuff about how difficult wiki markup is, is for practical purposes, fiction. Wiki technology succeeded in the first place because wiki markup is relatively easy to use; if that were a major cause of the decline of wikimedia projects, they'd be in far better shape than they are. Putting massive effort into moving away from wiki markup, and thereby witholding effort from much less labor-intensive things that would better exploit wiki markup's strengths, is... a deeply unfortunate trend. --Pi zero (talk) 23:20, 26 February 2014 (UTC)
- It does seem your list of things needed for a wizard may be underlain by a specific perspective. I sympathize with this difficulty. When one is accustomed to one way of doing things, it can be difficult to imagine a different approach, and all too easy to think of one's accustomed approach as intrinsic to the problem. --Pi zero (talk) 02:32, 28 February 2014 (UTC)
- I think you're absolutely kidding yourself about anyone contributing to the codebase. I see no reason wiki-based wizard-building tools should involve more syntax. And I think the stuff about how difficult wiki markup is, is for practical purposes, fiction. Wiki technology succeeded in the first place because wiki markup is relatively easy to use; if that were a major cause of the decline of wikimedia projects, they'd be in far better shape than they are. Putting massive effort into moving away from wiki markup, and thereby witholding effort from much less labor-intensive things that would better exploit wiki markup's strengths, is... a deeply unfortunate trend. --Pi zero (talk) 23:20, 26 February 2014 (UTC)
- Yeah, I think anyone who wants to work on our codebase is totally welcome to do so, and if their patches fall under my purview I'll do my best to help review them, especially if they ask me specifically to review. Interactive wizards aren't just links between pages, they involve adding validation, adding AJAX calls that perform actions (API calls, so programming structure), adding hints in the interface, adding the ability to perform actions on a grander scale, etc. etc.. I think all of this is totally possible through gadgets that help people through difficult-to-understand pages, but I don't think it's necessary or even desirable to put this sort of thing into wikitext. Wikitext is already hard enough to work with as it is, and as I keep saying, dumping even more syntax into it won't help. JavaScript is already the way we do this sort of thing, so let's just focus on making sure people can access the bits of the MediaWiki workflow they need to. --MarkTraceur (talk) 22:04, 26 February 2014 (UTC)
- Wiki-markup-based wizard-buliding tools don't involve "add[ing] programming structure" (no more than having two wiki pages link to each other constitues a divergent computation). "It's really not that hard to contribute to our codebase"? Seriously? Do you really believe most wiki contributors would have a hope of doing so, or even of gradually working up to doing so? For the wikimedian movement to survive and flourish in the long run, the entire community needs to control the process of capturing and disseminating expertise —the fashioning of interative wizards— as expertise is the limiting factor on the ability of the community to produce ever-better content, and the community is where the expertise resides. Nor are wizard-bullding tools even all that difficult... except, of course, that the effort involved is almost entirely in avoiding design mistakes, which is very difficult to do with a group of people; generally, with a group, everyone starts moving toward some poor strategy that's traditional, and the more effort people put into it, the more its advocates can resist exploring better choices with the objection that 'we've already put so much work into doing it this way'. --Pi zero (talk) 21:50, 26 February 2014 (UTC)
- There's no reason for anything involved in this to be Turing-complete. Turing-power seems obviously undesirable. However, moving away from wiki markup is a completely separate issue from avoiding Turing-power, and indeed wiki markup is ultimately the technical wellspring from which the open character of wikis flows. To move away from it is to move away from wiki-ness toward a high-priesthood model of content production. --Pi zero (talk) 17:02, 26 February 2014 (UTC)
which wizard are we talking about?
editthe page title and intro talk about "upload wizard". however, the "context" paragraph seem to be about the "article creation wizards". these are two wildly different discussions that, at least at first glance, seem to be hopelessly conflated. it's not at all clear from the discussion which wizard is being discussed here. please clarify. thanks, peace - קיפודנחש (talk) 23:21, 10 February 2014 (UTC)
- קיפודנחש, that's right. The title says: "scale upload wizard to sister projects", meaning "scale upload wizard for all wizard needs of sister projects". Gryllida 02:38, 11 February 2014 (UTC)
- Oh right, that really wasn't clear to me (and I suspect others). "Scale" doesn't seem the right word to use. Perhaps "Wizard framework" would be a more appropriate title, if that's what is being proposed? the wub "?!" 12:14, 11 February 2014 (UTC)
- Thanks. Feel free to move with leaving a redirect, no harm I can see in that. (I've thought a bit but didn't come up with a reasonably specific but short title.) :-) 101.164.32.121 21:17, 11 February 2014 (UTC)
- sorry, i still do not understand what is this discussion about. not trying to be difficult - maybe i'm just stupid. what does it mean "scale upload wizard for all wizard needs of sister projects"? the closest i could come up with, is some idea for "Wizard framework" development that will create toolset that will allow wikipedians to construct (all king of) wizards as needed. if this is really the case, then i fail to see what does this have to do with "upload wizard", and maybe it will be better to discuss "Wizard framework". if this is not so, then i'm completely lost, and have no idea what is the subject here. if everyone except myself understand what it's about, feel free to ignore me (i.e., ignore my ignorance...). peace - קיפודנחש (talk) 00:08, 12 February 2014 (UTC)
- Thanks. Feel free to move with leaving a redirect, no harm I can see in that. (I've thought a bit but didn't come up with a reasonably specific but short title.) :-) 101.164.32.121 21:17, 11 February 2014 (UTC)
- Oh right, that really wasn't clear to me (and I suspect others). "Scale" doesn't seem the right word to use. Perhaps "Wizard framework" would be a more appropriate title, if that's what is being proposed? the wub "?!" 12:14, 11 February 2014 (UTC)
Keep it simple, please...
editIt doesn't make much difference to me whether the markup or the JS will be used. But a unified wizard framework would be welcome, in a sense that developing a specific wizard would become as simple as writing a specific template. Existence of Template:Article wizard is encouraging, though having CSS hard-coded into it isn't - it would be preferable for it to adopt appearance of whatever theme is being used.
Yes, having Javascript code for active interaction with the user would be welcome, as long as the wizard isn't going to be broken by browser without Javascript. Given the complexity of a single wizard, and large number of wizards which would possibly be created in the future, it would make sense to develop separate modules (Javascript functions or server-side backend), APIs, so that a wizard would be well-structured and easier to maintain. Wikiwide (talk) 11:33, 13 February 2014 (UTC)
- specifically for wizards, i do not think JS dependency is a poison pill. if you are a purist, it's nice to have JS "optional" for all use cases, but in reality, people who disable JS are probably not all that interested in wizards. personally, i do not think that a limitation such as "wizards in MW only work if the user has JS enabled" is a real "limitation". one can summarize this limitation as "if you can't use visual editor, you can't use the wizard(s)". practically speaking, i think this is a non-issue. peace - קיפודנחש (talk) 18:49, 18 February 2014 (UTC)