Talk:ResourceLoader/Version 2 Design Specification

How shall gadget developers be given access to the Mediawiki namespace, and will they only get access to their own gadgets or to all gadgets? Jeblad 22:42, 10 December 2011 (UTC)

There is a new Gadget namespace, so they're out of MediaWiki namespace and you do not need the editinterface right to edit them.

Nice! What about localization?

And what about the interface messages used by a gadget? Per ResourceLoader/Version_2_Design_Specification#Messages, they will be on MediaWiki namespace, but will they be editable by users with the new permissions (gadgets-edit, gadgets-definition-create, gadgets-definition-edit and gadgets-definition-delete) or will it be required those users to be sysops?

Previously gadgets had no support for localization. For Gadgets 2.0 we're providing all ResourceLoader features for localization.

As for anything related to localization and interface messages, these are restricted to users with the 'editinterface' right. Depending on your wiki configuration this right could be given to groups like "translators" or "interface editors" or "administrators" (default).

Interface messages are intentionally not covered by the "gadgets-edit" right, that's exactly the point of separating these away from interface editors so that gadgets become more accessible.

Due to translations/localization being completely native, fallbacks to other languages and availability in mw.msg comes along for free.

The Gadgets "Export"/"Import" feature also covers messages in the MediaWiki:-namespace, so if a gadget is developed somewhere and imported by an admin, those will be imported as well.

This seems a bit strange to me. Gadgets pose a security risk, usually system messages is not a risk unless for those that isn't laundered properly.

I would say that an admin should not have rights as a gadget developer but a gadget developer should have the edit interface rights.

Agreed.

The main concern which comes to my mind is that while developing new gadget this system would force us to do a lot of coordination between the gadget creator and the interface editors (sysops et al...) and the gadget interface could be broken until a sysop has the time to do the edits on MediaWiki namespace pages.

I was talking about user rights, not user groups. I think you will agree that the "gadgets-edit" user right should have no influence on the MediaWiki-namespace, it's not related to that.

How your wiki configures their user groups is entirely up to you. You can create the user group "Gadget developers" any way you like. Just like "Sysop" is a user group with a bunch of rights assigned to it (check Special:ListGroupRights).

It makes perfect sense for the user group allowed to edit gadget script/styles to also be allowed to edit interface messages, but configuring that is up to the wiki configuration.

I'm interessted in how this will be solved at whatever repository WMF choses for Wikipedia.

The discussion at Thread:Talk:ResourceLoader/V2 testing/Questions about permission model and developer workflow may be relevant.

In a new gadget manager [or edit tool] there should be possible to generate a report of code style. For example a tool like [JS]lint could be used. It must be fairly easy for users to run a check on the code and some kind of report should be visible so coders would want to make good code, that is use social crediting for good code.

In the first iteration of the Gadgets rewrite ("Gadgets 2.0") we are focussing on the main functionality. Keeping as first priority to not break existing gadget infrastructures and making the new features stable as supposed to packed-with-features.

A nice code editor with syntax highlighting and validation of syntax or code style sounds like a good idea. However I'd recommend doing that as a gadget, not as part of this extension itself.

On MediaWiki.org, for example, is a gadget called "CodeEditor". If you activate that and go and edit your common.js you'll have live syntax highlighting.

Yeah I know the CodeEditor gadget (I set it up on a wp-project), but this isn't about editing the code but for the users of the gadget to build confidence that the coder has done a good job. It is also for identifying suspicious gadgets when something breaks.

I know what you meant. Building a gadget that validates code on the fly or enforces or recommends a certain coding style is, however, fairly similar to a syntax highlighting custom editor interface. Incorporating something like this is significant to the user, but is technically very similar.

For example JSBin automatically validates through JSLint when "Real-time preview" is enabled (it's in the bottom left corner).

Not sure if this is the correct page for this, but over a few years I have helped a lot of users that run into problems with gadgets. It seems like a common task is to identify which gadgets breaks the chain and why it happens. This has lead me to a conclusion that it would be nice to be able to have unit tests on gadgets. It could be a button besides the gadgets on the special page and if a user pushes the button the unit tests are run. In addition there should be a button to run a unit test on all gadgets in sequence. This would solve most of the problems when something croaks cleanly, but sometimes things croaks only in certain setups. In those cases I think it would be nice to have something like the debug url parameter, but instead run the sequence of unit tests on a specific page. This will make it possible to identify interdependences betweens code that makes it croak only in some contexts.

Closelly related to this is if it would be possible to post a report from a unit test directly from the gadget page, or some other place, to Bugzilla to report a problem. Make it extremely simple to report a problem, its difficult enough for the user to identify the problem at all.

A little off topic, but there is also a gadget for debugging scripts on pl:MediaWiki:Gadget-wikidebug.js / css.

In ver 1 of the gadget manager the whole descriptive string is used as a label. In a new system it should be possible to write a descriptive text with links. Hopefully you fine folks have already solved that problem! ;)

Short answer: Using descriptive text with links is still possible. In addition to that we've introduced the ability to give a gadget a "Title". These are now separate.

Long answer:

I assume you mean version 1 of the "Gadgets" extension. The "Gadget manager" is entirely new and hasn't existed until now.

Previously a gadget had a codename and a description:

-- MediaWiki:Gadgets-definition:
*codename

-- MediaWiki:Gadget-codename
Description here with ''style'' and [[links]].

-- Special:Preferences
Gadgets:

[_] Description here with ''style'' and [[links]].

In Gadgets 2.0 we've introduced a "title" so that the "codename" doesn't have to be used as much, and this also allows for:

• Titles to be localized in multiple languages (MediaWiki:Gadget-foo-title/fr)
• Title to change without breaking existing preferences (since Title is now separate from codename)

New structure:

-- Special:Gadgets
== Title (edit) ==
Description (edit)

-- MediaWiki:Gadget-codename-title
Title

-- MediaWiki:Gadget-codename-desc
Description here with ''style'' and [[links]].


-- Special:Preferences
[_] Title
    Description here with ''style'' and [[links]].

Good! I like it! =)

Just don't do as in the Special:Preferences where the descriptions are wrapped up in label-tags, it creates a lot of problems if some one iterating back and forth between the special page and a linked description page. Doing so they ticks and unticks gadgets without noticing.

Indeed. Maybe only the title should be clickable?

I don't want to wait till RL2 is finished. Can anyone point me to where I can force RL to load user scripts on top?

Simply not possible.

• Special:Preferences on repo1 is currently broken.
• client2 main page fails with "1054: Unknown column 'rev_sha1' in 'field list' (localhost)", presumably due to out of date schema.
• The behavior of the autocompletion in the gadget editor is confusing. When you manually type a valid script name, it is not accepted. Only the autocompleted script name is accepted. There are no clear success/failure indicators and invalid entry is quietly ignored.
• I've created a new gadget on repo1 and enabled sharing, but it does not appear in client1 shared gadget prefs.
• When viewing a gadget definition page, I would suggest making the default "edit" tab utilize the Special:Gadgets form editor, and stashing the raw JSON edit mode into the "additional actions" dropdown in Vector.

Thanks for your bug reports :)

• The error on the preferences page was caused by a gadget name containing a space ("Invalid name 'wpgadget-Quick Recent Changes' passed to HTMLFormField::__construct"). I superficially fixed this case in r102409 but now the display name is weird (contains underscores). This is easily fixed by editing the title in the gadget manager, though.
• client2 main page is fixed now, had to run update.php on all DBs
• The new gadget not appearing in "Shared gadgets" is probably a cache invalidation bug, I'll look into that. I know that ForeignAPIGadgetRepo doesn't have any proper cache invalidation at all at this time
• That's an interesting suggestion. Timo?

I was right about cache invalidation:

• client1 uses ForeignDBGadgetRepo, so hasSharedCache should have been set to true, but it was set to false. I set it to true so it now exploits the fact that repo1 and client1 use the same memcached server (and can therefore access each other's cache entries; this is also the case for WMF wikis and ForeignDBFileRepo exploits this as well) instead of maintaining its own cache of what repo1 has
• client2 uses ForeignAPIGadgetRepo, which does not offer shared cache usage. So this one still relies on using a mirror cache, which never expires (!!!), and so client2 doesn't see any of the new gadgets on repo1. I'll fix the never-expires bug in the code

Cache expiry bugs are fixed now. Changes should show up immediately on wikis pulling through the DB (like client1) and with a delay of no more than 10 minutes on wikis pulling through the API (like client2).

\o/ Shared gadget setup confirmed working now.

As an aside, the localization cache on the prototype wikis is borked right now, leading to message values randomly not being loaded ("Some UI text" becomes "<somemessagekey>" every now and then). NeilK ran into this with commons.prototype, and it's affecting your test wikis as well. He took a shortcut and just set, I think, $wgLocalisationCacheConf['forceRecache']=true; which seemed to at least suppress the issue.

I discovered and fixed this issue independently, using $wgLocalisationCacheConf['store'] = 'files';. The default backend for LC in trunk is APC, which doesn't seem to be working very well for us.

This post by Krinkle was moved on 2011-06-14. You can find it at Thread:Talk:ResourceLoader/Version 2 Design Specification/Top-loaded site scripts.

Tracked in Bugzilla
Bug 39025

The "hidden" option is not working for this gadget. It appears on Special:Preferences#mw-prefsection-gadgets even after I hide it. Indeed, the loop which adds a preference for each gadget doesn't check for its hidden property before adding it to Special:Preferences (BTW: it also doesn't check if the user has the appropriate rights).

The infrastructure is as good as ready and in place in the Gadgets 2.0 branch. It may not be used everywhere yet, we're working on it.

Thanks for the report (bugzilla:39025)

Hello? Now that we have ContentHandler would be a nice idea to define a Gadget: namespace and move Gadgets there, I think. But is there any planned / unplanned work for this at all?

It'd be nice, wouldn't it? But it seems to have missed being included in the Wikimedia Engineering/2013-14 Goals, which is quite unfortunate.

True true. But at least the Gadgets stuff (a Gadget: namespace, perhaps) is something that could be done without the help of WMF engineering, methinks.

Good point. However, I think there is a shortage of willing volunteers who could take up something like this and push it through to completion.

+1

Don't forget that Roan and myself already implemented the vast majority of the features for Gadgets 2.0 (see the branch).

However unfortunately we weren't able to finish it before VisualEditor development was scheduled to take the main priority.

Help is always welcome with testing the Gadgets 2.0 development branch (you'll have to install it locally to test it), reporting bugs to bugzilla and submitting patches to Gerrit.

I expect Roan, myself and/or someone else will be assigned to finish this at some point, but at the moment other projects take priority.

Is it correct to say that it's our goal to ensure that we have a pretty clean flow from getting your feet wet by writing a user script, to implementing a proper gadget, to getting that gadget officially accepted and possibly even activated by default?

If so, are there elements of the current permission model and workflow which present potential for friction?

More specifically:

• If I'm not mistaken, MediaWiki: namespace edit permission is managed through the 'editinterface' permission, which in Wikimedia wikis is only given to users with sysops-or-higher access privileges. Would this present a barrier for a non-sysop developer to prepare their gadget for proper i18n? If so, how could we reduce that barrier?

• With the current permission model, it looks like higher permission users (gadgetmanagers) have to perform work on behalf of lower permission users (gadgetauthors). Alternatively, might it make sense to enable some kind of gadget submission workflow where a gadget author can use all the same forms that a gadget manager can, but not activate the gadget (except perhaps for personal testing purposes)? This would mean that a gadget author could successfully take a gadget very close to activation, awaiting only review and acceptance by a gadget manager.

If we can build a workflow that makes it easy to do the Right Thing without jumping through hoops or obtaining extra permissions, it seems likely that we can get more casual user script authors to adapt their scripts to best practices and publish them through a shared repository.

Re 1, yes, that would be a barrier. It would be pretty hard to separate gadget messages from regular messages though. I'll ask Timo to share his thoughts on that.

Re 2, yes, I agree that we need that. From a backend perspective, hiding the gadget from all but a few users and disallowing a certain group to create gadgets that aren't hidden in that way wouldn't be hard. As for the frontend / user-observed workflow perspective, I once again defer to Timo.

I'll answer your points while at the same time writing some extra documentation that hasn't been described much so far (feel free anyone to re-use this on a documentation page)

---

One of the goals I had in the ResourceLoader 2 project (or rather "Gadgets 2.0 - made possible by a few ResourceLoader improvements" as Roan and I have been calling it lately).. is to improve the user's workflow in general (in addition to just the technical aspect).

Technical aspect being the efficient loading through ResourceLoader, proper dependency support, localization and what not. Although I think the technical aspect is the main focus and main motivation for the project, the user workflow definitely needs improvements and I took some of them into account when writing the specification.

The main change in the workflow is to branch off two very important abilities away from the "sysop" user group (aka "Wiki page/ wiki user administrator"). Note that no actual user rights are removed from the "sysop" user group (neither were they ever added in the past), they were simply implied by the fact that everything was controlled from MediaWiki-namespace pages. Any gadget guru sysop is free to join one or both of these new groups as soon as we're live.

• The first ability to be available independently from sysops is the ability to maintain gadgets. This was annoying so far because, like you say, the gadget resources are stored in a pseudo-namespace prefixed by "Gadget-" inside the actual MediaWiki-namespace, which is (and should) be restricted to translators and sysops. This made it impossible for anyone expert and trusted with JavaScript to easily help out. Often leading to pending "{{editprotected}}" messages on MediaWiki_talk-pages by users requesting that a sysop copies and pastes the latest version from page X.
  • This is now solved by having gadget resources live in their own "Gadget"-namespace. Among many advantages is that they can be easily filtered in Special:RecentChanges (What changes were made to gadgets recently), as well as the ability to grant users access to it (users that know JavaScript) without them having to be super well known by the community and becoming a "sysop" and all.
  • A new user group "Gadget authors" was introduced which grants the gadgets-edit user right (=ability to edit in NS_GADGET)

• The second ability is the ability to manage gadgets. Meaning that a set of js/css pages in NS_GADGET will be available as a bundle in everybody's preferences. As well as the ability to enable something by default (which is not something a developer should be needed for). This is improved/made possible by two things.
  • Special:Gadgets (Gadget manager): The management of gadgets is now much more intuitive and doesn't require any knowledge of some obscure syntax.
  • A new user group "Gadget managers" was introduced which grants the gadget-definition-* rights. This is what grants users the right to use the Gadget manager.

Note that the "Gadget authors" and "Gadget managers" user groups don't actually exist by default, they're commented-out examples when installing the Gadgets extension. Any wiki is free to configure this in any way they like. For instance, WMF may chose not to have "Gadget managers" and instead grant those rights to the "sysop" user group (Introducing only "Gadget authors" as a new user group). Of course separate user rights and groups can still be configured on a per-wiki base would a wiki wish to have a separate "Gadget managers" user group.

Also note that although a Gadget author is not able to enable a gadget by default, there is no difference between editing a JavaScript-page of a gadget that is or isn't enabled by default. So any Gadget author is therefor able to modify scripts that affect all users including anonymous users (assuming there is at least one gadget set to be enabled by default). This is why we shouldn't put the bar for Gadget authors too low. In order to attract new developers the bar should not be too high either. Previously the bar was being a sysop, so I'd say it's a significant improvement already.

If a Gadget author wishes to develop/test a gadget, they would previously do this on a subpage of their user page (i.e. User:Example/Gadget-Foobar.js). This made collaboration hard (since nobody else is allowed to edit that user subpage), as well as sharing. By having them in the Gadgets namespace one can create a set of JS pages that any Gadget author can improve and collaborate on, without it having to be an officially defined "Gadget". Anyone who wants to use this experimental bundle of JS-pages can put "importScript('Gadget:Foobar.js'); in their personal javascript.

Last but not least, development by users who have not been granted any non-default user rights. There hasn't been much improvement on that part, I do believe progress should take place in that area in the long term. However I believe right now it's more important that we finalize the existing (already fairly large) specification of ResourceLoader 2 and get it out there. And reserve the other workflow feature ideas for Gadgets 3.0, after we have experienced how Gadgets 2.0 is going to be used by the community.

The current workflow for "personal scripts" (or "potential gadgets") by users who are not Gadget authors or sysops works fairly well. They can still create as many js-subpages under their user page as they like, and have other users importScript these etc. Although these would not be loaded by ResourceLoader, the ResourceLoader library and functionality will be available there (in that they can use mw.loader.load, mw.loader.using, mw.messages.set, mw.msg, mw.html*, jQuery, jQuery UI etc. in their user subpage scripts just as if it were a Gadget in the Gadget namespace)

-- Krinkle

PS: For anyone using this for documentation, this post is not a summary of features new in ResourceLoader 2. There are many new features not even mentioned here at all. See RL/V2SPEC.

Makes sense. So I'm trying to figure out what group permission model would mean for a potential production deployment configuration on WMF sites.

It sounds like it would be simplest to create a 'gadget-developer' group which has all the permissions needed to develop gadgets, including

• editing definitions
• editing gadget source code
• editing interface messages (!).

This group would need to be highly trusted as, like you say, they'd be able to manipulate JS that gets run by default. For that reason, I don't think it makes a ton of sense to practically distinguish definition maintenance from code maintenance from interface message edits -- all of these are potentially highly destructive actions that should only be performed by trusted individuals. So you need to be familiar with policies/practices before you get those rights.

An evolving security model would allow us to lower the barrier over time, but for starters, we could give users with an established track record as gadget devs permission to do centralized gadget development work on MediaWiki.org (as a repo serving all projects), and keep gadget-related permissions limited to admins on all other wikis. That way, we don't introduce complex new roles all over the place: The only new role we're creating is that of a gadget-developer on MediaWiki.org to centralize gadget development by trusted users without confusingly making them all sysops.

Does that sound sensible? If so, that's the security model that I think we should build out in Labs, to demonstrate these concepts in an environment that resembles the targeted production deployment as much as possible.

Sounds good :)

Initially I wanted to separate the definition and development. I still think that is a good idea, however not the way I proposed it. Because it would've only introduced a burdon for developers by having to go through extra bureaucracy with users that likely don't understand it as good as they do. It also didn't make sense since it would allow them to unconditionally edit both existing gadgets and inexistant gadgets but can't define new gadgets (from a security POV, not helpful since harm could simply be done through an existing gadget). Anyway, we agree :)

The "Gadget developers" group makes sense, except for one detail though. It would not be a new group only on MediaWiki.org, but on all wikis. The gadget manager is for managing gadgets in general (replacing MediaWiki:Gadgets-definition), the only new thing is (aside from the visual difference), that wikis marked as "repo" wikis (MediaWiki.org in case of WMF) will have an option in the gadget manager to make them globally available.

Gadgets still need to be editable on local wikis, and after upgrading the Gadgets extension they will no longer be in the MediaWiki-namespace, and as such no longer by editable by a sysop through the 'editinterface' user right.

By default the Gadgets extension doesn't grant any user the "gadget-edit" and "gadget-definition-*" rights. So either

• Grant them to sysops on WMF (on wikis other than MediaWiki.org). That way no right changes are visible, sysops are still the only ones able to edit them. And on MediaWiki.org create the "Gadget developer" user group for it instead.
• Grant them to a new "Gadget developer" user group on all WMF wikis, this means until bureaucrats add users to it on a wiki, gadgets will be temporarily uneditable by local users (however the global WMF user group for gadget developers (currently has global right 'editinterface') will likely be expanded with the new gadget rights, so they will be able to help out on any wikis that don't have local gadget developers.

Either is fine by me, I have no preference.

Yeah, I understand the gadget manger would be available everywhere, sorry for being unclear. With regard to the permissions rollup, I was arguing for the first of the two scenarios you mention, to keep things simple for now.

Wow Krinkle, this looks like an awesome improvement. I have a couple questions. Will the new group created for this include the rights for managing tags (which would be very useful for some gadgets to use tags instead of taking up half the edit summary to indicate a gadget/tools was used)? Can access to this usergroup be a vetted process? I'm guessing the answer to that will be more of a "local" wiki choice than a global thing. Which leads me to a question of whether or not there will be a global-gadgetX usergroup for people who are interested an willing to help with maintaining gadgets on sites that may not have their own js/gadget personnel or even capable/willing administrators to take care of that stuff?

As far as I can see, the issue with i18n and lots of expected edit requests (one edit request per message per language!) is neither solved nor answered.

Maybe it would be possible to integrate the i18n of gadgets with translatewiki:?