Intro

Scope of the project is to make gadgets customizable, that is:

Allow gadget writers to easily export their gadget's customization variables.
Provide the users with a nice UI for customizing those variables.

This page contains a newer design than the one I proposed in my application, since the scope of the project has slightly changed.

Places

I'm working on this branch: http://svn.wikimedia.org/viewvc/mediawiki/branches/salvatoreingala/Gadgets/

From email 23 Aug:

[M]y plan [is] to merge it to trunk only after the Gadget rewrite was complete (or almost).... I expect integration to be easy, since most of the backend code is in a static class with just a few public methods; so the code to be rewritten just has to manage saving/loading and some other goodies which should be pretty straightforward.

Previous brainstorming was done on EtherPad. This is the last useful revision.

Roan Kattouw is working on merging this code as part of his ResourceLoader 2 work~~, probably in February~~. From Roan's email 14 Dec 2011: "There is lots of stuff to do in RL2 and integrating Salvatore's work is one of the tasks.". See MediaWiki's Roadmap for a more up to date schedule.

Features definition

These are the needed features:

Ability to specify what preferences do we have for a gadget;
- Will do this in JSON format.
- Where? For now, I'm using a fixed page MediaWiki:Gadget-mygadget.preferences or something similar. Likely to change (with low impact)
- STATUS: implemented (with a limited number of preference type, documented below; more will be added); server-side validation done; client site validation done with jquery.validate (link).
Hooking up to Special:Preferences to add per-gadget prefs
- Currently done like this: every enabled gadget shows a "configure" link near his checkbox in Special:Preferences. Clicking on the link pops up a modal dialog with the configuration interface. This may need partial rewrite after Gadgets rewrite in RL2.
- STATUS: working UI mock-up
Store preferences somewhere
- Using the existing user_properties table. Gadget preferences are retrieved and then hidden in UserLoadOptions hook, and reinserted in UserSaveOptions.
- Unused preferences need to be deleted (lazily is good enough). Saving policy to maintain coherency:
  - When saving the configuration of a gadget, throw away any previous configuration for that gadget (so to clean up old values for no-longer-existing settings).
  - When reading the configuration of a gadget, check it against its specification and assume default for missing values or values that fail validation (if specifications changed).
- STATUS: implemented
Provide the preferences to gadgets. Concepts:
- mw.gadgets.options.get( 'gadgetname' )?
- mw.loader.options.get( 'modulename' )?
- binding the configuration to this in gadget's context. (may have issues with RL2)
- STATUS: implemented; current solution is a combination of two ideas: gadget metadata (including configs) retrievable via mw.gadgets.info.get( 'gadgetname' ), but also bound to this in gadget's context.
Internationalization of strings used by the gadget
- There are two types of "messages" to handle:
  1. messages shown in the configuration dialog; these don't need to be passed to the gadget's module, and may be interpreted server-side, when building the dialog code during the AJAX request. Should fetch their names form the configuration JSON (maybe adding some common prefix);
  2. other messages used by the gadget; this is in the scope of RL2.
- STATUS: implemented (with some issues to fix)

Preference description syntax

The "preference description" is an object in JSON format which describes all relevant info about gadget's settings (and some other settings related to the configuration system); when all the rest of the project will be finished, a tool to create and maintain preference descriptions may be built, so to relieve gadget developers from the annoyance of learning its syntax.

The general format is this:

{
  "someGlobalSetting": "value",
  "anotherGlobalSetting": "anotherValue",
  // and so on...
  "fields": [
    {
      //option description 1
    },
    {
      //option description 2
    },

    //...

    {
      //option description n
    }
  ]
}

"global" settings will be settings that apply to the preference dialog (e.g.: min/max width, min/max height, title...). It will be useful for future advanced features, too.

fields is an array containing field description objects.

The syntax of a field description object is as follows:

{
  "type": "boolean", //or some other allowed types
  "member1": "valueOfMember1",
  "member2": "valueOfMember2", //or some other allowed types
  //other fields, as needed
}

type is compulsory for any field description. According to type, more members may be allowed (or needed).

Raw strings and messages

The ability to specify messages in the "MediaWiki:" namespace is absolutely necessary; nevertheless, sometimes raw strings are needed. Instead of making complex syntaxes to distinguish between raw strings and messages, a preprocessing of strings is done: if the string starts with "@", then it's intended as a message prefixed by "MediaWiki:Gadget-mygadget" (where mygadget is the name of the gadget, case-sensitive); otherwise, it's a raw string. Two "@" at the beginning escape for a single "@". This applies, for example, to labels and to options of select fields.

Global settings

Nothing, for now

Field definition specifications

This section describes the exact syntax for each field type.

`label`

Fields of this type don't encode for any gadget preference, and only add an arbitrary text. The syntax is the following

{
  "type": "label",
  "label": "@some-message", //raw string or message
}

Unary fields

All fields of this section encode for exactly one gadget preference, and support compulsory name, label and default members, that is:

{
  "type": "boolean", //or some other allowed types
  "name": "optionName",
  "default": true, //or any allowed value
  "label": "@some-message", //raw string or message
  //other fields, as needed
}

name, default, and label are compulsory for any option description in this section.

optionName must be a valid JavaScript identifier (that is, it must start with a letter or '_', then contain only letters, numbers or '_'), and must not be more than 40 characters long.

`boolean`

This field will be shown as a checkbox. No additional members are allowed.

`string`

This will be shown as a single-line text field. There are additional members:

required: (optional boolean, no default value). If true, a zero-length string will never be accepted; if false, a zero-length string will always be accepted (even when minlength is given). If it is not given, none of the above validations is done.
minlength (optional integer, defaults to 0): an integer number specifying the minimum length allowed for this option.
maxlength (optional integer, defaults to a 1024): an integer number specifying the maximum length allowed for this option.

TODO: think again for the default value of maxlength.

`number`

This will be shown as a single-line text field. The javascript representation of values of this field is as numeric datatypes (or null). There are additional fields:

required: (optional boolean, defaults to true). If true, a valid number must be given. If false, an empty string will be accepted (and will be delivered as null).
min (optional number): a number specifying the minimum allowed value.
max (optional number): a number specifying the maximum allowed value.
integer (optional boolean, defaults to false). If true, only integer numbers will be accepted (min and max must honor this, too). If false, decimal number will be allowed, too.

`select`

This will be shown as an HTML select element. There is only an additional compulsory field:

options: (compulsory array): an array with elements of the following shape:

{
  "name": "someStringOrMessage", //The string or message that will be shown
  "value": true                  //or false, or null, or a number, or a string: the actual value
}

All values should be different (according to the === operator), and all names too.

`range`

Shown as a jQuery.UI Slider, even if only its basic functionality is currently supported. There are other fields:

min: (compulsory number): Minimum allowed number.
max: (compulsory number): Maximum allowed number.
step: (optional number, defaults to 1): Gap between allowed numbers.

Negative or decimal numbers are valid values for min and max; positive decimal numbers are allowed for step.

Note that max must be coherent with min and step (that is, the difference between min and max must be an integer multiple of step).

`date`

Shown as a jQuery.UI [http://jqueryui.com/demos/datepicker/ datepicker. There are no other fields.

Dates are delivered as string with the format: [YYYY]-[MM]-[DD]T[hh]:[mm]:[ss]Z, which is an UTC time in ISO 8601 standard. It is a format recognized by Javascript's Date constructor. Empty dates are delivered as null.

TODO: add a required option.

`color`

Shows a color picker (farbtastic); colors are delivered as hexadecimal strings from "#000000" to "#ffffff" (uppercase letters are not allowed)

TODO: add a required option; add an option to allow choosing 'transparent' as a color?

`composite`

A composite preference is a javascript object, compound of several subfields; better to start with an example:

{
    "name": "position",
    "type": "composite",
    "fields": [
        {
            "name": "x",
            "type": "number",
            "label": "Abscissa:",
            "integer": true,
            "min":0,
            "max": 1024,
            "default":500
        },
        {
            "name": "y",
            "type": "number",
            "label": "Ordinate:",
            "integer": true,
            "min": 0,
            "max": 768,
            "default": 350
        }
    ]
}

This example encodes a composite field that encodes for a preference with name position, whose value is an object with two fields: position.x and position.y; in this example, both are numbers, each with his own specification, but the types may have been different and there could be an arbitrarily large number of "subfields".

A composite field has two compulsory members, name (with the same syntax and meaning as in the other fields), and fields, which is an array of fields, with the same syntax and semantic of an arbitrary set of preferences; any kind of field is allowed in the fields member (yeah, even a composite, recursively). It does not have a default member, and it deduces the default value as the composition of default values of its subfields.

Visually, all subfield are shown together, but the composite field is not visible to the end user, and its existence may be completely transparent.

`list`

The list field type allows the user to create homogeneous ordered sets of things (it implements a generic array). The user can add objects to the set, remove them, or change their order. It has the following members:

name (compulsory string), the name of the preference encoded by this field.
field (compulsory object), which is the complete description of an arbitrary unitary field, except that it does not have the name field.
default (compulsory array), which is the default value (an array of items of the type specified by field).
required: (optional boolean, no default value). If true, a zero-length array of items will never be accepted; if false, a zero-length array of items will always be accepted (even when minlength is given). If it is not given, none of the above validations is done.
minlength (optional integer, defaults to 0): an integer number specifying the minimum number of items allowed for this preference.
maxlength (optional integer, defaults to a 1024): an integer number specifying the maximum number of items allowed for this preference.

Example of list field:

{
    "type": "list",
    "name": "rainbow",
    "field": {
         "type": "color",
         "label": "Choose a color:",
         "default": "#ff0000" //default value of newly created items
    },
    "default": [ "#ff0000", "#00ff00", "#0000ff" ],
    "required": false,
    "minlength": 2,
    "maxlength": 5
}

This is an example of a field that allows the choice of an array of colors. The default value is a set of three colors, red, green and blue.

It is possible to create lists of any unitary type; for example, list of composites, or even list of lists, but don't try this at home...

TODOs:

Add an option to require that all items are different?
Add an option to sort items automatically instead of allowing the user to choose the :order?
Improve editor support, which is currently limited.

more to come...

Compound fields

Fields described in this section encode for several gadget preferences.

`bundle`

Bundles allow to subdivide gadget preferences in several named visually distinct sections. Each section has the same format of a full preference description. Bundles are implemented with jQuery.UI tabs, each section in his own tab.

Bundles are especially useful for complex gadgets with lots of preferences.

The syntax is as follows:

{
  "type": "bundle",
  "sections": [
    {
      //Section description, with the same syntax as a complete description of preferences
      "title": "@section1", //section's title (message or string)
      "intro": "@section1-intro" // optional introductory message
      "fields": [
         //... field descriptions ...
      ]
    },
    {
      "title: "@section2",
      "intro": "@section2-intro"
      "fields": [
         //... field descriptions ...
      ]
    },
    //... other sections ...
  ]
}

Bundles have only one member, named sections, which is an array of section descriptions; each section description defines fields of each section, with the same format of full preference descriptions.

Note that whether a gadget preference is encoded inside a bundle section only matters to improve user experience; there is no difference in what will be delivered to gadgets.

Easy to implement

multiselects
'url' fields

Ideas for further enhancements

What follows is a collection of unimplemented (and, possibly, not well defined yet) ideas.

Field dependencies

Sometimes one wants to enable a field (or a set of fields) only if some other fields satisfy some contraint. Most typical case is a checkbox which, if checked, enables one or more fields.

Proposed syntax for simple conditions:

{
	"name": "someField",
	"label": "@msg",
	"type": "string",
	"default": "foo",
	"requires": "otherField" //where "otherField" is an existing preference name
}

The field described by someField (in this case a string field) will be enabled only if the value of the field described by otherField is true (or something that evaluates to true, like non-empty strings or non-zero numbers).

A syntax for more advanced conditions may be added later, after evaluating if doing that is worth the added code complexity (it would not be difficult to allow arbitrary boolean predicates on other fields).

Extensibility

No matter how many features will be added to the preference description syntax, there will always be some specific use-case that still needs something that is not provided. Instead of adding tons of new features to cover each and every use-case, it may be better to allow gadget writers to enhance preference dialogs with their own code, which interacts with the dialog with some well defined interface.

This conflicts with an established (and well motivated) policy of not allowing any user defined code to be run on Special:Preferences (even if the "user" is the gadget's writer, which is usually more trusted than ordinary users). A way to solve this is putting user-provided code in an a ResourceLoader module which is registered on Special:Preferences, but not delivered; then, only if the user clicks the "Configure" link for that gadget, the module is downloaded and executed. In this way, the end user would be able to disable the gadget on Special:Preferences, without the risk of malicious code tampering with the interface.

Hidden/restricted fields

There should be a way of marking fields "hidden", so that default is always issued; moreover, there should be the possibility to show certain options only to specific users.

Known issues

Gadget preferences are deployed with a delay of 5 minutes after saving them, unless the user purges browser cache.
If memcached is enabled, changes to messages used by gadget preferences won't be visible to users untile memcached objects expire (generally 1 hour).

Working example: HotCat v2.9

NOTE: syntax of /HotCat.preferences changed in r93975; old syntax won't work anymore.

Steps to reproduce:

Download a copy from the branch I'm working on:

svn checkout http://svn.wikimedia.org/svnroot/mediawiki/branches/salvatoreingala/Gadgets

and install it.

Add this row to MediaWiki:Gadgets-definition:

*HotCat[ResourceLoader]|HotCat.js

and write a description for the gadget on MediaWiki:Gadget-HotCat.

Put the content of /HotCat.preferences to MediaWiki:Gadget-HotCat.preferences (NOTE: in this example all strings are hard-coded in preferences; in real use, MediaWiki messages would be used instead).
Put the content of /HotCat.js to MediaWiki:Gadget-HotCat.js.
Go to Special:Preferences and enable the HotCat gadget: a "Configure" link will (hopefully) appear.

To make the gadget use preferences, these changes were needed.

All the user configuration options documented in commons:Help:Gadget-HotCat#User_configuration (as of 9 Jul 2011) are implemented, with the limitation that the changedBackground option currently doesn't allow to choose 'transparent' as a color.

Not much care has been put on improving User Interface appearance and user experience; don't expect it to be great, yet! :)

Preferences editor

The page PreferencesEditor.js contains a simple GUI editor of preferences. After installing Gadgets from my branch, it may be used as a gadget. When ran, it adds a "Preferences editor" link to the Toolbox portlet.

All field types documented here are supported.

Integration howto

This section describes added code, and the changes made to existing classes. Paths refer to the new directory structure in my branch.

/ (root)
- Gadgets.alias.php: no changes
- Gadgets.i18n.php: trivial changes
- Gadgets.php: trivial changes
- Gadgets_tests.php: just added methods to test GadgetPrefs.php functions
- backend/
  - GadgetPrefs.php: main backend class; static class with these public methods:
    - isPrefsDescriptionValid($prefsDescription): checks if a preferences description is valid (must be called before using $prefsDescription with the other methods).
    - getDefaults($prefsDescription): returns the default values for $prefsDescription.
    - getMessages($prefsDescription): returns the messages needed to build the preference dialog for $prefsDescription.
    - checkPrefsAgainstDescription($prefsDescription, $prefs): checks if $prefs are valid preferences for $prefsDescription.
    - matchPrefsWithDescription($prefsDescription, &$prefs): changes the $prefs array, by putting default value for all missing or invalid values.
    - simplifyPrefs($prefsDescription, &$prefs): removes redundant values from $prefs, i.e. values that equal their default (called before saving, to remove unneeded values).
  - Gadget.php: old gadget class (was in Gadgets_body.php). Added three fields:
    - $prefsDescription: description of preferences (decoded from json, null if missing or invalid description)
    - $preferences: actual preferences for the current user (always valid, "fixed" if needed before saving this field)
    - $prefsTimestamp: timestamp of the last time preferences for this gadget have been saved (used to ensure module's freshness when delivering options )
  Added 8 methods:
  - getPrefsModule: returns the module that delivers preferences to the user (GadgetOptionsResourceLoaderModule).
  - getPrefsModuleName: obvious
  - getPrefsDescription/setPrefsDescription, getPrefs/setPrefs, getPrefsTimestamp/setPrefsTimestamp: accessors for the three new members; setPrefs can also save preferences back to DB (if the second param is set to true).
  Also changed things in loadStructuredList and newFromDefinition, to properly fill the those members ($preferences is filled with default values here; this is needed because gadgets may be enabled for anonymous users; prefs for other users are filled in hooks). Code of this methods was somewhat ugly, and now is even uglier: hopefully those don't survive the rewrite :P
  - GadgetsMainModule.php: simple module ('ext.gadgets') that creates that mw.gadgets object and some non gadget-specific info (e.g.: the list of configurable gadgets).
  - GadgetResourceLoaderModule: gadget's module ('ext.gadget.Foo'), was in Gadgets_body.php. Added dependencies to 'ext.gadgets' and to 'ext.gadget.Foo.config' (if needed); overridden getScript method to support delivering of preferences (gadget's code is wrapped in a closure).
  - GadgetOptionsResourceLoaderModule: actual module ('ext.gadget.Foo.prefs') with user preferences for a gadget; straightforward.
  - GadgetHooks.php: was in Gadgets_body.php. Changes made:
    - articleSaveComplete: if a MediaWiki:Gadget-Foo.preferences is saved, reload gadget (old code only looked for MediaWiki:gadgets-definition).
    - getPreferences: just replaced deprecated wfMsgExt with wfMessage.
    - registerModules: added code to register the 'ext.gadgets.preferences' module; done here because it needs messages returned by GadgetPrefs::getMessages(...) for all gadgets. Added code to register the gadgets options module (old code only registered gadget's module).
    - beforePageDisplay: only added a couple of lines to load the 'ext.gadgets.preferences' module on Special:Preferences.
    - userLoadOptions: completely new hook. This removes from the $options array all the preferences that looks like gadget preferences, unserializes them and saves them in gadget objects (aften ensuring correctness with GadgetPrefs::matchPrefsAgainDescription).
    - userSaveOptions: completely new hook. This reinserts gadget's preferences back in the $options array, after calling GadgetPrefs::simplifyPrefs, adding timestamp and serializing them.
    - applyScript: no changes.
- api/
  - ApiQueryGadgets.php and ApiQueryGadgetCategories.php: no changes.
  - ApiGetGadgetPrefs.php and ApiSetGadgetPrefs.php: straightforward new APIs to get preferences (and description) and to set preferences.
- ui/
  - SpecialGadgets.php: no changes.
  - resources/
    - jquery.farbtastic.js, jquery.farbtastic.css and the image/ folder: color picker, adapted from http://acko.net/dev/farbtastic
    - jquery.formBuilder.js and jquery.formBuilder.css: main cliet-side component, jquery plugin that builds dialogs from preference descriptions. Also contains code for the editor.
    - ext.gadgets.preferences.js and ext.gadgets.preferences.css: code tweaks for Special:Preferences.
    - jquery.validate.js: jQuery plugin for form validation, from http://bassistance.de/jquery-plugins/jquery-plugin-validation/.

User:Salvatore Ingala/Notes

Contents