Multilingual Templates and Modules
IntroductionEdit
What is it? This project makes it possible for Lua modules to be used on multiple wikis, without any modifications. All text strings are parameterized so they can be translated and localized when calling the modules from templates (or in some cases translations are stored in Commons) and the Synchronizer tool is used for keeping modules synced across Wikimedia wikis.
Why is this needed? Because we do not have a single Wikipedia, we have 300+ separate Wikipedias and other wiki projects, and every time someone creates a good new template or Lua module, it gets copied and translated 300+ times. Every translator has to thoroughly understand MediaWiki markup, making copying a very tedious and error-prone process, partially because template authors often assume their templates will be used in just one language. Once copied, the original templates are often improved, and each copy has to be updated while maintaining all existing translations. The pure human expense of copying templates and modules is so high that most of them are either never copied or never updated, especially for the smaller wikis.
Is this the best approach? No, but it is the best approach with the current technology. A significant MediaWiki rewrite is required to make this possible on the system level. Multi-site templates has been requested from the start in 2001, but not much was done simply because this is a hard problem to solve. With this approach it is possible to create multilingual content now, and once MediaWiki supports it, we can easily migrate multilingual content to the new system without much work.
ExampleEdit
Module:Excerpt and its partner Template:Excerpt can serve as an example of how to create a cross-wiki module and template:
- Module:Excerpt is copied to other wikis with exactly the same name and code. For example, in the Spanish Wikipedia it's called Módulo:Excerpt. Keeping the same name is important because dependencies between modules would break otherwise. Luckily, the Module namespace being localized causes no issues because "Module" is an alias for the Module namespace in all languages.
- Template:Excerpt, on the other hand, can have a completely different name and code in each wiki. For example, in the Spanish Wikipedia it's called Plantilla:Extracto. Templates call modules and in doing so they may localize parameters, set new defaults, and translate user-readable strings. For example, in the Spanish Wikipedia the wikitext of Plantilla:Extracto could be:
{{#invoke:Excerpt|main | lists = {{{listas|}}} | files = {{{archivos|1}}} | error-page-not-found = Página no encontrada }}
- The first parameter
lists
is localized to Spanish, since listas means lists in Spanish. - The second parameter
files
is localized to Spanish too, but also sets 1 as the default value. - The last parameter
error-page-not-found
controls the error message shown to the user by the module when a page is not found, and is here being localized to Spanish. An alternative way to localize user-readable strings is to create a translation table in Commons and have the module get the translations from there (example). See below for more.
- The first parameter
- The Synchronizer tool may then be used to keep the module in sync across all Wikimedia wikis:
Best practicesEdit
This section describes some of the current best practices while developing cross-wiki modules.
Initial commentEdit
Each module should start with a comment that includes a link to the main version of the module and some kind of warning asking to contribute to the main version rather than the local one, to foster collaboration rather than forking (example).
SandboxEdit
Each module should have a /sandbox subpage where to test out changes before deploying them on the main module and the other wikis.
TestcasesEdit
Each module should have a /testcases subpage with good unit tests to ensure high quality and stability of the shared module, see for example Module:No globals/testcases
- Testcases should use Module:ScribuntoUnit
- Testcases should run with both the main module and the sandbox versions, so that we can compare the results (see TNT or TableTools examples)
- Testcases should have require('Module:no globals') to avoid accidentally using non-declared variable
- Testcases should output their results both in testcases/doc and the main doc page of the module, to catch errors as early as possible
DocumentationEdit
Each module should have a /doc subpage with documentation that:
- Documents all public functions of the module.
- Includes a quality control section, listing the summary of the testcase runs for both the primary and the sandbox versions of the module. See Module:TNT for an example.
Localized user-readable stringsEdit
Many modules need to output user-readable strings, such as error messages and interface elements (like buttons). Hard-coding the text of these strings would force other wikis to modify the code in order to localize them, which would complicate or even prevent future synchronization. To avoid this, developers should provide a way to localize user-readable strings without having to modify the code itself. This section explains several ways to achieve this.
Template parametersEdit
Modules may allow templates to set the user-readable strings when calling the module. This approach is convenient when:
- The text is likely to vary with each template call.
- The text is likely to contain a magic word, a template call or some other wiki element.
An example of a module using this approach would be:
local p = {}
function p.main(frame)
local args = frame.args
local text = args['text'] or 'Example'
return text
end
return p
This way, every template may modify the text when calling the module, like so:
{{#invoke:Example|main
| text = Ejemplo
}}
Notice that in this example, if a template calls the module without specifying the text
parameter, then the hard-coded English text 'Example' will be used. This is not necessary. Modules may require template callers to set the text
parameter by throwing an error if they don't. However, it's often friendlier to fallback to English.
Config fileEdit
Another approach is to have a separate /config subpage where strings are localized. This approach is convenient when:
- The module is meant to be called by many templates per wiki, thus allowing localizations to be done once and reused again and again.
- There're many messages to localize, so it's easier to have them all together in their own place.
- There's already a need for a /config file for other reasons, so we might as well use it for localizations too.
An example of a module using this approach would be:
local config = require('Module:Example/config')
local p = {}
function p.main(frame)
local text = config.text or 'Example'
return text
end
return p
Then wikis would be able to create /config files like the following:
return {
text = 'Ejemplo'
}
Translation tablesEdit
It's also possible to create a central translation table at Commons and get localizations from there. This approach is convenient when:
- The strings should vary with the preferred language of the user, rather than the language of the wiki or the page.
- We want to centralize localization efforts on a single page.
The Module:TNT was created specifically to get strings from translation tables. An example module using TNT could look like this:
local TNT = require('Module:TNT')
local p = {}
function p.main(frame)
local text = TNT.format('I18n/Example', 'text')
return text
end
return p
See Data:I18n/Template:Graphs.tab for a simple but real example with two messages, each message having a single parameter. It's important to store parameters as parts of the string because in many languages the parameter would have to be placed at a different position in the string according to the norms of the language.
Translation tables should start with the "Data:I18n/..." prefix to separate them from other types of tabular data. If a message has not yet been localized, TNT will fallback to English (or other fallback language as defined by the language's fallback sequence). TNT also supports all standard localization conventions such as {{PLURAL|...}} and other parameters.
One downside of this approach is that translation tables cannot be loaded from non-Wikimedia wikis, so relying only on this localization method prevents third-party wikis from using the module as-is.
MediaWiki messagesEdit
In some cases, MediaWiki itself (or some extension) may have the messages we need already localized. For example, if we need the string "New page" we may use MediaWiki:Newpage, like so:
local p = {}
function p.main( frame )
local msg = mw.message.new( 'newpage' )
local text = msg:plain()
return text
end
return p
See Special:AllMessages for a list of all available messages.
All of the aboveEdit
Depending on the case, all of the above methods may be combined. For example, MediaWiki messages may be used when available, and when not, a translation table or config file is queried, and if no localization is found there, then a hard-coded English text is used, unless a template parameter overrides it.
Combining several methods can be effective, but the benefits should be weighted against the downsides of the increased complexity, which may cause performance loss and bugs, as well as more difficulty in maintaining the code and attracting new contributors.
Translating template parametersEdit
Template parameters are usually stored as a JSON templatedata block inside the template's /doc subpage. This makes it convenient to translate, but when a new parameter is added to a global template, all /doc pages need to be updated in every language. Module:TNT helps with this by automatically generating the templatedata block from a table stored on Commons. Placing this line into every /doc subpage will use Data:Templatedata/Graph:Lines.tab table to generate all the needed templatedata information in every language. Even if the local community has not translated the full template documentation, they will be able to see all template parameters, centrally updated.
{{#invoke:TNT|doc|Graph:Lines}}
DashboardEdit
Module:ArgumentsEdit
- Support - while we can adapt it as is, we may also enhance it to allow parameter name translation table to come from Commons (many templates localize their parameters). --Yurik (talk) 19:06, 7 May 2019 (UTC)
- Support --Obsuser (talk) 21:09, 11 July 2019 (UTC)
- Support, the same version is used in a bunch of wikis. There is a kind of related patch at https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Scribunto/+/158323/. --Amir E. Aharoni {{🌎🌍🌏}} 23:36, 24 January 2020 (UTC)
Module:Citation/CS1Edit
Module:Clickable button 2Edit
- Support, Module:Clickable button 2 (Q17170916) is used widely across multiple Wikis. And often this creates confusion as there is an unlinked version 1 as well and some templates use depreciated ui-button. A global version would provide constancy and use updated css class Vis M (talk) 19:53, 26 June 2021 (UTC)
Module:DateI18nEdit
- Support - This module localizes dates, is the latest incarnation of the code used for over a decade on majority of files on Commons. It is cloned to many other wikis. I recently rewrote it to use c:Data:DateI18n.tab and c:Data:I18n/MonthCases.tab to store data on language specific date formats. --Jarekt (talk) 03:58, 13 April 2020 (UTC)
Module:ExcerptEdit
- Maintainer: User:Sophivorus
Module:GraphEdit
Module:Location mapEdit
- Very strong support --Obsuser (talk) 21:09, 11 July 2019 (UTC)
- Support Thousands of subpages with the name being the only parameter needed to be translated to different languages. --94rain Talk 06:15, 6 August 2019 (UTC)
- Oppose location map -template will be eventually deprecated as development is already moving to dynamic maps. --Zache (talk) 06:34, 23 August 2019 (UTC)
Module:Lua bannerEdit
Module:Message boxEdit
Edit
- Support --Obsuser (talk) 21:09, 11 July 2019 (UTC)
- Strong support--BoldLuis (talk) 13:44, 20 April 2021 (UTC)
Edit
- Support --Obsuser (talk) 21:09, 11 July 2019 (UTC)
- Support -- Masumrezarock100 (talk) 11:20, 23 November 2019 (UTC)
Module:No globalsEdit
Module:StringEdit
- Support - The _error() function should be changed to accept (message_id,...) (id + optional params). If not ignoring errors, use TNT to translate messages. --Yurik (talk)
- Support --Obsuser (talk) 21:09, 11 July 2019 (UTC)
- Support --Jarekt (talk) 03:58, 13 April 2020 (UTC)
Module:SynchronizerEdit
- Maintainer: User:Sophivorus
Module:TNTEdit
Module:TableToolsEdit
Module:TranscluderEdit
- Maintainer: User:Sophivorus
Module:ScribuntoUnitEdit
- Support can be easily supported, not much site-specific info. Viztor (talk) 00:32, 6 August 2019 (UTC)
- Support --Jarekt (talk) 04:00, 13 April 2020 (UTC)
Template:BCP47Edit
Template:BotEdit
- Support It's one of the most commonly used templates in all fo Wikimedia's projects, perhaps the most used one. --Amir E. Aharoni (talk) 10:33, 25 September 2019 (UTC)
Edit
Template:TlEdit
- Comment Suggest to migrate this to be a magic word, instead of copy-pasting everywhere. --Liuxinyu970226 (talk) 14:51, 30 April 2020 (UTC)
Template:User linkEdit
Proposed modulesEdit
Modules and templates listed below don't yet exist on MediaWiki.org and thus cannot be synced from here. However, they are strong candidates to someday join the list.
- Wikidata:Module:Databox
- Wikipedia:Module:Infobox
- Neutral - eventually yes, but I suspect infobox has some dependencies we need to solve first. --Yurik (talk) 19:06, 7 May 2019 (UTC)
- Neutral - hard CSS dependencies. Missing navbar module can be handled with code. --Zache (talk) 02:01, 8 May 2019 (UTC)
- Strong support --Obsuser (talk) 21:09, 11 July 2019 (UTC)
- Support, of course. How can there even be any doubt. It's big and difficult, and local communities need to have the ability to override content and design in some places, but it's one of the most visible families of templates, and most of the functionality is the same in all wikis, even if the implementation and the content policies around using infoboxes may be different. For small wikis, the lack of infobox templates is one of the biggest pain points. --Amir E. Aharoni (talk) 10:33, 25 September 2019 (UTC)
- Support -- Masumrezarock100 (talk) 11:20, 23 November 2019 (UTC)
- Strong support and urgent. w:Module:Infobox depends on Module:Navbar--BoldLuis (talk) 13:18, 20 April 2021 (UTC)
- Wikipedia:Module:InfoboxImage
- Support Capankajsmilyo (talk) 01:06, 25 March 2020 (UTC)
- Wikipedia:Template:Convert
- Support It even has instructions for adapting to other languages. --Amir E. Aharoni (talk) 06:27, 26 September 2019 (UTC)
See alsoEdit
- Wishlist proposal 2019 (40 votes)
- T122086 - RFC: Sharing templates and modules between wikis - poor man's version (original idea for this bot)
- T121470 - Central Global Repository for Templates, Lua modules, and Gadgets ticket (main ticket for everything cross-site shareable)
- Extension:Scribunto/Central modules
- Global templates/Proposed specification, short version - a proposal to implement a similar idea comprehensively, without the need for the special tools, and with full support in MediaWiki core and extensions