Manual:Adding and removing languages/fr

MédiaWiki est fortement multi-langues et localisé. Y ajouter de nouvelles langues est une activité courante.� Parfois , il arrive aussi que des langes soient supprimés. Ceci est fait pour diverses raisons, et les procédures diffèrent beaucoup les unes des autres, autant du point de vue technique que au niveau des politiques de la communauté.� Cette page documente les méthodes d'ajout de langues.

Some conventions for this document:

• "qqf" is a generic example language code.

Noyau MediaWiki

Ajouter au noyau MediaWiki

MediaWiki core is usually not the first place to which a language is added; the first places are usually translatewiki.net and language-data.

A new language is usually added to MediaWiki core when the translation of the messages in the "MediaWiki core" group in translatewiki.net reaches the export threshold (as of February 2023, it's 13%; see :translatewiki:Translating:MediaWiki and the "mediawiki" section in repoconfig.yaml). When this happens, the translations are automatically exported to languages/i18n/qqf.json, a bot adds the new language to Translating:MediaWiki/New languages, and one of the translatewiki administrators creates a Phabricator task to add the new language (example: task T294729).

Preparation steps:

1. Determine the ISO 639 code. No ISO 639 code—no adding! (At this stage, it should have already been checked when adding to language-data and translatewiki, but you still need to double-check it, because Names.php is one of the most notable and stable locations for language configuration in MediaWiki.)
2. Determine the autonym. By this point, this should have already been done when the language was added to translatewiki or language-data, but it doesn't hurt to double-check. Check the patches that added the languages to those other places, and verify the sources for the autonym.
3. Do basic quality control on the translations: No one can know all the languages, but it's possible for anyone with MediaWiki experience to check simple things:
   1. Messages must be actually translated and not just copied from English.
   2. General syntax correctness is practiced with magic words, links, etc.
4. Decide whether the language needs a fallback language. English is the default final fallback language and doesn't need an explicit definition. Don't guess this, but ask native speakers whether it's better for most people who speak this language to see untranslated things in English or in some other language. Remember that one language may be spoken in several countries.
5. Optional, but highly recommended: Determine what are all the characters that are necessary for writing this language's words. This is necessary for defining the linktrail. The Wikipedia articles about the language is often a good source for the alphabet, but double-check them with external reliable sources. If the language is written in the plain 26-letter ASCII Latin alphabet without any diacritics or special characters, then an explicit linktrail is not necessary.
   • Note: Languages that are written in scripts of East Asia (Chinese, Japanese, Korean) or Southeast Asia (such as Thai, Burmese, Javanese, etc.) probably don't need a linktrail. If in doubt, ask a speaker.
6. Optional, but highly recommended: Ask people who know this language well (for example, trustworthy translatewiki translators or Incubator contributors) to give you translations for the namespace names. Useful documentation about this for translators can be found on the page Translating:MediaWiki#Translating namespace names. The list is short, but give the translators some time to think about it: it's a bit difficult to change it later, so it's important to get it right.
   • Note: Check that none of the namespaces are the same as language codes! This creates ambiguity with interlanguage links in wikitext.
7. Optional: Get date formats for the language.

Make a Gerrit patch in core:

1. Add an entry about adding this language to the "Languages updated" section in the newest version RELEASE-NOTES.
2. Edit includes/languages/data/Names.php. Add an entry for the language. Copy the autonym from translatewiki or language-data (they should be the same; if they aren't, something may be wrong somewhere). The name doesn't have to begin with a capital letter; English requires it, but most languages don't. Put the language's English name in a comment.
3. If the language is written in a writing system that requires it, define wider line-height. This is mostly needed for South Asian and South East Asian writing systems, such as Devanagari, Bengali, Thai, etc. This is done in the file resources/src/mediawiki.skinning/i18n-headings.less.
4. If you have any information to add there, create the file languages/messages/MessagesQqf.php. If you don't have any information to put there, skip this step. You can copy the boilerplate from a file for a similar language, but make sure to change all the necessary parts. Details:
   • Add fallback in the beginning, if necessary.
   • If the languages is written from right to left, add $rtl = true;.
   • Add namespace names in the variable $namespaceNames, if you have them. Make sure to use underscores instead of spaces in the strings.
   • If necessary, add gender aliases for them in the variable $namespaceGenderAliases.
   • If the new language uses a fallback that has gender aliases, such as French, Russian, Spanish, or Portuguese, but the new language itself doesn't need them, reset them by adding $namespaceGenderAliases = [];.
   • Add date formats in the variables $datePreferences, $defaultDateFormat, and $dateFormats, if you have them.
   • If the language needs different numerals, add them in the variable $digitTransformTable. Examples can be found in Persian (fa), N'Ko (nqo), Burmese (my). Don't guess this—consult with native and ask whether they actually use them. Some languages have traditional native numerals defined in the Unicode block for their writing system, but in practice they use Arabic numerals or some other system.
   • Add the $linkTrail in the end, if you have it.
   • Add magic words and special page aliases if you have them.

After this is done and deployed:

1. Remove the language from translatewiki.net configuration.
2. Test whether you can change your interface to this language using the "Internationalisation" section in Preferences, and using Universal Language Selector.
3. Check whether there are any special definitions for this language in Wikibase and Wikidata, and remove them. When the language is in Names.php, it's fully supported in Wikibase (and Wikidata), too.

Special scenarios for core

TODO

Removing from core

TODO

CLDR

The CLDR extension of MediaWiki holds a copy of the official Unicode CLDR Project. It is used to display language names and other information about languages, in a lot of languages. Examples:

• {{#language:fr|de}}= The language name of "fr" (French) in "de" (German) = Französisch
• {{#language:ko|en}}= The language name of "ko" (Korean) in "en" (English) = Korean
• {{#language:en|ar}}= The language name of "en" (English) in "ar" (Arabic) = الإنجليزية

It is used by Babel and Wikidata too (test page).

Therefore, if the new language code is not part of CLDR, only the autonym will be displayed instead of the name of language in the requested language or the user language.

To override this locally:

1. Check whether the language code is in CldrNames/CldrNamesEn.php and is correct:
   1. Yes: Go to the next section
   2. No: check if the language code is in LocalNames/LocalNamesEn.php. This is a MediaWiki-specific file for languages that are not part of Unicode CLDR or are wrongly translated.
      1. Yes: Go to the next section.
      2. No: Write a patch which adds the new language code to LocalNamesEn.php. Translations into other languages are welcome. Example commit: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/cldr/+/846592
   3. If the official Unicode CLDR translation is wrong, it would be a bit more complex: File a ticket at Unicode CLDR and write a patch that changes the language code in LocalNamesEn.php (and, if needed, in other languages). Sadly, the process of fixing a translation in the official Unicode CLDR is really, really slow.

jquery.ime

jquery.ime is a library for typing in various languages and alphabets. It is embedded into websites, and the users don't have to install anything or change their settings (the only condition is using a modern browser with JavaScript enabled).

In MediaWiki, jquery.ime is integrated into the UniversalLanguageSelector extension, but because jquery.ime is designed to be independently usable without MediaWiki, it has its own list of languages. Therefore, when adding a new keyboard layout for a language that is not supported yet, the language must be added to that list:

1. Determine the ISO 639 code and the autonym.
2. If it's not there already, add the language to Universal Language Selector (language-data, jquery.uls, ULS extension).
3. In the jquery.ime GitHub repository, add the input method's code (see the README in the repository). After that, edit src/jquery.ime.inputmethods.js. In the $.extend( $.ime.languages, { section, add a new block, ordered by the ISO 639 code. There, write the autonym in the autonym and list the input method identifiers in the inputmethods field.

To test, run a local webserver, open http://localhost/examples, and try selecting the language and using your new input method.

jquery.uls

jquery.uls is the generic MediaWiki-independent library that provides the core functionality of the Universal Language Selector extension. To add a language to jquery.uls:

1. Add it to the language to language-data.
2. After that, make a GitHub patch in the wikimedia/jquery.uls. In the repo's root folder, run the script ./scripts/fetch-language-data.sh.
3. Check the diff. In the commit message, describe the changes (using the English name of the language), and give a link to the latest language-data commit on GitHub.

Example commit: https://github.com/wikimedia/jquery.uls/pull/422

language-data

language-data is a library with a list of language codes. Its primary purpose is to make the language selectable in Universal Language Selector, although it may also be used in other contexts. As of late 2022, it's stored in the wikimedia/language-data repository.

It's one of the most "liberal" places for adding a language. Since the generic Universal Language Selector is used for various purposes in MediaWiki, and even outside the MediaWiki and Wikimedia world, it's not as strictly limited to new languages as, for example, the Wikimedia Language proposal policy for new wikis, and it may include ancient and constructed languages or variants that can be generally useful.

Despite being relatively liberal, it still strictly requires that only languages with a valid ISO 639 code are added. No ISO 639 code—no adding!

To add a language, determine the autonym and make a GitHub patch in the wikimedia/language-data repository. Before doing it for the first time, make sure that you are familiar with that repository's README and with the other documentation to which it links.

1. Update your copy of the the language-data repository.
2. Run the script php src/util/ulsdata2json.php and view the diff. If it creates any changes, make a patch. In the commit message, say that the change is automatic, but describe the changes.
   • The ulsdata2json.php script automatically downloads a file with data about languages from the CLDR server. Usually, these automatic changes are adding or removing languages used in countries.
3. When ulsdata2json.php script doesn't create any changes, edit the file data/langdb.yaml. Add a line for that language in alphabetical order of language codes. List the ISO 15924 code of the language's script, the continents where the language is spoken, and the autonym.
   • The ISO 15924 code must appear in one of the scriptgroups sections towards the end of the file. If the language is written in a script that doesn't yet appear in any of these groups, determine its writing system, make sure that it has a valid four-letter ISO 15924 code, and add it to a one of the groups. Use your judgment to put it in an appropriate group.
4. Run the script php src/util/ulsdata2json.php.
5. Using git diff, check that the file data/language-data.json was updated.
6. Run npm test. All the tests must succeed.
7. In the commit message, mention the reason for adding the language (using the language's English name), and the source for the autonym. (This is similar to adding a language to translatewiki.)
8. Submit the changes as a pull request.

After the pull request is merged, update jquery.uls.

Consider also adding a keyboard layout for the language using jquery.ime.

Names.php

See Core MediaWiki.

translatewiki

Adding to translatewiki

Adding new languages to translatewiki usually begins as a request on the page Support. The request must include the language's ISO 639-3 language code.

Before adding, read the policy at Translatewiki.net languages and make sure that the language fits it. Ask the requester for clarifications if needed. If the language doesn't fit the policy, politely decline the request.

Determine the autonym. The autonym should be mentioned in the request, but you still have to verify it according to the instructions on this page.

Make a Gerrit patch in the translatewiki repository:

1. Edit the file mw-config/LanguageSettings.php. Add a line for this language in alphabetical order of language codes. Write its autonym in the quotes. In a comment on the same line, add the English name of the language, and write your name and date.
2. Optional: Add default assistant languages. Edit the file mw-config/FallbackSettings.php, add a line for that language, and list the languages in the array. The language that is the most likely to be helpful to translators should be at the top. Even though the file where it's done is called FallbackSettings.php and the variable in question is called $wgTranslateLanguageFallbacks, this is not actually a fallback language, but an assistant language that is shown to translators as an aid, when a translation is available. This will make translation easier for people who didn't define assistant language preferences. Only languages that are already available on translatewiki should be added there. English doesn't have to be added there, because it's always shown. Add no more than six languages. Consider also adding the new language as a default assistant language to other languages. The languages to add there are:
   • A common foreign or official language in the country where this language is spoken. E.g., if a language is spoken in Indonesia, add Indonesian; if a language is spoken in a Francophone African country, add French; etc.
   • Languages from the same family. Don't just add everything from the same linguistic family; only add languages that the translators are likely to find helpful.
   • Other languages spoken in the same country or region.
3. If a language is a variant of another language, and needs only a partial translation, edit the file repoconfig.yaml and add the language code to the relevant always-export-languages sections. (Example: "pap-aw".)

In the commit message:

1. Give a reason for adding the language (using the language's English name). If it's a request on the Support page, give a direct permanent link to the relevant thread. If the reason is different, explain it clearly.
2. Give the source for the autonym.

See the git log for mw-config/LanguageSettings.php for example patches.

After the patch is reviewed and merged, deploy the updated configuration to the production translatewiki server. After the deployment, test that the language was added correctly: go to Special:Translate, click the target language selector, and type the new language's code. The new language should appear in the results.

If the language is written from right to left, add its code to the following local translatewiki.net pages:

• translatewiki:Template:Dir
• translatewiki:MediaWiki:Common.css - to the section that defines all :lang(qqf) elements as direction: rtl;.

Add the language to language-data and then to Universal Language Selector. The translatewiki.net change can be deployed before the Universal Language Selector change is done, and the language will be usable as a target language for localization, but selecting it can be broken in some cases, so don't forget it and do it as quickly as possible.

Make sure that there is a language portal for the language on translatewiki. The portals have names in the form "Portal:qqf". Some portals for languages that aren't yet configured already exist, but the language may be marked as "disabled". See the instructions at Template:Portal.

Add the language's native and transliterated autonyms to the Wikimedia Portals localization (not to be confused with translatewiki's own language portals).

Consider also adding a keyboard layout for the language using jquery.ime.

Removing from translatewiki

Languages are removed from translatewiki in two cases:

1. If the language has crossed the export threshold (see Translating:MediaWiki) and was added to MediaWiki core's Names.php, it should be removed from translatewiki.net's own configuration. In this case:
   1. Make sure that the language is actually in Names.php, and that the MediaWiki revision that includes it has been deployed to translatewiki!
   2. Remove the language's entry from the file mw-config/LanguageSettings.php.
   3. Check the language's entry in the file mw-config/FallbackSettings.php. If the language has no entry, you're all set. If it has an entry, check whether any of the assistant languages has been added to core MediaWiki as the fallback language, remove that language from FallbackSettings.php. Leave the rest of the languages, as they may still be useful as assistant languages.
   4. Remove special definitions for this language code from the translatewiki:MediaWiki:Common.css page.
2. Sometimes, the translatewiki administrators decide that MediaWiki and other projects hosted on translatewiki shouldn't be localized into that language. This may happen, for example, if the language had been added not according to policy, if grave mistakes were made while adding the language, or if the localizations are very low quality and it's better to remove them and start the work in that language from scratch.
   1. Remove the language's entry from the file mw-config/LanguageSettings.php and from mw-config/FallbackSettings.php.
   2. If the language is being removed because it was added by mistake, check whether it appears in any special configurations in repoconfig.yaml, and remove them if needed (to be extra-sure, grep the repository for more appearances).
   3. Consider also deleting all the translations from the site.
   4. Delete the language's portal page or mark the language as disabled.
   5. TODO: anything more to document here?

Universal Language Selector

Universal Language Selector is the MediaWiki extension that provides language selection functionality in various contexts.

To add a language to Universal Language Selector, first add it to language-data and then to jquery.uls. After that is merged, make a Gerrit patch for Universal Language Selector:

1. From the repo's root folder, run the script scripts/update-jquery-uls.sh.
2. Check the diff. In the commit message, describe the changes (using the language's English name), and give a link to the latest jquery.uls commit on GitHub.

Example commit: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/UniversalLanguageSelector/+/793082

Furthermore, if the language's autonym has commonly used aliases or alternate spellings, add them to the $specialLanguages array in the file data/LanguageNameIndexer.php. This is done to ensure the ULS's search box usefulness: many people may use a name that is significantly different from the autonym to search for the language they need. You can see the existing examples of cases when this is needed in the file itself:

• A language that has a common alias in the language itself. For example, in Spanish, the Spanish language itself is often called both "español" and "castellano". The name "español" is used as the autonym, but since "castellano" is a common alias, it is also added here.
• Some languages that aren't written in the Latin alphabet are frequently searched using the Roman alphabet, such as Armenian or Japanese.

Wikibase

TODO

Lexemes

TODO

Remove wikibase-lexeme-language-name-qqf

Wikimedia Portals

Wikimedia Portals are the main language-neutral pages of Wikipedia, Wiktionary, and some other Wikimedia projects. They have www in the beginning of their URL, rather than a language code.

To make sure a language can be properly supported in portals, do the following steps:

1. Make sure that the language is configured in translatewiki.net.
2. If the autonym is written in an alphabet that has uses letter casing, such as Latin or Cyrillic, you should begin the autonym with a capital letter, unless the language specifically requires that it be a small letter. If in doubt, verify with someone who knows the language well.
3. Go to Special:Translate on translatewiki.net.
4. Select "Wikimedia Portals" in the message group selector.
5. Select the target language.
6. Click "..." in the toolbar (next to "Translated") and check the "Optional messages" box.
7. In the key portals-language-name, add the autonym.
8. If the autonym is not in the Latin alphabet: In the key portals-language-name-romanized, add a transliteration of the autonym. It may include diacritics and various special character.
9. If the key portals-language-name or portals-language-name-romanized includes any characters outside the basic 26-letter Latin alphabet: In the key portals-language-name-romanized-sorted, add a transliteration of the autonym without any special characters.

Make sure to read the documentation (qqq) for each of the messages above!

The rest of the "Wikimedia Portals" message group should also be translated, but this, of course, should be done by people who know the language.

If there is a project in that language, but its name appears incorrectly, doesn't appear on the portal at all, or appears in the wrong section, please create a Phabricator ticket with the tag "#wikimedia-portals".

TODO:

• Describe the actual fixing for cases when the name doesn't appear or appears in the wrong section.
• Describe rtl language handling (they probably have to be added while creating a new wiki, but verify this).
• Describe various caveats, exceptions, overrides.

Determining the autonym

When people ask to add a language or to change the autonym of an already-configured language, please do your best to verify the autonym. This is sometimes challenging: MediaWiki already supports well over 400 languages, which are, naturally, the world's better-documented languages. This means that many of the new languages that are being added are less well documented, so it's generally harder to find information about them. In particular, their autonyms may be hard to find, or different sources may cite different autonyms, and when adding the language you'll have to make a decision without actually knowing the language. Try to use your best judgment and to reach a reasonable compromise between the information in available third-party sources and the information given to you by the requesters.

Autonyms usually don't have to be written with capital letter. English spelling conventions require that names of languages be written with a capital letter in English, but most languages don't have such a requirement. Use a capital letter only if this is specifically required by the language's orthography or if you are doing it in an environment where a capital letter is needed for another specific reason.

Autonyms usually appear in lists of languages, so they have to be unique. The users must be able to choose the precise language that they need, and not something with a similar name.

Some sources where you may find autonyms:

• A good autonym may appear in the Wikipedia article about the language in major languages, such as English, French, German, Portuguese, Spanish, or Russian. You can also look for it at the Wikidata item page for the language. It may be correct, but as with every statement in Wikipedia and Wikidata, they are often good starting points, but you have to verify it with an external reliable source.
• The best source for autonyms is a professionally written and published online or printed book about the language: a dictionary, a grammar reference, a standard orthography guide, a history of the language, etc. Academic articles about the language are a good source, too. Glottolog often has titles of such sources, although it doesn't always have a link to an online copy. A good source where actual books or at least parts thereof may be found are the Internet Archive (many such books are available for free time-limited lending in the e-book library). Some providers in The Wikipedia Library, such as L'Harmattan, Cambridge, and others have free access to relevant e-books. Google Books and Amazon.com may have enough in the free preview to find the autonym.
• Other websites and apps in this language can be a very good source, especially if they have a language selector. One particular kind of app to check is Android keyboard apps, such as Gboard: install the app, enable the language, and check how does its name appear when you choose to type in it. Check also the language settings on Windows—recent versions have a very large selection of languages.
• The UN's Universal Declaration of Human Rights is one of the most translated documents in history, and translations are available online. Unfortunately, the lookup system for languages is not so convenient, and the presentation is non-standardized and quite inconsistent. Nevertheless, it is sometimes a useful source.
• Ethnologue: the autonym is usually available as part of the freely-shown pages, although it's not always the best option. Try to check in other sources, too.
• JW.org: this site is available in more than a thousand languages and has an autonym for each of them, but they are sometimes written differently from other websites or books. Do your best not to rely only on this source.

Note that when writing in English, for example in English-language discussions, code comments, Git commit messages, etc., you should use the English name of the language and not the autonym.