Help:Extension:Translate/Validators

Translatable strings often contain markup that should be retained as-is in the translation. Typing that markup can be slow and difficult because special characters are common. Translate extension can provide translators a button clicking on which inserts the piece of markup into the translation to the current cursor position. In addition, if a translation is missing that specific markup, the Translate extension can either warn the translator or simply reject the translation, since such markup is usually mandatory to display the messages properly to the end user.

For example in the string,

Adapted by %{name} from a work by %{original}

There are two insertables - %{name} and %{original}. If the translator does not add them to their translation, the end user using the software will not see a proper message.

The MessageValidator framework has been added with the intent of helping with validating translations. Validators run on the translated message and based on the configuration, a warning or error message is shown to the translator. Translations with warnings can still be saved, but ones that have error cannot. Only a user with translate-manage permission can save translations that have errors.

When configuring a validator, a regex is defined to identify markup that is mandatory. The validator can also be marked as insertable, in which case a button will be displayed to the translator to add that markup into the translation.

Adding a custom validators is still possible and will be needed for more specialized validations.

ConfigurationEdit

Following is a summarized validator configuration,

 1 VALIDATORS:
 2     # Example 1
 3     - id: InsertableRegex
 4       enforce: true
 5       insertable: true
 6       params: /\$[a-z0-9]+/
 7       keymatch:
 8         - 'untranslated' # Matches key untranslated directly
 9         - 
10           type: 'wildcard'
11           pattern: '*translated*' # Matches any key that has the translated in it
12     # Example 2
13     - id: InsertableRegex
14       insertable: true
15       params:
16           regex: /(?<pre>\[)[^]]+(?<post>\]\([^)]+\))/
17           display: $pre $post
18           pre: $pre
19           post: $post
20     # Example 3
21     - class: MathJaxMessageValidator
22       enforce: true
23     # Example 4
24     - id: BraceBalance

In the example above,

  1. InsertableRegex is a bundled validator that can accept a custom regex and run validations.
  2. MathJaxMessageValidator is a custom validator class.
  3. BraceBalance is another bundled validator.

VALIDATORS uses an array format. Lets look at the various parameters being used here in each array item,

ParametersEdit

Property Type Description
id string Incase a bundled / pre-provided validator is being used, the ID of the validator. Required if class is not specified.
class string If a custom validator is being used, then use this option instead of id. Specifies the name of the validator class See example #3 in the above config. The AUTOLOAD option can be used to load the class. Required if id is not specified.
enforce bool Whether the validator should be enforced. If set to true, and a translation fails validation, an error will be displayed which must be fixed in order to save the translation.
insertable bool Whether the validator should also be an insertable.
keymatch array With this option it is possible to limit certain validations to certain messages. Keymatch is an array with each option being either a string or a prototype. If it is a string, a direct comparison with the message key is done. See example #1 in the above config.
keymatch[i].type string Type is either regex or wildcard. This is the approach that will be used to check if the message key matches a given pattern.
keymatch[i].pattern string Pattern is a string that will be used for matching.
params string / associative array If params is specified as a string, it is used as the regex. See example #1

In this case if insertable is true,

  1. display is the first value from the regex match.
  2. pre is also the first value from the regex match.
  3. post is left empty.

If params is specified as an associative array (see example #2), see below for further details.

params.regex string The regex to be used for validator. Must use named captures. When specifying named captures, do not use the $ symbol in the name.

In example #2, two named captures are used - pre and post.

params.display string Mandatory value. The display value for the insertable. Named captures prefixed with $ are used here. See example #2.
params.pre string The pre value for the insertable. Value inserted before the cursor position. Named captures prefixed with $ are used here. If not specified, is set to the display value. See example #2.
params.post string The post value for the insertable. Value inserted after the cursor position. Named captures prefixed with $ are used here. See example #2. If not specified, defaults to an empty string.

Pre-provided / Bundled validatorsEdit

Following is a list of bundled validators,

BraceBalanceEdit

ID: BraceBalance

Ensures that the number of open braces / brackets, matches the number of closed braces / brackets in the translation.

For example, the following translations would pass,

  • {{ }}
  • [ ]

whereas, the following would fail,

  • [ ]]
  • {{ }

This validator cannot be marked as insertable.

EscapeCharacterEdit

ID: EscapeCharacter

The validator ensures that only the specified escape character are present in a translation.

The allowed escape characters can be specified when adding the validator, and can only include,

  • \t
  • \n
  • \'
  • \"
  • \f
  • \r
  • \a
  • \b
  • \\

This validator is not insertable.

GettextNewlineEdit

ID: GettextNewline

This works specifically for GetText based message groups.

Ensures that the translation has the same number of newlines as the source message at the beginning and end of the string.

GettextPluralEdit

ID: GettextPlural

This works specifically on GetText based message groups.

Ensures that if the source / definition contains a plural in the format - foo {{PLURAL:GETTEXT|one|many}} bar, the translation must contain it as well. Based on the language this also checks if the translation has the correct number of plural forms. For example, English has two, but Hebrew has four.

InsertableRegexEdit

ID: InsertableRegex

A generic reusable validator that can be used to specify custom validations and insertables.

For example, take the following configuration where the validator is marked as insertable and enforced,

- id: InsertableRegex
  enforce: true
  insertable: true
  params: "/\$[a-zA-Z0-9]+/"

Given the following source message - Hello $name. My name is $myName. that is being translated, the translation must have the parameters - $name and $myName. They will also be displayed as insertables to make it easier for translators to use them in the translation. An absence of these parameters will cause an error to be displayed to the translator.

InsertableRubyVariableEdit

ID: InsertableRubyVariable

This is a validator that matches ruby variables in the translations. Internally it extends InsertableRegexValidator and uses the following regex - %{[a-zA-Z_]+}. This validator is insertable.

Example: %{abc}.

MatchSetEdit

ID: MatchSet

Ensures that the translation is present in the list of values. Also takes a parameter - caseSensitive that can be either true (default) or false.

For example the following configuration, the validator will validate the message with key - html.dir and ensure that the values for it can either be ltr or rtl. Note that LTR or RTL will not be valid values, since caseSensitive is true by default.

1   - id: MatchSet
2     enforce: true
3     keymatch:
4       - html.dir
5     params:
6       values:
7         - ltr
8         - rtl

MediaWikiMiscEdit

ID: MediaWikiMisc

Provides validations for expiry options and IP block options specified in the MediaWiki core. These are usually in the format,

indefinite:indefinite,3 hours:3 hours,12 hours:12 hours,24 hours:24 hours,31 hours:31 hours,36 hours:36 hours,48 hours:48 hours,60 hours:60 hours,72 hours:72 hours,1 week:1 week,2 weeks:2 weeks,1 month:1 month,3 months:3 months,6 months:6 months,1 year:1 year,2 years:2 years,3 years:3 years,infinite:indefinite

The validations ensure that the translations have the exact same number of key-value pairs. These validations are run only on messages with keys,

  1. protect-expiry-options
  2. ipboptions

MediaWikiPluralEdit

ID: MediaWikiPlural

Ensures that if the source / definition contains a {{PLURAL:$1|message|messages}}, the translation should also have it. It can also be used as an insertable. Based on the language this also checks if the translation has the correct number of plural forms. For example, English has two, but Hebrew has three.

NewlineEdit

ID: Newline

Ensures that the translation has the same number of newlines as the source / definition message at the beginning of the string. This validator is not insertable.

PrintfEdit

ID: Printf

This validator checks for missing and unknown printf formatting characters in translations. This validator is insertable.

Example: %2$f, %d etc.

PythonInterpolationEdit

ID: PythonInterpolation

This validator matches python string interpolation variables by using the following regex: /\%(?:\([a-zA-Z0-9]*?\))?[diouxXeEfFgGcrs]/U. This validator is insertable.

Example: %s, %(name)s

SmartFormatPluralEdit

ID: SmartFormatPlural

This works specifically on SmartFormat based message groups.

Ensures that if the source / definition contains a plural in the format - {1:test|tests}{0:message|messages}, the translation must contain it as well. Based on the language this also checks if the translation has the correct number of plural forms. For example, English has two, but Hebrew has four.

UnicodePluralEdit

ID: UnicodePlural

Ensures that if the source / definition contains a plural in the format - foo {{PLURAL|one=one|many}} bar, the translation must contain it as well. Based on the language this also checks if the translation has the correct number of plural forms. For example, English has two, but Hebrew has three.

WikiParameterEdit

ID: WikiParameter

This is a validator that matches wiki parameters in the translations. Internally it extends InsertableRegexValidator and uses the following regex - /\$[1-9]/. This validator is insertable.

Example: $1, $2.

User interfaceEdit

The user interface has been updated to differentiate between errors and warnings.

 
A warning and error shown on top of a translation

During translation, if an error is noticed with the translation, the Save translation button is disabled unless the user who is translating has translate-manage permission.

Additionally validation is also done on the server when the user is saving the translation. This will still allow users who have the translate-manage permission to save the translation even if it has errors.

Custom validatorsEdit

Certain complicated validations might still require a custom validator to be written. Custom validators must implement the MediaWiki\Extensions\Translate\MessageValidator\Validator interface [1]. They can use the trait MediaWiki\Extensions\Translate\MessageValidator\ValidationHelper [2] that contains some commonly used methods. Below is an example of a custom validator,

 1 <?php
 2 // Filename: Validator.php
 3 use MediaWiki\Extensions\Translate\MessageValidator\Validator;
 4 use MediaWiki\Extensions\Translate\MessageValidator\ValidationHelper;
 5 
 6 /**
 7  * My Custom Validator
 8  */
 9 class MyCustomValidator implements Validator {
10 	use ValidationHelper;
11 	
12 	public function validate( TMessage $message, $code, array &$notices ) {
13 	    // Validation code goes here. Push notices into the notices array
14 	}
15 }

The format for the $notices array,

<?php
// $key is the message key
$notices[$key][] = [
    # check idenfitication
    [ 'printf', $subcheck, $key, $code ],
    # check notice message
    'translate-checks-parameters-unknown',
    # optional special param list, formatted later with Language::commaList()
    [ 'PARAMS', $params ],
    # optional number of params, formatted later with Language::formatNum()
    [ 'COUNT', count( $params ) ] ],
    'Any other parameters to the message',

The add the custom validator in the configuration file,

VALIDATORS:
  - class: MyCustomValidator
    enforce: true

AUTOLOAD:
  MyCustomValidator: Validator.php