Help:Extension:Translate/API
WORK IN PROGRESS
Introduction
editThe Translate extension is a complex extension. Some major functionality groups in it are:
- creation, modification and deletion of different kinds of messages groups
- parsing and generating various file formats
- loading a collection of messages from the database
- translation aids like translation memory
- translation editor
- message mass processing capabilities (import, export, fuzzying)
- statistics collection and presentation
- page translation (splitting and generating pages, creating message groups)
The functions are accessed either directly via PHP code (Internal-API) or via the web facing API framework (Web-API). There are also other web facing entry points that do not use the W-API framework (Special-API) and command line scripts (Cli-api). It is in principle possible to modify the database tables or files directly, but that is not recommended nor aim to be supported.
Users access the functionality via dedicated interface on MediaWiki itself, which consists of special pages and added/removed/modified interface elements on other pages, added via various hooks that MediaWiki provides. Some of the interface code calls the W-API to execute its actions, providing ideal separation of concens with the downside that W-API always needs JavaScript support on the browser (but it can also be used from other software without this issue), necessitiating S-API kind of alternative for users without JavaScript. Some of the newest features do not fallback gracefully when JS is not available.
The order of importance to third party developers is assumed to be Special-API, Cli-API, Internal-API and Web-API. I will present in this order what is already available, how flexible they are and how well they are documented. After that I will switch to functional inspection which includes features that are not yet available at all, and on which layers they should be.
Cli-API
editCommand line scripts are suitable for executing time consuming operations, like exporting translations into files or bootstrapping translation memory database. They are not suitable for small actions due to (1) difficulty of passing parameters (2) no standard way of doing error reporting and (3) startup overheard gets significant when doing thousand of actions. Cli-API is primarly intented for system administrators, advanced translation administrators and automatic tasks.
Each command line script has --help switch that tells what the command does and what are the parameters. There is general help somewhere how to run command line scripts, but not special help for Translate extension, though it doesn't differ from MediaWiki itself, but finding out that information might be a problem for new users.
There is no detailed overview of the command line scripts in Translate extension. The following list can be used as a staring point.
Maintenance scripts
- createCheckIndex.php - Runs message checks for each message updating the database state
- createMessageIndex.php - Message index is needed internally by Translate and it is not always regenerated automatically
- messadeDust.php - Find unused messages or messages in wrong place
- populateFuzzy.php - Updates the database fuzzy tag status for each messages
- ttmserver-export.php - Fills the translation memory with current translations
Message group related
- export.php
- fuzzy.php
- magic-export.php
- processMessageChanges.php
- sync-group.php
Sysadmin stuff
- logfilter.php - for sysadmins
- migrate-schema2.php - database schema upgrade
- pagetranslation-check-database.php - database consistency check script
- list-mwext-i18n-files.php - needed for MediaWiki i18n files for some time yet
Stats
- languageeditstats.php
- groupStatistics.php
Tests (should be moved to unit tests)
- pagetranslation-test-parser.php
- yaml-tests.php
Other
- plural-comparison.php
- cldr-plural-to-yaml.php
A very limited set of functionality is accessible via cli scripts.
Special-API
editAlmost all of the user facing functionality is primarly exposed through the special pages with some extensions done as command line scripts, due to restrictions of web environment. Special pages interfaces doesn't really constitute an API that third party users can rely on. Using web pages programmatically is called screencraping and that is highly discouraged. Having good API coverage that is well documented and advertised helps to avoid others to rely on screenscaping, which might cause a compatability problem which refactoring the special pages.
Very high level overview of the special pages:
- SpecialAggregateGroups.php - Message group management
- SpecialImportTranslations.php - Message group management
- SpecialManageGroups.php - Message group management
- SpecialMagic.php - Very little relevant for 3rd party use
- SpecialMessageGroupStats.php - Statistics
- SpecialSupportedLanguages.php - Statistics
- SpecialTranslations.php - Statistics kind of
- SpecialLanguageStats.php - Statistics
- SpecialTranslationStats.php - Statistics
- SpecialMyLanguage.php - Page translation related
- SpecialPageTranslation.php - Page translation related
- SpecialTranslate.php - The main translation interface, including the editor
- SpecialFirstSteps.php - Translatewiki.net specific signup page
Special pages expose only very little information and functionality, very often not enough to do alternative implementations. Special pages have quite good user documentation scattered around different help pages of Translate extension in mediawiki.org. The calling conventions of them are not documented and wont be documented. One specific case is the, or actually, are the translation editors, which are provided as HTML code by Special:Translate in a way that is highly tied to rest of the Translate extension.
Internal-API
editInternal API offers access basically to all functionality what Translate extension has - and I'm not going to repeat that. But there are certain ways and patterns how the internal api is made available and intented to be used.
Hooks. Hooks are way for programmers to execute their own code in predefined places decided by the Translate extension authors. Hooks provide well-defined injection points - at least in principle. Often the relevant data is not passed to the users code, or the parameters are highly tied to the current implementation and those might break when the implementation is refactored. Translate extension uses many hooks of MediaWiki, but only provides few new hooks itself which are currently only used internally by the extension. Because the extension code itself can access everything it needs, hooks aren't usually added except in obvious cases, or when somebody requests them. The hooks in Translate extension are not documented anywhere. The most important hook that currently exists is the hook which can be used to register new message groups.
Interface and subclasses. Another way to extend the Translate extension code is to write classes that implement some pre-defined interface (in general sense). Translate extensions currently has four interfaces (specific sense):
- iTTMServer
- StringMangler (making sure the message keys conform to limitations of wiki titles)
- MessageGroup
- FFS (file format support)
The interface defines what methods the user's code must implement, what parameters they get and what they return (not enforceable in PHP). The most useful interface currently is the FFS interface that defines how a file format parser and generator is called. Also MessageGroup interface is common use.
But just providing an interface is not all we can do. Translate extension comes with many classes that implement those interfaces, like SimpleFFS, FileBasedMessageGroup, AggregateMessageGroup and WikiMessageGroup. The user code and extend these classes, just overriding specific parts to implement the behaviour the user wants. This sometimes also works for classes that don't implement any predefined interface (specific sense). It should be noted that the Translate extension must be coded in a way that expects different classes by providing way to register those custom written classes and by providing a way to indicate in some configuration which class to use. The YAML based message group configuration allows registering new classes and defining which class to use for MessageGroup, FFS and StringMangler.
In theory one could also replace one of the core classes with custom class, but this is not recommended nor supported. The target should be that the core classes are extensible enough with hooks, and things like FFS use interfaces and subclassing.
The interfaces and classes are reasonable well documented via code documentation. There could be an overview interfaces and tutorials for them.
Web-API
editSince Translate extension is often provided as service and it acts as the interface (general sense) for the data it contains, being able to call that service over web (in this case http(s)) is of utmost importance.
Here's a list of currently available Web-APIs:
- ApiAggregateGroups.php - Creation and deletion of aggregate groups and defining their subgroups (write only)
- ApiGroupReview.php - Changing of message group states (write only)
- ApiHardMessages.php
- ApiQueryLanguageStats.php
- ApiQueryMessageCollection.php - Querying of message of a group, provides filtering and stuff (read only)
- ApiQueryMessageGroups.php - Information about message groups (read only)
- ApiQueryMessageGroupStats.php - Statistics of message groups (read only)
- ApiQueryTranslationAids.php
- ApiQueryMessageTranslations.php - Special:Translations equivalent (read only)
- ApiStatsQuery.php
- ApiTranslateSandbox.php
- ApiTranslateUser.php
- ApiTranslationStash.php
- ApiTranslationReview.php - Reviewing of translations (write only)
- ApiTTMServer.php - Querying of translation memories (read only)
These are somewhat documented when you access http://translatewiki.net/w/api.php, e.g. [1], though we understand it's hard to get the high level picture of what each module does.
Auto generated documentation
editThe following documentation is the output of Special: |
action=groupreview
- This module requires read rights.
- This module requires write rights.
- This module only accepts POST requests.
- Source: Translate
- License: GPL-2.0-or-later
Set message group workflow states.
- group
Message group. See action=query&meta=messagegroups.
- This parameter is required.
- language
Language code.
- Default: en
- state
The new state for the group.
- This parameter is required.
- token
A "csrf" token retrieved from action=query&meta=tokens
- This parameter is required.
- Mark the state of the German translation for the message group "group-Example" as ready
- api.php?action=groupreview&group=page-Example&language=de&state=ready&token=foo [open in sandbox]
The following documentation is the output of Special: |
action=translationreview
- This module requires read rights.
- This module requires write rights.
- This module only accepts POST requests.
- Source: Translate
- License: GPL-2.0-or-later
Mark translations reviewed.
- revision
The revision number to review.
- This parameter is required.
- Type: integer
- token
A "csrf" token retrieved from action=query&meta=tokens
- This parameter is required.
- Review revision 1
- api.php?action=translationreview&revision=1&token=foo [open in sandbox]
The following documentation is the output of Special: |
The following documentation is the output of Special: |
action=searchtranslations
- This module requires read rights.
- Source: Translate
- License: GPL-2.0-or-later
Search translations.
- service
Which of the available translation services to use.
- One of the following values: codfw, default, eqiad
- Default: default
- query
The string to search for.
- This parameter is required.
- sourcelanguage
The language code of the source text.
- Default: en
- language
The language code to search string for.
- Default: (empty)
- group
The group ID to search string in. See action=query&meta=messagegroups.
- Default: (empty)
- filter
Message translation status filter.
- One of the following values: Can be empty, or fuzzy, translated, untranslated
- Default: (empty)
- match
Match any/all search words.
- Default: (empty)
- case
Case (in)sensitive search.
- Default: 0
- offset
Offset for the translations.
- Type: integer
- Default: 0
- limit
Size of the result.
- Type: integer or max
- The value must be between 1 and 50.
- Default: 25
- Show translations for the language.
- api.php?action=searchtranslations&language=fr&query=aide [open in sandbox]
- Show untranslated messages matching the query in the source language.
- api.php?action=searchtranslations&language=fr&query=edit&filter=untranslated [open in sandbox]
The following documentation is the output of Special: |
The following documentation is the output of Special: |
action=translationaids
- This module requires read rights.
- Source: Translate
- License: GPL-2.0-or-later
Query all translations aids.
- title
Page title of a known message, with namespace and language code.
- This parameter is required.
- group
Message group the message belongs to. If empty then primary group is used.
- prop
Which translation helpers to include.
- Values (separate with | or alternative): definition, definitiondiff, documentation, editsummaries, gettext, groups, inotherlanguages, insertables, mt, support, translation, ttmserver
- Default: groups|definition|translation|inotherlanguages|documentation|mt|definitiondiff|ttmserver|support|gettext|insertables|editsummaries
The following documentation is the output of Special: |
action=ttmserver
- This module requires read rights.
- Source: Translate
- License: GPL-2.0-or-later
Query suggestions from translation memories.
- service
Which of the available translation services to use.
- One of the following values:
- sourcelanguage
The language code of the source text.
- This parameter is required.
- targetlanguage
The language code of the suggestion.
- This parameter is required.
- text
The text to find suggestions for.
- This parameter is required.
- Get suggestions for translating "Help" from English to Finnish
- api.php?action=ttmserver&sourcelanguage=en&targetlanguage=fi&text=Help [open in sandbox]
The following documentation is the output of Special: |
meta=messagetranslations (mt)
- This module requires read rights.
- Source: Translate
- License: GPL-2.0-or-later
Query all translations for a single message.
- mttitle
Page title of a known message, with namespace and language code.
- This parameter is required.
- mtoffset
When more results are available, use this to continue. More detailed information on how to continue queries can be found on mediawiki.org.
- Type: integer
- Default: 0
- List of translations in the wiki for "MediaWiki:January"
- api.php?action=query&meta=messagetranslations&mttitle=MediaWiki:January [open in sandbox]
The following documentation is the output of Special: |
list=messagecollection (mc)
- This module requires read rights.
- This module can be used as a generator.
- Source: Translate
- License: GPL-2.0-or-later
Query MessageCollection about translations.
- mcgroup
Message group. See action=query&meta=messagegroups.
- This parameter is required.
- mclanguage
Language code.
- Default: en
- mclimit
How many messages to show (after filtering).
- Type: integer or max
- The value must be between 1 and 5,000.
- Default: 500
- mcoffset
When more results are available, use this to continue. More detailed information on how to continue queries can be found on mediawiki.org.
- Default: (empty)
- mcfilter
Message collection filters. Use ! to negate condition. For example !fuzzy means list only all non-fuzzy messages. Filters are applied in the order given.
- fuzzy
- Messages with fuzzy tag.
- optional
- Messages which should be translated only if changes are necessary.
- ignored
- Messages which are never translated.
- hastranslation
- Messages which have a translation regardless if it is fuzzy or not.
- translated
- Messages which have a translation which is not fuzzy.
- changed
- Messages which have been translated or changed since last export.
- reviewer:N
- Messages where the user number N is among reviewers.
- last-translator:N
- Messages where the user number N is the last translator.
- Separate values with | or alternative.
- Maximum number of values is 50 (500 for clients that are allowed higher limits).
- Default: !optional|!ignored
- mcprop
Which properties to get:
- definition
- Message definition.
- translation
- Current translation (without !!FUZZY!! string if any, use the tags to check for outdated or broken translations).
- tags
- Message tags, like optional, ignored and fuzzy.
- properties
- Message properties, like status, revision, last-translator. Can vary between messages.
- revision
- Deprecated. Use mcprop=properties.
- Values (separate with | or alternative): definition, properties, tags, translation, revision
- Default: definition|translation
- List of supported languages
- api.php?action=query&meta=siteinfo&siprop=languages [open in sandbox]
- List of non-optional message definitions for the group "page-Example"
- api.php?action=query&list=messagecollection&mcgroup=page-Example [open in sandbox]
- List of optional messages in Finnish with tags for the group "page-Example"
- api.php?action=query&list=messagecollection&mcgroup=page-Example&mclanguage=fi&mcprop=definition|translation|tags&mcfilter=optional [open in sandbox]
- More information about latest translation revisions for the group "page-Example"
- api.php?action=query&generator=messagecollection&gmcgroup=page-Example&gmclanguage=nl&prop=revisions [open in sandbox]
The following documentation is the output of Special: |
meta=languagestats (ls)
- This module requires read rights.
- Source: Translate
- License: GPL-2.0-or-later
Query language stats.
- lsoffset
When more results are available, use this to continue. More detailed information on how to continue queries can be found on mediawiki.org.
- Default: 0
- lslanguage
Language code.
- This parameter is required.
- lsgroup
Message group ID. See action=query&meta=messagegroups.
- List of translation completion statistics for Finnish
- api.php?action=query&meta=languagestats&lslanguage=fi [open in sandbox]
- List of translation completion statistics for Finnish for group A
- api.php?action=query&meta=languagestats&lslanguage=fi&group=A [open in sandbox]
The following documentation is the output of Special: |
meta=messagegroups (mg)
- This module requires read rights.
- Source: Translate
- License: GPL-2.0-or-later
Return information about message groups.
Note that the uselang parameter affects the output of language dependent parts.
- mgdepth
When using the tree format, limit the depth to this many levels. Value 0 means that no subgroups are shown. If the limit is reached, the output includes a "groupcount" value, which states the number of direct children.
- Type: integer
- Default: 100
- mgfilter
Only return messages with IDs that match one or more of the inputs given (case-insensitive, separated by pipes, * wildcard).
- Separate values with | or alternative.
- Maximum number of values is 50 (500 for clients that are allowed higher limits).
- Default: (empty)
- mgformat
In the tree format message groups can exist in multiple places in the tree.
- One of the following values: flat, tree
- Default: flat
- mgiconsize
Preferred size of rasterised group icon.
- Type: integer
- Default: 64
- mgprop
What translation-related information to get:
- id
- Include ID of the group.
- label
- Include label of the group.
- description
- Include description of the group.
- class
- Include class name of the group.
- namespace
- Include namespace of the group. Not all groups belong to a single namespace.
- exists
- Include self-calculated existence property of the group.
- icon
- Include URLs to icon of the group.
- priority
- Include priority status like discouraged.
- prioritylangs
- Include preferred languages. If not set, this returns false.
- priorityforce
- Include priority status - is the priority languages setting forced.
- workflowstates
- Include the workflow states for the message group.
- sourcelanguage
- Include the source language for the message group.
- subscription
- Include whether the user is subscribed to the group.
- Values (separate with | or alternative): class, description, exists, icon, id, label, namespace, priority, priorityforce, prioritylangs, sourcelanguage, subscription, workflowstates
- Default: id|label|description|class|exists
- mgroot
When using the tree format, instead of starting from top level start from the given message group, which must be an aggregate message group. When using flat format only the specified group is returned.
- Default: (empty)
- mglanguageFilter
Only return message groups that can be translated to the given language.
- Default: (empty)
- Show message groups
- api.php?action=query&meta=messagegroups [open in sandbox]
The following documentation is the output of Special: |
meta=messagegroupstats (mgs)
- This module requires read rights.
- Source: Translate
- License: GPL-2.0-or-later
Query message group stats.
- mgsoffset
When more results are available, use this to continue. More detailed information on how to continue queries can be found on mediawiki.org.
- Default: 0
- mgsgroup
Message group ID. See action=query&meta=messagegroups.
- This parameter is required.
- mgssuppresscomplete
Do not display languages which have completed translation
- Type: boolean (details)
- mgssuppressempty
Do not display languages which do not have any translations
- Type: boolean (details)
- List of translation completion statistics for the group "page-Example"
- api.php?action=query&meta=messagegroupstats&mgsgroup=page-Example [open in sandbox]