Extension:External Data/Configuration

External Data configuration settings consist of two parts: configuration for data sources and a few other settings.

Data sources

Most of the extension settings regulate calls to {{#*_external_table:}}/{{#external_value:}} in standalone mode, or {{#get_*_data:}} parser functions in compatibility mode, or the corresponding Lua functions, configuring so called data sources. There are at least two data sources relevant to any parser function call; a more specific data source overriding a more universal one; and the last ('*') containing global fallback settings.

Settings for the data sources are stored in the associative array $wgExternalDataSources. Its keys are names of data sources and values are arrays containing settings for the data source.

The relevant data sources for the data retrieval parser functions (and their Lua analogues) and the relevant settings are:

Function	`$wgExternalDataSources` index (parser function `parameter`)				Settings	Description
Function	first	then	then	last	Settings	Description
Any data retrieving function	Data source ID (`source`, etc.)			`'*'`	`params`	An array of additional parameters to the data retrieving function used to substitute wildcards (like $param$ ) in configuration settings. Array member like `'param' => 'default'` will set a default `'default'`value for `'param'`, while `'param'` will make `'param'` required.
					`param filters`	An array of validators for additional parameters. A validator may be a string with a delimited regular expression that should match on the parameter value, or a callable that should return `true`. It is important to valdate parameters that are substituted for wildcards to prevent injections.
					`hidden`	If set to `true`, this data source can only be called with `{{#get_external_data:source=id\|...}}` or `{{#get_external_data:id\|...}}`, and the error messages will be suppressed, as if `suppress error` has been passed to the parser function.
`{{#get_web_data:}}` `mw.ext.externalData.getWebData()`	URL (`url`)	host (from `url`)	second level domain (from `url`)	`'*'`	`replacements`	Replacements in the URLs
					`allowed urls`	A whitelist of URLs
					`encodings`	A list of charsets to try
					`allow ssl`	Whether to allow SSL
					`options`	HTTP options
`{{#get_soap_data:}}` `mw.ext.externalData.getSoapData()`					`throttle key`	Throttle key
					`throttle interval`	Interval between two throttled calls, in seconds
					`always use stale cache`	Always allow stale cache
					`min cache seconds`	Cache for at least so many seconds
`{{#get_file_data:}}` `mw.ext.externalData.getFileData()`	File ID (`file`)/ directory ID (`directory`)			`'*'`	`path`	File or directory path
`{{#get_file_data:}}` `mw.ext.externalData.getFileData()`	File ID (`file`)/ directory ID (`directory`)			`'*'`	`depth`	Allowed directory iteration depth
`{{#get_ldap_data:}}` `mw.ext.externalData.getLdapData()`	LDAP domain (`domain`)			`'*'`	`server`	LDAP server
					`user`	LDAP user
					`password`	LDAP password
					`base dn`	Base DN
`{{#get_db_data:}}` `mw.ext.externalData.getDbData()`	Database connection ID (`db`)			`'*'`	`server`	Database server
					`type`	Database type
					`name`	Database name
					`user`	Database user
					`password`	User password
					`directory`	SQLite directory
					`flags`	Database flags
					`prefix`	Table prefix
					`prepared`	Prepared statement(s). If a string, this is the only prepared statement for the connection, and the `query` parameter in the wikitext is not needed. If an associative array, there are several prepared statements, indexed by `query`. Each of them can be, in turn, a string containing a prepared statement with no parameters or only string parameters, or an array of the form `[ 'query' => 'SELECT ...', 'types' => 'si' /* or other parameters types */ ]`
					`types`	Parameter types for the prepared statements
					`cache seconds`	Cache MongoDB result for so many seconds
`{{#get_program_data:}}` `mw.ext.externalData.getProgramData()`	Program ID (`program`)			`'*'`	`command`	Shell command
					`input`	Parameter name that shall be fed into program's standard input
					`temp`	Name of the temporary file to be used instead of standard output
					`limits`	Resource limits
					`env`	Environment variables
					`ignore warnings`	Ignore warnings that a successfully executed program may send to stderr
					`preprocess`	A callable that preprocesses program's standard input
					`postprocess`	A callable that postprocesses program's standard output
					`name`	Program name for Special:Version
					`program url`	Program website fot Special:Version
					`version`	Program version for Special:Version
					`version command`	A shell command that outputs program version for Special:Version
					`tag`	Tag for the tag emulation mode
					`throttle key`	Throttle key
					`throttle interval`	Interval between two throttled calls, in seconds
					`always use stale cache`	Always allow stale cache
					`min cache seconds`	Cache for at least so many seconds

Remember that {{#*_external_table:}}/{{#external_value:}} in standalone mode or {{#get_external_data:}} can replace any of the {{#get_*_data:}} parser functions, as well as mw.ext.externalData.getExternalData() can replace any of the mw.ext.externalData.get*Data() Lua functions.

Any parameter of {{#get_*_data:}} can be omitted, provided it is set in the corresponding $wgExternalDataSources['…'] array. The obvious exception is the parameter used as the key to $wgExternalDataSources: url, directory, file, domain, db, program and source (see below).

The parameter source can replace url, directory, file, domain, db and program, provided that the corresponding $wgExternalDataSources['source'] contains all the settings necessary to choose and initialise the proper connector. Furthermore, if the value of source does not contain equal signs, source = can be omitted, i.e., this parameter can be passed anonymously.

Any configuration setting can include wildcards surrounded by dollar signs, like this: 'url' => 'https://raw.githubusercontent.com/lipis/flag-icons/main/flags/4x3/$iso2$.svg'. These wildcards will be substituted from additional parameters to {{#get_*_data:}}. The additional parameters should be declared as required or receive a default value in $wgExternalData['…']['params'], e.g.: $wgExternalData['…']['params'] = [ 'iso' ];. It is important that a validator is set up for these parameters: $wgExternalData['…']['param filters'] = [ 'iso' => '/^[a-z]{2}$/' ];. This mechanism allows formation of shell commands used by server-side programs.

With 'hidden' => true, wiki admin can define hidden data sources, the very nature of which is hidden from wiki user. Example of such a source:

$wgExternalDataSources['flags'] = [
	'url' => 'https://raw.githubusercontent.com/lipis/flag-icons/main/flags/4x3/$iso2$.svg',
	'params' => [ 'iso2' ],
	'param filters' => [ 'iso2' => '/^[a-z]{2}$/' ],
	'format' => 'text',
	'hidden' => true
];

Is such a source is defined, the following wikitext will show the SVG code for Israeli flag:

{{#get_external_data: flags | iso2 = il }}
{{#external_value:__text}}

Hidden data sources can only be called with {{#*_external_table:}}/{{#external_value:}} in standalone mode, or {{#get_external_data:source=(id)|...}}, or {{#get_external_data:(id)|...}}, not using other specific {{#get_*_data:}} functions and there identifiers like db, domain, etc.

The default value for $wgExternalDataSources is:

$wgExternalDataSources = [
	'*' => [
		'min cache seconds' => 3600,
		'always use stale cache' => false,
		'throttle key' => '$2nd_lvl_domain$',
		'throttle interval' => 0,
		'replacements' => [],
		'allowed urls' => [],
		'options' => [ 'timeout'=> 'default' ],
		'encodings' => [ 'ASCII', 'UTF-8', 'Windows-1251', 'Windows-1252', 'Windows-1254', 'KOI8-R', 'ISO-8859-1' ],
		'params' => [],
		'param filters' => []
	]
];

Other settings

Other settings are:

$wgExternalDataVerbose = true; — show error message, if an internal variable is not set. Note also, that {{#external_value:}} allows passing a default/fallback value as its second parameter,
$wgExternalDataAllowGetters = true; switches on the compatibility mode, under which the {{#get_…_data:}} data retrival functions are still available, as well as mw.ext.getExternalData.get…Data() functions other than mw.ext.getExternalData.getExternalData().
Without the compatibility mode, the only way of accessing external data is the standalone mode of {{#…_external_table:}} and mw.ext.getExternalData.getExternalData().
When wikipages are parsed with Parsoid, only the standalone mode or using Lua guarantee that the data is fetched prior to its display,
$wgExternalDataIntegratedConnectors = [...]; — a set of rules regulating the choice of class to handle connection to a data source depending on parameters of the {{#…_external_data:}} working in standalone mode. Injecting a new rule calling a new class extending EDConnectorBase allows to add new functionality to this extension,
$wgExternalDataConnectors = [...]; — an additiona set of rules regulating the choice of class to handle connection to a data source depending on the parser function invoked and its parameters in compatibility mode,
$wgExternalDataParsers = [...]; — a set of rules regulating the choice of text parser to convert text, returned by an external service, to variables. Injecting a new rule calling a new class extending EDParserBase allows to add new functionality to this extension.

For default values of these three variables refer to the file extension.json.