User:Santhosh.thottingal/WikiFamily

This is a guide to set up a wiki family with multiple mediawiki instances with different languages. We will use mwcli for this.

Download latest mwcli application from its releases page. Installation instructions can be found on Cli/guide/Installation#Install.

Download and Install mediawiki

mw docker mediawiki create

This will ask many questions related to the installation, including path to checkout the code. Then install the mysql database

mw docker mysql create

Now, we can create the wikis as per our requirement.

mw docker mediawiki install --dbtype=mysql --dbname=en

We gave en as dbname just to associate it with English language wiki. Using the dbname same as the language name also helps to map languages to wikis. We can create any number of wikis as per our requirement as follows

mw docker mediawiki install --dbtype=mysql --dbname=es
mw docker mediawiki install --dbtype=mysql --dbname=ml
mw docker mediawiki install --dbtype=mysql --dbname=ta 
mw docker mediawiki install --dbtype=mysql --dbname=fr

You can access each of this wikis using the following URL http://en.mediawiki.mwdd.localhost:8080, http://ig.mediawiki.mwdd.localhost:8080 etc. Note that all of them will be in English. We will soon map them to corresponding languages. mwcli will create a LocalSettings.php in the root folder of mediawiki source code it fetched in first step. It will have only one line

<?php

//require_once "$IP/includes/PlatformSettings.php";

require_once '/mwdd/MwddSettings.php';

Below this line you can add extension installations and their configurations as usual. To map each with with different languages, add the following line to LocalSettings.php

# Site language code. Same as $wgDBname or $dockerDb for this setup
$wgLanguageCode = $wgDBname;

Now all the wikis created above will have its own language just like production wikipedia has.

Additional Services

To add new services that are not part of mwcli, follow these steps.

Let us add cxserver nodejs service to mwcli.

As a first step, setup cxserver (MediaWiki-Docker/Extension/ContentTranslation).

Then, find out the location of mwcli docker configuration files:

mw docker where

On Linux systems, it will be ~/.config/mwcli/mwdd/default. Add the source code path to .env file in the same directory. For example:

CXSERVER_VOLUMES_CODE=/home/santhosh/work/wiki/cxserver

Then create a file named custom.yml and add the service docker-compose. Example:

version: '3.7'
services:
  cxserver:
    build:
        context: ${CXSERVER_VOLUMES_CODE}
    entrypoint: npm start
    environment:
      - VIRTUAL_HOST=cxserver.mwdd.localhost, cxserver.mwdd
      - VIRTUAL_PORT=8080
    volumes:
      - "${CXSERVER_VOLUMES_CODE:-cxserver-code}:/opt/cxserver"
      - /opt/cxserver/node_modules
    depends_on:
      - nginx-proxy
    dns:
      - 10.0.0.10
    networks:
      - dps

volumes:
  cxserver-code:

The service can be started using

mw docker custom create

And it can be accessed by using the URL http://cxserver.mwdd.localhost:8080

Installing Adminer for database administration

Adminer is very useful to interact with mysql database

mw docker adminer create

Adminer can be now accessed from http://adminer.mwdd.localhost:8080/

Note the the username is root, password is toor, server is mysql and database is en or other language codes we used when creating wikis

Installing eventlogging service for EventLogging

Create the eventlogging service

mw docker eventlogging create

The events registered by mediawiki and extension can be viewed by looking at the logs of eventlogging service.

mw docker docker-compose logs eventlogging

Installing ContentTranslation extension

To the extensions folder of mediawiki, clone the extension and dependent extensions

git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/Cite" extensions/Cite
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/ContentTranslation" extensions/ContentTranslation
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/UniversalLanguageSelector" extensions/UniversalLanguageSelector
git clone --recursive "https://gerrit.wikimedia.org/r/mediawiki/extensions/VisualEditor" extensions/VisualEditor
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/BetaFeatures" extensions/BetaFeatures
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/GlobalPreferences" extensions/GlobalPreferences
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/MobileFrontend" extensions/MobileFrontend
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/EventLogging" extensions/EventLogging
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/EventBus" extensions/EventBus
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/EventStreamConfig" extensions/EventStreamConfig
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/WikimediaEvents" extensions/WikimediaEvents

Then add the required settings to LocalSettings.php

wfLoadExtension( 'GlobalPreferences' );
wfLoadExtension( 'UniversalLanguageSelector' );
wfLoadExtension( 'BetaFeatures' );
wfLoadExtension( 'Cite' );
wfLoadExtension( 'MobileFrontend' );
wfLoadExtension( 'VisualEditor' );
$wgDefaultUserOptions['visualeditor-enable'] = 1;

wfLoadExtension( 'ContentTranslation' );
$GLOBALS['wgContentTranslationDatabase'] = 'shared';
$GLOBALS['wgContentTranslationAsBetaFeature'] = true;
$wgContentTranslationVersion = 2;
$wgContentTranslationEventLogging = true;
$wgContentTranslationEnableMT = true;
$wgContentTranslationEnableSectionTranslation = true;
$wgContentTranslationEnableAnonSectionTranslation = true;
$wgContentTranslationTranslateInTarget = true;
$wgContentTranslationDefaultParsoidClient = 'direct';
$wgContentTranslationSiteTemplates = [
	"view" => "//$1.mediawiki.mwdd.localhost:8080/wiki/$2",
	"action" => "//$1.mediawiki.mwdd.localhost:8080/w/index.php?title=$2",
	"api" => "//$1.wikipedia.org/w/api.php",
	"cx" => "http://cxserver.mwdd.localhost:8080/v1",
	"cookieDomain" => "mediawiki.mwdd.localhost",
	"restbase" => "https://$1.wikipedia.org/api/rest_v1"
];
$GLOBALS['wgContentTranslationCXServerAuth'] = [
	"algorithm" => "HS256",
	"key" => "secret",
	"age" => "3600"
];
$wgContentTranslationContentImportForSectionTranslation = true;
$wgSectionTranslationTargetLanguages = ['bn',"ml", "is", "ig"];

// Enable event logging
wfLoadExtensions( [
	'EventBus',
	'EventStreamConfig',
	'EventLogging',
	'WikimediaEvents'
] );
$wgEventServices = [ '*' => [ 'url' => 'http://eventlogging:8192/v1/events' ] ];
$wgEventServiceDefault = '*';
$wgEventLoggingStreamNames = false;
$wgEventLoggingServiceUri = "http://eventlogging.mwdd.localhost:" . parse_url($wgServer)['port'] . "/v1/events";
$wgEventLoggingQueueLingerSeconds = 1;
$wgEnableEventBus = defined( "MW_PHPUNIT_TEST" ) ? "TYPE_NONE" : "TYPE_ALL";

The database tables required for CX wont be created by the above steps. We have to create them manually. Adminer can be very helpful there. Go to adminer, create a new database names shared as we gave it in the above settings. Use the sql scripts from sql folder of ContentTranslation database to create tables.

In the above configuration, please note the wgContentTranslationSiteTemplates. It uses the multi-language wiki URL patterns by mwcli so that CX's inter domain navigation works well. We also set the cookie domain to support this. We also used the Cxserver URL that we created above.

Updating

To run update script in the mediawiki instances, follow these steps

mw docker docker-compose exec mediawiki php maintenance/run.php update --  --wiki en

Provide your own wiki names instead of bn above. In this tutorial, we were using wikinames same as language codes.

Running phpunit

For integration tests with phpunit, provide the wikiname as follows. For unit tests, this is not required. Not providing PHPUNIT_WIKI will result 'Database not found' errors.

PHPUNIT_WIKI=en composer phpunit:integration extensions/ContentTranslation/tests/phpunit/integration/RestbaseClientTest.php

Step Debugging

To get step debugging working using XDebug, you are welcome to view the documentation on Cli/guide/Docker-Development-Environment/MediaWiki#XDebug. In a nutshell, it is important to include

MEDIAWIKI_XDEBUG_CONFIG

which informs the docker containers of your local IP and what port XDebug will be running on. Your .env file under your mwcli docker configuration files which you can get a path to using:

mw docker where

should look something like this:

MEDIAWIKI_VOLUMES_CODE=/path/to/mediawiki/
MEDIAWIKI_VOLUMES_DOT_COMPOSER=/path/to/.composer
CXSERVER_VOLUMES_CODE=/path/to/cxserver
NETWORK_SUBNET_PREFIX=10.0.0
PORT=8080
MEDIAWIKI_XDEBUG_MODE="develop,debug,profile"
MEDIAWIKI_XDEBUG_CONFIG="client_host=192.168.100.3 client_port=9003"

After, it's important to restart your mediawiki services using

mw docker mediawiki create