MediaWiki

User:DWalden (WMF)/Wiki family

< User:DWalden (WMF)

Setup edit

The below steps should create:

Some problems:

  • Logins often won't work. I don't know why. Just keep retrying until they do. This problem seems to have sorted itself out.
  • I don't know if the job runner will work properly. This works as long as you run the appropriate commands in #Recommended way of starting the wiki family at the bottom of the page.
  • I think it might be accessing the database as a root user.

Clone all the necessary repos: 

git clone "https://gerrit.wikimedia.org/r/mediawiki/core" family
cd family
git clone "https://gerrit.wikimedia.org/r/mediawiki/skins/Vector" skins/Vector
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/CentralAuth" extensions/CentralAuth
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/AntiSpoof" extensions/AntiSpoof

Create a docker-compose.override.yml with this in it: 

version: '3.7'
services:
  mariadb-main:
    image: 'bitnami/mariadb:latest'
    volumes:
      - mariadbdata:/bitnami/mariadb
    environment:
      - MARIADB_REPLICATION_MODE=master
      - MARIADB_REPLICATION_USER=repl_user
      - MARIADB_REPLICATION_PASSWORD=repl_password
      - MARIADB_ROOT_PASSWORD=main_root_password
      - MARIADB_USER=my_user
      - MARIADB_PASSWORD=my_password
      - MARIADB_DATABASE=enwiki
  mariadb-replica:
    image: 'bitnami/mariadb:latest'
    depends_on:
      - mariadb-main
    volumes:
      - mariadbdatarep:/bitnami/mariadb
    environment:
      - MARIADB_REPLICATION_MODE=slave
      - MARIADB_REPLICATION_USER=repl_user
      - MARIADB_REPLICATION_PASSWORD=repl_password
      - MARIADB_MASTER_HOST=mariadb-main
      - MARIADB_MASTER_PORT_NUMBER=3306
      - MARIADB_MASTER_ROOT_PASSWORD=main_root_password
volumes:
  mariadbdata:
    driver: local
  mariadbdatarep:
    driver: local

Create .env file: 

MW_SCRIPT_PATH=/w
MW_SERVER=http://localhost:8080
MW_DOCKER_PORT=8080
MEDIAWIKI_USER=Admin
MEDIAWIKI_PASSWORD=dockerpass
XDEBUG_CONFIG=
XDEBUG_ENABLE=true
XHPROF_ENABLE=true
MW_DOCKER_UID=1000
MW_DOCKER_GID=1000

In the mediawiki core repo, run these commands to start the docker container and install PHP libraries: 

docker compose up -d
mv composer.local.json-sample composer.local.json
docker compose exec mediawiki composer update

Run this command to access the database: 

docker compose exec mariadb-main mariadb -u root -pmain_root_password

Run these SQL commands to create the databases: 

CREATE DATABASE loginwiki;
GRANT ALL ON loginwiki.* TO 'my_user'@'%' IDENTIFIED BY 'my_password';
CREATE DATABASE centralauth;
GRANT ALL ON centralauth.* TO 'my_user'@'%' IDENTIFIED BY 'my_password';

Exit the database and run these commands: 

source .env
docker compose exec mediawiki php maintenance/install.php --dbname=enwiki --dbuser=my_user --dbpass=my_password --dbserver=mariadb-main --server="${MW_SERVER}" --scriptpath="${MW_SCRIPT_PATH}" --lang en --pass ${MEDIAWIKI_PASSWORD} mediawiki ${MEDIAWIKI_USER}
mv LocalSettings.php LocalSettings_enwiki.php
docker compose exec mediawiki php maintenance/install.php --dbname=loginwiki --dbuser=my_user --dbpass=my_password --dbserver=mariadb-main --server="${MW_SERVER}" --scriptpath="${MW_SCRIPT_PATH}" --lang en --pass ${MEDIAWIKI_PASSWORD} mediawiki ${MEDIAWIKI_USER}

Run this command: 

docker compose exec mariadb-main mariadb -u root -pmain_root_password -e "CREATE TABLE centralauth.objectcache LIKE enwiki.objectcache;"

Add this to the end of LocalSettings.php: 

wfLoadSkin( 'Vector' );

wfLoadExtension( 'CentralAuth' );
wfLoadExtension( 'AntiSpoof' );

$wgSharedDB = 'centralauth';
$wgSharedTables = array( 'objectcache' );
$wgCentralAuthSessionCacheType = CACHE_DB;

if ( defined( 'MW_DB' ) ) {
    $wgDBname = MW_DB;
} elseif ( isset($_SERVER['SERVER_NAME']) && count(explode(".", $_SERVER['SERVER_NAME'])) > 1 ) {
    $wgDBname = explode(".", $_SERVER['SERVER_NAME'])[0];
} else {
    $wgDBname = 'loginwiki';
}

$wgConf->settings = [
    'wgServer' => [
        # If you want to allow also usage of https, just use '//localhost'
        #  and set 'http://localhost' at 'wgCanonicalServer'
        'default' => 'http://localhost:8080',
    ],

    'wgCanonicalServer' => [
        'default' => 'http://localhost:8080',
    ],

    'wgScriptPath' => [
        'default' => '/w',
    ],

    'wgArticlePath' => [
        'default' => '/wiki/index.php/$1',
    ],

    'wgSitename' => [
        'default' => 'Wikipedia',
    ],

    'wgLanguageCode' => [
        'default' => 'en',
    ],
];

$wgLocalDatabases = [
    'enwiki',
    'loginwiki',
];

foreach ( $wgLocalDatabases as $db ) {
    $wgConf->settings['wgServer'][$db] = 'http://' . $db . '.localhost:8080';
    $wgConf->settings['wgCanonicalServer'][$db] = 'http://' . $db . '.localhost:8080';
    $wgConf->settings['wgSitename'][$db] = $db;
}

$wgConf->wikis = $wgLocalDatabases;
$wgConf->suffixes = [ 'wiki' ];
$wgConf->localVHosts = [ 'localhost' ];

function efGetSiteParams( $conf, $wiki ) {
    $site = null;
    $lang = null;
    foreach( $conf->suffixes as $suffix ) {
        if ( substr( $wiki, -strlen( $suffix ) ) === $suffix ) {
            $site = $suffix;
            $lang = substr( $wiki, 0, -strlen( $suffix ) );
            break;
        }
    }
    return [
        'suffix' => $site,
        'lang' => $lang,
        'params' => [
            'lang' => $lang,
            'site' => $site,
            'wiki' => $wiki,
        ],
        'tags' => [],
    ];
}

$wgConf->suffixes = $wgLocalDatabases;
$wgConf->siteParamsCallback = 'efGetSiteParams';
$wgConf->extractAllGlobals( $wgDBname );
$dockerMainDb = [
	'host' => "mariadb-main",
	'dbname' => $wgDBname,
	'user' => 'root',
	'password' => 'main_root_password',
	'type' => "mysql",
	'flags' => DBO_DEFAULT,
	'load' => 0,
];
$dockerReplicaDb = [
	'host' => "mariadb-replica",
	'dbname' => $wgDBname,
	'user' => 'root',
	'password' => 'main_root_password',
	'type' => "mysql",
	'flags' => DBO_DEFAULT,
	'max lag' => 60,
	'load' => 1,
];
// Integration tests fail when run with replication, due to not having the temporary tables.
if ( !defined( 'MW_PHPUNIT_TEST' ) ) {
	$wgDBservers = [ $dockerMainDb, $dockerReplicaDb ];
} else {
	$wgDBserver = $dockerMainDb['host'];
	$wgDBuser = $dockerMainDb['user'];
	$wgDBpassword = $dockerMainDb['password'];
	$wgDBtype = $dockerMainDb['type'];
}

$wgCentralAuthDatabase = 'centralauth';
$wgVirtualDomainsMapping['virtual-centralauth'] = [ 'db' => 'centralauth' ];
$wgCentralAuthCookies = true;
$wgCentralAuthAutoMigrate = true;
#$wgCentralAuthCookieDomain = '.example.org';
$wgCentralAuthLoginWiki = 'loginwiki';

Run these commands: 

docker compose exec mediawiki php maintenance/sql.php --wikidb centralauth extensions/CentralAuth/schema/mysql/tables-generated.sql
docker compose exec mediawiki php maintenance/sql.php --wikidb enwiki extensions/AntiSpoof/sql/mysql/tables-generated.sql
docker compose exec mediawiki php maintenance/sql.php --wikidb loginwiki extensions/AntiSpoof/sql/mysql/tables-generated.sql
docker compose exec mediawiki php maintenance/sql.php --wikidb centralauth extensions/AntiSpoof/sql/mysql/tables-generated.sql
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass0.php --wiki loginwiki
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass0.php --wiki enwiki
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass1.php --wiki loginwiki
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass1.php --wiki enwiki

It should now all be setup. Go to http://enwiki.localhost:8080/wiki/Main_Page.

Adding a new wiki edit

Create a new database for the wiki. Run this command to access the database: 

docker compose exec mariadb-main mariadb -u root -p

Run these SQL commands to create the databases: 

CREATE DATABASE <wiki database name>;
GRANT ALL ON <wiki database name>.* TO 'my_user'@'%' IDENTIFIED BY 'my_password';

mv LocalSettings.php LocalSettings_old.php
source .env
docker compose exec mediawiki php maintenance/install.php --dbname=<wiki database name> --dbuser=my_user --dbpass=my_password --dbserver=mariadb-main --server="${MW_SERVER}" --scriptpath="${MW_SCRIPT_PATH}" --lang en --pass ${MEDIAWIKI_PASSWORD} mediawiki ${MEDIAWIKI_USER}
mv LocalSettings_old.php LocalSettings.php

Add the new wiki's database name to $wgLocalDatabases in LocalSettings.php. 

docker compose exec mediawiki php maintenance/sql.php --wikidb <wiki database name> extensions/AntiSpoof/sql/mysql/tables-generated.sql
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass0.php --wiki <wiki database name>
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass1.php --wiki <wiki database name>
docker compose exec mediawiki php maintenance/run.php --wiki <wiki database name> update.php --quick

(Optional) Add the wiki database name to the script in #Recommended_way_of_starting_the_wiki_family.

Access the new wiki via http://<wiki database name>.localhost:8080.

Having a different configuration per wiki edit

If you want to add specific configuration options that are only for certain wikis, add something like this to you LocalSettings.php: 

if ( $wgDBname == 'loginwiki' ) {
// Config only for loginwiki
} elseif ( $wgDBname == 'enwiki' ) {
// Config only for enwiki
} elseif ( $wgDBname == '<wiki database name>' ) {
// Config only for other wiki
}

You could also potentially do some of this with Manual:$wgConf.

Recommended way of starting the wiki family edit

Recommended way of updating all the code, installing php dependencies, updating databases and starting this family of wikis. You can save this to a file locally and run like sh <filename> or you can copy and paste this into your terminal. Should be ran from the mediawiki core repository: 

#!/bin/bash

for dir in ./ skins/*/ extensions/*/
do
    cd $dir
    git checkout master
    git pull
    cd -
done

docker compose up -d
docker compose exec mediawiki composer update

# When you create a new wiki, add its database name to the end of this list
for wiki in loginwiki enwiki
do
    docker compose exec mediawiki php maintenance/run.php --wiki $wiki update.php --quick
    docker compose exec -d mediawiki-jobrunner php maintenance/run.php --wiki $wiki runJobs.php --wait
done
Retrieved from "https://www.mediawiki.org/w/index.php?title=User:DWalden_(WMF)/Wiki_family&oldid=6526784"