Reading/Web/Setting up a staging environment

Setting up a staging environment for the MobileFrontend projects is trivial with MediaWiki-Vagrant in a Cloud VPS, though there are some minor manual steps that you'll need to perform.

Defining an instance

An instance should have the following properties:

medium ("large" instances are rarely needed and space is precious and limited in our project environment)
Image id: debian-9.4-stretch
Edit instance and ensure security groups: web and default
It should be configured (in puppet configuration) to ensure that the role::labs::vagrant is enabled.

Check the instance is working

Follow instructions on wikitech

Confirm your new instance is accessible from the web.

Reading-web-ify the instance

Once you've confirmed the instance is running you'll want it useful to your team! The following commands are recommended:

vagrant enable-role mobilefrontend minerva
# For testing the language switcher feature.
vagrant enable-role interwiki
# For testing the lazily-loaded images on pages with equations.
vagrant enable-role math
# Recommended installations
# For testing diffs
vagrant enable-role wikidiff2
vagrant enable-role cite echo svg eventlogging
# Recommended reading web projects
vagrant enable-role popups wikimediaevents relatedarticles
# The proverbial kitchen sink (optional extras)
vagrant enable-role geodata visualeditor
# Keep trying until everything downloads
until vagrant up --provision; do :; done

Set up regular codebase updates

The simplest solution to this is to regularly run vagrant git-update. So, as the MediaWiki-Vagrant user (sudo su mwvagrant) add the following to the crontab file (crontab -e):

*/10 * * * * cd /srv/mediawiki-vagrant; vagrant git-update

Set up the Selenium_newuser and Selenium_user accounts

After you've set up the instance, you'll notice that a Selenium_user account has already been created. You'll need to promote this user, and to create and promote Selenium_newuser as well:

vagrant ssh
mwscript maintenance/createAndPromote.php --server wiki --force --bureaucrat --sysop Selenium_user "<password>"
mwscript maintenance/createAndPromote.php --server wiki --force --bureaucrat --sysop Selenium_newuser "<password>"

For more information on how to use this password in the browser tests, see the docs.

Adding another admin account

You can add another admin account using the createAndPromote.php script as above:

vagrant ssh
mwscript maintenance/createAndPromote.php --server wiki --force --sysop <username> "<password>"

Set up a handful of interlanguage prefixes

Visit the Special:Interwiki page while logged in as a user with the "interwiki" right and add a handful of stub interlanguage prefixes. The selenium tests check the language picker by adding an interlanguage link with the prefix es , so add that. If you try to add an interlanguage link to a page (e.g [[es:Selenium language test page]] it may not show up in the mobile view immediately. It may be necessary to run vagrant ssh && mwscript maintenance/refreshLinks.php for it to be available with the langlinks api.

Configure the MobileFrontend extension

See https://phabricator.wikimedia.org/P936 for the production-like configuration for the MobileFrontend extension. Since you're using MediaWiki-Vagrant to set up the instance, you can isolate the MobileFrontend configuration from other extension configurations by creating a file in the /vagrant/settings.d directory, i.e. /vagrant/settings.d/10-MobileFrontend.php.

Configure proxies

Since the mobilefrontend role depends on the centralauth role, your staging server will have the CentralAuth extension installed and configured. This means that you'll have to create proxies for both the default wiki and the login wiki, which is managed by the centralauth role, e.g. reading-web-staging.wmflabs.org, and login-reading-web-staging.wmflabs.org.

Import articles

You should import articles from a Special:Export dump with the importDump.php maintenance script:

mwvagrant ssh
mwscript maintenance/importDump.php --conf LocalSettings.php /path/to/dump.xml

(Alternative) Configure MobileFrontend to fetch content from another wiki's API

Per Extension:MobileFrontend#Testing with articles on a foreign wiki (live data), add the following to the config file you created in the Configure the MobileFrontend extension section above:

$wgMFContentProviderClass = 'MobileFrontend\ContentProviders\MwApiContentProvider';
$wgMFMwApiContentProviderBaseUri = 'https://<host>/w/api.php';

For testing Extension:Graph you will need to manually set a value for:

mw.config.get( 'wgGraphSpecs' )

Alternative lightweight throwaway setup

Initial configuration

git clone --recursive https://gerrit.wikimedia.org/r/mediawiki/vagrant
cd vagrant &&
./setup.sh &&
vagrant config nfs_shares no &&
vagrant enable-role mobilefrontend minerva popups &&
vagrant up  --provision

Edit Vagrantfile-extra.rb

Vagrant.configure('2') do |config|
  config.vm.provider :virtualbox do |vb|
    vb.customize ['modifyvm', :id, '--memory', '4096']
  end
end

(equivalent to running vagrant config vagrant_ram 4096 )

Edit mediawiki/LocalSettings.php

$wgDebugToolbar = true;
include_once "$IP/extensions/MobileFrontend/tests/browser/LocalSettings.php";

// Fetch remote wiki content.
$wgMFContentProviderClass = 'MobileFrontend\ContentProviders\MwApiContentProvider';
$wgMFMwApiContentProviderBaseUri = 'https://en.wikipedia.org/w/api.php';

// Use the mobile site formatter (which supports remote wiki content) even for desktop.
$wgMFAlwaysUseContentProvider = true;

// Configure logo
$wgMinervaCustomLogos = [
  'copyright-width' => 100,
  'copyright-height' => 18,
  'copyright' => 'https://en.m.wikipedia.org/static/images/mobile/copyright/wikipedia-wordmark-en.svg',
];

// For diffs

// For Popups (disable if using Special:Block)
$wgArticlePath = "/wiki/$1";
$wgPopupsOptInStateForNewAccounts = '1';
$wgPopupsVirtualPageViews = true;
$wgPopupsGateway = "restbaseHTML";
$wgPopupsRestGatewayEndpoint = 'https://en.m.wikipedia.org/api/rest_v1/page/summary/';

// For A-B tests
$wgMinervaABSamplingRate = 1;

Halt the VM

vagrant halt
# Optionally save the VM state here.

Update

git pull &&
git submodule update --recursive --init &&
vagrant up  --provision &&
vagrant git-update
# Optionally install dependencies from any secondary package managers like NPM
# in select repos such as MobileFrontend.

Stephen's set up

Warning: Stephen manages the stephen server! It's ok to change it in a pinch but let them know so they can wipe it and start over. This is supposed to be a well defined configuration!

Most of the set up is in MediaWiki-Vagrant in a Cloud VPS. Follow the screenshots repeated below (now on debian-10.0-buster).

Launch instance part 1: fill out the name.

Launch instance part 2: select an image.

Launch instance part 3: choose a resource size.

Launch instance part 4: edit the instance just created.

Verify the mediawiki_vagrant role is enabled.

Set up a proxy to Vagrant.

After configuring everything in Horizon:

ssh stephen
sudo puppet agent --test --verbose
exit

ssh stephen
sudo su -s /bin/bash -l mwvagrant
cd /srv/mediawiki-vagrant &&
time vagrant up &&
vagrant enable-role cite echo interwiki geodata math mobileapp mobilefrontend minerva popups svg visualeditor wikimediaevents &&
time vagrant provision
cat >> settings.d/99-stephen.php << 'eof'
<?php
$wgMFNearbyEndpoint = 'https://en.wikipedia.org/w/api.php';
$wgMFMwApiContentProviderBaseUri = 'https://en.wikipedia.org/w/api.php';
$wgMFContentProviderClass = 'MobileFrontend\ContentProviders\MwApiContentProvider';
$wgMFAlwaysUseContentProvider = true;
$wgCrossSiteAJAXdomains = [ '*' ];
$wgPopupsGateway = 'restbaseHTML';
$wgPopupsRestGatewayEndpoint = 'https://en.wikipedia.org/api/rest_v1/page/summary/';
$wgMinervaDownloadIcon = true;
$wgMinervaApplyKnownTemplateHacks = true;
$wgMFLazyLoadImages = [ 'base' => true, 'beta' => true ];
$wgMinervaABSamplingRate = 1;
$wgShowExceptionDetails = true;
$wgShowDBErrorBacktrace = true;
$wgShowSQLErrors = true;
$wgMFEnableMobilePreferences = true;
$wgMinervaErrorLogSamplingRate = 1;
$wgWMEReadingDepthEnabled = true;
$wgWMEReadingDepthSamplingRate = 1;
eof
# Add if wanted (unstable):
crontab -e
# */15 * * * * bash -lc 'cd /srv/mediawiki-vagrant && time { echo; date; echo; git pull; vagrant provision; vagrant git-update; } >> cron.log 2>&1'
# You should be able to login (from HTTPS only) without changing anything.

Test the instance https://stephen.wmflabs.org.

Troubleshooting

https://wikitech.wikimedia.org/wiki/Help:MediaWiki-Vagrant_in_Cloud_VPS#Setting_up_your_instance_with_MediaWiki-Vagrant is useful.
Ports sometimes hang open after vagrant up but restarting fixes them.
No wiki is sometimes shown after provisioning but vagrant destroy and restarting fixes it.
"Vagrant cannot forward the specified ports on this VM, since they

would collide with some other application that is already listening on these ports." - run sudo netstat -nlp | grep :8080 and then sudo kill the pid

the wiki not found (showing in web interface): vagrant ssh, then run sudo service apache2 restart
Visiting https://horizon.wikimedia.org/project/instances/ and soft rebooting may help you with any port issues. Try multiple times if necessary!

Setting up wiki farms

For testing certain extensions such as GlobalPreferences and Wikidata, follow the instructions in wikitech:Help:MediaWiki-Vagrant_in_Cloud_VPS#Run_a_wikifarm