User:CFeng (WMF)/MachineVisionSetup

Introduction

This page introduces you to install mediawiki-docker and set up the MachineVision extension.

Install Docker Compose

https://docs.docker.com/compose/install/

Clone mediawiki-docker-dev

Clone the code to the path you would like to work on, I would recommend to clone it into ~/Documents:

git clone git@github.com:addshore/mediawiki-docker-dev.git

And now, it is in the ~/Documents/mediawiki-docker-dev/

Please note this might be the offical one: https://www.mediawiki.org/wiki/MediaWiki-Docker

Clone MediaWiki and the skin

Since the repository above only contains settings and scripts about setting up MediaWiki on Docker, we will need to install the MediaWiki by ourselves.

Clone it into ~/Documents:

git clone git@github.com:wikimedia/mediawiki.git 

And now, it is in the ~/Documents/mediawiki/

Also, you will need a default skin for the entire MediaWiki page to the skins folder:

 cd ~/Documents/mediawiki/skins/</code>
 git clone git@github.com:wikimedia/mediawiki-skins-Vector.git Vector

Install Composer

Before running the MediaWiki installation scripts, you will need the Composer. Please download and install it on your machine.

https://getcomposer.org/download/

Run composer install for MediaWiki

Run the followingg script to install MediaWiki

 cd ~/Documents/mediawiki
 composer install

Create a LocalSettings.php for MediaWiki

The LocalSettings.php contains the information that MediaWiki needs and you will need to update this file every time you add extensions.

Since the MachineVision needs a lot of extensions and other settings, here is the copy of my LocalSettings.php and please copy that into the ~/Documents/mediawiki/

User:CFeng_(WMF)/MachineVisionSetup/LocalSettings.php

Install extensions for mediawiki

The LocalSettings.php has the following code:

 wfLoadExtension( 'CirrusSearch' );
 wfLoadExtension( 'Elastica' );
 wfLoadExtension( 'MachineVision' );
 wfLoadExtension( 'UniversalLanguageSelector' );
 wfLoadExtension( 'UploadWizard' );
 wfLoadExtension( 'WikibaseCirrusSearch' );
 wfLoadExtension( 'WikibaseMediaInfo' );
 wfLoadExtension( 'WikimediaEditorTasks' );

which you will need to install the extensions manually.

Please go to the https://www.mediawiki.org/wiki/Special:ExtensionDistributor to find the extensions and copy them to the ~/Documents/mediawiki/extensions/

or, you can run the following scripts:

 cd ~/Documents/mediawiki/extensions
 git clone git@github.com:wikimedia/mediawiki-extensions-CirrusSearch.git CirrusSearch
 git clone git@github.com:wikimedia/mediawiki-extensions-Elastica.git Elastica
 git clone git@github.com:wikimedia/mediawiki-extensions-MachineVision.git MachineVision
 git clone git@github.com:wikimedia/mediawiki-extensions-UniversalLanguageSelector.git UniversalLanguageSelector
 git clone git@github.com:wikimedia/mediawiki-extensions-Wikibase.git Wikibase
 git clone git@github.com:wikimedia/mediawiki-extensions-WikibaseCirrusSearch.git WikibaseCirrusSearch
 git clone git@github.com:wikimedia/mediawiki-extensions-WikibaseMediaInfo.git WikibaseMediaInfo
 git clone git@github.com:wikimedia/mediawiki-extensions-WikimediaEditorTasks.git WikimediaEditorTasks

Setting up Wikibase

Include the following into composer.local.json at the ~/Documents/mediawiki/:

{
  "extra": {
    "merge-plugin": {
      "include": [
        "extensions/Wikibase/composer.json"
      ]
    }
  }
}

and run the script to apply the changes at the ~/Documents/mediawiki/ installation

composer install

MediaWiki-Docker environment configuration

Copy the default.env in ~/Documents/mediawiki-docker-dev and into a new file called local.env.

Please make sure to update the mediawiki path of your installation.

DOCKER_MW_PATH=/Users/[USERNAME]/Documents/mediawiki

Some other settings in the local.env you might need to update, for example, the PHP version you've installed, or the port has already taken by other services.

Create and start the Docker container

cd ~/Documents/mediawiki-docker-dev
./create

The script waits up to 120 secs (4 x 30 seconds) for the database containers to initialize and respond.

Host name resolution configuration

Adding the following configuration to your /etc/hosts

127.0.0.1 default.web.mw.localhost # mediawiki-docker-dev
127.0.0.1 proxy.mw.localhost # mediawiki-docker-dev
127.0.0.1 phpmyadmin.mw.localhost # mediawiki-docker-dev
127.0.0.1 graphite.mw.localhost # mediawiki-docker-dev

Please note that you might need sudo when editing the hosts.

For more details of mediawiki-docker configuration, please visit: https://github.com/addshore/mediawiki-docker-dev/blob/master/README.md

MachineVision extension configuration

Run composer install in the MachineVision extension folder:

cd ~/Documents/mediawiki/extensions/MachineVision
composer install

and then go to the mediawiki-docker to ssh into the webserver container

cd ~/Documents/mediawiki-docker-dev
./bash
php maintenance/update.php

Import the MachineVision mapping data

Download the mapping data from Google: https://developers.google.com/freebase/#freebase-wikidata-mappings

Extract the downloaded file from its archive and place it somewhere that MediaWiki can access it. To import, run the following maintenance script:

php maintenance/populateFreebaseMapping.php --mappingFile [FILE_PATH].nt

This process may take some time to complete.

Apply and enable the Google Cloud Vision API

Go to https://console.cloud.google.com and create a project, for example, MachineVision and then click on APIs & Services on the left menu to enable the API we need.

In the API list, search Cloud vision and enable it.

To use the API, we will NEED to add credentials to the project. Once you have created a credential, you can choose the format of the credential to be JSON, and download it to your machine.

Not sure if this is associate with the service, but you might need to activate the free trial status with $300.00 credit on your account in order to activate the service successfully.

Since we have already set the credential path in LocalSettings.php, we will need to copy the credential to ~/Documents/mediawiki and rename the file as machine-vision-credentials.json to match the following setting:

 $wgMachineVisionGoogleCredentialsFileLocation = '/var/www/mediawiki/machine-vision-credentials.json';

For more details please vision here: https://github.com/wikimedia/mediawiki-extensions-MachineVision/blob/master/README.md

Upload images and test the API

Go to http://default.web.mw.localhost:8080/, create an account and log in. Once you have logged in, you can upload images to the MediaWiki.

After the images being uploaded, you will need to run a script to communicate with Cloud Vision API.

cd ~/Documents/mediawiki-docker-dev
./bash
php maintenance/runJobs.php

and, check the local ImageLabel API to see if you have done all the configurations right.

http://default.web.mw.localhost:8080/mediawiki/api.php?format=json&formatversion=2&errorformat=plaintext&action=query&generator=unreviewedimagelabels&guillimit=10&prop=imagelabels%7Cimageinfo&iiprop=timestamp%7Cuser%7Curl%7Cmime&iiurlwidth=320&uselang=en

If you see an empty response, please make sure you have set up the Cloud Vision API credential correctly.