Extension:GrowthExperiments/developer setup

Other languages:
English • ‎français • ‎polski • ‎العربية • ‎日本語

InstallationEdit

If you're not using vagrant, download MediaWiki and set it up with either MediaWiki-Docker or a local server using the manual installation instructions.

Download and install the necessary skins and extensions:

git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/GrowthExperiments" extensions/GrowthExperiments
git clone "https://gerrit.wikimedia.org/r/mediawiki/skins/Vector" skins/Vector
git clone "https://gerrit.wikimedia.org/r/mediawiki/skins/MinervaNeue" skins/MinervaNeue
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/PageViewInfo" extensions/PageViewInfo
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/MobileFrontend" extensions/MobileFrontend
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/Elastica" extensions/Elastica
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/CirrusSearch" extensions/CirrusSearch


  • If using Vagrant , install with vagrant roles enable growthexperiments addlink --provision
Manual installation
  • Download and place the file(s) in a directory called GrowthExperiments in your extensions/ folder.
  • Add the following code at the bottom of your LocalSettings.php:
    wfLoadExtension( 'GrowthExperiments' );
    wfLoadExtension( 'MobileFrontend' );
    wfLoadExtension( 'GrowthExperiments' );
    wfLoadExtension( 'PageViewInfo' );
    wfLoadExtension( 'Elastica' );
    wfLoadExtension( 'CirrusSearch' );
    
  • Run the update script which will automatically create the necessary database tables that this extension needs.
  •   Done – Navigate to Special:Version on your wiki to verify that the extension is successfully installed.

Suggested EditsEdit

There are three basic approaches for setting up suggested edits, the main functionality of GrowthExperiments, in a developer setup:

  1. Have the extension use the search API of a remote (production) wiki. You won't have to deal with setting up search locally, and you will get a wide range of realistic task suggestions, but the articles suggested for those tasks won't exist on your wiki so most editing-related functionality won't work. This is the easiest way to get the homepage and guidance working in general, and to QA task suggestions. It doesn't work for structured link recommendations though since those aren't enabled on any production wiki yet.
  2. Set up search locally, copy-paste or import a couple articles from some real wiki. This is quite a bit more effort but doable (on Vagrant it should mostly work out of the box). It's good for backend development, probably not really worth the effort for frontend development.
  3. Mock all the backend logic involved with static PHP code. This is nice for frontend work as you get direct and full control over the responses from the backend.

Use remote searchEdit

use GrowthExperiments\AqsEditInfoService;
use MediaWiki\MediaWikiServices;

# Use the test config for en.wikipedia.org
$wgGENewcomerTasksConfigTitle = 'mw:Growth/Personalized_first_day/Newcomer_tasks/Prototype/templates/en.json';

# Search for tasks on en.wikipedia.org
$wgGENewcomerTasksRemoteApiUrl = 'https://en.wikipedia.org/w/api.php';

# Get extra data for the task cards from en.wikipedia.org
$wgGERestbaseUrl = 'https://en.wikipedia.org/api/rest_v1';
$wgHooks['MediaWikiServices'][] = function ( MediaWikiServices $services ) {
	$services->redefineService( 'GrowthExperimentsEditInfoService', function ( MediaWikiServices $services ) {
		return new AqsEditInfoService( $services->getHttpRequestFactory(), 'en.wikipedia' );
	} );
};

Set up search locallyEdit

In Vagrant search can be set up with vagrant roles enable cirrussearch --provision (but it will be set up automatically as a dependency if you provision the growthexperiments role). For Docker, see MediaWiki-Docker/Configuration recipes/ElasticSearch. For other setups, a basic configuration is:

wfLoadExtension( 'CirrusSearch' );
require_once "$IP/extensions/CirrusSearch/tests/jenkins/FullyFeaturedConfig.php";
$wgCirrusSearchServers = [ 'elasticsearch' ];
$wgCirrusSearchWMFExtraFeatures = [
	'weighted_tags' => [ 'build' => true, 'use' => true ]
];

See the setup notes of Extension:CirrusSearch for more details.

You can use the CirrusSearch maintenance script UpdateWeightedTags.php for setting the ORES topics of wiki pages.

To use settings from cswiki:

  1. Follow the ElasticSearch instructions from MediaWiki-Docker/Configuration_recipes/ElasticSearch.
  2. Copy the contents of MediaWiki:NewcomerTopicsOres.json to http://localhost:8080/w/index.php?title=MediaWiki:NewcomerTopicsOres.json&action=edit. (must be signed in as an admin on MediaWiki)
  3. Copy the contents of MediaWiki:NewcomerTasks.json to http://localhost:8080/w/index.php?title=MediaWiki:NewcomerTasks.json&action=edit. (must be signed in as an admin on MediaWiki)
  4. Add $wgGENewcomerTasksTopicType = 'ores' to LocalSettings.php.
  5. Add $wgGERestbaseUrl = 'https://cs.wikipedia.org/api/rest_v1'; to LocalSettings.php.
  6. Add $wgGENewcomerTasksGuidanceEnabled = true; to LocalSettings.php.

Mock the backendEdit

use GrowthExperiments\NewcomerTasks\AddLink\SubpageLinkRecommendationProvider;
use GrowthExperiments\NewcomerTasks\ConfigurationLoader\StaticConfigurationLoader;
use GrowthExperiments\NewcomerTasks\TaskSuggester\StaticTaskSuggesterFactory;
use GrowthExperiments\NewcomerTasks\TaskSuggester\TaskSuggesterFactory;
use GrowthExperiments\NewcomerTasks\TaskType\LinkRecommendationTaskType;
use GrowthExperiments\NewcomerTasks\Task\Task;
use GrowthExperiments\NewcomerTasks\TaskType\TaskType;
use MediaWiki\MediaWikiServices;

# Enable under-development features still behind feature flag:
$wgGENewcomerTasksLinkRecommendationsEnabled = true;
$wgGELinkRecommendationsFrontendEnabled = true;

$wgHooks['MediaWikiServices'][] = function ( MediaWikiServices $services ) {
	$linkRecommendationTaskType = new LinkRecommendationTaskType( 'link-recommendation', TaskType::DIFFICULTY_EASY, [] );

	# Mock the configuration, which would normally be at MediaWiki:NewcomerTaskConfig.json, to have just one 'link-recommendation' task type.
	$services->redefineService( 'GrowthExperimentsNewcomerTasksConfigurationLoader', function ( MediaWikiServices $services ) use ( $linkRecommendationTaskType ) {
		return new StaticConfigurationLoader( [ $linkRecommendationTaskType ] );
	} );

	# Mock the task suggester to specify what article(s) will be suggested.
	$services->redefineService( 'GrowthExperimentsTaskSuggesterFactory', function ( MediaWikiServices $services ) use ( $linkRecommendationTaskType ): TaskSuggesterFactory {
		return new StaticTaskSuggesterFactory( [
			new Task( $linkRecommendationTaskType, new TitleValue( NS_MAIN, 'Douglas Adams' ) ),
		] );
	} );
};

# Set up SubpageLinkRecommendationProvider, which will take the recommendation from the article's /addlink.json subpage,
# e.g. [[Douglas Adams/addlink.json]]. The output of https://addlink-simple.toolforge.org can be copied there.
$wgHooks['MediaWikiServices'][] = SubpageLinkRecommendationProvider::class . '::onMediaWikiServices';
$wgHooks['ContentHandlerDefaultModelFor'][] = SubpageLinkRecommendationProvider::class . '::onContentHandlerDefaultModelFor';

Link recommendationsEdit

Your local development environment can query the "external traffic" production release of the link recommendation service. Access is proxied via the api-gateway (api.wikimedia.org) which requires an access token to utilize the API. That means you need to generate a personal access token (https://api.wikimedia.org/wiki/Documentation/Getting_started/Authentication#Personal_API_tokens), when you will use the access token as the value for GELinkRecommendationServiceAccessToken in LocalSettings.php:

$wgGELinkRecommendationServiceWikiIdMasquerade = 'cswiki';
$wgGELinkRecommendationServiceAccessToken = "#access token from api.wikimedia.org";
$wgGELinkRecommendationServiceUrl = 'https://api.wikimedia.org/service/linkrecommendation';

// Optionally set the fallback on DB miss variable. In practice, if you are
// using refreshLinkRecommendations.php and importOresTopics.php maintenance
// scripts locally, then you probably want this config flag set to false (default)
// $wgGELinkRecommendationFallbackOnDBMiss = true;

To allow non-existent pages (red links) as suggested edits, add the following to LocalSettings.php

$wgGEDeveloperSetup = true;

Seeding ArticlesEdit

See also https://phabricator.wikimedia.org/T274198#6972115

  1. Get article titles for Growth tasks from Czech Wikipedia.
    curl -s "https://cs.wikipedia.org/w/api.php?action=query&format=json&export=1&exportnowrap=1&exportschema=0.11&generator=growthtasks" > tasks.xml
    
  2. Import XML dump by running the following command inside mediawiki installation directory
    php maintenance/importDump.php --report=10 tasks.xml
    
  3. Create "Module:Wikidata" on your local site. Copy/paste the source of https://cs.wikipedia.org/wiki/Module:Wikidata and save it.
  4. Update secondary tables (to get accurate information on Special:RecentChanges and Special:Statistics)
    php maintenance/rebuildrecentchanges.php
    php maintenance/initSiteStats.php
    
  5. Update ElasticSearch
    php maintenance/update.php --quick
    php extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php && \
    php extensions/CirrusSearch/maintenance/ForceSearchIndex.php --skipLinks --indexOnSkip && \
    php extensions/CirrusSearch/maintenance/ForceSearchIndex.php --skipParse && \
    php maintenance/runJobs.php
    

Seeding ORES topicsEdit

To import ORES topics for articles imported from a production wiki (English Wikipedia in the example below), run

php /path/to/mediawiki/extensions/GrowthExperiments/maintenance/importOresTopics.php --apiUrl https://en.wikipedia.org/w/api.php --wikiId enwiki --count 100 --verbose

To set ORES topics for some article manually, use

php /path/to/mediawiki/extensions/CirrusSearch/maintenance/UpdateWeightedTags.php --tagType classification.ores.articletopic --tagName 'Regions.Americas.Central America' --weight 750 --tagName 'Media.Media*' --weight 800 --page={yourPageTitle}