Extension:GrowthExperiments/developer setup

Other languages:

InstallationEdit

  • If using Vagrant , install with vagrant roles enable growthexperiments addlink --provision
Manual installation

git clone "https://gerrit.wikimedia.org/r/mediawiki/skins/Vector" skins/Vector git clone "https://gerrit.wikimedia.org/r/mediawiki/skins/MinervaNeue" skins/MinervaNeue git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/PageViewInfo" extensions/PageViewInfo git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/MobileFrontend" extensions/MobileFrontend git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/Elastica" extensions/Elastica git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/CirrusSearch" extensions/CirrusSearch

  • Download and place the file(s) in a directory called GrowthExperiments in your extensions/ folder.
  • Add the following code at the bottom of your LocalSettings.php:
    wfLoadExtension( 'GrowthExperiments' );
    wfLoadSkin( 'Vector' );
    wfLoadSkin( 'MinervaNeue' );
    wfLoadExtension( 'PageViewInfo' );
    wfLoadExtension( 'MobileFrontend' );
    wfLoadExtension( 'Elastica' );
    wfLoadExtension( 'CirrusSearch' );
    
  • Run the update script which will automatically create the necessary database tables that this extension needs.
  • Install and set up VisualEditor .
  • Install and set up CirrusSearch .
  •   Done – Navigate to Special:Version on your wiki to verify that the extension is successfully installed.

Other extensions (optional)

Enabling Growth featuresEdit

  1. Go to Special:Preferences (while logged in)
  2. Under User profile section: check Display newcomer homepage and Default to newcomer homepage from username link
  3. Under Editing section: check Enable the editor help panel

Alternatively, run the following in the browser console:

new mw.Api().saveOptions({
	'growthexperiments-help-panel-tog-help-panel': '1',
	'growthexperiments-homepage-enable': '1',
	'growthexperiments-homepage-pt-link': '1',
});

location.reload()

Suggested EditsEdit

There are three basic approaches for setting up suggested edits, the main functionality of GrowthExperiments, in a developer setup:

  1. Have the extension use the search API of a remote (production) wiki. You won't have to deal with setting up search locally, and you will get a wide range of realistic task suggestions, but the articles suggested for those tasks won't exist on your wiki so most editing-related functionality won't work. This is the easiest way to get the homepage and guidance working in general, and to QA task suggestions. It doesn't work for structured link recommendations though since those aren't enabled on any production wiki yet.
  2. Set up search locally, copy-paste or import a couple articles from some real wiki. This is quite a bit more effort but doable (on Vagrant it should mostly work out of the box). It's good for backend development, probably not really worth the effort for frontend development.
  3. Mock all the backend logic involved with static PHP code. This is nice for frontend work as you get direct and full control over the responses from the backend.

Use remote searchEdit

use GrowthExperiments\AqsEditInfoService;
use MediaWiki\MediaWikiServices;

# Use the test config for en.wikipedia.org
$wgGENewcomerTasksConfigTitle = 'mw:Growth/Personalized_first_day/Newcomer_tasks/Prototype/templates/en.json';

# Search for tasks on en.wikipedia.org
$wgGENewcomerTasksRemoteApiUrl = 'https://en.wikipedia.org/w/api.php';

# Get extra data for the task cards from en.wikipedia.org
$wgGERestbaseUrl = 'https://en.wikipedia.org/api/rest_v1';
$wgHooks['MediaWikiServices'][] = function ( MediaWikiServices $services ) {
	$services->redefineService( 'GrowthExperimentsEditInfoService', function ( MediaWikiServices $services ) {
		return new AqsEditInfoService( $services->getHttpRequestFactory(), 'en.wikipedia' );
	} );
};

Set up search locallyEdit

In Vagrant search can be set up with vagrant roles enable cirrussearch --provision (but it will be set up automatically as a dependency if you provision the growthexperiments role). For Docker, see MediaWiki-Docker/Configuration recipes/ElasticSearch. For other setups, a basic configuration is:

wfLoadExtension( 'CirrusSearch' );
require_once "$IP/extensions/CirrusSearch/tests/jenkins/FullyFeaturedConfig.php";
$wgCirrusSearchServers = [ 'elasticsearch' ];
$wgCirrusSearchWMFExtraFeatures = [
	'weighted_tags' => [ 'build' => true, 'use' => true ]
];

See the setup notes of Extension:CirrusSearch for more details.

You can use the CirrusSearch maintenance script UpdateWeightedTags.php for setting the ORES topics of wiki pages.

To use settings from cswiki:

  1. Follow the ElasticSearch instructions from MediaWiki-Docker/Configuration_recipes/ElasticSearch.
  2. Copy the contents of MediaWiki:NewcomerTopicsOres.json to http://localhost:8080/w/index.php?title=MediaWiki:NewcomerTopicsOres.json&action=edit. (must be signed in as an admin on MediaWiki)
  3. Copy the contents of MediaWiki:NewcomerTasks.json to http://localhost:8080/w/index.php?title=MediaWiki:NewcomerTasks.json&action=edit. (must be signed in as an admin on MediaWiki)
  4. Add $wgGENewcomerTasksTopicType = 'ores' to LocalSettings.php.
  5. Add $wgGERestbaseUrl = 'https://cs.wikipedia.org/api/rest_v1'; to LocalSettings.php.
  6. Add $wgGENewcomerTasksGuidanceEnabled = true; to LocalSettings.php.


To use settings from other wikis, replace "cs" with the language code of the desired wiki (for MediaWiki:NewcomerTasks.json and $wgGERestbaseUrl).

Mock the backendEdit

use GrowthExperiments\NewcomerTasks\AddLink\SubpageLinkRecommendationProvider;
use GrowthExperiments\NewcomerTasks\ConfigurationLoader\StaticConfigurationLoader;
use GrowthExperiments\NewcomerTasks\TaskSuggester\StaticTaskSuggesterFactory;
use GrowthExperiments\NewcomerTasks\TaskSuggester\TaskSuggesterFactory;
use GrowthExperiments\NewcomerTasks\TaskType\LinkRecommendationTaskType;
use GrowthExperiments\NewcomerTasks\Task\Task;
use GrowthExperiments\NewcomerTasks\TaskType\TaskType;
use MediaWiki\MediaWikiServices;

# Enable under-development features still behind feature flag:
$wgGENewcomerTasksLinkRecommendationsEnabled = true;
$wgGELinkRecommendationsFrontendEnabled = true;

$wgHooks['MediaWikiServices'][] = function ( MediaWikiServices $services ) {
	$linkRecommendationTaskType = new LinkRecommendationTaskType( 'link-recommendation', TaskType::DIFFICULTY_EASY, [] );

	# Mock the configuration, which would normally be at MediaWiki:NewcomerTaskConfig.json, to have just one 'link-recommendation' task type.
	$services->redefineService( 'GrowthExperimentsNewcomerTasksConfigurationLoader', function ( MediaWikiServices $services ) use ( $linkRecommendationTaskType ) {
		return new StaticConfigurationLoader( [ $linkRecommendationTaskType ] );
	} );

	# Mock the task suggester to specify what article(s) will be suggested.
	$services->redefineService( 'GrowthExperimentsTaskSuggesterFactory', function ( MediaWikiServices $services ) use ( $linkRecommendationTaskType ): TaskSuggesterFactory {
		return new StaticTaskSuggesterFactory( [
			new Task( $linkRecommendationTaskType, new TitleValue( NS_MAIN, 'Douglas Adams' ) ),
		] );
	} );
};

# Set up SubpageLinkRecommendationProvider, which will take the recommendation from the article's /addlink.json subpage,
# e.g. [[Douglas Adams/addlink.json]]. The output of https://addlink-simple.toolforge.org can be copied there.
$wgHooks['MediaWikiServices'][] = SubpageLinkRecommendationProvider::class . '::onMediaWikiServices';
$wgHooks['ContentHandlerDefaultModelFor'][] = SubpageLinkRecommendationProvider::class . '::onContentHandlerDefaultModelFor';
# Same for image recommendations, with addimage.json and http://image-suggestion-api.wmcloud.org/?doc
$wgHooks['MediaWikiServices'][] = SubpageImageRecommendationProvider::class . '::onMediaWikiServices';
$wgHooks['ContentHandlerDefaultModelFor'][] = SubpageImageRecommendationProvider::class . '::onContentHandlerDefaultModelFor';

Link recommendationsEdit

Your local development environment can query the "external traffic" production release of the link recommendation service. Access is proxied via the api-gateway (api.wikimedia.org) which requires an access token to utilize the API. That means you need to generate a personal access token (https://api.wikimedia.org/wiki/Documentation/Getting_started/Authentication#Personal_API_tokens), when you will use the access token as the value for GELinkRecommendationServiceAccessToken in LocalSettings.php:

$wgGELinkRecommendationServiceWikiIdMasquerade = 'cswiki';
$wgGELinkRecommendationServiceAccessToken = "#access token from api.wikimedia.org";
$wgGELinkRecommendationServiceUrl = 'https://api.wikimedia.org/service/linkrecommendation';

// Optionally set the fallback on DB miss variable. In practice, if you are
// using refreshLinkRecommendations.php and importOresTopics.php maintenance
// scripts locally, then you probably want this config flag set to false (default)
// $wgGELinkRecommendationFallbackOnDBMiss = true;

To allow non-existent pages (red links) as suggested edits, add the following to LocalSettings.php

$wgGEDeveloperSetup = true;

To enable link recommendation tasks for the currently logged in user, run the following from the browser console (while on Special:Homepage):

ge.utils.setUserVariant('linkrecommendation')

Image recommendationsEdit

To use the live image suggestions API with the Wikipedia matching the wiki's language, set

$wgGEImageRecommendationServiceUrl = 'https://image-suggestion-api.wmcloud.org';

To use Commons as a file repository, set

$wgUseInstantCommons = true;

To enable image recommendation tasks for the currently logged in user, run the following from the browser console (while on Special:Homepage):

ge.utils.setUserVariant('imagerecommendation')

Seeding Articles from Czech WikipediaEdit

See also https://phabricator.wikimedia.org/T274198#6972115

  1. Make sure that http://localhost:8080/wiki/MediaWiki:NewcomerTasks.json is the same as that in Czech Wikipedia.
  2. Get article titles for Growth tasks from Czech Wikipedia.
    curl -s "https://cs.wikipedia.org/w/api.php?action=query&format=json&export=1&exportnowrap=1&exportschema=0.11&generator=growthtasks" > tasks.xml
    
  3. Import XML dump by running the following command inside mediawiki installation directory
    php maintenance/importDump.php --report=10 tasks.xml
    
  4. Create "Module:Wikidata" on your local site. Copy/paste the source of https://cs.wikipedia.org/wiki/Module:Wikidata and save it.
  5. Update secondary tables (to get accurate information on Special:RecentChanges and Special:Statistics)
    php maintenance/rebuildrecentchanges.php
    php maintenance/initSiteStats.php
    
  6. Update ElasticSearch
    php maintenance/update.php --quick
    php extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php && \
    php extensions/CirrusSearch/maintenance/ForceSearchIndex.php --skipLinks --indexOnSkip && \
    php extensions/CirrusSearch/maintenance/ForceSearchIndex.php --skipParse && \
    php maintenance/runJobs.php
    

Seeding ORES topicsEdit

To import ORES topics for articles imported from a production wiki (English Wikipedia in the example below), run

php /path/to/mediawiki/extensions/GrowthExperiments/maintenance/importOresTopics.php --apiUrl https://en.wikipedia.org/w/api.php --wikiId enwiki --count 100 --verbose

To set ORES topics for some article manually, use

php /path/to/mediawiki/extensions/CirrusSearch/maintenance/UpdateWeightedTags.php --tagType classification.ores.articletopic --tagName 'Regions.Americas.Central America' --weight 750 --tagName 'Media.Media*' --weight 800 --page={yourPageTitle}