XTools/Development
Thanks for contributing to XTools! This guide gives a high-level walkthrough of everything you need to contribute to XTools in a local environment.
Prerequisites
edit- PHP 7.4+
- Composer
- Node with the version specified by the .nvmrc file
- MySQL 8.0 or higher, or MariaDB 10.5 or higher (the latter is what's used on production)
- A Wikimedia developer account
- Access to Toolforge so that you can connect to the Toolforge wiki replicas
Running a development server
edit- Clone the repository:
git clone https://github.com/x-tools/xtools.git && cd xtools
- Install PHP dependencies:
composer install
- Run
cp .env .env.local
and fill in the missing values, using /Configuration as a guide. npm install
to install Node dependencies- Assets are generated with Symfony Encore. If you're working on JavaScript, CSS, or adding images, compile the assets with:
npm run watch
.- Before submitting your pull request,
npm run build
to build the production assets.[1]
- Before submitting your pull request,
- Launch Symfony's built-in server:
symfony serve
- Assuming the configuration was done correctly and as desired, you should be up and running at http://localhost:8000
The Simple Counter is the simplest tool and should work as soon as you set up XTools. Test it by going to http://localhost:8000/sc and put in Jimbo Wales
as the Username and en.wikipedia.org
as the wiki. After submitting you should quickly get results.
The development server does not cache application data; any changes you make are visible after refreshing the page. However when you modify things in .env
or otherwise aren't seeing changes you're making, you may need to clear the cache with symfony console cache:clear
.
The logs are in var/logs/dev.log
. If things are acting up unexpectedly, you might try clearing the cache or restarting the server.
Developing against Toolforge replicas
editFirst make sure you have a Wikimedia developer account and access to Toolforge. Then you can set up the necessary tunnels and a shell session with:
ssh -L 4711:s1.web.db.svc.wikimedia.cloud:3306 -L 4712:s2.web.db.svc.wikimedia.cloud:3306 -L 4713:s3.web.db.svc.wikimedia.cloud:3306 -L 4714:s4.web.db.svc.wikimedia.cloud:3306 -L 4715:s5.web.db.svc.wikimedia.cloud:3306 -L 4716:s6.web.db.svc.wikimedia.cloud:3306 -L 4717:s7.web.db.svc.wikimedia.cloud:3306 -L 4718:s8.web.db.svc.wikimedia.cloud:3306 -L 4720:tools.db.svc.eqiad.wmflabs:3306 username@login.toolforge.org
Replace username
with your Toolforge Unix shell username. Note our tunnel has to connect to each database slice. The ports used here should match up with the corresponding options in your .env
, like so:
DATABASE_REPLICA_USER=<your-username>
DATABASE_REPLICA_PASSWORD=<your-password>
DATABASE_REPLICA_HOST_S1=127.0.0.1
DATABASE_REPLICA_PORT_S1=4711
DATABASE_REPLICA_HOST_S2=127.0.0.1
DATABASE_REPLICA_PORT_S2=4712
DATABASE_REPLICA_HOST_S3=127.0.0.1
DATABASE_REPLICA_PORT_S3=4713
DATABASE_REPLICA_HOST_S4=127.0.0.1
DATABASE_REPLICA_PORT_S4=4714
DATABASE_REPLICA_HOST_S5=127.0.0.1
DATABASE_REPLICA_PORT_S5=4715
DATABASE_REPLICA_HOST_S6=127.0.0.1
DATABASE_REPLICA_PORT_S6=4716
DATABASE_REPLICA_HOST_S7=127.0.0.1
DATABASE_REPLICA_PORT_S7=4717
DATABASE_REPLICA_HOST_S8=127.0.0.1
DATABASE_REPLICA_PORT_S8=4718
DATABASE_TOOLSDB_HOST=127.0.0.1
DATABASE_TOOLSDB_PORT=4720
Change the your-*
bits to your own values, which you can find in your replica.my.cnf
file in the home directory of your account on Toolforge.
Caching
edit
XTools should probably take advantage of Doctrine's built-in caching mechanism, but it doesn't. Instead it is done in a somewhat manual fashion. This is only done in Repository
classes, using this pattern:
public function doSomething($input)
{
$cacheKey = $this->getCacheKey(func_get_args(), 'component_method');
if ($this->cache->hasItem($cacheKey)) {
return $this->cache->getItem($cacheKey)->get();
}
$results = 'results of big query';
return $this->setCache($cacheKey, $results);
}
The cache key can be anything, so long as it is unique to the specific method. A third parameter can be passed to setCache
to set the TTL, using the same syntax from the DateInterval class (e.g. P1D
is one day, PT1H
is one hour).
The above methods are just wrappers around a PSR-6 implementation, intended to reduce the repetition of similar lines of code.
Style guidelines
edit- It's called "XTools", with two capital letters.
- XTools conforms to PSR2 and Slevomat coding standards; use
./vendor/bin/phpcs -s -p .
to check your code. - HTML routes must begin with the tool name, i.e.
EditCounterYearCounts
, while API routes begin with either Project, User, or Page. - Version numbers follow Semantic Versioning guidelines.
Tests
editTests are located in the tests/
directory, and match the src/
directory structure. They are built with PHPUnit. Repositories only handle fetching data and do not need to be tested. Controllers also interact with the database, and while tests are most welcomed for these, they will not run on the CI server (GitHub Actions or Scrutinizer) due to limitations. Instead, put all things that query the replicas behind a app.is_wmf
check (example).
There are also tests for linting, phpDoc blocks, and file permissions.
Use composer test
to run the full suite, or ./vendor/bin/phpunit tests/
to run just the unit tests.
Releases
editReleases are made by tagging commits in the master branch. Before tagging a new release:
- Update the
APP_VERSION
in the.env
file - Check the copyright year in
README.md
- Update
RELEASE_NOTES.md
with any notable new information for the end user.
Then tag the release (follow the Semantic Versioning guidelines, and annotate the tag with the above release notes) and push it to GitHub.
Lastly, update the version
and updated
parameters at XTools.
Glossary
editThe following terms are used throughout the XTools codebase. These words may have different meanings in other MediaWiki-related software.
- Anon / anonymous
- Internal term referring to both temporary accounts and IPs. IP-specific methods will have "IP" in the name.
- ArticleInfo
- The old internal name for the Page History tool.
- Length
- In the context of edits, this is the size of the edit in bytes.
- Model
- A class holding the business logic for a tool.
- Registered
- Refers to users, edits or pages created by permanent registered accounts.
- Repository
- A class responsible for communication with the database, API, or external service.
- Template
- Refers to either the view (a Twig template), or the Template namespace.
- Unregistered
- Refers to users, edits or pages created by temporary accounts and IPs.
Additional help
edit- Email:
tools.xtools
@toolforge.org
- IRC: ##wikimedia-xtools connect
- MediaWiki talk page: Talk:XTools
Notes
edit- ↑ We commit the assets to avoid having to build them on the production servers. Nothing in the public/build/ directory should be changed directly, including images. All such changes go in the assets/ directory then are compiled and/or copied to the build directory with Encore.