Manual talk:JavaScript unit testing/Flow

Previous page history was archived for backup purposes at Manual talk:JavaScript unit testing/LQT Archive 1 on 2015-07-10.

Is Chrome supposed to be included Vagrant? I get the following error

Terminal

$ npm run qunit

> qunit
> grunt qunit

Running "assert-mw-env" task

Running "karma:main" (karma) task

START:
21 07 2023 11:13:21.118:INFO [karma-server]: Karma v6.4.1 server started at http://localhost:9876/
21 07 2023 11:13:21.121:INFO [launcher]: Launching browsers ChromeCustom with concurrency unlimited
21 07 2023 11:13:21.127:INFO [launcher]: Starting browser ChromeHeadless
21 07 2023 11:13:21.128:ERROR [launcher]: No binary for ChromeHeadless browser on your platform.
  Please, set "CHROME_BIN" env variable.

Finished in 0 secs / 0 secs @ 11:13:21 GMT+0000 (Coordinated Universal Time)

Warning: Task "karma:main" failed. Use --force to continue.

Aborted due to warnings.

What steps are missing from the instructions to make it work?

I know to to get a coverage report for PHP. But how to get a coverage report for JavaScript?

Flexibility, environment and extension hook

Right now there's three factors that need some love that need to be fixed in the medium-long term. These are the:

• Flexibility of the test suite
  • Currently not so due to hardcoding in index.html
  • And will remove the need to keep track of dependancies
• Environment of the test suite
  • Right now there are no portals, tabs, bodyContent etc.
• Extensions
  • Extensions should be able load their modules in the test suite, and add a test suite for them

The way I'm planing to do this is by a SpecialPage. There would be a global variable (perhaps a hook is better / less evil regarding the killing of globals), that keeps track of where files containing test suites are located, and which module they test.

The SpecialPage would output the structure for QUnit (simple html snippet) and perhaps a self-submitting dropdown menu with available skins (submitting through GET as useskin), just like the SpecialBlankpage/"mwutiltest" hack had before I removed it. This in order to verify functions work in other layouts as well.

Also, this SpecialPage must unload all modules added before OutputPage->addModules to avoid conflicts with gadgets, user scripts and (parts of) extension modules, and only load the ones that have a test suite.

Issues

The main reason I didn't do this initially and went for a static html file is devided:

• It was a rushed proof-of-concept for the Berlin hackathon
• The mw.loader object needs to be tested as well, (read below), which means we need a more neutral environment anyway (atleast for those tests)
• The way the TestSwarm is currently set up it must work directly from an svn checkout. This isn't the case if there's a Special page that needs load.php to work and what not. This means there's a database needed and an otherwise "installed" MediaWiki instance.

mw.loader

The way we unit test the loader (currently it doesn't have that many tests but this is expected to grow in the future) must (obviously) not depend on itself. So we can either:

• keep the static index.html file but only for the loader
• or, we give our SpecialPage a mode in which, instead of unloading all added modules and adding our own, we add good old <script> tags to jquery.js and mediawiki.js from /resources and mediawiki.test.js from <ttt>tests/qunit/

I think the latter is the better option but would need a few changes in OutputPage probably.

TestSwarm

As mentioned, right now our TestSwarm configuration checks out /resources and /tests/qunit and adds a job for each test suite to http://mediawiki-testswarm/r12345/tests/qunit/index.html?filter=testmodulename.

In order to make this work from a SpecialPage the wiki needs to be installed, which also means a much larger check out. And due to TestSwarms nature of keeping (almost) all revisions[1] (which is a good thing), it may become very big.

Then we'd probably do a one-time install with a config we want (ie. private/read-only) and make the update/check/svn-co script in such a way that it will alias or copy the LocalSettings.php file to the directory. We may also need an .htaccess to hide the rest of the wiki.

Alright, done. --
Flushed 3,415 bytes of English from Krinkle's brain

---

1. ↑ all, as in, all revisions it checks out. It only checks out if the revision touched /resources or /tests/qunit

Brion added the following:

Test files often have the same name as the files they are testing, eg 'mediawiki.js' or 'jquery.colorUtil.js'. This makes code maintenance more difficult because bare filenames are often shown when navigating or working with code. Consider using the word 'test' visibly in the filename.

The reason I made it like that is twofold:

1. Because it makes lazy comparison of directories easy ( glob() the dirs and loop through to detect modules without a test suite[1])
2. Because we do the same in the PHP Unit framework testing.

I dont have any personal preference, if you think it's better, go ahead and make the change. Please ping me about/when one would commit this, so I can stand by and make the neccecary changes in the configuration of our TestSwarm (since the function that scans the svn directory for modules would have to strip the '.test' suffix when filtering the modules that are going to be distributed, because the module() call doesn't contain ".test", example).

1. ↑ ofcourse adding a concatenation with ".test" in the target of the comparison isn't complicated, but it's one extra difference that has to be accounted for everywhere.

Almost everything in tests/phpunit/includes is named with a 'Test.php' suffix; just yesterday I spent plenty of time fiddling with Block.php (Block class) and BlockTest.php (Block's test class), which was very pleasant to work with. It seems that most of the unit test files are similarly named: 'GlobalTest', 'ExternalStoreTest', 'HttpTest', 'MessageTest', 'MWNamespaceTest', 'SanitizerTest', 'RevisionTest', 'TitleTest', 'UploadTest', 'SearchDbTest', 'SearchEngineTest', 'SearchUpdateTest', 'DatabaseSqliteTest', 'DatabaseTest', 'ApiBlockTest', 'ApiWatchTest', 'ApiUploadTest', etc.

I would have been surprised and annoyed if the test class had been in another Block.php, and it would have made my life harder switching between a "Block.php" tab and another "Block.php" tab in my editor, trying to keep them straight. Yes the paths are different -- paths that are rarely shown in key parts of a programming editor's UI... When I encounter files that are confusingly named, I immediately want to change them to match our regular convention: with 'test' in the name.

Directory comparison, if that's something that's going to be done frequently, should probably be scripted up so people don't have to manually construct a command line every time -- even the simple case of matching names is not necessarily trivial.

In that case I blame the documentation on Manual:PHP unit testing:

The unit tests are located in the tests/phpunit directory Tests are organized into a directory structure that matches the directory structure of the code that they are testing. For example: The unit tests for file includes/IP.php can be found in tests/phpunit/includes/IP.php.

– oldid 405067 / edit

Thanks, I'll make the switch later tonight.

Looks like the current script is kinda hardcoded to not allow this. Rather than changing everything in a hurry I suggest postponing this to July (which, when all is good, is when I'm in SF). Then we'll be finishing up TestSwarm to move to a WMF host. Then in a bit of a rewrite we'll take this into account.