Talk:Selenium/PHP/Selenium Framework

Previous page history was archived for backup purposes at Talk:Selenium Framework/LQT Archive 1 on 2015-07-10.

An older comment also exists at Talk:Selenium Framework

How are we going to run selenium tests for extensions? As of right now, extension tests must be hard coded in. We need to figure out a way of integrating the tests automatically when MediaWiki is undergoing selenium testing.

Idea:

• Each extension has a hook that is called for adding/running tests.
• The hooks are only called when MediaWiki is configured to use Selenium.

On the test framework side, running tests for extensions can be done by adding the test suite name to the selenium_settings.ini. Eg:
testSuite[WikiEditorTestSuite] = "extensions/WikiEditor/tests/selenium/WikiEditorTestSuite.php"
On the test wiki side, seems like the setup needs to happen before the Setup.php. So instead of using hooks this is what I'm thinking of doing:

• LocalSettings.php should contain

require_once("extensions/WikiEditor/tests/selenium/WikiEditorSeleniumConfig.php");
$wgSeleniumTestConfigs['WikiEditorTestSuite'] = 'WikiEditorSeleniumConfig::getSettings';

WikiEditorSeleniumConfig::getSettings returns an array of include files and configuration variables needed by the extension.

• If a cookie has been set, indicating this is part of the WikiEditorTestSuite test run, we use the configuration we get from WikiEditorSeleniumConfig::getSettings

I'm doing some final testing and cleanup, and then I'll update the page and check in.

I only read the implementation details part, and it looks mostly sane to me. I don't think it's a good idea to have test suite names double as hook names; you could use "SeleniumTest$testsuite" or whatever to avoid scary naming collisions, but even then I think it'd be better to just call something like Selenium::doSetupFor( $testsuite ) (calling a hook makes no sense, because there'd be exactly one listener) and have that function know where to look. The latter could be implemented with a one-dimensional equivalent of $wgHooks (i.e. array( 'testsuite' => xallback ) ) if you like, just don't pollute $wgHooks itself with this that aren't really hooks.

I like how you're doing all this with a cookie, that's smart. It'll break in subtle ways if Squid caching is used (and in even more interesting ways if Selenium is hitting production URLs through Squid). It's possible to fix this, but as far as I'm concerned it can go in the "don't be stupid" category.

So after 1.14, MediaWiki started returning 404 status codes for pages that don't yet exist. Selenium seems to die when it encounters a 404. Are you all dealing with that in some way? For example, how do you test browsing to a page that doesn't exist and clicking the create link? --Cneubauer 13:42, 28 September 2010 (UTC)

I try to test SimpleSeleniumTestSuite so I execute :

php tests/RunSeleniumTests.php

• Error1 : PHP Fatal error: require_once(): Failed opening required 'PHPUnit/Runner/Version.php' (include_path='.:/usr/share/pear:/usr/share/php') in /home/karima/git/wikimedia-git-repos/core/tests/RunSeleniumTests.php on line 34
  • test (maybe) : pear install -f phpunit/PHPUnit_Selenium
  • result : may be ok but new error (Error2)
• Error2 : PHP Warning: Invalid argument supplied for foreach() in /home/karima/git/wikimedia-git-repos/core/tests/RunSeleniumTests.php on line 134
  • test : maybe the position of config file but with the argument seleniumConfig, I have same error

php tests/RunSeleniumTests.php --seleniumConfig /home/.../core/selenium_settings.ini

An idea ?

I found, there is error "foreach" because the parameter $seleniumTestSuites is NULL when [SeleniumTests] is empty in selenium_settings.ini.

ah, cool. Thanks for sharing :)