Account creation user experience/Testing/Experiment

Also see Extension:E3 Experiments/Testing

This describes testing for the experiment that modified "Create account" for new users of English Wikipedia by adding JavaScript code to Extension:E3Experiments that modified the page appearance and behavior. The experiment has been running since late in 2012.

The full test query string

?title=Special:UserLogin&type=signup&acux_active=1&acux_on=1&acux_eligible=1&acux_debug=1&acux_bucket=acux_3

use acux_bucket=control_3 to see the unmodified form. This will force the page regardless of schedule and even if you're logged in.

To watch bucketing happen, clear userbuckets and clicktrackingDebug cookies, log out, and visit

?title=Special:UserLogin&type=signup&acux_active=1&acux_on=1&acux_eligible=1

to force the experiment on and eligibility

Append

• &acux_bucket=acux_3 or control_3 to force bucketing (the number increases each time we change the presentation of the form)
• &acux_debug=1 to see console output
• &debug=1 to make MW send the JS separately. Only this mode will report JS errors on the console, so try both ways.

Upon form error, the form is redisplayed without these, so you lose eligibility dates, etc. — all reasons you'll see the form in the old style.

• You can always enter mw.e3.acux.modifyPage() in a JS console to restyle the page.
• You can add &campaign=TEST to the query string, see campaign section

Campaign testing

ACUX version 4 added support for campaign tracking, initially for the AFTv4 "create an account" Call To Action. If the query string contains campaign=Something, ACUX

• always shows the fancy acux_N form
• adds XXXX COMPLETE to userbuckets but ?? doesn't add ACUX bucket.
• remembers the campaign in its form so that on error the user still sees t

Testing account creation variations

A registered user doesn't ordinarily create an account (she has one!), but can navigate to the form and then

• can create account [By e-mail] (unstyled button)
• is prompted for Reason (not part of design)
• may see a checkbox to override AntiSpoof (not part of design)

sometimes

• may be asked for Real name (not part of design)

For all of these

• Test if an anonymous user creating her account can see any of these, we have a bug

• Test weird edge cases when you log out but don't clear cookies, or vice-versa.

Testing username validation

See Account creation user experience/Testing/Usernames page for more usernames info

Situation	Test username	Desired behavior	Status
# in username	Any#	Client-side invalid	validation succeeds. MW has 'usernamehasherror' => 'Username cannot contain hash characters' , but it's unused?
# at start of username	#Any	Client-side invalid	x
/ in username	Any/	Client-side invalid
Username already exists	Is there a guaranteed username?	Client-side invalid	✔
leading/trailing whitespace	Any	??
lowercase first letter, username already exists	admin, maybe? sysop?	Client-side validation should fail	Error from server on submit
Reserved usernames	ScriptImporter	$wgReservedUsernames
Username like existing (AntiSpoof)	Swalling (vs. Swailing on en-wiki)	Client-side validation should fail	Error from server on submit
Other special characters	Hello@There	Reject characters in $wgInvalidUsernameCharacters (default is just '@')
Illegal username & password combinations	elsewhere (/Engineering ?)	invalid after submit (client-side validation deferred)	✔

Other

• test length (existing form behavior, should be unchanged)
• test int'l characters , may mess up length restrictions
• valid username PHP code fails on "invalid due to empty, IP, slash, length, or lowercase"

Testing password validation

Not implemented yet

• test length (existing form behavior, should be unchanged)

• username & password combos (see above)
• minimum password length
• password strength? (deferred? en-wiki allows 1-character passwords)

Testing server API validation

Currently only the username is validated on the server before submit

• Client timeout (suspend browser?)
• Server timeout (simulate with break in JS debugger?)
• Slow link (change api/ApiAccountCreation.php to sleep( rand( 1, 3 ); before returning

Automated testing

PHP unit testing

The e3acuxvalidate API has some tests. Once you have set up PHPUnit:

$ ../../tests/phpunit/phpunit.php tests/phpunit/api/ApiAccountCreationTests.php

if you set the environment variable MW_INSTALL_PATH. you can also run

$ cd tests/phpunit
$ make

JavaScript unit testing

If you set $wgEnableJavaScriptTest = true; in LocalSettings.php, you can run Special:JavaScriptTest/qunit Current test/ subdirectory is not set up for qunit.

Browser testing

Most automated browser testing uses the Selenium web driver to drive a special version of the browser. QA has developed Cucumber-Watir-RSpec tests for the MediaWiki Userlogin that could be extended to account creation.

Meanwhile https://github.com/atdt/karaga is E3Experiments' own Python test framework for making simple assertions ("JavaScript modules loaded without errors", "there should be a cookie", etc.) about browser pages running on https://saucelabs.com.

You can also manually fire up a browser on http://www.browserstack.com/ and look for JavaScript errors while filling out the form.