Quality Assurance/Exploratory testing
This page is obsolete. It is being retained for archival purposes. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on the information here being up-to-date. |
Exploratory testing is "simultaneous learning, test design and test execution" or "test design and test execution at the same time". Exploratory testing is a powerful approach that everyone should know.At WMF, a very structured form of Exploratory testing is maintained for some teams such as Editing team.
Types of testing
editExploratory testing can include different testing approaches or types. The different types of testing that are followed in Editing team is Regression testing, Sanity testing, Negative testing, Feature focused testing and verifying bug fixes. These testings are done on different browsers,OS and on different devices for both desktop and mobile web version of VE.
Test Environments
editThere are two test environments which are regularly tested. The test environments are :https://test2.wikipedia.org/wiki/ and http://en.wikipedia.beta.wmflabs.org/wiki/
Test schedules
editUsually Regression testing and Sanity is done almost everyday on two major browsers (Chrome and Firefox) on Betalabs environments.Also, whenever there is a new bug fix , it gets verified on Betalabs first which runs pre-wmf (n+1) until every Thursday.
Every Thursday when wmf (n+1) and wmf (n) gets deployed on test2 and all production wikis respectively, Regression testing,Sanity testing and Bug verification is done on those environments covering up major browsers and devices. The list of bug fixes or new feature implementations are maintained primarily through Bugzilla with each bug getting tagged under a specific target milestone.After verifying each bug, it is advisable to put a comment that the tester has verified in a certain environment. If the issue is still reproducible, the status of the bug gets changed to "Reopened" .When a bug is verified in production , in addition to adding a comment specifying that, the status of the bug also gets changed to "Verified".Note that, this is done only when a bug fix makes it to Production, not on any test environments.
Feature focused testing can be done, once a stable version of the new feature gets merged to Betalabs and the testers are notified of it. Of course, when the new feature gets deployed on test2 on Thursday of the current week and the next thursday on production, it gets thoroughly tested on those two environments too.
Also, sometimes proper steps to reproduce a bug reported by a user is needed for developers to understand the bug, so it is also a part of QA task to try to do some combinatorial testing to figure out the exact steps to reproduce.