User:Rachel99/proposal1

Browser Test Automation for VisualEditor (Outreach Program for Women - Round 6)

Name: Rachel Thomas

Email: rachelqa99 [at] gmail [dot] com

IRC handle: rachel99

Location: Boston, MA (UTC -4:00 )

Timezone: EDT (UTC -4:00)

Typical working hours: Flexible, but most likely would work 8:30am-4:30pm EDT

Background

As the major issues in VisualEditor have been fixed, the VisualEditor has recently been deployed to 14 non-English Wikipedias so that users can become familiar with the editor. It was already been deployed to the English Wiki in December 2012. VisualEditor will eventually become the default editor for Wikipedia, and thus it is important that its current features are tested thoroughly. These tests can then be run as regression tests to make sure nothing breaks as more new features are deployed to VisualEditor in future phases. As the editor is finally stable enough to have automated tests written and run against it, now is a good time to set up test cases that confirm VisualEditor is working the way it should.

VisualEditor is a great target for testing because it matches the desired criteria exactly: http://www.mediawiki.org/wiki/QA/Browser_testing#Rationale. VE should work in many browsers, is javascript-heavy; requires navigation to test properly; is likely to have regression issues, and is a critically important project.

Goals

To construct a battery of web automation tests for the thorough testing of VisualEditor. These would be Accepted Test Driven Development tests written in Cucumber and falling under the same framework as the currently running web automation tests. The Cucumber tests would be coded using Ruby Page Object Design, and then run under Jenkins. The tests should cover all the current features of VisualEditor.

Implementation

A top-level overview of what needs to be done:

Come up with the features needed to be tested for VE functionality
Add these VE features to current QA/Backlog page under VE section
Write Cucumber test cases for each feature
Run each test case, and write Ruby code to execute it properly
Check into Gerrit, and have it approved by Chris or Zeljko
Add it to Jenkins

Some examples of features needing to be tested:

Test that VE works in User and Article namespaces

Test that VE doesn't work for talk pages, templates, categories, etc.

Test Vector skin to make sure VisualEditor is working

Test Monobook skin to make sure VisualEditor is working

VisualEditor works in the most modern versions of Firefox, Chrome and Safari

Test that editing references or templates that cannot be edited yet, will be cross-hatched, have a tooltip and can be deleted

Language tests:

multiple character sets
right-to-left languages like Hebrew

Integration tests

Wikimedia projects that may deploy VE in the future (i.e. Wikisource)

Consider open bugs https://bugzilla.wikimedia.org/buglist.cgi?list_id=179710&product=VisualEditor&query_format=advanced&resolution=---&resolution=LATER&resolution=DUPLICATE&order=bug_id%20DESC&query_based_on=

Timeline

This a tentative timeline, and is subject to change. Note that I believe I overestimated how much time it takes to code Cucumber tests.

Dates	Activity
May 8- June 6	Continue to code features for QA web automation tests in the current backlog, in order gain more familiarity with Ruby and Selenium. Will seek help from Zeljko when needed.
June 7- June 14	Work with Visual Editor to develop expertise in using VE.
June 17 - June 24	Come up with other features to test in VE as I look at VE more closely. Also add feature testing based on critical bugs found in Bugzilla. Add the complete list of VE features to be tested to current QA/Backlog page under VE section. Sequence the tests from most important to test to least important to test, getting input from Zeljko and Chris. Divide these tests into chunks, and prepare to complete one chunk before doing the next chunk. Start with the most important chunk first.
June 25 - July 2	Take the first chunk of tests and code them into Cucumber language. I suggest chunks should be about 4 -6 tests, depending on complexity.
July 3- July 31	Run each test case in Cucumber, and write the corresponding code in Ruby. Check code into Gerrit for code review after each Ruby sequence for the test script is written. When each test case is complete, and successfully passed code review, add it into Jenkins to be deployed continuously as a automatic test.
August 1 - August 10	Take the second chunk of tests and code them into Cucumber language.
August 11 - Sept 2	Write the corresponding code in Ruby. Deploy in Gerrit for code review and Jenkins.
Sept 3 - Sept 23	Clean-up period for any remaining tests to be written and coded, including important bugs added to Bugzilla in past 2 months. Buffer period.

Past Open Source Experience

I have been volunteering at Wikipedia for the past 3 months in the Browser web automation area to gain more experience in QA automation, as well as to engage in working on Open Source. I have always been a supporter of free and open software, so I am really excited to be working with the Wikimedia Foundation. I have really enjoyed my time volunteering here. I have been mentored by Chris Mcmahon and Zeljko Fillipin as I have gone through the process of learning the environment at Mediawiki, and then learning Cucumber and Ruby to write and code the test cases. I have done the test case for the SpecialNewsFeed feature and search feature in Cucumber, and have also coded the corresponding search.feature script in Ruby. Zeljko and I paired up to work on the search.feature script, and using IRC, etherpad, and our own local running environments we were able to complete it successfully

I have also learned how to use git and gerrit in order to contribute tests for Search that we are running right now.

Browser automation skills and experience are not specific to any given test target, so my contributions need not be limited to VisualEditor or Search or any other particular target, but VisualEditor is just a particularly good match for browser tests, and those tests do not exist right now. As VisualEditor testing is now becoming a higher priority, it seems like a good time for me to start writing tests for it.

Past Contributions

https://gerrit.wikimedia.org/r/#/q/owner:%22Rachel99+%253Crachelqa99%2540gmail.com%253E%22,n,z

About Me

I have a B.S in Mathematics, Minor Computer Science from SUNY Albany and have over 20 years experience working in IT at various positions including Linux System Administration, Release Engineer, QA tester and Technical Support Engineer. As the lead QA tester for a small startup, I developed test cases, performed functional, regression and integration tests, and filed bug reports in Bugzilla.

I have always been interested in Quality Assurance, particularly in QA automation. I really enjoy coding and have done coding in Shell scripting, Perl, and Java throughout my career. I am adept at learning new coding languages quickly.

I hope to pursue a career in QA automation in the future, while still staying involved as a volunteer in Wikipedia. I would like to continue to help the Wikipedia QA automation group by adding test cases to their backlog in the future, or by doing other volunteer work of interest.

I am also interested in community building, particularly among women in technology, and I was a member of Society of Women Engineers for many years. During that time, I managed a committee of volunteers at SWE-Boston that were responsible for re-doing the SWE Boston website. I received the "Outstanding SWE-Boston Member Award" in 2003 for my work in getting the new website constructed.

Other community organizations that I am currently involved in are my temple, where I am on the Board of Directors, and the Software Quality Group of New England, where I am a member.

Monthly Contributions

June

completed my Visual Editor Charter which lists the areas of Visual Editor that I will be writing test cases for
familiarized myself with new features in Visual Editor
Found 2 Visual Editor bugs, and reported them on Bugzilla
am working on automation scripts to test the references feature of Visual Editor.
have completed the VE_references feature file that sets up the tests for VE references
worked in Cucumber, Ruby and with Page Objects to translate the cucumber language to Ruby for several cases.
Pair programmed with Chris McMahon several times to learn more about Cucumber/RubyPage Objects.
Reported a problem running my tests on test2 wiki
continue doing exploratory testing on VE to look for bugs
participate on QA mailing list
attended OPW kickoff meeting, 2 Engineering Community Team Meetings, and GSOC/OPW all hands meeting.

July

make improvements to my Visual Editor Test plan, and add new features to be tested
write Visual Editor links test (first scenario), and have it merged with master branch
write test for testing VE for logged-in user and anonymous user
add to VE links test a test for internal links, and links that don't exist
start work on VE test for headings
test Mediawiki-vagrant Windows install
help out with QA automation workshop, and participate on QA mailing list
attend 4 ECT meetings
file several Bugzilla bugs on Visual Editor

August

polish my Visual Editor Test plan
write a VE headings script to test the different VE headings, and get it merged to master
start work on VE script to test bullets, numbering, indents, outdents
train a QA volunteer on writing cucumber/ruby/selenium in a pairing session
participate on QA mailing list
after VE scripts are moved to a new repo, copy the new repo to my computer, and set it up to work with Gerrit
create login on Sauce Labs to view instances of failed tests
attend 3 ECT meetings
write a blog post on the tool Cucumber

September

finish work on VE script to test bullets, numbering, indents, and outdents
participate on QA mailing list
attend wrap up OPW meeting on IRC
attend 2 ECT meetings
write my wrap up blog post