Manual:Coding conventions/Selenium-Ruby

This page describes coding conventions we follow when writing browser tests. The links section of the mediawiki-selenium README file lists the repositories that contain browser tests. There are three types of Cucumber test files in the MediaWiki codebase: Cucumber features files, Cucumber step definition files and page object files.

Željko Filipin comments

I pair with people from different teams and I work on the mediawiki-selenium gem, so I tend to see a lot of Ruby, Selenium, Cucumber and page-object code. I have noticed some code that I like and some code that I do not like.
To move the code towards what I like, I have been working on this page. I plan to implement the changes myself, if nobody does it before I have the time to do it. I do not have a timeline for that.
I would really appreciate if you would read the page (it should take you 5 minutes or so) and let me know if you agree or disagree with my thoughts, if I missed something, if something needs to be explained, if the page should have more text and less code or vice versa...
You can let me know in the page discussion section, at QA mailing list, at #wikimedia-qa ^connect freenode IRC channel (I am zeljkof there). Also, feel free to update the page.

General

Whitespace

Selenium tests should use the same whitespace convention (tabs, spaces...) that the repository already uses. That makes it easy for developers to work on tests. This convention is currently used in most places. ✔

tests/browser

If possible, Selenium tests should be located in tests/browser folder. This convention is currently used in most places. ✔

Cucumber feature files

For example file see any file in features folder of qa-browsertests repository. Feature files usually contain features and scenarios.

Scenarios

Every scenario should be as simple as possible. It is better for a feature to have more smaller scenarios than just a few big scenarios that try to test everything. This convention is currently used in most places. ✔
The scenarios are supposed to be as human-readable as possible. Do not turn them into a programming language - they are supposed to be a communication tool between users, product managers, testers and developers.
Avoid mentioning implementation details in the scenarios.
Whenever possible, specify actual test data strings in Scenarios and pass those to the tests using the regex capture aspect of Cucumber.

Example of article title defined in the page object (bad)

When I go to a nonexistent page

When(/^I go to a nonexistent page$/) do	
  visit(NonexistentPage)
end

class NonexistentPage < ArticlePage
  def self.url
    URL.url("Nonexistent_page_ijewrcmhvg34773")
  end
  page_url url

Fixed: article title is passed from the Scenario to the step to the page object. (good):

When I go to an uncreated page using URL Nonexistent_page_ijewrcmhvg34773

When(/^I go to an uncreated page using URL (.+)$/) do |article|
  visit(NonexistentPage, :using_params => {:article_name => article})
  end

class NonexistentPage < ArticlePage
  include PageObject
    include URL
      page_url URL.url("<%=params[:article_name]%>")
     end

also

Example of text typed in to page only in a step and checked only in a step (bad):

When I type a math expression

Then alt for that img should be the math expression

When(/^I type a math expression$/) do
  on(EditPage).article_text=<math>3 + 2</math>
end

Then(/^alt for that img should be the math expression$/)
  on(EditPage).math_image_element.element.alt.should == "3 + 2"
end

Fixed, text to be typed into page is specified in the Scenario and result to be checked also specified in the Scenario (good):

When I type <math>3 + 2</math>

Then alt for that img should be 3+2

When(/^I type (.+)$/) do |write_text|
  on(EditPage).article_text=write_text
end

Then(/^alt for that img should be (.+)$/) do |alt|
  on(EditPage).math_image_element.element.alt.should == alt
end

Alphabetically sorted

If a feature or scenario has more than one tag, they should be sorted alphabetically. This convention is currently used in most places. ✔

Example (good):

@clean @custom-browser @en.wikipedia.beta.wmflabs.org @firefox @login @phantomjs @test2.wikipedia.org

Required tags

Every feature in a feature file should have site- and browser- specific tags. Scenarios inherit tags from the features they belong to (see Cucumber tags documentation), so if for example the entire feature runs on a specific browser, you only need tag the feature at the top of the file, not individual scenarios.

A site-specific tag is for example @en.wikipedia.beta.wmflabs.org or @test2.wikipedia.org. The tag specifies where the feature or scenario should run. This convention is currently used in most places. ✔
- @clean is a special case of a site-specific tag is . If the feature or scenario runs fine on a clean wiki, it should be tagged @clean. This convention is currently not used. N Except for a single test in the /qa/browsertests repo, at the moment we tag features or scenarios that are known to fail on a clean wiki with @needs-custom-setup.
A browser-specific tag is for example @firefox or @phantomjs. These tags specify which browsers can run the feature or scenario. This convention is used in most projects that have CI browser tests. ✔ We also tag features or scenarios if they are known to fail with a specific browser, for example @phantomjs-bug.

Optional tags

Some features or scenarios can have an optional tag.

If the feature or scenario requires the user to log in, it should have @login tag. This convention is currently used in most places. ✔
If the feature or scenario requires custom browser configuration, it should be tagged @custom-browser. This convention is currently used in most places. ✔
We need to tag tests that are quick or slow to run. My suggestion is @quick and @slow tags. @quick tag could be used to create a Jenkins job that would run after every patch set submission to Gerrit, or every time a commit is merged into master branch. @slow tag could be used to create a Jenkins job that would run once a day. This convention is currently not used. N

Extension tags

If your feature requires the presence of optional MediaWiki extensions, be sure to include an @extension- tag for each. These tags will help keep your test suite more deterministic by allowing Cucumber to skip features that would otherwise fail falsely due to the wiki's configuration.

@extension-visualeditor
Feature: VisualEditor Mobile

Scenario: VisualEditor Provides Bold
  Given I am logged into the mobile website
   When I look at the VisualEditor toolbar
   Then I see a bold button

Before attempting to execute your feature, Cucumber will check that the wiki has these extensions installed and enabled. If any of the dependencies aren't met, the runner will skip the feature and warn the user.

Cucumber step definition files

For example file see any file in step_definitions folder of qa-browsertests repository. Step definition files usually contain Given, When and Then steps.

Page objectives pattern

Direct calls to Selenium function are not supposed to be used. Use PageObject. This convention is currently used in most places. ✔

Example (good):

Given(/^I am at Log in page$/) do
  visit LoginPage
end

Example (bad):

Given(/^I am at Log in page$/) do
  @browser.goto "#{ENV['MEDIAWIKI_URL']}Special:UserLogin"
end

Simplicity

In general, code in step definition files should be as simple as possible. Ideally, just one or two lines per step. All complicated code should be moved to page objects. This convention is currently used in most places. ✔

Example (good):

Given(/^I am at Log in page$/) do
  visit LoginPage
end

When(/^I log in with incorrect password$/) do
  on(LoginPage).login_with(ENV["MEDIAWIKI_USER"], "incorrect password")
end

Grouped by type

Steps should be grouped by type. Given and When steps should be grouped in one group, Then steps should be grouped separately.

Example (good):

Given(/^I am at Log in page$/) do
  visit LoginPage
end
When(/^I log in with incorrect password$/) do
  on(LoginPage).login_with(ENV["MEDIAWIKI_USER"], "incorrect password")
end

Then(/^feedback should be (.+)$/) do |feedback|
  on(LoginPage) do |page|
    page.feedback_element.when_present.click
    page.feedback.should match Regexp.escape(feedback)
  end
end
Then(/^Log in element should be there$/) do
  on(LoginPage).login_element.should exist
end

This convention is currently not used. N At the moment we do not combine Given and When steps in one group.

Alphabetically sorted

Inside a group, steps should be sorted alphabetically by step name. That moves steps with similar name close to each other. Steps with similar name usually have similar functionality, making them good candidates for merging. This convention is currently used in most places. ✔

Example (good):

Then(/^Log in element should be there$/) do
  on(LoginPage).login_element.should exist
end
Then(/^Log in page should open$/) do
  @browser.url.should match Regexp.escape("Special:UserLogin")
end

Assertions

We are using rspec-expectations assertions in step definition files. Assertions should be used only in Then steps. Using assertion in Given or When step is usually a sign that the scenario is too big and should be split into two or more smaller scenarios. This convention is currently used in most places. ✔

`expect` or `should`

We should use rspec-expectations expect syntax.

Example (good, expect syntax):

Then(/^Log in element should be there$/) do
  expect(on(LoginPage).login_element).to exist
end

This convention is currently not used. N At the moment we are using old should syntax.

Example (bad, should syntax):

Then(/^Log in element should be there$/) do
  on(LoginPage).login_element.should exist
end

Page object files

For example file see any file in features/support/pages folder of qa-browsertests repository. Step definition files usually contain page URL, page elements and methods.

URL

Page URL is optional. It is used in step definitions when a Given or When step needs to go directly to a page, or when a Then step needs to check page URL. This convention is currently used in most places. ✔

Example (good):

class LoginPage
  include URL
  page_url URL.url("Special:UserLogin")
end

Page elements

Simple

Simple page elements should be defined using page-object Ruby gem API. This convention is currently used in most places. ✔

Example (good):

a(:edit, text: "Edit source")

page-object gem shortcuts

Code is more readable if shortcuts are not used.

Example (good):

page.edit_element.click

Example (badbad):

page.edit

Complicated

Elements that are complicated to find should pass blocks finding the elements to the page-object API. This convention is currently used in most places. ✔

Example (good):

unordered_list(:search_results, class: "mw-search-results")
li(:second_result_wrapper) do |page|
  page.search_results_element.list_item_element(index: 1)
end
link(:second_result) do |page|
  page.second_result_wrapper_element.div_element(class: "mw-search-result-heading").link_element
end

Methods

If a page has complicated functionality, a method in its page class is usually the best place for it. This convention is currently used in most places. ✔

Example (good):

class LoginPage
  include PageObject

  def login_with(username, password)
    self.username_element.when_present.send_keys(username)
    self.password_element.when_present.send_keys(password)
    login_element.fire_event("onfocus")
    login_element.when_present.click
  end
end

If the method returns page element, it's name should end in _element. This convention is currently used in most places. ✔

Example (good):

class LoginPage
  include PageObject

  def statement_name_element(group_index)
    @browser.element(css: ".wb-claimlistview:nth-child(#{group_index}) div.wb-claim-name")
  end
end

Example (bad):

class LoginPage
  include PageObject

  def statement_name(group_index)
    @browser.element(css: ".wb-claimlistview:nth-child(#{group_index}) div.wb-claim-name")
  end
end

Hooks

Complex, exceptional behavior needing to be used by multiple tests may be specified in the file under support/hooks.rb

Some examples from the VisualEditor repository:

A hook to keep the browser open if an env var is set:

at_exit do
  $browser.close unless ENV["KEEP_BROWSER_OPEN"] == "true"
end

A "Before" hook that will set up an edited page for manipulation by tests:

Before("@edit_user_page") do
  if (!$edit_user_page or !(ENV["REUSE_BROWSER"] == "true")) and @browser
    step "I am logged in"
    step "I am at my user page"
    step "I edit the page with Editing with"
    $edit_user_page=true
  end
end

A "Before" hook that sets up a page and selects a particular string on the page to be manipulated by tests:

Before("@make_selectable_line") do
  if (!$make_selectable_line or !(ENV["REUSE_BROWSER"] == "true")) and @browser
    step "I am logged in"
    step "I am at my user page"
    step "I click Edit for VisualEditor"
    step "I type in an input string"
    step "select the string"
    $make_selectable_line=true
  end
end

Manual:Coding conventions/Selenium-Ruby

General

Whitespace

tests/browser

Cucumber feature files

Scenarios

Alphabetically sorted

Required tags

Optional tags

Extension tags

Cucumber step definition files

Page objectives pattern

Simplicity

Grouped by type

Alphabetically sorted

Assertions

expect or should

Page object files

URL

Page elements

Simple

page-object gem shortcuts

Complicated

Methods

Hooks

`expect` or `should`