Hi All,

When you are using watir-webdriver for web testautomation, you might encounter problems that are not be easy to debug. For example interaction with page elements such as links or buttons that are hidden. Or locating elements in other iframes. One easy way to see what is going on is by interacting with your web browser through the command line!

To make this happen, make sure you have;

1. a recent version of ruby installed (1.9.3 or later)
2. a recent version of firefox installed
3. the ruby gem ‘watir-webdriver’ installed;

gem install 'watir-webdriver'

Now, start the Interactive Ruby Shell…
In this irb, after loading ‘watir-webdriver’, you can start firefox with one command and interact with it right away!

sC-MBP:~ sCblogger$ irb
irb(main):001:0> require 'watir-webdriver'
=> true
irb(main):002:0> b = Watir::Browser.new
=> #
irb(main):003:0> b.goto "www.google.com"
=> "http://www.google.nl/"
irb(main):004:0> b.button(:text => /Google zoeken/).id
=> "gbqfba"
irb(main):005:0>

NOTE: with ruby 1.8.7 or earlier, you’d have to require ‘rubygems’ first before require ‘watir-webdriver’

Introduction

The following post makes recommendations for platforms, operating systems and browsers to use when testing commercial marketing web sites aimed at the general public in Europe.

Our recommendations are based on usage figures widely available on the Internet, our experience and analysis of clients needs. Since modern browser versions change frequently, we review and update these recommendations regularly.

Continue reading »

Viral
Wouldn’t this almost be a great viral campaign? This new campaign of one of the world’s leading companies only works properly in Internet Explorer 9.0. In other browsers this is what the Wallmart Video Download service looked like right after launch.

Even though errors like this don’t occur that much anymore most Chrome and Safari users know that there are still quite some annoying bugs when surfing the internet.

Continue reading »

Much of the software development in website development is done using agile methods. Automation is usually necessary to keep up with the speed.

It’s not really a question of whether automation is useful. Only a few people would rather repeat the same tasks many times every day then to let a computer do them. The question is whether the benefits are bigger then the costs. There will always be costs; automation is an investment. Businesses expect a return for their investment. There are several advantages:

• Requiring less manual work to reach the same test coverage
• Being able to use testing resources for higher value work improving the user experience
• Happier users -> more users

How to get started?

One of the critical parts in building test automation and automation suites is to have the right people to do it. There are other factors too, which I’ll briefly go through in this post. Tools, planning and working methods are other important factors.

Test automation suite?

What is it?

A test automation suite can be many things. It can and will evolve during its lifetime. The word test automation suite, at least for me, conveys a message of something massive, something that took a long time to build, covering the whole site functionality. That’s one definition, but there is no reason not to use a simpler, more agile approach.

A suite can be as small of automating a specific part of a site, for example, top navigation or a registration form. Anything, which needs a lot of steps to cover with every build, is a good candidate for automation. Can’t these things be tested manually then? Yes, but testers are humans too, you know. They will not be happy clicking the same 100 links or entering the 100 different email addresses required for good coverage. Does it matter that the tester is not happy about something. Well, that’s more of a HR question, but for me, it does matter. People need to be motivated to achieve the best results.

And the main reason for doing any testing is to achieve good quality. Saving the time from manually testing the forms and navigation will allow the tester to really do his/her job and concentrate on the more challenging test areas, for example, making sure the user experience of your site’s users is as good as possible.

And to be honest, no one will test all the links manually with every build. It’s only a matter of time before there is a serious problem when something important is missed in testing.

Okay, okay, we need automation. What next?

Requirements for successful automation project?

Here is a list of things that are important for a successful automation project.

1. Deep understanding of web technologies, HTML page elements. Mind you, this is hopefully something your testing team already understands.
2. Co-operation between development and QA teams in terms of communication about upcoming chances
3. Time. It will take some time to implement the automation. It will still be worth it, in many cases. Just remember to allocate some time for it. It’s important to keep working on automation iteratively, building more coverage as the site and the development work progresses. There will never be a two to four week period in the end of the project where you can just concentrate on building automation. Really. There isn’t.
4. Stability. If the site changes every day, it’s better to not start building an extensive suite. Again, start with the areas that are more stable. If not, you might face the chicken and the egg problem. You need a stable site to build automation but the site does not get stable when you don’t have it.
5. Iterative (agile) approach. Start from a single area in one browser. While doing this, you will see how well the automation works with your site. Once you are comfortable with the automation and how it works with your site, increase the coverage and expand to other browsers and platforms.
6. Tools. Use a tool set that’s easy to maintain. Use modern tools like Cucumber for writing the test cases and combine it with either Watir or Selenium. Whatever tool you use, make sure you can reuse your code. Cucumber is essentially a collection of navigation commands on your site combined with test cases that can be understood by other people, especially business analysts. They could even give you a hand in the test case creation by documenting the requirements in a format that can be used for test cases. Ideally, when using Cucumber, the requirements are the test cases.
7. Test automation is a tool to increase your test coverage, do not make it the software development project. You will not be allowed to keep working on the automation for months and months. While it is important to get the automation done properly, there is a risk that too much effort will be allocated for it. Like in any other form of testing, there is a trade-off between the time spent and the results achieved. 80-20 rule applies here too. Test automation is a tool to make your development project more successful in terms of quality and schedule. When you achieve that, you can be satisfied with the results.

The people

Can your average QA person do automation in a maintainable and reusable way? Most likely not. It requires a different skill set then manual testing. Or should I say, other skills in addition to the functional tester skills. A good solution is to have a team where some of the people are good in planning testing and test cases while others have the coding skills to maintain and build the test automation. If the team does not have the people with coding skills, most people start to automate testing with the help of a tool recording test cases. This will in most cases lead to major difficulties in maintaining the tests in the future. It can also be difficult to customize the tests for data-driven testing.

Required competencies

Automation requires development skills. You don’t need to be a high-level C++ programmer to create automation scripts. However, knowing how to implement code in a managed way will be necessary to get the automation done. And it is not only about completing the initial set of automation tests. The software/website you’re testing will typically receive updates every day/week. Some of those changes are going to break your test automation. Even if all the functionality on the site still works, some elements could have been moved or renamed causing the test automation not to find the element anymore. Elements on the page will need to be identified with the element attributes (for example, link text) or XPATH expressions, which are powerful but rigid. This is really the part where good understanding of HTML is required.

Software development practices

Maintaining any software, also test automation will benefit from good development process. One of the most important ones is using version control systems, for example, Subversion or git. It will enable maintaining the current version of the automation and tracking the changes done to it as well as branching your scripts for the current production system and the development version if there are major differences.

Development guidelines

Having a set of development guidelines to support the automation is needed to minimize unnecessary changes to the site. A big part of the technical implementation of test automation has to do with locating HTML elements on the page. It is good to establish naming conventions for the elements, and then stick to it. While it might be / is a good idea to fix naming of an element from something that’s obviously wrong, it can potentially cause test cases to fail. A typical case of this is could be something like renaming an element to something that’s clearer to other users:


<div>
  <ul class="top-nav">
    <li>Item 1</li>
    <li>Item 2</li>
  </ul>
</div>


Renaming the above unordered list would require the following change to an XPATH expression identifying the element:

//ul[@class='top-nav']  ->  //ul[@class='top-navigation']


While this change does make sense, it can easily cause automation tests to fail. The fact that test cases sometimes fail is not really a problem; it’s a fact of life that needs to be managed by allocating time for maintenance work.

Allocating time for automation

The project manager creates a plan for his project. The plan includes time for QA after the development work is completed.

Project of 6 months:

• 2 month for requirements / user stories
• 3 months for development
• 1 month for QA

Now, what happens when development time is increased by 30%. That’s not at all unrealistic, is it? Even with this relatively low delay percentage, the whole time to QA is basically eaten up by the development work. And there is of course a commitment for a customer to deliver on the agreed date. An additional complication is that there are most likely areas that have not been tested before the QA cycle is supposed to end.

The team will find defects on the last week of the project, things that should have been discovered 3 months earlier. Maybe there are areas that cannot be implemented at all with the current architecture.

This can, especially for a new team, lead to a situation where test automation is never taken into use in a proper way. There is not a lot of time, people are not familiar with the tools and enough progress is not made. In such a situation, management can easily loose faith in the usefulness of automation in general.

  spriteCloud Team

Agile testing methods

Test driven development

Behaviour driven development

Cucumber

Feature

Feature: Website Navigation
  In order to navigate website pages
  I need to be able to click on menu item

Scenario

Scenario: Going to page Services
  Given that I am on spriteCloud Home
  When I click on link Services
  Then the page title should be "spriteCloud - Services"

Step definitions

require 'spec'
require "watir-webdriver"

SITE = "www.spritecloud.com"
BROWSER = Watir::Browser.start(SITE, :firefox)
PAGES = {
  "spriteCloud Home" => "http://www.spritecloud.com/"
}

The hash PAGES is not really necessary in this case, but can be useful in a case when there is more data and pages involved. It is used for looking up a page URL based on page name in the scenarios.

Given /^that I am on (.*)$/ do |page|
  BROWSER.goto(PAGES[page])
end

Notice that the step syntax is formatted as a regular expression. Given is a keyword but everything else is basically just something that Cucumber can recognize as being part of your test. Let’s break that part down a bit:

• Sentence starts with “Given” keyword. This is mandatory
• What follows is a space and the words “that I am on”
• Another regular expression (.*), which matches everything from “on ” until the end of the line “storing” it for later use for the variable page

Using the capturing group (.*) is important because it allows us to write more generic steps, where for example, the page where we are or the link that we are following is defined in the scenario, not in the step. This again allows for the less technical person to write tests without having to write Ruby code. It is of course also better programming practice not to write the same things multiple times and just changing a value.

When /^I click on link (.*)$/ do |link|
  BROWSER.link(:text, link).click
end

Then /^the page title should be "([^\"]*)"$/ do |title|
  BROWSER.title.eql?(title).should == true
end

Test Execution

This type of test can be run either during the implementation phase, which is the original purpose or later on when the site is fully ready. Even if the test was created during the implementation phase, it should still be run with every new release that goes out. It can even be used as a part of a stand-alone test set for the site.

Part of the power of Cucumber is the nicely formatted output displaying immediately the successful and the failed tests and test steps. Successful tests are colored green on the terminal and failed one, you guessed it, red.

mbp-2:features marko$ cucumber .
Feature: Website Navigation
	In order to navigate website pages
	I need to be able to click on navigation

  Scenario: Going to page Services                        
    Given that I am on spriteCloud Home                   
    When I click on link Services                        
    Then the page title should be "spriteCloud - Services"

1 scenario (1 passed)
3 steps (3 passed)
0m1.261s

Summary

This post only scratches the surface of what can be done with either Watir or Cucumber. They are both powerful test tools by themselves and combining them can be the basis for a successful test automation project. It all depends on the needs in your project.

Marko