How come Ruby ?

 
There are several important Ruby projects, which are also increasing the popularity of the Ruby language in general:

http://www.ruby-lang.org/en/libraries/top-projects/

The most interesting projects from my (and this blog posts) point of view are Watir and Ruby on Rails, the former from a testers point of view, the latter from web developers.

Good projects don’t just come from nowhere, either. The language itself is very elegant and enjoyable to use. Having a programming language that works for you instead of against you is going to leave more time for innovation and thinking about the next big features for your product. The Ruby community is very active and provides very nice modules (=gems), which provide a specific functionality and usually do it very well.

What is Watir?

 
Watir stands for Web Application Testing in Ruby, http://watir.com. It’s a framework, which drives the browser programmatically. Currently the officially supported browsers are IE and Firefox, but gems also exists for Chrome and Safari. Also Opera is working on a port of their own, which they are using in their own development (http://my.opera.com/core/blog/2009/03/06/test-automation-with-operawatir). Unfortunately, it is not available for outsiders yet, which makes guaranteeing websites work on Opera more difficult than it should be.

What about Ruby on Rails?

 
The single most important reason for Ruby gaining a lot of attraction in the recent years is the Ruby on Rails web development framework, http://rubyonrails.org. The framework enables quicker web development with it’s inbuilt support for database access. It’s actually not just support, but ActiveRecord is one of the fundamental technologies behind RoR and makes it especially suited for database driven applications.

How is that related to testing?

 
Watir is great for web automation, and we have used it in extensive automation projects. There is lots of documentation available for the basic usage, but as always, best practices for implementing automation test projects in different types of situations are more difficult to find and require some trial and error to find the best approach.

Typically, one of the main points to consider is how easy the test cases will be to maintain. Writing one or ten test cases is easy without any structure, but scaling the automation suite for hundreds or thousands of test cases requires careful planning. Many of the same rules apply as for any software development project. As most of the testers do not have a developer background, this can be a challenge in the beginning.

There’s is nothing stopping testers taking advantage of the already well thought out features of Ruby on Rails what comes to database access (ActiveRecord) and for sending email (ActionMailer). Database access can be used to verify that a form actually posted the values and they were stored correctly in the database. Email can be, for example, used for sending test reports directly from test automation. There are clear synergies in understanding both of these technologies when aiming the create good test automation projects. And when using the already available gems for the tasks, you’ll save time for actual automation work.

We will cover several of these topics in a series of blog posts.

 
Marko

Building a Ruby on Rails test server on FreeBSD 8.0 RC1

We’re a Ruby on Rails shop at our place and we work on a number of different platforms for developing and testing our stuff; the devs normally work on macs, while the staging environment – this week – is Ubuntu’s Jaunty Jackalope (9.04) distribution. However I’m a big FreeBSD fan, and have been for a number of years. Speaking personally for myself, FreeBSD makes sense in every aspect of its operation and configuration, and the ports collection is second to none for all aspects of application management. (disclaimer – *nix distributions are more like pairs of jeans than they are operating systems in the hearts and minds of regular users; what fits well on one user, doesn’t fit at all on another. So I’m absolutely not saying FreeBSD is best, it’s just best for me). When the FreeBSD 8.0 RC1 build was made available I decided to put together a test server for deploying new builds of our apps for first round QA work. More or less from a standing start this is the road I travelled, which at the end resulted in a fully operational test server.

First off was the installation of the 8.0-RC1 build itself on the test server machine, which was my Lenovo X60 laptop with 2GB of RAM. I got a hold of a build ISO image from a mirror detailed on the main “Get FreeBSD” site page, and took disc 1 only. The idea was to make a minimal install which meant only the first disc was necessary. With a minimal install you don’t get add the ports collection by default, so the first post installation task was getting this added. At this point, I was ready to install Rails, but as I also wanted a desktop I went ahead and installed KDE4. I’m not going to go into the details here, as there are other fine posts about this easily found with a Google search, but the process is fairly straight forward with no major gotchas yet takes around 10 – 12 hours to complete.

Starting to test a project is always a challenge. You don’t necessarily know what to expect of the software. It is supposed to be almost ready, but the definition of ready varies a lot based on the individual developer. And not only of the developer. There are many external factors affecting the project and pushing things along. A project can have, and usually has, schedule pressures and any single person does not want to be the one to blame for the delays. For example, a release or iteration could have 20 new features, which are supposed to be completed on a specific date. The development team does not want to be the one delaying the project and releases the software to QA. As it turns out, only 10 of the features are really implemented and 5 actually work. This is a bit of an extreme scenario, but nonetheless one that many QA people have faced. If external factors have too much affect on the project, the software quality will be a bit of a hit and miss.

Even with all of the problems mentioned here, the initial period on working on the new software is very rewarding. Testers learn how the new software works and find plenty of defects. They look at the product with an open mind and get new ideas and approaches to testing it. A lot of progress can be made in a short time. When the work becomes more routine later on, many times there will also be less innovation.

It’s also a time when test cases evolve rapidly. The initial set is based on the specifications, which don’t necessarily completely reflect the real functionality. The test cases are going to be refined during the initial testing rounds. More test cases will also be added. Very rarely will the initial set of tests have full coverage for the software although it is possible in highly organized development team where quality is what matters most. In practice, most organization can not spend the time it takes to plan for full test coverage, and they probably should not either. Test planning is a continuous process, where both more details to the test cases for current functionality and more test cases for new functionality in new releases are added.

The fact that test cases are in practice a result of a long continuous process makes it even more important to keep track of the early test cases, which can be used to build upon. The cases might not be perfect, but will serve as a foundation for future work.

Marko