Marco Valtas

This week I had to rewrite a bunch of functional tests, mostly Selenium stuff. When I think about Java Unit tests the first thing that comes to my mind is the JUnit framework. All tests that I have written until past week were with JUnit, and was using it to run Selenium.

But I got one more requirement, since I was rewriting all tests I would like to change environments (development and pre-production) without dealing in the code where was running the tests. The problem was a simple one, just write configurations with URLs, expected values and so on, for each of my environments. A trivial example is using different URLs in selenium for the same tests. Well, how do this with JUnit?

First approach, using JUnit 3

In JUnit 3 (old one) you have Test Suites. Programatically you have a chance to read a properties file and give the information to your test classes. Well, this could work if I had not the principle: "Do not write test code if isn't a test." When I caught myself creating a "little" framework to test my application I realized "I'm in the wrong way". Not that you should never code a "little" framework to simplify even more the tests, but i hadn't the time to do it.

Second approach, using JUnit 4

Well, JUnit 4 is more flexible, newer and uses annotations; and I didn't explored it well. After some digging i found JUnit's Parameterized Tests, but again I had (bear with me in this one) "to code, code to test code". And one more thing, JUnit 4 have Test Suites but, AFAIK, you only have a annotation version to define the classes for a suite.

JUnit wasn't solving the problem

After some thought i had two choices, forget about environments, or spend some time coding new stuff that wasn't tests or my application. Justice has to be made, JUnit is a excellent test framework but I think my objectives weren't too "Unitwise".

TestNG, next generation?

I heard about TestNG about a year ago and didn't gave it enough attention, I was happy with JUnit. But, everyday is a new day and I had a problem, so I started to read about TestNG. I was skeptic at first, thinking "maybe this can do it, but I don't have time to learn it all". I was wrong, happily wrong.

TestNG, next generation!

I have to confess, there's nothing to write here, maybe just some code. Remember the problem? "Two environments, one test, test it all". First, let me show the test example.

package com.keepcoding.test;

import static org.testng.AssertJUnit.*;
import org.testng.annotations.*;
import com.thoughtworks.selenium.*;

public class Google {

 private Selenium selenium;

 @BeforeClass
 @Parameters({"selenium.host", "selenium.port", "selenium.browser", "selenium.url"})
 public void startSelenium(String host, String port, String browser, String url) {

  this.selenium = new DefaultSelenium(host, Integer.parseInt(port), browser, url);
        this.selenium.start();
        this.selenium.open(url);
 }

 @AfterClass(alwaysRun=true)
 public void stopSelenium() {
  this.selenium.stop();
 }

 @Test
 @Parameters({"search","expected"})
 public void googling(String search, String expected) {
  try {
   selenium.type("name=q", search);
   selenium.click("name=btnG");
   selenium.waitForPageToLoad("3000");
   assertTrue(selenium.isTextPresent(expected));

  } catch (SeleniumException e) {
   fail(e.getMessage());
  }
 }
}

This is a simple test, will start Selenium, open a URL (to be defined), type something (to be defined) and finally check if a text is present. Some notes:

• @AfterClass is defined with "always=true" because Selenium has to stop, even on test a failure.
• Selenium commands are surrounded with a try/catch because I want to avoid ERROR statuses, better a FAILED status if Selenium fails.

Now let's look on TestNG configuration file:

<suite name="DevelEnvSuite" verbose="3">
 <parameter name="selenium.host" value="localhost"></parameter>
 <parameter name="selenium.port" value="4545"></parameter>
 <parameter name="selenium.browser" value="*firefox"></parameter>
 <parameter name="selenium.url" value="http://www.google.com"></parameter>

 <test name="Google">
 <parameter name="search" value="Keep Coding"></parameter>
 <parameter name="expected" value="keepcoding.blogspot.com"></parameter>
 <classes>
 <class name="com.keepcoding.test.Google"></class>
 </classes>
 </test>
</suite>

Here I defined all parameters and what tests should run, as you probably guessed the test search on Google for "Keep Coding" and checks if this blog URL is in the results. The key thing is not the test itself, but it's setup. We have defined Selenium to open a certain URL and if we want change the URL is a simple case of editing the XML file.

Using Ant to test all environments

Finally, how test all environments? First, create another TestNG configuration file, change the parameters accordingly, and use Ant to run it all. Here's a build file example:

<project name="testng_selenium" basedir="." default="build-all">

  <property name="src.dir" value="src"></property>
  <property name="reports.dir" value="testng_reports"></property>
  <property name="build.dir" value="bin"></property>
  <property name="lib.dir" value="lib"></property>

  <taskdef resource="testngtasks" classpath="${lib.dir}/testng-5.8-jdk15.jar"></taskdef>

  <path id="master-classpath">
    <fileset dir="${lib.dir}">
      <include name="*.jar"></include>
    </fileset>
    <pathelement path="${build.dir}"></pathelement>
  </path>

  <target name="clean" description="Removes build directory and test results.">
    <delete dir="${build.dir}"></delete>
    <delete dir="${reports.dir}"></delete>
  </target>

  <target name="build" depends="clean" description="Build Java files.">
    <mkdir dir="${build.dir}"></mkdir>
    <javac destdir="${build.dir}" debug="true" deprecation="false" optimize="false" failonerror="true">
      <src path="${src.dir}"></src>
      <classpath refid="master-classpath"></classpath>
    </javac>
  </target>

  <target name="build-all" description="Build and test all, implies clean." depends="clean,build,tests">
  </target>

  <target name="tests" description="Run testNG tests.">
    <mkdir dir="${reports.dir}"></mkdir>
    <testng classpathref="master-classpath" outputdir="${reports.dir}" sourcedir="${src.dir}" haltonfailure="true">
      <xmlfileset dir="${src.dir}" includes="*_suite.xml"></xmlfileset>
    </testng>
  </target>

</project>

This is a simple ant file, the highlighted (line 36) part shows where you can have as much environments you want, for each one create a file changing parameters, skipping (not in production) tests and so on. TestNG will run one file after another for you.

Conclusion

TestNG is really impressive, from very simple tests to complex environment tests, you have in hand a set of options to make it happen. When you mix TestNG with Selenium and Ant is possible to build very complex testing. TestNG goes beyond the "units" and provide what you need to think about "collections".