10x More Productive Blog!

In a typical enterprise application, there are many areas requiring testing. Starting from the simplest components, classes, the developers or specialized test developers will need to program unit tests to ensure that the smallest units of the application behave correctly. Potentially each component can pass the unit tests alone - however the developers need to make sure that they work together as expected, as part of a subsystem, and as part of the whole application, hence integration tests must be performed. In some projects performance requirements must be fulfilled, so the QA engineers perform load tests to verify and document how the application performs under various conditions. During the development of the application, QA engineers perform automated and manual functional tests to test the behavior of the application from the view point of the end user. When a development project is nearing completion of a specific milestone, acceptance tests can be performed to verify that the application fulfilled the requirements.

HttpUnit is a framework based on jUnit, which allows the implementation of automated test scripts for web applications. It is best suited for the implementation of automated functional tests, or acceptance tests. As the name suggests, it can be used for Unit testing, however my opinion is that typical web layer components like JSP pages, servlets, or any other template components do not lend themselves to unit testing too well. As for various MVC framework based components, these are better suited for testing with other test frameworks. Struts actions can be unit tested with StrutsUnit, and WebWork 2 actions can be unit tested without a web container for example.

Before we jump into the architecture and implementation details, it's important to clarify exactly what the test scripts will need to prove about the web application. It is possible to just simulate the behavior of a casual web site visitor, just clicking on interesting links, and reading pages in a random order, but the result of these random scripts would not describe the completeness and the quality of the application.

A typical enterprise web application (or a complex web site) has several documents describing the requirements of the various users or maintainers of the application. These may include Use Case specifications, Non-functional requirements specifications, Test Case specifications derived from the other artifacts, User Interface Design documents, mockups, Actor Profiles, and various additional artifacts. For a simple application, the whole specification could possibly consist of a simple text file with a list of requirements.

From these documents, we have to create an organized list of Test Cases. Each Test Case would describe a scenario which can be carried out by a web visitor through a web browser. It is a good practice to aim for similar sized scenarios - larger scenarios can be broken down to smaller chunks. There are many excellent books and articles regarding the creation of Test Case specifications. For this article, let's assume that you have a set of things you want to test for your web application, organized into sets of Test Case scenarios.

OK, now we know the boring stuff, let's download some cool toys! First of all, we need an installed Java 2 SDK, to be able to compile and execute our tests. Then we need to download the HttpUnit framework - currently at version 1.5.5. The binary package contains all the required third party libraries. We will also need the Ant build tool to run the tests and generate reports automatically. Probably any fairly recent version of these tools would work; I just prefer to use the latest-and-greatest version from everything.

To write and execute our tests, I would recommend using an IDE which has a jUnit test runner embedded. I use Eclipse 3.0 M7 to develop my test scripts, but intelliJ has jUnit support as well, as do most recently released IDEs.

As we want to test web applications, ideally the test tool should behave exactly as the web browsers of the end users would. Our application (the target of the test) should not be aware of any difference when serving up pages to a web browser or the test tool. That's exactly what HttpUnit provides: it simulates the GET and POST requests of a normal browser, and provides a nice object model to code our tests against.

Figure 1. The HttpUnit classes I use frequently

Check out the detailed API guide for the rest of the classes and methods, this is just a brief overview for the classes I use most frequently. A user session (a sequence of interactions with the web application) is encapsulated with a WebConversation. We construct WebRequests, typically configuring the URL and the parameters, and then we send it down through the WebConversation. The framework then returns a WebResponse, containing the returned page and attributes from the server.

Here's a sample HttpUnit test case from the HttpUnit docs:

    /**
     * Verifies that submitting the login form with the name "master" results
     * in a page containing the text "Top Secret"
     **/
    public void testGoodLogin() throws Exception {
        WebConversation     conversation = new WebConversation();
        WebRequest  request = new GetMethodWebRequest( 
	  			"http://www.meterware.com/servlet/TopSecret" );

        WebResponse response = conversation.getResponse( request );
        WebForm loginForm = response.getForms()[0];
        request = loginForm.getRequest();
        request.setParameter( "name", "master" );
        response = conversation.getResponse( request );
        assertTrue( "Login not accepted", 
	           response.getText().indexOf( "You made it!" ) != -1 );

        assertEquals( "Page title", "Top Secret", response.getTitle() );
    } 
  

Notice how the Java sample above contains the domain name of the server running the application. During the development of a new system, the application lives on multiple servers, and the servers my run multiple versions. Obviously it's a bad idea to keep the server name in the Java implementation - for each new server we'd need to recompile our sources. There are some more items that should not live in the source files: things like user names, passwords should be configurable for the specific deployment. On the other hand, we should not over-architect a simple Test Case implementation. Normally the TC specification already contains most of the system state and specific parameter descriptions for our scenario, so there's no point making everything parameterizable in the implementation.

During the coding, you'll realize that there are many code sections which appear in more than one TC implementation (potentially in all of the TCs). If you're an experienced OO developer, there'll be great temptation to create class hierarchies and create common classes for these things. In some cases, that makes a lot of sense - for example the login procedure should be a common method available for all the TCs. However you need to step back a bit and realize that we're not building a new production system on top of the target-of-test application - these Java classes are no more than test scripts to validate the output of the web site. Exercise common sense, and aim for simple, sequential, and self-contained test scripts. The TCs are typically very fragile. If a developer changes an URL, reorganizes the <table> structure of the layout, or changes an ID for a form element, the human visitor will probably not see any difference, but our test scripts WILL be blown. Expect a lot of rework and change for each TC implementation. OO design could reduce the effort of reworking common parts in the TCs, but from the perspective of a QA Engineer or tester, I'm sure that a simple, sequential script that interacts with a web site is easier to maintain and fix.

Traceability is very important for our Test Cases. If something goes KA-BOOM, or a for example a calculation result is wrong, it's important to point the developer to the corresponding Test Case specification, and the Use Case specification for a quick bug resolution. Therefore it's a good idea to annotate our implementation with references to the original specification documents. It's also useful to include the version number of those documents. This could be just a simple code comment, or a complex mechanism where the test reports themselves link to the documents; the important thing is to have the reference in the code, and to keep the traceability.

Now that we're aware of the requirements (Use Case docs, and corresponding Test Case specifications), we understand the Framework's basics, and we have a set of architectural guidelines, let's get to work.

For the development of the Test Case implementations, I prefer to work in Eclipse. First of all, it has a very nice jUnit test runner. You can select a Java class, and from the Run menu, you can run it as a jUnit Unit Test. The runner will display the list of recognized test methods, and the result of the execution. When everything was OK during the test run, it will give us a nice green line. In case an Exception occurred, or an assertion failure happened, it will display a very distressing red line. I think the visual feedback is really important - it gives a nice sense of accomplishment especially when writing Unit Tests for our own code. I also like to use Eclipse for the nice refactoring capabilities. If I realize that within a Test Case class I need to copy-and-paste code sections, I can just use the Refactoring menu to create a method from the code section instead. If I realize that a large number of TCs would use the same method, I can use the menu to pull up my method into my base class.

Based on the architectural requirements above, for each project I typically create a base Test Case class, which extends the jUnit TestCase class. I call it ConfigurableTestCase. Each Test Case implementation extends this class.

Figure 2. Configurable Test Case implementations

ConfigurableTestCase typically contains the common methods, and initialization code for the test case. I use a property file to store the server name, the application context, various login names for each role, and some additional settings.

The specific Test Case implementations will contain one test method per Test Case scenario (from the Test Case specification document). Each method typically logs in with a specific Role, and then executes the interaction with the web application. Most Test Cases do not need a specific user to carry out the activities, they typically require a user in a specific Role, like Administrator, or Visitor, or RegisteredUser. I always create a LoginMode enum, which contains the available roles. I use the Jakarta Commons ValuedEnum package for creating Enums for the Roles. When a specific test method in a TC implementation logs in, it has to specify which Login Role is required for that particular test scenario. Of course it should be also possible to log in with a specific user, for example to verify the "Register User" Use Case...

After each request and response cycle, we typically need to verify if the returned page contains an error, and we need to verify our assertions about what content the response should contain. We need to be careful here as well: we should only verify things that are not variable, and are not too fragile in the application. For example, if we assert specific page titles, our tests will probably not run if the language is selectable in the application, and we want to verify a different language deployment. Similarly, there's little point checking an item on the page based on its position within a table layout; table based designs change frequently, so we should strive to identify elements based on their IDs. In case some important elements on the page don't have IDs or names, we should just ask the developers to add one, rather than trying to work around it. JUnit assertions are a poor way to check if the look-and-feel, the layout, and the design of the page are compliant to the requirements. It could be done, given an infinite amount of time for the test development, but a good human tester can assess these things way more efficiently. So concentrate on verifying the functionality of the web application, rather than checking everything possible on the page.

Here's an updated test scenario based on our TC architecture. The class extends ConfigurableTestCase, and the login details are handled in the base class.

    /**
     * Verifies that submitting the login form with the name "master" results
     * in a page containing the text "Top Secret"
     **/
    public void testGoodLogin() throws Exception {
        WebConversation     conversation = new WebConversation();
        WebResponse response = login(conversation, LoginMode.ADMIN_MODE);
	assertTrue( "Login not accepted", 
		    response.getText().indexOf( "You made it!" ) != -1 );
        assertEquals( "Page title", "Top Secret", response.getTitle() );
    } 
  

Most scenarios can be handled quite easily by setting WebForm parameters, and then looking for specific elements with results in the WebResponse pages, but there are always some challenging Test Cases.

Now we know how we can write and execute our tests in the IDE. However these tests should be also executed automatically from time to time. Based on the development method, this could be after a daily build, or during a testing period, but the best is to execute the full Test Suite or at least a subset many times a day, hourly for example, or after each commit in the Version Control system.

Most web applications require a database, and the Test Case scenarios typically have pre-conditions, or initial system state. So any automated test run should be executed on a cleanly built system, using a fresh database, which may already contain some test data specified in the Test Cases. Actually there are other alternatives, like each Test Case can prepare the test data for itself during the setup method, but I prefer to have the DB scripts inserting the test data. This way the QA Engineers manually testing the application can see the same initial state and configured pre-conditions for their tests.

Some tests may put the system in an invalid state. Typically Web Applications should not allow input which can cause a system-wide failure, but perhaps the reason of the automated Test Case is to highlight such an issue in the application. Running this TC with the rest of the Test Cases could heavily influence many other tests. These "viral" tests can be excluded from the batch test run, and they can be run separately, restoring the initial database state after running them.

The Ant build system has everything that is needed to do these test runs: there are available tasks which can load up the test data in the database, there is a <junit> task which can execute our test cases, and there is a task to format a HTML report from the result of our Test Cases. So all we need to do is to create a build.xml file, which compiles the TC classes, executes the DB init scripts, executes the tests, and then outputs the report for us. It's useful to have up-to-date HTML reports available on a project-specific intranet site. This way the developers and the management can see the status of the application.

Here's a sample section from an Ant build.xml file:

<!-- runs all unit test -->
<target name="junit" depends="jar_test">
      <mkdir dir="${log.dir}" />
             <junit printsummary="yes" haltonfailure="no">
	     <classpath refid="compile.class.path"/>
	     <classpath>
	     <fileset dir="${dist.dir}" includes="*.jar" />
	     <pathelement location="$(conf.dir}" />
	     </classpath>
            <batchtest fork="yes" todir="${log.dir}">
          <formatter type="xml" />
          <fileset dir="${src.test.dir}">
            <include name="**/*Test.java" />
          </fileset>
        </batchtest>
      </junit>
    <junitreport todir="${log.dir}">
         <fileset dir="${log.dir}">
           <include name="TEST-*.xml"/>
         </fileset>
         <report format="frames" todir="${log.dir}"/>
    </junitreport>
</target>

While writing unit tests is a popular and fashionable activity, most software developers I know either hate or simply don't like to do functional testing on other people's applications. While using HttpUnit will not replace manual testing, it is nevertheless a great way to create automated tests programmatically, with much more freedom than the typical visual test scripting environments. While tools recording the Browser activity tend to record every insignificant detail, with big and hard-to-modify recorded scripts, using HttpUnit will allow a developer to create readable, concise, and cleanly structured code, which can focus on validating the important details. So if you get a task to do functional tests for a few weeks, I can assure you that you'll encounter enough challenges to keep you interested, and to enjoy your work.