How I used Zinc and Soup with Sunit to automate Seaside UI testing

This is a revision of the prior article replacing WebClient with Zinc.

The Zinc package includes a simple and easy to use HTTP client. Soup is a port of the Beautiful Soup Python HTML/XML parser designed for screen-scraping. The combination of these two made it possible to perform black-box testing of my Seaside web application.

My environment is Seaside 3.0.3 and Pharo 1.1.1.

Load the Zinc and Soup packages

Evaluate the following statements to load the Zinc and Soup packages.

Gofer it
    squeaksource: 'ZincHTTPComponents';
    package: 'Zinc-HTTP';
    package: 'Zinc-Tests';
    load"

Gofer it
    url: 'http://www.squeaksource.com/Soup';
    package: 'Soup';
    load.

Prepare the Seaside app for automated testing

While Soup is designed to allow you to navigate the HTML of any arbitrary web page, automated testing can be made much easier with a little cooperation from the system under test. My goal was to make every input element that will be involved in the testing as easy as possible to identify and make the tests resistant to unimportant changes in the application such as re-wording or translation of screen text.

To do this, I assigned a unique ID attribute to every input element in my forms. I think it is also a good idea to assign a unique ID attribute to the form itself, especially if there may be more than one form on a page.

Similarly, to determine whether a particular text message is present on a page, it is easier and more resistant to change if the enclosing div has a unique ID. Soup can find text, but if the message is re-worded or translated in the future and the test is looking for the old wording, the test will fail unnecessarily.

It is tedious to go back and retro-fit this into existing code, but it is very little trouble if it is done when the form is first coded.

Here's an example:

    html form
        id: 'registerform';
        with: [
            userExistsError ifTrue: [
                html div
                    class: 'registererror';
                    id: 'usernameexistserror'; 
                    with: 'That username is already taken. Try another.']
            html table: [
                html tableRow: [
                    html tableData: 'Username:'.
                    html tableData: [
                        html textInput
                            id: 'registeruser';
                            callback: [ :v | user := v ]].
                html tableRow: [
                    html tableData: 'Password:'.
                    html tableData: [
                        html passwordInput
                            id: 'registerpswd';
                            callback: [ :v | pswd := v ]];
                . . .

CSS class names may be duplicated, while IDs should not be. This makes IDs a better choice for identifying elements in an automated test in an unambiguous way.

The IDs added to the above form for automated testing are registerform, usernameexistserror, registeruser, and registerpswd.

The testing sequence

The basic sequence that I have employed to test a Seaside web app is as follows.

1. Request the first page with an HTTP GET.
2. Verify that we received the expected page.
3. Provide values for form elements in an HTTP POST request to the URL specified in the form's action attribute.
4. Receive the HTTP code 302 response from Seaside that is normally returned after the callbacks have executed. (This is done automatically by the Zinc client.)
5. Request the page at the URL provided in the Location header attribute of the 302 response with an HTTP GET request. (This is done automatically by the Zinc client.)
6. Verify that the page we receive contains the expected content.
7. Repeat from step 3 until we reach the target page.

SetUp and TearDown

In my TestCase subclass #setUp method I placed code like the following. My goal here was to help other who might load my package figure out why the tests might be failing.

setUp
    "We will need the port on which our Seaside server is listening."
    self port: WAKomEncoded default port.		

    "Make sure the Seaside server is up and running."
    self
        assert: (WAKomEncoded default isRunning)
        description: 'This test needs a Seaside server.'.

    "Make sure the packages we need are present."
    self
        assert: ((Smalltalk at: #Soup ifAbsent: [nil]) isNil not)
        description: 'This test requires Soup: http://www.squeaksource.com/Soup'.
    self
        assert: ((Smalltalk at: #ZnClient ifAbsent: [nil]) isNil not)
        description: 'This test requires Zinc http://www.squeaksource.com/Zinc'.

   "Here's the web client we'll be using."
    znClient := ZincHttpClient new.
    znClient settings timeout: 5.

Utility methods

I created the following methods to allow my tests to focus on the information pertinent to the test and make them concise and easy to read.

baseUrl
   "Answer the base URL for our local Seaside server."
   ^ 'http://localhost:', self port asString

findForm: content
   "Find the first form on the page or nil."
   ^ (Soup fromString: content) findTag: 'form'

findTag: tag in: content withId: anId
   "Answer the tag with id anId or nil."
   ^ ((Soup fromString: content) findAllTags: tag) detect: [ :each |
       (each attributeAt: 'id') = anId] ifNone: [ nil ]

findForm: content withId: anId
   "Answer the form with id anId or nil."
   ^ self findTag: 'form' in: content withId: anId

findTag: tagName In: content withClass: aClassName
   | tag |
   ^ (tag := ((Soup fromString: content) findTagByClass: aClassName)) name = tagName
       ifTrue: [ tag ]
       ifFalse: [ nil ]

findTag: tagName In: content withId: id
   ^ ((Soup fromString: content) findAllTags: tagName)
           detect: [ :each | (each attributeAt: 'id') = id ]
           ifNone: [ nil ]

findFormIn: content withClass: className
   ^ self findTag: 'form' In: content withClass: className 

actionUrlFor: aForm
   "Answer the complete action URL attribute of the form."
   ^ self baseUrl, (aForm attributeAt: 'action'). 

namesById: aForm
   "Answer a dictionary of all input and button tag name attributes keyed
   by id. The names are assigned by Seaside and are used to identify values
   sent when POSTing the form to the server."
   | tags dict |
   dict := Dictionary new.
   tags := #( 'input' 'button').
   tags do: [ :eachTag |
       (aForm findAllTags: eachTag) do: [ :each |
           | id |
           id := each attributeAt: 'id'.
           id isNil ifFalse: [ dict at: id put: (each attributeAt: 'name')]]].
   ^ dict

clickOn: anId in: content with: valuesById
   "Send the POST request that would be sent when a user clicks on the
   form element with anId after filling in the values contained in the
   valuesForId dictionary."

   | form names actionUrl args |
   form := self findForm: content.
   actionUrl := self actionUrlFor: form.
   names := self namesById: form.
   args := Dictionary new.
   valuesById
       keysAndValuesDo: [ :k :v | 
           args at: (names at: k) put: v].
   args at: (names at: anId) put: ''.
   ^ (znClient post: actionUrl data: args) contents

getStartPage
    "A convenience method to get the starting page of the web app."
    ^ (znClient get: self baseUrl, '/', appName) contents.

clickOn: anId in: content
   "Convenience method for when there is no form content."
   ^ self clickOn: anId in: content with: Dictionary new.

Finally, a test case

Here's a test case for a user registration form. First we connect to the server and click on the Register button with the ID registerbtn. In the page that is returned, we fill in the form and click on the Register button in that form (it happens to have the same ID.) Finally, we verify that a div with the class registrationok is present in the returned page.

testRegistration
   | content |

   content := self getStartPage.
   self assert: (self findForm: content withId: 'loginform') notNil.
   content := self clickOn: 'registerbtn' in: content.
   self assert: (self findForm: content withId: 'registerform') notNil.
   content := self
       clickOn: 'registerbtn'
       in: content
       with: ((Dictionary new)
           at: 'registeruser' put: 'TestUser1';
           at: 'registerpswd' put: 'testpass';
           at: 'registerconfirmpswd' put: 'testpass';
           at: 'registeremail' put: 'myemail@server.com';
           yourself).
   self assert: ((Soup fromString: content) findTagByClass: 'registrationok') notNil.

Conclusion

I found that if I write the tests at the same time as I code the forms, this all goes pretty fast. The advantage is that instead of using the browser for minutes to manually test the few things that I think I might have affected, I can run a whole regression suite in a few seconds.

There is of course no substitute for actually using your web app from a browser, but after making a change to the code, seeing all my tests run green gives me some confidence that I didn't break anything inadvertently.