Functional Test Driven Development with Grails and WebTest

EDIT: Most of this functionality is now available in 0.6 Alpha: This is not a new idea by any stretch of the imagination, however, I thought I'd post about how I've been doing it recently. The Grails WebTest plugin comes with a script which will start your application, run all your test, and shut down the application. This is great for checking your code before checking in or for running on your continuous integration server. However, if you would like to use a WebTest to drive out some functionality it is far too slow. To get around this I have been using a custom script and parent class for my WebTests. The script is basically a copy of RunWebtest.groovy supplied by plugin with the start/stop application code removed. It also has a small section of code to parse a class patttern and method pattern as arguments. It needs to be placed in the plugins/webtest-0.5/scripts directory of your application so that it can access the required WebTest configuration and resources. This allows for a command such as: grails run-webtest-only MyDomain edit which will run all test methods containing the word edit but only those methods found in classes with a name containing the word MyDomain. In order for this to work, you also need to modify call-webtest.xml, found in the plugins/webtest-0.5/scripts directory. Add clonevm="true" to the java tag at line 10. This is required so that the system properties set in RunWebtestOnly.groovy are passed through the java process that actually runs the tests. This allows you to pick out one or two key tests that you are working on and run them quickly and repeatedly without restarting the application. It also has a nice side effect of forcing you to write robust, repeatable tests as you will be running the test multiple times against the same database without refreshing the data. The second part of the solution is the custom parent class for the WebTests. By default, the WebTest plugin requires you to add a suite method to your WebTest to specify the order in which to run the test cases contained within it. This is very useful if there are dependencies between the cases and testA must run before testB for example. To me, this is a bit of a smell that your tests are too fragile. If testA must run before testB in order to set up some state, refactor the code that creates that state into a method and re-use it in testB with unique values so that you can run testB regardless of database state. Of course there are always exceptions, especially when it comes to keeping test execution time to a minimum, and in reality it may be better to set up common data once and re-use it in several tests. My opinion above is given with a "in an ideal world" disclaimer. Apologies, I'm getting side tracked! What my custom parent class does is model JUnit in that it automatically builds a suite from all methods beginning in 'test'. It also applies the class and method filters you gave to the run-webtest-only script which are passed through from Gant via system properties. Now with the help of the WebTestRecorder firefox plugin I can start to drive out a new page, menu item or behaviour. I find this method very useful in forcing a methodical, logical approach to developing new code and also makes it very easy to 'do the simplest thing' as you drive all code changes from the front end from the users point of view. As usual, there are always exceptions, and you should always refactor and add any additional integration or unit tests required to cover exception handling or complicated logic that you cannot drive out from a WebTest. My internal dialog normally goes something like this: "Right, so I need to add a new field to X. First, I need to record browsing to the list screen and creating a test X" click, click, click, cut and paste "Ok, next I want to be able to enter a value for field Y when creating an X." setInputField(name:'Y',value:'someValueForY') clickButton('Save') "This should now fail..." run-webtest-only XTests editY "Excellent, I can now go and modify the GSP to add the input field" and so on and so on... Make sure you you only do enough at each stage to make the test pass, in the above example, I could go off and add the field to the domain class and update the controller if necessary, however all the test is making me do so far is add an input field to the GSP. Once the test is checking that the value of Y is persisted and displayed on the show or list screen, then I can go and make those changes. It can be hard to stay disciplined but hopefully you have access to a pair and can keep each other honest. I openly admit to cutting corners when working on my own. It's not my fault, Grails makes my actual coding so fast, the tests just slow me down!! ;)

Audi helps you time the lights!

The latest technology to come out of Audi shows the driver the perfect speed to drive in order to avoid red lights at upcoming traffic lights! Admittedly, the traffic lights in a German town were modified for the technology to work, but it's still an awesome idea! Source: http://blogs.cars.com/kickingtires/2008/07/audi-takes-on-s.html

Full dyno day results

Full results from the dyno day are now online: http://www.petrolmonkey.com/dyno-results.html

Dyno Day Result

I organised a second OZVR4 dyno day for today and it went very well. 19 Cars were there (17 8th gen and 2 6th gen). Shots from the day will hopefully end up in High Performance Imports (hpi.com.au). Here's my graph, I was 4th for day, which is pretty respectable seeing as I have no aftermarket fuel control like 1st, 2nd and 3rd do :)

Switchable Grails DataSource

As part of my day job I needed to write an administration app that could be easily pointed at multiple environments. To achieve this I used a technique I found a while ago based on Spring's AbstractRoutingDataSource. It basically works by using a facade to switch the connection between one to many child data sources. First off SwitchableDataSource.groovy

package com.leebutts 
import org.springframework.jdbc.datasource
                 .lookup.AbstractRoutingDataSource 
import com.leebutts.Environment 
import org.springframework.context.ApplicationContextAware 
import org.springframework.context.ApplicationContext 
import javax.sql.DataSource 
import org.springframework.jdbc.datasource.DriverManagerDataSource 

class SwitchableDataSource extends AbstractRoutingDataSource 
    implements ApplicationContextAware { 
    def applicationContext 
    
    public void setApplicationContext(ApplicationContext 
                                           applicationContext) { 
        this.applicationContext = applicationContext 
    } 

    protected DataSource determineTargetDataSource() { 
        DriverManagerDataSource ds = 
                        super.determineTargetDataSource(); 
        def env = EnvironmentHolder.getEnvironment() 
        if (env && env.passwordRequired && ds) { 
            ds.setPassword(env.password) 
        } 
        return ds 
    } 

    protected Object determineCurrentLookupKey() { 
        def env = EnvironmentHolder.getEnvironment() 
        return env?.id ?: Environment.list()[0]?.id 
    } 
}

SwitchableDataSource is the facade that delegates to the list of standard DriverManager data sources. They are defined in grails-app/conf/spring/resources.groovy using the environment settings from the Environment class. Here's my spring config in resources.groovy:

import com.leebutts.SwitchableDataSource 
import com.leebutts.Environment 
import org.springframework.jdbc.datasource.DriverManagerDataSource 

beans = { 
    parentDataSource(DriverManagerDataSource) {
        bean -> bean.'abstract' = true; 
        driverClassName = 'com.mysql.jdbc.Driver' 
        username = "root" 
    } 
    
    Environment.list().each {env -> 
        "${env.prefix}DataSource"(DriverManagerDataSource) {bean -> 
            bean.parent = parentDataSource 
            bean.scope = 'prototype' 
            def port = env.port ?: 3306 
            url = "jdbc:mysql://${env.host}:${port}/switchingDemo" 
            if (env.user) { 
                username = env.user 
            } 
            if (env.password) { 
                password = env.password 
            } 
        } 
    } 

    def dataSources = [:] 
    Environment.list().each {env -> 
        dataSources[env.id] = ref(env.prefix + 'DataSource') 
    } 

    dataSource(SwitchableDataSource) { 
        targetDataSources = dataSources 
    } 
}

Environment currently uses a static list to hold the environment config. This could be done better via a reloadable properties file or by adding a view/controller to modify the environment settings on the fly. Environment.groovy:

package com.leebutts

class Environment {

    static environments = []

    static {
        environments << [id: 1, name: 'local', prefix: 'local', 
                          host: 'localhost']
        environments << [id: 2, name: 'UAT', prefix: 'uat',
                            host: 'uat.leebutts.com']
        environments << [id: 3, name: 'Testing', prefix: 'testing',
                         host: 'testing.leebutts.com'] 
        environments << [id: 4, name: 'Beta', prefix: 'beta',
                         host: 'beta.leebutts.com', 
                         passwordRequired: true] 
        environments << [id: 5, name: 'Prod', prefix: 'prod', 
                         host: 'db.leebutts.com', user:'grails', 
                         port: 13306, 
                         passwordRequired: true] 

        //unique id check 
        environments.each {env ->
            assert environments
                .findAll {it.id == env.id}.size() == 1}
    }

    static list() {
        return environments
    }
}

SwitchableDataSource needs a way to determine which environment the current request wishes to use. It does this via a ThreadLocal holder EnvironmentHolder.groovy:

package com.leebutts

class EnvironmentHolder {

    private static final ThreadLocal contextHolder
                                = new ThreadLocal();

    static void setEnvironment(Map environment) {
        contextHolder.set(environment);
    }

    static getEnvironment() {
        return contextHolder.get();
    }

    static void clear() {
        contextHolder.remove();
    }
}

Now that the infrastructure is in place, the next step is to add a controller to allow users to select the environment they wish to use and a filter to set a default environment if one has not yet been chosen. The controller is called via ajax (in my application) but could be used as a standard controller just as easily. It looks up the Environment based on an ID and then tests the connection to make sure the password supplied is valid (if a password is required as specified in the environment config). If a password is being used it takes a copy of the environment config map, adds the password and stores it in the session so that the user doesn't have to re-enter the password on every screen and so that only this user has access to the password. EnvironmentController.groovy:

import com.leebutts.Environment
import com.leebutts.EnvironmentHolder
import javax.servlet.http.HttpServletResponse
import org.codehaus.groovy.grails.commons.ApplicationAttributes
import org.codehaus.groovy.grails.web.context.ServletContextHolder

class EnvironmentController {

    def change = {
        if (params.environment) {
            def env = Environment.list()
                     .find {it.id == new Integer(params.environment)}
            if (env) {
                if (env.passwordRequired) {
                    if (params.password) {
                        //take a copy and add a pword
                        env = addPasswordToEnvCopy(params, env)
                    } else {
                        render 'PASSWORD REQUIRED'
                        response.setStatus(
                          HttpServletResponse.SC_UNAUTHORIZED)
                        return
                    }
                }

                //test connection
                def oldEnv = EnvironmentHolder.getEnvironment()
                EnvironmentHolder.setEnvironment env
                def ds = getDataSourceForEnv()
                try {
                    def con = ds.getConnection()
                    session.environment = env
                    render 'Environment change complete.'
                } catch (e) {
                    EnvironmentHolder.setEnvironment oldEnv
                    render 'Unable to connect to database: '
                              + e.message
                    response.setStatus(
                       HttpServletResponse.SC_UNAUTHORIZED)
                    return
                }
            } else {
                render 'No such environment'
                response.setStatus(
                        HttpServletResponse.SC_BAD_REQUEST)
            }
        } else {
            render 'Missing parameter environment'
            response.setStatus(HttpServletResponse.SC_BAD_REQUEST)
        }
    }

    private def getDataSourceForEnv() {
        def servletContext = ServletContextHolder.servletContext
        def ctx = servletContext
                    .getAttribute(
                         ApplicationAttributes.APPLICATION_CONTEXT)
        return ctx.dataSource
    }

    private Map addPasswordToEnvCopy(Map params, env) {
        def myEnv = [:]
        env.each {key, val ->
            myEnv[key] = val
        }
        myEnv.password = params.password
        return myEnv
    }
}

As mentioned, there is also a simple filter for defaulting the environment to the first one in the list if one has not been selected and storing it in the ThreadLocal holder. Filters.groovy:

import com.leebutts.EnvironmentHolder
import com.leebutts.Environment

class Filters {
    def filters = {
        all(uri: '/**') {
            before = {
                if (!session.environment) {
                    session.environment = Environment.list()[0]
                }
                EnvironmentHolder.setEnvironment(session.environment)
            }
        }
    }
}

The final step is to add an environment selection form to your layout so that users can choose their environment. views/layouts/main.gsp:

<html>
<head>
    <title>Administration System</title>
    <link rel="stylesheet"
            href="${createLinkTo(dir: 'css', file: 'main.css')}"/>
    <g:layoutHead/>
    <g:javascript library="application"/>
    <g:javascript library="prototype"/>
    <script type="text/javascript">
        function refresh()
        {
            window.location.reload(false);
        }

        function loading() {
            document.getElementById('spinner').style.display = 'inline';
            document.getElementById('error').style.display = 'none';
        }
      
        function showError(e) {
            var errorDiv = document.getElementById('error')
            errorDiv.innerHTML = '<ul><li>'
                     + e.responseText + '</li></ul>';
            errorDiv.style.display = 'block';
        }
    </script>
</head>
<body>
<div class="logo">
    <div style="margin-left:10px;">
        <h1>Current Environment: 
               ${session.environment?.name ?: 'None'}</h1>
        <form action="">
            <g:select name="environment" 
                   from="${com.leebutts.Environment.list()}" 
                      optionKey="id" optionValue="name" 
                       value="${session.environment?.id}"/>
            <g:passwordField name="password"/>
            <g:submitToRemote value="Select" controller="environment" 
             action="change" update="currentEnv"
             onLoading="loading();"
             onComplete
               ="document.getElementById('spinner').style.display='none';"
             onFailure="showError(e)"
             onSuccess="refresh()"/> <br/>
            <div class="errors" id="error" 
              style="display:none;width:500px;">

            </div>
            <img id="spinner" style="display:none;"
              src="${createLinkTo(dir: 'images', file: 'spinner.gif')}"
              alt="Spinner"/>
        </form>
    </div>
</div>
<g:layoutBody/>
</body>
</html>

The finished screen looks something like this:







As this is an internal admin application it has not been tested as thoroughly as standard prod code so please do so before using it in a "live" application. Please post any corrections/bug fixes.

Drag racing photos

I found some "official" of me at Willow bank the other day

Unreal 3D Car Art

I think this is the best CG car art I've ever seen! Deviant art gallery: http://dangeruss.deviantart.com Article about him here: http://www.nextautos.com/sexy-digital-sheetmetal-the-art-dangeruss

New PB at Willowbank yesterday

I went out to Willowbank Raceway yesterday for some 1/4 mile fun. My best time was 13.791, about 1/2 a second quicker than last time thanks to a new exhaust, front mount intercooler and clutch.

I haven't found any video of my runs but here's some I took of the other VR-4s and Evos that were there. The last clip is of an R35 GT-R which ran an 11.3!

The only SL Mercedes I'd be seen in

Definitely not a hair dresser's car.... AMG SL65 Black

Honey, I shrunk my drift car

One word. Awesome: http://speedhunters.com/archive/2008/07/10/random-snap-gt-gt-drifting-kei-truck.aspx

Photos from BarCamp

DJ has posted his great photos from BarCamp. They are in his photostream starting from here: http://flickr.com/photos/djp72/2644670844/ Hopefully he'll tag them all soon. My two favourites below :)

Some initial Grails Portlets documentation

I've started documenting how the Grails Portlets plugin currently (hopefully?) works. http://grails.org/Portlets+Plugin You can try it out (with some effort) by checking it out from subversion: http://svn.codehaus.org/grails-plugins/grails-portlets/trunk. You'll also need to checkout and build a Grails 1.1 snapshot: http://svn.codehaus.org/grails/branches/GRAILS_1_1

BarCamp Gold Coast

I had a great time at BarCamp Gold Coast yesterday. A big thanks to the un-organisers!