Grails Amazon S3 0.7.4 Released

see http://refactor.com.au/blog/grails-amazon-s3-074-released

Adding pagination to a Grails WebFlow state

I noticed a few unanswered questions relating to paginating a WebFlow state on the users mailing list and thought I'd post the solution I came up with for a Refactor project I was working on last week.

It basically involves splitting the behaviour into an action and a view state with a transition between them called 'paginate'

loadResults {
 action {
    def criteria = MyDomain.createCriteria(){
        eq('someField','someval')
    }
    flow.myDomainResults  = criteria.list([max: 9, offset: flow.myDomainOffset ?: 0])   
    flow.totalMyDomains = flow.myDomainResults.totalCount
    return success()
 }
 on('success').to('browse')
 on(Exception).to "handleError"
}
browse {
 on('paginate') {
    flow.myDomainOffset = params.offset
 }.to('loadResults')
 on('selectItem').to('someOtherState')
 //...
}

One point to note is that I use a criteria in order to get the totalCount property populated for me automatically so that I don't have to run a separate query. Also, the above only works if your domain class is Serializable, in my real application this wasn't the case so I stored a list of maps containing the properties I wanted to display instead of the list of domain objects.

Then in the view all we need to do is to pass the current page from the flow scope to the paginate tag. The paginate tag also contains an extra parameter in order to trigger the paginate event when the generated links are clicked.

<g:each in="${myDomainResults}" status="i" var="myDomain">
    <g:form action="myFlow">
        <input name="id" value="${myDomain.id}" type="hidden">
        URL: ${myDomain.url}
        <g:submitbutton name="selectItem" value="Select"/>
    </g:form>
</g:each>
<div class="paginateButtons">
   <g:paginate max="9" offset="${myDomainOffset}" total="${totalMyDomains}" params="${[_eventId_paginate:true]}" />
</div>

You can also see I render each item as a separate form with a hidden id field and a submit button to trigger the 'selectItem' event and move onto the next state.

I haven't tried messing with all the options for g:paginate to make sure they still work so please let me know if you find any problems with the above code.

Grails JSR-186 Portlets Plugin Released

Kenji Nakamura has taken the reins of my initial work on the Grails Portlets plugin and done an awesome job of refactoring it into more manageable sub-plugins.

You can now plug-in portal implementations with Pluto and Liferay supported initially.

JSR-286 will be supported when the underlying version of Spring used by Grails is updated to Spring 3.0.

Check it out at the Grails plugin portal.

Great work Kenji!

WebTest Plugin 1.1.4.2 Released

A new minor release of the Grails WebTest plugin to allow the plugin to download the latest WebTest snapshot (which uses HtmlUnit 2.5 which fixes issues with testing JQuery 1.3.2).

It will download the latest snapshot by default when installed. If you would prefer the latest stable release (3.0) you can answer no when prompted.

Testing a JQuery-UI Drag and Drop using WebTest

Just a quick post regarding using WebTest with JQuery-UI Draggable and Droppable.
The way that WebTest performs a drag and drop is as follows:

from.mouseDown()
to.mouseOver()
to.mouseUp()

Note that there is no intermediate move events between mouse down on the thing you're dragging and mousing over the target element.
What this means is that you need to configure your Draggable to start dragging on mousedown without delay. You can do this by setting the distance and delay options to zero (delay is zero by default):

$('.block').draggable({distance:0});

WebTest will now be able to click and drag it correctly. Unfortunately due to HtmlUnits 'renderless' behaviour you cannot specify exactly where you want to drop it so if you require precise targeting to properly test your application you're out of luck.

Grails WebTest Plugin 1.1.4 Released

Another WebTest plugin release, this time with some new functionality!

Soenke Sothmann has contributed a nice patch to allow you to run custom suites of tests:

run-webtest -suite=MyTestSuite

Note that under windows you will need to quote the -suite argument due to an issue with Gant's argument parsing e.g. "-suite=MyTestSuite"

import grails.util.WebTest

class MyTestSuite extends WebTest {

    static void main(args) {
        new MyTestSuite().runTests(args)
    }

    void suite() {
        new SomeTest1(ant:ant).suite()
        new SomeTest2(ant:ant).suite()
        // You could also load up certain tests based on naming conventions or annotations
    }
}

In addition to this patch, there were some bug fixes relating to running webtests under Hudson and also around setting a custom port when starting up the server.

Sydney Groovy/Grails User Group

In between meeting all the interesting folk at OSDC, I attended the Sydney Groovy (and Grails) User Group. I presented a short talk on the new features in Grails 1.1 then demoed the Portlets Plugin and new features in the WebTest Plugin 0.6. Dave Peterson then gave a great Intro to Gant talk with comparisons to buildr. Come along next time to hear more from Dave about Gradle :) The group is organsied by Nick Carrol from Thoughtworks and the venue (in The Rocks near Sydney harbour) was great. If you're ever in the area, get onto the mailing list and see when the next meeting is.

Grails WebTest Plugin 0.6 alpha

I've been working on committing some improvements to the Grails WebTest Plugin. You can download the alpha version here (I haven't yet released it to the plugin repository). I'd appreciate feedback from users with existing applications with non-trivial webtests regarding any regressions or upgrade issues they have. If all looks ok I will update the official repository version shortly.

Release Notes

• setUp/tearDown at the method and class level

  • classSetUp() and classTearDown() are run as individual test cases. SetUp() and tearDown() are run as the first and last steps of each test case.

• New superclass AutoWebTest

  • This new superclass will automatically run all methods starting with test. This saves you having to manually maintain the suite method unless you really want to for test order reasons.

  • AutoWebTest will also generate the test case name from the class and method name removing the need for repetitive webtest('blah'){...} code. The generated test name also makes it much easir to find the failing test from the generate reports.

  • MethodMissing code has been added so you can refactor a group of steps without having to wrap them in and ant.group closure.

  • You can now call config() as the first step in your test method to set WebTest options like host, port and ajax support

• -nostart option allows you to runthe tests against a server that is already running. It should come after run-webtest on the command line

• System parameters now passed through to WebTest. They need to be placed directly after grails on the command line e.g. grails -Dwt.headless=true run-webtest

  • -Dwt.headless=true to hide Swing monitor and stop browser launching

  • -Dserver.port=XXXX to get the tests to run against a server on a non-default port

• The plugin has been updated with the latest WebTest release which includes an update of HtmlUnit to version 2.3

• Application lib folder now on WebTest classpath. This avoids the need to duplicate/move libraries into webtest/home/lib

• Custom steps

  • You can now extend com.canoo.webtest.steps.Step by placing groovy classes in webtest/tests/step. They will be automatically loaded at runtime and allow for easy testing of complicated scenarios such as JSON interfaces and email integration

    • The last project I worked on used these custom steps to start, check then stop an embedded Wiser SMTP server for testing email functionality.

Upgrade Instructions

delete plugins/webtest-0.x

svn delete webtest/home, commit.

This avoids svn issues as the install script deletes the folder and extracts the latest build over the top, removing the .svn directories

grails install-plugin grails-webtest-0.6.zip

You need to copy the zip file into the root folder of your project and run the command there.

Open Source Developers Conference - Sydney Dec 2008

Myself and colleagues from Refactor will be attending the Open Source Developers Conference (OSDC) in Sydney from 2-5 Dec. It's a great conference to get in touch with developers who use and contribute to open source and find out what's on the horizon. I didn't get organised early enough to present any main sessions on Grails but am keen to do some lightening talks about it and also demo Balsamiq MockUps which I came across a few weeks ago. I think it's a great tool for agile BA's and product owners to really get their customers engaged. If you see me there, make sure you come over and say hi!

How to WebTest a site using an invalid SSL certificate

UPDATE: there is a option you can pass to the config step which should achieve the same thing: useInsecureSSL

A very short post regarding an issue I recently came across while testing an application I'm working on.

My application needs to interface with and existing PHP application that uses SSL. To acceptance test this functionality I am writing a WebTest that drives both applications to assert information is flowing correctly between them.

Unfortunately the test instance of the PHP application I have been given to use has a self-signed SSL certificate which causes WebTest to fail with a SSLHandshakeException.

To ignore the self signed certificate, add the following line to your test:

groovy('step.context.currentWebClientContext.webClient.useInsecureSSL = true')

Having trouble installing the Quartz plugin? Read on...

For those of you who don't follow the grails-users list, I thought I'd post a quick solution to solving an odd-looking compile error when installing the Quartz plugin into a Grails application that also uses the JSecurity plugin.
The compile error you will see is:

GrailsJobFactory.java:75: cannot find symbol
symbol  : constructor JobExecutionException(java.lang.String,java.lang.Exception)
location: class org.quartz.JobExecutionException
    throw new JobExecutionException(e.getMessage(), e);

It's caused by a library clash between the two plugins as JSecurity also ships with quartz.jar. Luckily (according to Les from the JSecurity project), JSecurity doesn't actually rely on that jar so you can delete it from plugins/jsecurity-x.x/lib which will solve the compile issue.

Jasper Reports Grails Plugin - sub-reports

Here's something that tripped me up when developing my first JasperReport containing a sub-report for use with the JasperReport grails plugin. In order for the sub-report to resolve correctly you need to add a parameter to your parent report called SUBREPORT_DIR. This is then set by the JR plugin, and makes everything work as expected. When specifying the sub-report location, set it to

$P{SUBREPORT_DIR}  + "mysubreport.jasper"

(assuming your subreport is in the same directory as the parent report and called mysubreport.jasper) Don't forget to give the parameter a default value of empty string so that it still works when previewing in iReport. You also need to remember that the plugin runs the compiled versions (*.jasper not *.jrxml) so you need to compile the reports with iReport before running the application.

Jasper Reports Grails Plugin Gotcha

Thought I'd blog this quickly: If you're using iReport to develop reports for use with the Jasper Reports grails plugin, make sure you download the version that matches the version of Jasper that the plugin uses (currently 2.0.5). Otherwise you get terribly unhelpful exceptions like:

java.lang.NullPointerException
 at net.sf.jasperreports.engine.JRPropertiesMap.readObject(JRPropertiesMap.java:185)

Functional Test Driven Development with Grails and WebTest

EDIT: Most of this functionality is now available in 0.6 Alpha: This is not a new idea by any stretch of the imagination, however, I thought I'd post about how I've been doing it recently. The Grails WebTest plugin comes with a script which will start your application, run all your test, and shut down the application. This is great for checking your code before checking in or for running on your continuous integration server. However, if you would like to use a WebTest to drive out some functionality it is far too slow. To get around this I have been using a custom script and parent class for my WebTests. The script is basically a copy of RunWebtest.groovy supplied by plugin with the start/stop application code removed. It also has a small section of code to parse a class patttern and method pattern as arguments. It needs to be placed in the plugins/webtest-0.5/scripts directory of your application so that it can access the required WebTest configuration and resources. This allows for a command such as: grails run-webtest-only MyDomain edit which will run all test methods containing the word edit but only those methods found in classes with a name containing the word MyDomain. In order for this to work, you also need to modify call-webtest.xml, found in the plugins/webtest-0.5/scripts directory. Add clonevm="true" to the java tag at line 10. This is required so that the system properties set in RunWebtestOnly.groovy are passed through the java process that actually runs the tests. This allows you to pick out one or two key tests that you are working on and run them quickly and repeatedly without restarting the application. It also has a nice side effect of forcing you to write robust, repeatable tests as you will be running the test multiple times against the same database without refreshing the data. The second part of the solution is the custom parent class for the WebTests. By default, the WebTest plugin requires you to add a suite method to your WebTest to specify the order in which to run the test cases contained within it. This is very useful if there are dependencies between the cases and testA must run before testB for example. To me, this is a bit of a smell that your tests are too fragile. If testA must run before testB in order to set up some state, refactor the code that creates that state into a method and re-use it in testB with unique values so that you can run testB regardless of database state. Of course there are always exceptions, especially when it comes to keeping test execution time to a minimum, and in reality it may be better to set up common data once and re-use it in several tests. My opinion above is given with a "in an ideal world" disclaimer. Apologies, I'm getting side tracked! What my custom parent class does is model JUnit in that it automatically builds a suite from all methods beginning in 'test'. It also applies the class and method filters you gave to the run-webtest-only script which are passed through from Gant via system properties. Now with the help of the WebTestRecorder firefox plugin I can start to drive out a new page, menu item or behaviour. I find this method very useful in forcing a methodical, logical approach to developing new code and also makes it very easy to 'do the simplest thing' as you drive all code changes from the front end from the users point of view. As usual, there are always exceptions, and you should always refactor and add any additional integration or unit tests required to cover exception handling or complicated logic that you cannot drive out from a WebTest. My internal dialog normally goes something like this: "Right, so I need to add a new field to X. First, I need to record browsing to the list screen and creating a test X" click, click, click, cut and paste "Ok, next I want to be able to enter a value for field Y when creating an X." setInputField(name:'Y',value:'someValueForY') clickButton('Save') "This should now fail..." run-webtest-only XTests editY "Excellent, I can now go and modify the GSP to add the input field" and so on and so on... Make sure you you only do enough at each stage to make the test pass, in the above example, I could go off and add the field to the domain class and update the controller if necessary, however all the test is making me do so far is add an input field to the GSP. Once the test is checking that the value of Y is persisted and displayed on the show or list screen, then I can go and make those changes. It can be hard to stay disciplined but hopefully you have access to a pair and can keep each other honest. I openly admit to cutting corners when working on my own. It's not my fault, Grails makes my actual coding so fast, the tests just slow me down!! ;)

Switchable Grails DataSource

As part of my day job I needed to write an administration app that could be easily pointed at multiple environments. To achieve this I used a technique I found a while ago based on Spring's AbstractRoutingDataSource. It basically works by using a facade to switch the connection between one to many child data sources. First off SwitchableDataSource.groovy

package com.leebutts 
import org.springframework.jdbc.datasource
                 .lookup.AbstractRoutingDataSource 
import com.leebutts.Environment 
import org.springframework.context.ApplicationContextAware 
import org.springframework.context.ApplicationContext 
import javax.sql.DataSource 
import org.springframework.jdbc.datasource.DriverManagerDataSource 

class SwitchableDataSource extends AbstractRoutingDataSource 
    implements ApplicationContextAware { 
    def applicationContext 
    
    public void setApplicationContext(ApplicationContext 
                                           applicationContext) { 
        this.applicationContext = applicationContext 
    } 

    protected DataSource determineTargetDataSource() { 
        DriverManagerDataSource ds = 
                        super.determineTargetDataSource(); 
        def env = EnvironmentHolder.getEnvironment() 
        if (env && env.passwordRequired && ds) { 
            ds.setPassword(env.password) 
        } 
        return ds 
    } 

    protected Object determineCurrentLookupKey() { 
        def env = EnvironmentHolder.getEnvironment() 
        return env?.id ?: Environment.list()[0]?.id 
    } 
}

SwitchableDataSource is the facade that delegates to the list of standard DriverManager data sources. They are defined in grails-app/conf/spring/resources.groovy using the environment settings from the Environment class. Here's my spring config in resources.groovy:

import com.leebutts.SwitchableDataSource 
import com.leebutts.Environment 
import org.springframework.jdbc.datasource.DriverManagerDataSource 

beans = { 
    parentDataSource(DriverManagerDataSource) {
        bean -> bean.'abstract' = true; 
        driverClassName = 'com.mysql.jdbc.Driver' 
        username = "root" 
    } 
    
    Environment.list().each {env -> 
        "${env.prefix}DataSource"(DriverManagerDataSource) {bean -> 
            bean.parent = parentDataSource 
            bean.scope = 'prototype' 
            def port = env.port ?: 3306 
            url = "jdbc:mysql://${env.host}:${port}/switchingDemo" 
            if (env.user) { 
                username = env.user 
            } 
            if (env.password) { 
                password = env.password 
            } 
        } 
    } 

    def dataSources = [:] 
    Environment.list().each {env -> 
        dataSources[env.id] = ref(env.prefix + 'DataSource') 
    } 

    dataSource(SwitchableDataSource) { 
        targetDataSources = dataSources 
    } 
}

Environment currently uses a static list to hold the environment config. This could be done better via a reloadable properties file or by adding a view/controller to modify the environment settings on the fly. Environment.groovy:

package com.leebutts

class Environment {

    static environments = []

    static {
        environments << [id: 1, name: 'local', prefix: 'local', 
                          host: 'localhost']
        environments << [id: 2, name: 'UAT', prefix: 'uat',
                            host: 'uat.leebutts.com']
        environments << [id: 3, name: 'Testing', prefix: 'testing',
                         host: 'testing.leebutts.com'] 
        environments << [id: 4, name: 'Beta', prefix: 'beta',
                         host: 'beta.leebutts.com', 
                         passwordRequired: true] 
        environments << [id: 5, name: 'Prod', prefix: 'prod', 
                         host: 'db.leebutts.com', user:'grails', 
                         port: 13306, 
                         passwordRequired: true] 

        //unique id check 
        environments.each {env ->
            assert environments
                .findAll {it.id == env.id}.size() == 1}
    }

    static list() {
        return environments
    }
}

SwitchableDataSource needs a way to determine which environment the current request wishes to use. It does this via a ThreadLocal holder EnvironmentHolder.groovy:

package com.leebutts

class EnvironmentHolder {

    private static final ThreadLocal contextHolder
                                = new ThreadLocal();

    static void setEnvironment(Map environment) {
        contextHolder.set(environment);
    }

    static getEnvironment() {
        return contextHolder.get();
    }

    static void clear() {
        contextHolder.remove();
    }
}

Now that the infrastructure is in place, the next step is to add a controller to allow users to select the environment they wish to use and a filter to set a default environment if one has not yet been chosen. The controller is called via ajax (in my application) but could be used as a standard controller just as easily. It looks up the Environment based on an ID and then tests the connection to make sure the password supplied is valid (if a password is required as specified in the environment config). If a password is being used it takes a copy of the environment config map, adds the password and stores it in the session so that the user doesn't have to re-enter the password on every screen and so that only this user has access to the password. EnvironmentController.groovy:

import com.leebutts.Environment
import com.leebutts.EnvironmentHolder
import javax.servlet.http.HttpServletResponse
import org.codehaus.groovy.grails.commons.ApplicationAttributes
import org.codehaus.groovy.grails.web.context.ServletContextHolder

class EnvironmentController {

    def change = {
        if (params.environment) {
            def env = Environment.list()
                     .find {it.id == new Integer(params.environment)}
            if (env) {
                if (env.passwordRequired) {
                    if (params.password) {
                        //take a copy and add a pword
                        env = addPasswordToEnvCopy(params, env)
                    } else {
                        render 'PASSWORD REQUIRED'
                        response.setStatus(
                          HttpServletResponse.SC_UNAUTHORIZED)
                        return
                    }
                }

                //test connection
                def oldEnv = EnvironmentHolder.getEnvironment()
                EnvironmentHolder.setEnvironment env
                def ds = getDataSourceForEnv()
                try {
                    def con = ds.getConnection()
                    session.environment = env
                    render 'Environment change complete.'
                } catch (e) {
                    EnvironmentHolder.setEnvironment oldEnv
                    render 'Unable to connect to database: '
                              + e.message
                    response.setStatus(
                       HttpServletResponse.SC_UNAUTHORIZED)
                    return
                }
            } else {
                render 'No such environment'
                response.setStatus(
                        HttpServletResponse.SC_BAD_REQUEST)
            }
        } else {
            render 'Missing parameter environment'
            response.setStatus(HttpServletResponse.SC_BAD_REQUEST)
        }
    }

    private def getDataSourceForEnv() {
        def servletContext = ServletContextHolder.servletContext
        def ctx = servletContext
                    .getAttribute(
                         ApplicationAttributes.APPLICATION_CONTEXT)
        return ctx.dataSource
    }

    private Map addPasswordToEnvCopy(Map params, env) {
        def myEnv = [:]
        env.each {key, val ->
            myEnv[key] = val
        }
        myEnv.password = params.password
        return myEnv
    }
}

As mentioned, there is also a simple filter for defaulting the environment to the first one in the list if one has not been selected and storing it in the ThreadLocal holder. Filters.groovy:

import com.leebutts.EnvironmentHolder
import com.leebutts.Environment

class Filters {
    def filters = {
        all(uri: '/**') {
            before = {
                if (!session.environment) {
                    session.environment = Environment.list()[0]
                }
                EnvironmentHolder.setEnvironment(session.environment)
            }
        }
    }
}

The final step is to add an environment selection form to your layout so that users can choose their environment. views/layouts/main.gsp:

<html>
<head>
    <title>Administration System</title>
    <link rel="stylesheet"
            href="${createLinkTo(dir: 'css', file: 'main.css')}"/>
    <g:layoutHead/>
    <g:javascript library="application"/>
    <g:javascript library="prototype"/>
    <script type="text/javascript">
        function refresh()
        {
            window.location.reload(false);
        }

        function loading() {
            document.getElementById('spinner').style.display = 'inline';
            document.getElementById('error').style.display = 'none';
        }
      
        function showError(e) {
            var errorDiv = document.getElementById('error')
            errorDiv.innerHTML = '<ul><li>'
                     + e.responseText + '</li></ul>';
            errorDiv.style.display = 'block';
        }
    </script>
</head>
<body>
<div class="logo">
    <div style="margin-left:10px;">
        <h1>Current Environment: 
               ${session.environment?.name ?: 'None'}</h1>
        <form action="">
            <g:select name="environment" 
                   from="${com.leebutts.Environment.list()}" 
                      optionKey="id" optionValue="name" 
                       value="${session.environment?.id}"/>
            <g:passwordField name="password"/>
            <g:submitToRemote value="Select" controller="environment" 
             action="change" update="currentEnv"
             onLoading="loading();"
             onComplete
               ="document.getElementById('spinner').style.display='none';"
             onFailure="showError(e)"
             onSuccess="refresh()"/> <br/>
            <div class="errors" id="error" 
              style="display:none;width:500px;">

            </div>
            <img id="spinner" style="display:none;"
              src="${createLinkTo(dir: 'images', file: 'spinner.gif')}"
              alt="Spinner"/>
        </form>
    </div>
</div>
<g:layoutBody/>
</body>
</html>

The finished screen looks something like this:







As this is an internal admin application it has not been tested as thoroughly as standard prod code so please do so before using it in a "live" application. Please post any corrections/bug fixes.

Some initial Grails Portlets documentation

I've started documenting how the Grails Portlets plugin currently (hopefully?) works. http://grails.org/Portlets+Plugin You can try it out (with some effort) by checking it out from subversion: http://svn.codehaus.org/grails-plugins/grails-portlets/trunk. You'll also need to checkout and build a Grails 1.1 snapshot: http://svn.codehaus.org/grails/branches/GRAILS_1_1

First post!

Well, figured I should really have a blog as occasionally I do something that others may find vaguley interesting :) Stuff currently on the go: Grails JSR168/268 Portlet Support (more on this later hopefully) Rewrite of an existing PHP site in Grails OZVR4 QLD dyno day