Running Saucelabs in a Geb, Spock, Groovy and Gradle stack

Gradle

Gradle is an open source polyglot build automation system. I find it more powerful than maven. The best features for me are its build reporting and its fully programmable build capabilities. You can do many complex stuff that was not possible with Maven.

SauceLabs is an awesome tool where one can run automated/manual tests on numerous differentSauceLabs combinations of OS, browsers and devices. It only makes sense that functional tests written for a project to run on the configurations your customers use no matter how diverse.

Apache Groovy integrates smoothly with any Java program, and delivers powerful features, Groovyincluding scripting capabilities, Domain-Specific Language authoring, runtime and compile-time meta-programming and functional programming*. It allows developers to quickly prototype, which is perfect when writing tests for software that is constantly changing.

Spock is a testing and specification framework for Java and Groovy applications. What makes it Spockstand out from the crowd is its beautiful and highly expressive specification language*. Spock gives us the capability to use the same language for both unit and functional tests. It also has awesome extensions that could be used. Such as; @Issue to point to JIRA issues or its very powerful Mocking capabilities. I hope to do a more detailed post later on regarding this.

gebGeb, is an awesome browser automation tool. It has a built in powerful approach called Page Object modeling. It also uses WebDriver, the elegance of jQuery for content selection and with Groovy it is simply the perfect match.

Ok, now we know what all the moving pieces do let us get cracking.

First order of business is to bring all the requirements to get SauceLabs working. Now the thing is I wouldn’t want to make QA/Developers beholden to Sauce only. So this will allow one to switch to using a local Firefox installation. You will also notice the usage of Xvfb. This will allow one to run the Firefox tests without the need of a screen which is much needed when you are running the tests on a CI or on a Vagrant server box.

For those wondering what the runXvfb.sh script looks like here it is:

Now the thing is that you can if you want create the SauceLabs driver GebConfig.groovy. However this causes issues especially with SauceLabs time limits. Once you create the driver SauceLabs will assume that is a single test. So if you have like 200-300 functional tests which might take 3 hours plus SauceLabs will timeout.

The best way we found out how this could be handled is by creating a BaseSpec where all other Specs extend from and create the driver on a Spec to Spec basis. This allowed us to both separate tests and have more segregated test results.

So now to the Gravy, I mean Groovy file. See what I did there… You know because what I am about to show is the gravy of  this whole post… I mean you saw it right? 😉

So you might notice that there is a special test watcher called SpecialSauceOnDemandTestWatcher the reason is simply due to how SauceLabs handles test results. We wanted to surface up the Spock test results in SauceLabs. It turned out to be a futile. More on that later.

So the TestWatcher allows us to surface up the results tests.

Now as you might have seen in the comments stated in the code above there is a bug(feature?) in Spock that doesn’t correctly surface up errors. So even though the test fails the report in SauceLabs shows it as passed. Is this too big of an issue? Not really as Geb/Spock comes with its own reporting tools. And there are some solutions out there to create way better reports than what SauceLabs can provide.

Now how does all this tie in? Well below is a dummy test for the fun of it:

This was an exciting find. Looking through multiple documentations, issues, code examples provided by SauceLabs. Couldn’t find anything that worked with exactly with this stack. So I hope this post will help feature developers that want to go with this awesome stack.

As always have fun!