defining test categories in SoapUI

Categories are a feature of most modern test automation frameworks, which allows you to assign an arbitrary tag to any test, and then subsequently select a group of tests to run only by that tag. JUnit introduced this feature in version 4.8, both TestNG and NUnit have had it for a while. In SoapUI you have to do some tricky naming of your test case and suites in order to be able to achieve this. Or … you could use a custom event.

You want to check for the categories before every test run, so the event has to be a TestRunListener.beforeRun event. Leave the target blank (any test case), and name it anything you like – I used “categories”.

The category tags will be specified in three places: each test is going to be tagged with a category in a property called “categories”. Then at the project level, we want to specify properties “includeCategories” and “excludeCategories”, which will list categories to include and categories to exclude, respectively. Each of these will be just a comma-separated list. First part of our event script is to read all that stuff in:

def testCategories = []
def excludeCategories = []
def includeCategories = []

def tstCats = testRunner.testCase.getPropertyValue("categories")
def exCats = testRunner.testCase.testSuite.project.getPropertyValue("excludeCategories")
def inCats = testRunner.testCase.testSuite.project.getPropertyValue("includeCategories")

if(tstCats != null) tstCats.split(',').each { testCategories << it.trim() }
if(exCats != null) exCats.split(',').each { excludeCategories << it.trim() }
if(inCats != null) inCats.split(',').each { includeCategories << it.trim() }

The first three lines each define an empty List. Next three lines read all the properties in, and the last three lines parse them. We do not want to bother the user with defining a proper List of Strings, such as ["category1", "category2", "etc."], the above code expects that the categories property just specifies something like: category1, category2, etc., no quotes, no brackets.

First let’s deal with exclude categories. The meaning of these is usually that if a test is tagged with any of these, do not run it.

// exclude categories
excludeCategories.each {
	if(testCategories.contains(it))
		testRunner.cancel("${context.testCase.name} TestCase cancelled; excludeCategory = ${it}!")
}

And now the include categories. The meaning of these is that if the test is not tagged, skip it. Only if a test is tagged with any of these, then run it.

// include categories
if(includeCategories.size() != 0) {
	def cancelTest = true
	includeCategories.each {
		if(testCategories.contains(it))
			cancelTest = false
	}

	if(cancelTest)
		testRunner.cancel("${context.testCase.name} TestCase cancelled; includeCategories = ${includeCategories}!")
}

That is it!

As an added bonus, I can also have some automatic category handling, such as:

// Do not bother running LONG tests during work hours.
if(testCategories.contains("LONG")) {
	def rightNow = new GregorianCalendar()
	if(rightNow.get(Calendar.HOUR_OF_DAY) < 17)
		testRunner.cancel("${context.testCase.name} test too long to bother during the day!")
}

Enjoy! 😀