Randomize test execution in TestNG to improve test isolation.

Reading Time: 3 minutes
TestNG randomize test execution

Photo by http://testingfreak.com/wp-content/uploads/2015/01/testng1.png

Recently I watched this awesome talk by Simon Stewart which is kind of old, but still awesome, since I am quite green yet in the area of test automation or at least the meaningful and useful test automation. In it he used to talk about test isolation and how important it is. In a few words test isolation is the ability of your tests to run in an isolated environment, but also to run separately, non-depending on each other. So, there’s many ways to do so, may be if you’re interested you should watch the talk, but one of them is to randomize test execution. So, I decided to incorporate this using TestNG, since we are using it in our code and it’s kinda interesting to have this in my tool set, I didn’t find it a lot on the web.

How to randomize test execution using Java and TestNG.

Here are some code examples that I made just for the demo, I repeat for the demo, so don’t mind the moronish names that I’ve chosen for my classes. And it’s fairly simple example – we have a test class with some code, in our case it’s just outputting “in method …”. That’s just to keep track on which test is executed when, what’s the sequence of our test execution. So here’s the code of our test class:

As I said, simply moronish code example. So if we run this as testNG test we get the following output in the console:

And it does that every time we run them. So what we need is to randomize test execution in our class.

Why randomize test execution? How is this important?

Well, if you have a test suite of 30-40 test or more and you run them every time in the same order, try applying this solution and you will find out why. Turns out we as coders assume that we have many things for granted, for example – a specific state of the data in the DB or a specific screen that we land or start from. Turns out, this is not like that and unknowingly we’ve incorporated some dependency on our tests, which is not good if we want to use them in continuous integration or in some other environment. So, the biggest plus from having a good test isolation is – we don’t want to fix the damn thing every fucking time we change the environment it runs at.

See also  Organic testing

So, I’ve searched around and I found this Stackoverflow question, which had my solution.

In a separate package let’s say it’s called utils you may add the following class:

And in order to randomize test execution of certain class you have to add the following annotation to your test class:

So, when your run the tests now, the output looks like this.

Which is a bit better. The cool part of randomizing test class like this is the nanoseconds feed of the random, as you see it is logged on the console and if you find anything wrong in a specific sequence you could always copy the feed from console and hard code it in our random feed, in order to re-create the exact same scenario. Which is fucking awesome.

See also  Mentoring question - where do I start learning testing

Thank you all for reading and hope that was interesting and helpful for all of you. If you liked it, please share it with your  friends and/or share your thoughts in the comments.

Mr.Slavchev

Mr.Slavchev

Senior software testing engineer at https://siteground.com.
Experience in mobile, automation, usability and exploratory testing.
Rebel-driven tester, interested in the scientific part of testing and the thinking involved.
Testing troll for life.
Retired gamer and a beer lover.
Martial arts practitioner.

More Posts - Website

Follow Me:
TwitterFacebookLinkedInGoogle Plus


Leave a Reply

Your email address will not be published. Required fields are marked *

I am speaking at

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Goodreads Read books

Follow me on Twitter