Reading Time: 6 minutes

After I am done with the topics like “why is automation important” and where should our automation effort be allocated, I am getting in more specific advices, related to how to create tests that are useful and easy to support. In this chapter, we will focus on the AAA principle. Of course, as all articles in the series, this relates to my experience and knowledge, if you have better advice, please share it.

What are automation principles?

Principles are something good to follow, they are definitely not “best practices”, they are sort of guidelines. They are normally empirical, meaning they are based on knowledge of the thousands who already fucked themselves. There’s plenty of these and I will try to cover the ones I know about in few short posts.

Why the basic stuff?

Probably the ones having experience in automation are already rolling eyes, saying “C’mon man, this is kinder garden for test automators”, and that’s true, the AAA principle is as basic as it gets, it’s foundational principle not only for automation, but also in unit testing. The reason why I am talking about it – I see so many people focusing on advanced topics such as – running tests in parallel, hubs, containerization, in the cloud, Page Objects, Factories, Screenplay etc. These are all cool, but they are meant to be the “cherry” on your testing framework and if you don’t know the basic principles, you are practically building your framework on an unstable foundation. So, I prefer taking baby steps and provide knowledge of fundamental principles, rather than on frameworks and patterns, that might change over day.

The bottom line is – every principle helps to solve or avoid eventual problem.

What is the problem?

One of the most discussed topics is that of “flaky checks” and checks with inaccurate and unreliable results. As I pointed out in the previous chapter of the series the term “flaky tests” is in fact an attempt to pass responsibility from the tester to the tool/environment/framework or whatever. In their essence, flaky checks are failed test design. So, the purpose of the AAA principle is to make checks cleaner and more definitive in the actions they take.

AAA principle.

Triple A principle stands for Arrange, Act, Assert. What this means is – the structure of the test should contain 3 logical components:

Arrange – part where you set up your environment

Act – part where you perform the action that is of interest in your test

Assert – part that will provide definitive answer if desired/expected and actual results match.

The idea behind it is to make very explicit when you are performing a setup action, to reach a desired state, for example, vs. when you are performing and action and when you are asserting a result. This helps identify key aspects of your tests and helps improve the review, maintenance and debugging aspects of your tests.

Now let’s look at each separate phase.

Arrange

Arrange is the initial phase of your check – normally here you will be focusing on setting up the environment to a specific state, desired for the needs of your application and context. This might be, but is not limited to:

• Import desired test data.
• Open Chrome Driver.
• Navigate to page “/basket”
• Login as user X.
• Insert money amount X

Why is arrange important?

Arrange can be very helpful in tidying up your framework and figuring out what do you need to perform every time before a test. Very often you will find out that your Arrange step will be common or the same for many tests, it makes sense to pull it into the @before, @beforeSuite or SetUp method, if the framework that you are using is offering you one. This again can help you prevent code duplication, spaghetti logic and other “smells” of you code.

Act

So, let’s say you already have your data set up, created your users, records or whatever and you are ready to perform the actual test or check, as I prefer to call it.

In testing, as humans, we normally perform actions or sets of actions and we observe the behaviour of the system under tests, comparing it to a specific expected state – an oracle. Automated checking, called later just automation, is also performing actions, but cannot actually “observe behaviour”, it can only assert specific state – hence the term check. To have a meaningful check, we should perform an action that will yield a definitive, binary true/false answer.

And this is our “Act” part. Obviously, we should be very explicit in our actions here. Good example for “Act”:

• I change the customer’s country and want to check if VAT percentage is the correct one.
• I add item to basket and check if total has increased exactly with the price of the new item.
• I switch client type from “personal” to “business” and check if additional field “Company Name” is visible.

Acts are normally very variative and in tight relation with context, that’s why they belong to the logic of your tests. Normally, I’d recommend having one action and one assertion per check, as I said, one of the aspects of good quality checks is for them to be definitive.

Assert

We had our setup, we performed an action, we have a result. Now, for this check to be useful, it needs some sort of evaluation, comparing the actual result of our experiment, with the expected one, or in broader terms – with our oracle. This is a vital part of a check, this is what fails a check when we run it. There’s few important “gotchas” that I learned through some unpleasant experience.

• Don’t use verify. Verify is sort of a non-alcoholic version of assertion, where tests don’t fail, and they continue executing. My opinion – if you need this your design is wrong.
• Produce meaningful failing message
  This is a bit annoying, but it will save your ass. Don’t just rely on the output of the test to tell you what was wrong, you are the designer of the check, you know the business logic and context, help yourself.
• Try to avoid AssertTrue/False
  Not that they are not useful, but when they fail they produce these ridiculous errors “Failed to assert that true matches false” which is hell of a useful information. If you don’t have any other options, try to write a meaningful message.
• AssertEquals is quite powerful
  I found out, based on experience that assert equals helps a lot when debugging a failure, because it provides both actual and expected, so if you can design your tests in a way to use It, it helps a lot. Anyway, it’s not universal, as sometimes actual will be an exception having 10000 lines in it.
• One assertion per test
  Yes, I know we can put more than one, but that doesn’t mean it is useful. We mentioned a test should be definitive, meaning also, that when it fails it should give exact information what went wrong, right away. When a test fails I want to say “Ah, this value returned by the service is wrong”, not “Oh, let’s open the tests rerun it 10 times in debugging mode and see which one of these 10 assertion fails”.
• SYTF – See Your Tests Fail
  I am surprised there’s people committing tests that they never seen fail. This is quite amusing, because we always say it is normal to have bugs and nobody writes perfect code from the first time and yet, when automators write code, they treat it like it’s God made. WTF?! Test your tests, make them fail on purpose, sometimes you will rely on implicit expectations that you are not realising. We are fooling ourselves pretty well, sometimes.

Summing up, how would a good test look like?

The following is a pseudo code, as I want to demonstrate a principle, I don’t care about syntax.

public function checkVatPercentageAfterUpdate(){
   //Arrange, might be in @before or @beforeMethod
Importer.importTestData(data);
Client testClient = new Client(username, password);
LoginAsClient(username, password);

//Act
NavigateTo(Profile)
SelectCountry("Ireland")
SubmitProfile();
string actualPercentage = GrabDataFromField(vatPercentage); 

//Assert
AssertEquals(20%, actualPercentage, "The actual percentage of profile, mismatches the expected result")
}

Where can this go wrong?

AAA principle is meant to make you have one arrange, one action and one assertion. The reason why I dislike end-to-end concept of automation, aside from anything else, is that they chain “A”s endlessly like this Arrange, Act, Assert, another Act, one more Act, Assert, Assert something else, Assert if the page is still the page, etc.

This leads to absolute mess, especially in UI tests and you don’t really need it. This also, considering automated checks are code, is deviation with a fundamental principle in programming and it’s called “single responsibility principle”, stating that one method should be responsible for one and one only action. Our test methods, being methods or functions should implement the same principle – one test per function – we act, we assert, done. If we chain a lot of actions and assertions, we are creating code that’s hard to understand, hard to maintain and hard to debug, when failing.

So, this is it for the AAA principle for this time. I will continue with another one next time. I hope this is useful for new members of the automation craft, but may be for more experienced, as well.

Thanks for reading! 😉