In this part of “Hindsight lessons of automation” I will try to focus on another key question that is crucial for automation and it is – what’s worth automating? Why automate this, over that? Why are people wasting so much time writing UI tests and so on. In order to get to that, we will talk about interfaces.
What is an interface?
What is it really? We are using it everyday, interface that, interface this, UI, GUI, voice interface etc., but what is it? This is a question I like to give to my students in order to make them think over terminology, they very often take for granted and so do many experienced testers.
So, if you Google what an interface is, you will probably get definition like this:
In computing, an interface is a shared boundary across which two or more separate components of a computer system exchange information.(Source: Wikipedia)
In order to give this definition a little bit broader context I can paraphrase it like this:
An interface is a shared boundary, a medium, via which two systems exchange information.
They come in various kinds and types – we have hardware or physical interfaces – like RJ 45 (the standard internet cable connector) or HDMI, but we also have graphical user interfaces, command line interfaces, voice interfaces like the one Google assistant has, we have touch interfaces, etc.
The few common things that are specific for interfaces and we have to point out as important are:
- Interfaces hide complexity
Here’s the fun part, even in programing we use interfaces to hide complexity. Whenever we encounter a class that implements the interface ICountable, for example, we know that this class has some specific functions that we can use, if we know what ICountable means. We don’t really have to care about the internal workings of the class, but we know what behavior we can expect.
Now let’s take a look at the real-life examples – when we see an interface, we know what to do, but without having to care how it operates on the inside. Look at your screen, ironically you are looking at it right now, but try to think about it – could you explain how any key you press appears on the screen, how your mouse is moved on it? Do you have to know how it operates on the inside in order to use it? No, you are just aware that whenever you move your mouse, the magic works and the small cursor on your screen moves accordingly. Same is with electricity sockets, HDMI, keyboards, touchscreens, etc. - Interfaces give clear call for action.
Another interesting characteristic of interfaces is that they give us clear call for action how to use them. For example, you have the RJ-45 connector and you know you need something that looks alike to plug it inside, you can try plug you HDMI cable there, but it wont work, no matter how hard you try.
Touch interfaces have big screens, with big items that are inviting you to interact with them by touching, voice interface looks like microphones, etc. - Interfaces have very specific purpose, if you try to misuse them, you might look like an idiot.
What do I mean – earlier I said, that interfaces are boundary between two systems, and they are build specifically to communicate clearly with these systems in a very hassle-free manner. For that purpose, if we try to misuse interfaces, you risk looking very silly.
Imagine this – what will happen if you try to talk to a touch interface, that has no microphone? Nothing, and you will look like an idiot, remember this guy?
What will happen if you try to press touch interface controls on a voice interface, if there isn’t any? You will look like an idiot, again.
Do you feel where I am going? If, yes, hold that thought for a moment, I will get back to it.
What the does that have to do with testing?
And that’s a valid question, why is all that bullshit about interfaces? Well, interesting enough as testers, like it or not, we are using interfaces to interact with our product. Wow, what a discovery!
If you are following this series you may be read the article about layers of automation, if not, I hope you are lucky enough to know there’s more than the UI to automate. So, in this article I listed a couple of different layers – code layer, represented by unit and integration, service or API layer and UI layer. So, let’s see what these offer us as interfaces.
Code layer – Not particularly an interface, but it is offering us a way to manipulate it directly.
Is this meant to be manipulated by code? Of course! This is what code does, manipulating other code, all the time.
Is it fast and efficient? Absolutely, it’s just code, it’s as fast and as efficient as its compiler or interpreter allows it to run.
Does it make sense to automate? Absolutely, it’s easy to manipulate by code and fast to run.
Service/API layer
This is an interface already, it’s having it in its name (Application Programing Interface)
Is it meant to be manipulated by code? Well, it’s in the name, again – Programing, API were created to be operated by programs, e.g. by code.
Is this fast and efficient?
Yes, it is efficient, being meant to be contacted programatically, APIs provide very protocol-like way to interact with and obtain response by them. And responses are very definitive, there is no doubt if you got success or failure.
As for speed – it is as fast as you connection allows it, if you are performing your tests in a local network, isolated from the outside work, it might be relatively fast, but out in the wild, who knows.
Does it make sense to automate?
Yes, it does. It provides easy way to use it and easy way to interpret it’s responses automatically.
UI layer/user interface
Yes, it is again an interface and it’s one of the most common. In fact, there’s users who have no idea you can operate an application in another way than the user interface.
Let’s get back to my tirade about interfaces earlier – I said interface was boundary between two systems. Is the user a system? Of course, and it’s a very complicated one. Is the user a machine? Rarely, even if it is, we are not building user interfaces for it.
Is this meant to be manipulated by code?
At this point, this should be a self-answering question. Of course, not – User interface, it is an interface to the user, which is normally a human. Yet, we can build tools and abstractions by which we can interact to the interface with code.
Is this fast and efficient?
Well, having in mind that it is not meant to be operated by code, in order to do it, we are building additional layer of abstraction to achieve it. And this comes at a price – it’s slower, it’s more complicated, it’s more error prone.
Does it make sense to automate?
Not really. Why? How would you feel if you try to use voice commands on your keyboard? Like a complete idiot?
How would you feel if you try to write code in order to operate an interface that was originally created to be used by humans and humans, only?
And in order to give even better visualization of this, I will use an example I heard recently:
Both a bird and a plane fly, right? And planes were made by the model of a bird, but in order to make a plane fly, we don’t have to make it flap its wings.
Creating code to test user interfaces, by executing user-like interactions, is like trying to make a plane that flaps its wings. It’s simply not efficient.
Anyway, people are doing it
And there are valid reasons why they are doing it – because it’s useful. Here’s how:
- Ensuring functional correctness of our UI components – you might have a piece of art API and a piece of art business logic, but if your UI doesn’t work, nobody is going to give a fuck about it. UI is what the user is using.
- Testing closer to the user – in my previous post I mentioned why is it so vital to test in the UI – because we have better visibility for the impact of the errors.
- We use UI tests like tripwire – or change detectors, normally we plant these tripwire UI test around our product hoping that they will make a lot of noise if something bad happens. Just as with tripwire, the fact a test signaled doesn’t mean we are safe, it just mean we have to go for an investigation.
Do UI tests make good candidates for long tests? Considering all the above – NO, hell no, please don’t do it, in the name of everyone’s sanity. This is where flaky tests are born. I don’t know whose brilliant idea was it to come up with the E2E tests, I don’t believe in it and I don’t think it’s worth it.
Considering the complexity of UI and the fact it is not meant to be interacted with from code perspective, spending more time interacting, changing pages, taking values, checking them in other pages is simply an open invitation to get in deep sh.. trouble.
What can we get from this?
Eat your carrots, they are good for your eyes! 🙂
There’s a beautiful variety of ways to perform testing, there’s easy ways, harder ways, and much harder ways. Counter intuitively, in automation we decided to take the hardest option and make it a standard for the skill level of automation engineers. I find this amusing, but I believe there’s much better ways to invest time in automation, then simply the UI. Yet, it makes sense to automate the UI, as well, but we must be very careful.
I hope that helps someone out there, trying to figure out what the hell to do with these tests.
Any form of comments will be very welcome, along with shares in the social media.
Thanks for reading. 😉
You really make me thinking about my approach to automated UI tests. Where i had some foggy ideas yet, i think about it more structured now.
May actual thoughts:
– test most logic and its reactions to user input with the API, its faster/cheaper.
– UI tests should mostly concentrate on the UI itself, the interaction between the different components, and the reaction to the backend (e.g. show an error dialog for an error message). Maybe with a mocked backend, so i don’t have to rely on other slow UI actions for test preparation / arrangement.
– How much automated integration test between API and UI will i need? My example is an input field which allows more characters (length or type) than the API. A first manual test should found an error here fast so it think its a bad test (high stability, mid to high investment)