Hindsight lessons about automation: Start automating

Reading Time: 6 minutes

The first topic of the series will be how to start automating. As I gave a promise to myself, I will put some effort in clarifying, mainly to myself and whoever wants to read, knowledge of automation as a specifically targeted activity towards the goals of automation and in support of testing.

So, this series of blog posts aim to be:

• Short, about 5 mins reads, although that’s very hard for me.
• Practical – no bullshit, just empirical, productive advice
• Based on personal experience and bad decisions. I think that helps s lot.

Why hindsight?

Last summer, I read a book called “Thinking fast and slow” by Daniel Kahneman, it’s a psychology book and a really good one, if you have a chance definitely read it. I learned a new word from the book and it was “hindsight” it was used in a bias called “hindsight bias”. Hindsight bias occurs normally in a negative aspect whenever something  bad happens, there’s always that one person saying: “I knew it, I knew it would happen.” Turns out, he or she didn’t knew, it’s just a bias, but more on psychology – read Kahneman’s book.

So, what does this have to do with testing?

Once I had a discussion with another tester about a talk I gave at a conference called “Test automation – the bitter truth”. He asked what was it all about. I answered it was about people’s expectations about automation and how we should approach it with realistically, not treat it as it was a magic. His answer was: “Well isn’t this obvious?” Well, turns out it’s not.

Another example – Maaret Pyhäjärvi made an observation, while we talked at a conference few months ago – very often the things we consider common sense are not so common for new comers in our craft and this applies to automation, too.

So, turns out, there’s some layer of knowledge we all think we know and share, but we still fail miserably to apply it. That’s why  I decided to start with basic stuff.

Formal definition of hindsight.

Recognition of the realities, possibilities of events, situations,decisions etc.. after they have occurred

Or in other words – we all fucked up doing automation, we had to fuck up over and over again to learn a lesson. It will be useful if we share this lessons, so we don’t have to be the one saying “I should have known”.

Anyway – there’s even better definition of “hindsight” in urban dictionary:

After you screw something up, you magically and conveniently gain a vast amount of knowledge on what you SHOULD have done to avoid screwing up.

… and sounds very familiar to me unless I read the 3rd definition:

The universe rubbing your face in your own fuck up.

Isn’t that lovely, isn’t that “the story of my life” in everyone’s testing? 😀

So, the inspiration for this blog post series is – what valuable lessons have we learned and can we transfer, after we got our faces rubbed in our own fuck up? 🙂

Last before I dive into concrete stuff, I assume few things:

• You’ve read this article first, otherwise you kind of miss the context.
• You are aware there’s no such thing as “manual” testing.
• You are aware automation supports testing, but can’t replace it.
• You figured out the right reasons to do automation or the right reasons to do testing, using automation as a support.

Where to start automating?

[Note: This particular topic might be a bit boring for experienced automators, but I  hope you understand why I write about it.]

Although it might seem like a lame question, a basic one to people with experience in automation, I really want to clear it out. The answer is really simple – programming.

In order to think of dealing with automation, you should have clear idea in your head about the following:

• Automation is programming
• Automation code is a program
• You are a coder
• Professional automation is written in code
• Automation has bugs, always!
• Most of the time you will be dealing with bugs in your code, rather than bugs in production code.

I know there are tons of tools that offer alternative to coded testing via any sort of record and playback, support frameworks, visual testing, etc.

Unfortunately, when somebody speaks of automation developer or software developer in test, nobody means “record and replay” developer.

Something that is probably obvious for anyone  in testing and automation, but totally not transparent to new people is – automation community is sort of layered. Whenever you speak of automation, people will expect you to mean “coded automation”. If you want to just use record and replay tools, you will be treated as second class citizen in the automation community.

The reason for this is that in time record and replay tools didn’t prove reliable in the long-term. They are really easy to use for short and fast tests, but they are brittle and hard to support when project scales.

Another reason why record and replay tools are not desirable step of your automation career is because they inherently support the false claim that we can automate human activities by simply recording and repeating them and may be that’s why they failed (Yoda citation intended).

Get your testing hat off and put your developer hat on.

As you see in the article so far, I am avoiding the use of terms “test automation” and “automated testing” and I will continue to avoid them, on purpose. The reason why automation is in trouble, I think, is mainly because of these terms. What I prefer speaking of is “programmatic testing”.  I offered this term once, but it didn’t really sunk, that’s why I decided to forget about it. What I mean is – being an automation developer you are much more developer than being a tester. What you do, writing automation, in fact is performing testing by programmatic approaches. Now, I think when we speak of “programmatic testing” or “performing testing by using programmatic approach”, something slightly changes in the way we think about the activity compared with the term “automated testing”.

When we speak of “programmatic testing” we kind of realize the limits that we are facing by using coded approach in testing, because every programming framework, language, API or library has its limitation, and that’s totally OK.

Of course, using programming languages we can do almost everything, but considering the time limits we face, the quality criteria, the limitations of tools and frameworks, the variability of behaviors and the vastness of conditions that we have to reproduce, performing our tests, trying to achieve “almost everything” isn’t so much of a good idea, anymore.

That’s why I brought the idea about the two mind sets in the previous article – there’s simply difference in the approach we use being a human exploratory tester and being a programmatic tester.

So, I conclude, people willing to do automation are dealing with it, because they like solving testing problems in programmatic way.

If you are trying automation, because you think it will replace testing, you probably don’t have good understanding about testing.

If you are into automation simply, because of a better salary, you probably lack the proper motivation to do it.

Again, success in automation, according to my humble opinion is – to do it because you love it and because you like solving testing problems programatically.

What language to start with?

This is a big and complex topic and the more people you ask about it, the more biased answers you’ll get.

Simply put, there’s automation framework for almost any programming language that’s out there. If you have experience in a language and you like it, go for it. The main thing in programming and the main reason why people have “religious fights” over programming languages is – you stick with the language/framework you like and find comfortable using. That’s why people fight over languages, because they fail to realize they simply like the language, there’s no one ultimate supreme feature that their favorite language has over others.

If you don’t have any idea where to start and don’t know any programming language, my advice is start with Java. Statistically, demand looks higher for Java automators, rather than any other kind. The good thing with programming languages is, once acquired the basics, you can easily learn another, as it’s simply differences in syntax, strict or loose type and compiled vs interpreted languages.

Also, if you want to do automation, you will need basic knowledge in:

• HTML5, CSS and JS – simply because (sigh) sooner or later you will be digging in the UI, so better be prepared for it. Good news is, HTML and Css are fairly easy to learn, if you don’t want to make too complex stuff. Not the case with JS, though, but I won’t spoil your pleasure. 😀
• Databases – not only because of the interview questions, but database knowledge is needed when you have to deal with data management for your tests.
• Linux + shell scripting – not that it’ll be expected of you to have an admin skill set, but being a developer you are expected to be able to set up your own environment. Mostly it will run/have a server on a Linux machine. So, it’s way better if you can deal with it, before it turns into a huge pain in the ass.
• Web services and API knowledge – almost in any case, you will be required to use, integrate with a web service or at least, know how it is used by the application. It’s good idea to be aware what a service is, how it is used by the application, what possible methods there is in an API, what file formats could be used and how we can use or produce them, with a programming language.

I know this is probably too basic, but I humbly hope it might help some poor soul just stepping in our craft to get a little bit of a clue what they are dealing with from a meaningful perspective and not from all the bullshit out there.

Stay with me for the next chapter where I’ll focus on what types of automation are there. Until then – comments, shares and productive opinions are highly appreciated.

Thanks for reading. 🙂