Software testing is not … part 3.

Reading Time: 4 minutes

Links to the previous two parts:

Software testing is not…

Software testing is not … part 2.

Software testing is not activity that could be performed by irreplaceable robots.

I can say for sure that this is one of the most incorrect statements in testing and also one of the most toxic ones. Of course no body is saying that, no body is that dumb, but it is based on a false assumptions and it produces a lot of other false assumptions which become popular for the only reason that someone trusts them blindly, without asking questions. So, I split these assumptions in two groups:

1. Everyone can test, testing is easy. Your manager can test, the dev can test, the designer can test, the client can test, your managers dog can test, etc, etc… But, they can’t do it, because they are busy doing their important stuff, so you seem to be the guy/girl who has to do it.
2. Testing is easy, because a well written test is described in a finite number of steps or instructions, and if these instructions are followed exactly you can test the problem without any impediments.

Now, what I want you to do is – write down these on a piece of paper, bring it outside and burn it, while chanting some evil spell. 😀

Believe it or not, the above statements somehow made it in our craft and are even institutionalized, meaning a lot of people take them for granted. Which leads to a whole  bunch of false assumptions that derive from these, such as:

• Every step in testing needs to be documented in a script or a test case.
• If you don’t have test cases, you are doing testing wrong.
• If you don’t have test cases, you can’t prove that you did any job on the project.
• Testing is expensive.
• Testing is unnecessary.
• Testing can be performed automatically, so we reduce cost.

It’s easy to understand why these myths made it into our craft – one reason is, the senior management doesn’t really care about testing, they care about budgets and efficiency. Another reason is – we fail to provide the true story behind testing, how it is efficient and beneficial for the product/project, if we fail to do that, it’s our fault. And third reason – there are way too many vendors of tools and software testing courses and academies that have interest to sell you “snake oil” – the ultimate solution that will solve all your testing problems, whether it is a tool or methodology, it doesn’t matter.

Why testing can’t be performed by irreplaceable robots, and here I mean both human robot-like acting specialist and actual automation? One good reason for that is the fact that testing isn’t based on instructions, it’s based on interaction, simultaneous information gathering, evaluation and action according to that information. Testing is experiment, investigation and decomposition of statements that the others (development, management, you name it) believe are true(paraphrasing James Bach). Using all this, we have to provide relevant information, about the risk and the product itself. So, how all that aligns with the “concept” of following instructions?

And there’s one more general reason why testing can not be represented as a limited set of connected steps or instructions, which causes also the impossibility to fully automate testing – converting these instructions to machine instructions. The reason is the way knowledge works. More specifically the impossibility to convert tacit knowledge into explicit or even explicable knowledge(Collins, “Tacit and Explicit knowledge”).

This might be explained very simple – we all done cooking, sometimes, right? Well, we all know a cooking recipes are sort of algorithmic, step-by-step guide how to make a dish X. And they could be written by a really good expert, like a master-chef, for example. Yet, following his advice, even to the smallest detail, you might be able to make a decent dish, but not one made by the expert. You wouldn’t notice the small tweaks that the master chef will make, depending on his environment or the product that he/she uses, or the freshness of the spices, all based on a “gut feeling”. Why is this so? We will call that experience or routine, what it really is –  the huge underwater part of iceberg of knowledge, called tacit knowledge.

Same thing is applicable to any area, testing as well. “We can know more than we can tell” (Polanyi), therefore in testing we do much more than we can actually put in words, scenarios, cases, steps, scripts or you name it. Testing is an intellectual process and it has organic nature, it is normal to be inexplicable and for us to have trouble describing it or documenting it. That’s how knowledge and science work.

The reason why this is not a shared belief in the mainstream testing is the fact that many consultants and certification programs like to give a short and exciting story on “how can you become great tester by following this simple set of rules”. And it  is “sold” to us under the cover of the words “best practices”, meaning “I have nothing to prove to you, this is what everyone is using”. Well, newsflash, this is the testing community, you do a statement here, you have to be ready to defend your position, no body cares how much of a guru or an expert you are.

That’s it for this part, would like to read you comments and reactions on the social media. Thanks for reading. 🙂