Software testing is not … part 2.
Software testing is not … part 3.
Software testing is not… finding bugs part 4.
Software testing is not … meant to add value
I know this is controversial claim and many testers might object it. So, let me explain.
This is my position on a silly claim that was trying to prove that human testing is not useful, because it doesn’t add value to the product, the way that software development does, therefore, we don’t need it and we can absolutely replace it with machine driven checks.
Now, I want to make an important distinction here. Testing is being valuable as being part of the development process, but that doesn’t mean our testing performance and expertise adds monetary value to the product itself. On the opposite, I think our job as investigators and analysts, providing information, assessing risk and quality is actually to help the product to keep its value and help the team to prevent devaluation.
In other words, testing is not meant to add monetary value from which our product or company can benefit. It is meant to provide information and insight about the product, about the processes, about the risks that can harm the value or compromise it and/or our credibility or our client’s as provider of a software solution. And let’s not forget, software is a solution to a problem our clients have, we as testers are not the solution itself, but we are involved the solution as far as we have to make sure whether or not the solution we provide solves the right problem, if it solves it at all and if it solves it in the most optimal way we can provide as a team, given the time, resources and constraints that we have. And this is not an easy task.
Therefore, saying that testing isn’t valuable because it doesn’t provide value is like saying breaks in the car are not useful, because they don’t add horse power. Also person claiming this must have very limited or shallow view on the software testing profession and its expertise.
… playing with the product
Many of the things I can add to this will overlap with the ones I mentioned in the first part of these series, that’s why I want to add some different point of view.
One of the reasons why people see testing as an easy activity that everyone can do is, because it looks easy, in fact to some non-testers it looks like playing with the product. Tell me how many times did you hear that phrase from a developer: “Just play with it a little bit, to see if it works” or “just check if it works”.
Pretty recently Ministry of testing made a list of the icky words in software testing and “checking” was one of them. Michael Bolton stated his disagreement with the presence of “checking” in the list, then he clarified he meant checking in the RST context.
Yet, I think words like “checking” do belong to the icky list and the reason why they belong is the reason why Michael Bolton and James Bach did so much work on checking and testing series – and the reason is – we, as professional testers, don’t like someone to downgrade our effort and expertise to simply checking. In other words testing is not limited to checking simple facts that can be formalized to the pattern:
if (condition) check that result == something.
Same thing applies to the term “playing”. We might look like playing with the product, but we don’t do it to waste our time or to entertain ourselves. Anytime you see a tester or a QA specialist “playing” with the product, you should be aware that there’s way much more analytical thinking, planing and structured actions behind what looks to you as “playing”.
In fact, every educated approach towards an activity looks like playing to a viewer non-familiar to the area of the expert. Have you ever seen a master chef preparing a meal or a professor in the university explaining on a subject? They always look like they are doing minor effort doing what they do, almost like playing. Yet, what they do is the result of years of experience, improvement, planned actions and most important – errors. In fact, this is one way to make sure you are actually becoming skillful in an area, if I have to rephrase Einstein’s words, is to make your field of expertise look like you are playing.
This is it for this part. Any comments and shares are welcome, thanks 🙂
Mr.Slavchev
Senior software engineer in testing. The views I express here are mine, they don't represent any position held by any of my employers. Experience in mobile, automation, usability and exploratory testing. Rebel-driven tester, interested in the scientific part of testing and the thinking involved. Testing troll for life. Retired gamer and a beer lover. Martial arts practitioner.
Follow Me:
I used to think about testing and value in a similar way to the one you present here, but while I was trying to articulate my thoughts, I realized I actually do think that testing adds value – not to the product (unless you are selling testing services), but to the company.
In a nutshell, testing provides value in the form of being able to sleep better at night (I define value as “something people are willing to pay for”). I think I make a slightly better case for this at a post I had (http://always-fearful.blogspot.com/2016/03/why-do-we-test.html)
As for playing with the product – while testing *isn’t* doing that, playing with the product is an important part of testing. In fact, sometimes, the appropriate approach to testing something (usually low risk areas) is exactly “Just play with it a bit to see if anything is broken too badly” (in fact, I’ve used it yesterday). We might have fancy names around it, and call it “exploring” or “highly aware touring”, but the fact remains that part of testing is exactly playing with the software. What should be noted, is that when a professional tester plays with software, the result is that much better than it would be if someone who isn’t a professional tester got to play with the system. I don’t feel the need to coin a different term just to say “I’m doing this better than your average non professional”
Hey, there!
“I realized I actually do think that testing adds value – not to the product (unless you are selling testing services), but to the company.”
That’s correct, that’s why I made that distinction above between being valuable – for you team, for your company, but it isn’t necessarily a monetary value.
“I define value as “something people are willing to pay for””
I agree, but I hope you know many companies actually don’t want to pay for value which drives them into trying to cut off human testing and replace it with automated checks or omit testing at all. But that’s a bit off topic. Again I think we are on the same page here.
“Playing with the product is an important part of testing” correct, and I want to emphasize that it’s part of testing, not the testing itself. In general, I don’t mind the term “playing” when it means, similar to what the term “hacking” meant, when it first came into existence in the software world – playing with the software in an educational way in order to expose more information about its use. What I don’t support is scaling down whole testing to “just playing”, because it’s not. And yes, I do feel the need to explain that expert testing is doing “playing” in a different way, because we have to be the source of knowledge on what testing is or is not, other wise someone else not so aware as, us will do it for us.
Thanks for the comment.
Hi,
We indeed see this very much in a similar manner.
However, I don’t accept the distinction between “being valuable” and “adding value” – Developers does not add value, they create code. A salesperson does not add value – they can only sell what there is. and yet – without developers there would be no product, and without sales the product won’t add any revenue. Companies hire as few people as possible in order to get the work done, each of them is providing a service the company is willing to pay for – a developer provides coding services, a salesperson proves sales service, and a tester provides testing services. For each of these, if a company can get the same results for a lower price – it should.
Even if we exclude code reduction from “value” and narrow “something people are willing to pay for” to something along the lines “something people are willing to pay in order to get” (so they pay for having something they didn’t previously have, which is not the case of cost-reduction), testing still adds value in the sense that it provides a certain level of ease-of mind and enables managers and developers to sleep better at night. After all – if you have a strong smoke test-suite, you can commit your code at the end of the day and the chances you will break something that will cause someone to wake you in the middle of the night are smaller.
I don’t get that weird claim, that development don’t add value, but testing does. Using the way that you express it, isn’t it true that testing is just providing “testing service” how is it different from development, then?
Isn’t adding features by developers adding value? Isn’t it normal to sell version 1.2 higher than version 1.0, because it has more features, it is more stable probably improved behavior and so on. Yes, developers provide service and produce code, but code is what our product consists of, and this is what we ship to our client.
The part when you are talking about “cost reduction” and I quote “testing still adds value in the sense that it provides a certain level of ease-of mind and enables managers and developers to sleep better at night”, you are talking about adding value to the team, not to the product, that’s why I make the distinction between being valuable and adding monetary value to the product.
Yes, someone is willing to pay for testing services, your boss, the developers, the company anyone who is involved in producing the product. That’s the definition of being valuable.
On the other hand, our client that will pay $100 for the program.exe file for example, normally gives 0 f*cks if we tested the product or not, if the result of our job is good development practices and testing, or if it’s solely due to some magnificent developer’s job, because he doesn’t write bugs, our client doesn’t care.
All he/she asks for, is a working product without bugs. And we all know that 0 bugs product is impossible, so to translate that, what he asks from us is a product that works stable enough to let him do the task that it was meant to solve, without any unexpected surprises. In other words, our client asks from us a product that wont make him feel like an idiot for giving his money to us.
Now, we can’t charge our client for testing the product very well, we can’t go to him and say: “Sir, you owe us $50% more for program.exe, because we tested it very well, it has no bugs”, we simply can’t. It is meant not to have bugs, or at least not ones blocking him to do what software promises to do. That’s what I mean by “we don’t add monetary value”. What we do, that concerns our clients, is we try not to screw up too bad in front of them, by letting some silly bug creep into production, that’s what I refer to as “preventing the product from devaluation”. The code is there, the functionality is there, we don’t add up anything “sell-able” to the product while testing, we just make sure what is already there works as our client expects. Even if we do the testing of our life, we wont change that fact, the feature will still be the same, the functionality will still be the same.
The only thing that can happen if we don’t test well enough is, we can loose value, by failing to provide what our product promised to our client. Is it more clear now?
In summary, what I am trying to say is “value” is a term that has different definitions for our team and for our client, and what is valuable for our team, might have different value for the client that pays for the product we produce. That’s not bad, we just have to be aware where we belong and for what reasons we are valuable. I hope you understood, that the heading of that part was meant to be contradictory and draw attention to some facts like this. Thanks for the comment.
Of course you don’t get that weird claim – me neither!
Development and test are in the same boat – either both of them “add value” or none does.
The end user does not care for the testing, and does not care for the coding as well – you can’t go and charge him saying “the underlying code is simply beautiful” no more than you can charge extra for testing the sh*t out of it.
You can, however, get more money for saying “I have a cool feature that my competitor does not” or by saying “my product does not crash as often as my competition”. And while it might seem that on claim is purely deveopment and the other is purely test (You can’t claim such a thing with testing to provide you with the confidence to say that) In both cases, you can’t attribute that saying to neither the developer, nor the tester. It takes the whole team – product, design, dev & test to get this feature out properly (BTW, It might be that all of those are done by the same person). A nice quote I found (don’t recall where I got it, but I remembered enough to google it) – Business people does not want code[…], they want a solution to their business problem. http://essenceoftheba.com/?page_id=15
A programmer is an essential mean to achieve this end, and so is everyone else in the team – if they are not essential to get right product delivered to the right customer, they should not be working on this product.
Any work that I do as a tester is as important to the product as any work done by a developer. Saying that one is “adding value” while the other is “valuable, but not adding value” is a meaningless distinction that causes damage.
“Of course you don’t get that weird claim – me neither!”
Few comments above you said that testers are “adding value” and devs don’t add value, then you say this. Are you aware what are you talking about?
“Development and test are in the same boat – either both of them “add value” or none does.”
I totally agree with that, the idea of the post, though, was to prove how testing is adding value, not just if it adds value. Oversimplifying that can only make it worse.
The end user does not care for the testing, and does not care for the coding as well – you can’t go and charge him saying “the underlying code is simply beautiful” no more than you can charge extra for testing the sh*t out of it.
You can, however, get more money for saying “I have a cool feature that my competitor does not”
I don’t see how this brings anything new to your point or to the topic at all, practically you are saying what I posted in the previous comment, but in other words.
“… or by saying “my product does not crash as often as my competition””
You won’t get a penny if you are selling it for this, the product is supposed to work, and it is supposed to work as expected. In other words, person would buy a car for what it enables him to do, to move from point A to point B, not because it has smaller chances to explode.
“And while it might seem that on claim is purely deveopment and the other is purely test (You can’t claim such a thing with testing to provide you with the confidence to say that) In both cases, you can’t attribute that saying to neither the developer, nor the tester.”
I don’t understand what you are trying to say here, but I think I get the bottom line:
“It takes the whole team – product, design, dev & test to get this feature out properly (BTW, It might be that all of those are done by the same person). A nice quote I found (don’t recall where I got it, but I remembered enough to google it) – Business people does not want code[…], they want a solution to their business problem.”
Nobody claims the opposite. Yet, the problem that we are discussing on is how testing fits in that picture, are you sure you are following the conversation? Again I don’t see how that brings anything new, related directly to the topic.
“Any work that I do as a tester is as important to the product as any work done by a developer.”
Again, we are not discussing if it is valuable, we both agree on that, but how.
“Saying that one is “adding value” while the other is “valuable, but not adding value” is a meaningless distinction that causes damage.”
It is meaningless to you, but it makes perfect sense to me. Also, you are taking my words out of context and distorting them. About two or three times I mentioned that by being valuable I mean being valuable for the work of the team, while by adding value I am contradicting the claim that testing doesn’t add monetary value, to the product, that might increase its price. And I explained that testing’s job is not to make the product more “rich” of sell-able items, but to make sure the one that exist are working well enough to satisfy our clients needs.
One more thing, you continue to threat the term “value” like it has some universal abstract meaning that everyone agreed upon. Value has different meaning for the dev team, for the project manager, for the CEO of the company, it has different meaning for the client, as well. It often has different meaning for two testers or two developers on the same team.
That’s why I make that distinction. Nowhere in my post I say or try to say that testing isn’t valuable or is less valuable than development. I do say, though that it is valuable in a different way than development. If I have to use a metaphor for that – the developers are the ones that create the building blocks for the product, we are the ones that make sure they are aligned properly.
Anyway, I think we are theorizing way too much for something that has a concrete reason, it wrote this, because of a tweet I saw and I explained in what context do I mean it. Getting my words out of the context might lead to misinterpretations.
Yet, I feel you continue to get what I say out of context and make me repeat same stuff a thousand times. This doesn’t add anything new to the topic, so I prefer to end the conversation, as it becomes counter productive.
Thanks for your time and for the feedback.
Best regards,
Viktor Slavchev
Hi, I feel that we’ve lost each other a while ago, so I’ll backtrack a bit and start over –
In your post, you claim (if I understood you correctly) that developers add monetary value to the product, while testers do not.
In response, I claimed that there is no difference between testing and development in any relevant manner – so either none of the two activities is adding monetary value, or both of them do. I tried to support my claim by providing a perspective where development is not considered as adding monetary value, and a different perspective where testing is considered as providing monetary value (It seems that I caused some confusion by switching between those two incompatible viewpoints – sorry).
Second, I refuted the distinction you made between “being valuable” and “adding monetary value to the product”. I claim that:
1. There in no gain in making this distinction, testing would not be any different if we assume it has monetary value or if we assume the opposite (And if I’m missing something here – this is probably the best spot to see our differences).
2. Saying that testing “does not add value to the product” is risky at best, since the distinction between “being valuable” and “adding monetary value to the product” is a fine distinction (in this context, “fine distinction” is anything that needs to be explained and is not intuitive to outsiders that have yet to learn about it), and so it might propagate an approach of “testing has no value”
3. due to the combination of 1 and 2, we should not be saying that testing does not add value
4. In addition (and here I’m adding something new) – I think that testers should add monetary value to their product, since they are part of an execution team – even if we accept definitions under which *testing* adds no monetary value to the product, *testers* still can, and do, add that value, or help in adding it. If we keep that motto, it will blind us to that aspect.
At any rate – thank you for the discussion.
I love your series of blogs on that topic. This is the first post I have to contradict.
Testers can add value, if testing is done right and the feedback is transformed into according actions. They might not add value to the product directly, but there are many ways to influence the value of the product/project indirectly.
And a tester digging up the right information can absolutely add value, in my opinion. The product will be better after testing AND taking action on the information than without testing.
And in regards to playing, well, my counter-example would be professional football/soccer-players. They are still playing football, but on another level. So I agree with always-fearful, testers play with a product, but in a more sophisticated way than what is sometimes meant.
Looking forward to the next post of this series.
Patrick
Hey, Patrick! Thanks for the comment and for the kind words.
I knew it, that claim like this will have reactions that might disagree, which is OK, I like the fact that people protect the craft and it’s value and are ready to dig deeper to get to the bottom of testing’s purpose and real value in the development process.
I agree with your opinion, that’s why I made the distinction between being valuable and adding monetary value. What I meant with that is – testing is not convertible to money, we can’t charge our clients more for the fact we tested the software, it’s not a feature it’s not even an improvement. We can’t say “our software costs $200 more than company X, because we tested it better”. Of course, testing is calculated in the effort to produce it, but it doesn’t add to products price. It’s because people normally expect that what we provide is stable and reliable, not full of defects. That’s why I mention we are more into adding value to the team and the company, but in matters of product value, I think our job is more to prevent devaluation (screwing up too bad in front of our clients), rather than adding monetary value.
As for playing, yes, I agree it is “playing” in a more sophisticated level, but I don’t think that’s transparent for non-testers. I don’t agree with that nuance of the term “playing” that presumes non-directed, shallow not planned actions that don’t seem to have purpose or structure.
Thanks 😉
I still see top level leaders struggling to understand how testing fits in the Agile world. There are many Fortune 500 companies whose leaders think that once they are Agile, they don’t need a testers and that developers will handle everything. They fail to see the value that testing adds to the process.
Sometimes I feel that some of these leaders have gone 20 years back and need to remember why the whole SDLC came into being and why we need testing. I hope more of them come across this article.
Thanks for the kind words, mate! 🙂