Software testing is… part 2 – rooted in social science

This is the second part of the series software testing is, based on the mind map I provided in the initial post, you can take a look here: http://mrslavchev.com/2016/11/04/software-testing-is-part-1/

I bet that big part of the readers of this blog will be puzzled by the presence of the words software and social science in the same sentence and that’s OK. Here’s how it goes.

The current state of thinking about software testing.

When I was a newbie, not that I consider myself experienced now, I am just an old newbie, so when I was a newbie and first started studying the software testing craft I was presented with the following picture of software testing.

Software testing is a technical activity that relies on understanding of the technical side of the product, such as:

  • How the product is built – structure and frameworks
  • How specific functionalities are implemented?
  • What protocols are used for communication?
  • How does an application works – databases, interfaces, memory, processor usage?
  • How the internet operates – basic networking knowledge?
  • How is the application dependent on its environment – OS, hardware, 3-rd party libraries?
  • and many more

All of this is correct, but far from enough. It would be enough, if we only accept the definition that an application is just a series of instructions executed following a specific logic.

I like the example that Cem Kaner gives in the BBST foundation course, saying the above is like saying a house is just a bricks and materials ordered in a specific way, which is not true. A house is a place where someone can live and feel at comfortable, so there’s something more to a house than just the materials. Same with a software product – it’s not just the code, but it comes to partially replace a human interaction, but I will get into more depth in this a bit later.

The soft skills people.

Suddenly, when speaking of testing a person or two will show up saying that testing is not just technical, but incorporates some so-called “soft skills”. By “soft skills” they normally mean some part of negotiation skills, some part of team work abilities, some part of leadership, presentation skills, communication, etc. I say “some part”, because, so far, I didn’t hear or read any credible explanation what “are soft skills made of”, exactly, normally they are a small amount of this and that, etc.

I also agree, that this is important, too, but too general, in my opinion. It’s expected by a good professional in almost any area, interacting with clients/colleagues or any other human beings, to be a good communicator or a good negotiator and a good team player. So, I personally consider these – basic corporate survival skills, they are not something that’s strictly specific to software testing. So, what is it?

Software testing is rooted in social science.

Social science cloud
Source: http://www.proprofs.com/flashcards/topic/social-science

I mentioned earlier that a software product is not just a bunch of computer instructions ordered following specific logic. Software applications came to existence in order to replace, although partially, a human interaction, therefore, software applications are not only there to do what they are supposed to do, but they also have to bring us comfort and satisfaction. To understand how an application comforts our clients and how we can measure this in a reliable way, we need to understand the human nature and this is where social sciences come to help us. Or, if I have to use the house analogy Kaner made, it will be more useful to know how to evaluate a comfortable house, rather than just knowing how it’s built.

So, many times, under the influence of Kaner, Bach, Bolton and the CDT community in general, I have claimed that software testing is a science, or may be not a science, but uses many scientific methods and approaches and it corresponds with other sciences, not only but we can apply knowledge from other sciences in testing, so our testing can evolve and be more diverse. The aim of this topic is to give some perspective how testing corresponds with social science and which branches of it, I believe were useful for me until now. It’s important to mention that I am not considering myself an expert in any of the following, so all of the conclusions are made by my personal, non-expert, observations.

Psychology

In fact psychology is a science that can help any aspect of life and it has so many branches and sub-branches, that it will be practically impossible to cover all the possible applications it can have on software testing.

Psychology is science about the mind and behavior and that’s something that we rely on very heavily in testing, in two aspects:

Clients

In order to satisfy our clients’ needs, producing specific software product, we want to make them happy, we want to influence their emotions. And in no way, we want to influence their emotions in a negative way. That’s why, psychological knowledge can be useful in software testing, specifically in areas like usability evaluation. If you want more information on this I recommend you the book “The Design of Everyday Things” by Don Norman where you can learn how design of some utensils we use in our daily life has actually the ability to call for specific action.

Ourselves

The second aspect in which psychology can be very useful to software testing is sort of reflection oriented. We are professionals in assessing quality and performing experiments, but how can we say if we make the right experiments, if we have the correct judgement or in other words – how do we test a tester ?

Well, there’s no guideline actually, how to be a better tester or how to have the proper judgement, but we can be aware of what our own weak sides are or what known “defects and features” (because most of them are actually features) our minds have. And this is where cognitive psychology can help a lot.  We need to know how our minds can trick us, how our perception might be distorted, absolutely naturally and how our judgement can become biased. Another interesting topic that cognitive psychology reviews is the heuristic method for solving complex problems. This is a topic that is often discussed in the CDT community and a popular approach of solving testing problems, even when people are unaware they are doing it and don’t call it a heuristic.

A book that might be of great help to those curious of you, which makes a great analysis of all of these is: “Thinking, Fast and Slow” by Daniel Kahneman

Quantitative and qualitative research

Again, without any ambition to be an expert – this is something that is widely used in other fields like statistics and marketing and believe it or not a problem that we solve in testing every day.

In testing we often have that argument on whether or not our testing process should be structured around trying to provide some specific metrics or focus on experiment that is profound and provides valuable contextual information.

Seems that in science this is a problem that existed for ages in the face of quantitative and qualitative scientific research methods. And we deal with it in testing as well. How many times have you heard the argument on whether or not we should use metrics to direct our testing and if yes what kind of metrics, how do we translate these metrics in the language of quality and risks? How are they meaningful for us and for our peers?

On the other hand, there’s this group of people saying that metrics are easily manipulatable and they give us a good statistical results, but bad human or contextual results. They advice us to more “qualitative”, human and contextual approach in testing, which is in its nature experimentation.

Another question arises on how we choose one or the other, or if we have to choose at all? Do we need to combine them? If yes, how do we combine them, which is first, which is more important.

Seems to me we have a lot to learn from the field of qualitative and quantitative scientific research, in fact while preparing this article I reviewed some materials that made me think of a separate post on this topic only, so expect a follow-up on this in the near future.

Linguistics

This might seem a little bit odd to you may be, but expressing ourselves in testing is actually a vital part of our job. I often see people having arguments on the words that we use in testing or what a specific testing term means, what is the purpose of using this term over that term, I also often see these distinctions being qualified as “just semantics”, but I don’t think people realize how vital are these semantics for the quality of our craft. And believe me, this comes from me – a person that normally communicates his thoughts in testing in a language that’s not his native, so I am used to saying something stupid or incorrect and having to deal with the result.

Here’s how proper use of the language can be crucial for testing:

Bug advocacy:

Bug reporting is not just a routine task that we do in our day-to-day activities, although it looks like one. Quality bug report is not “just a bug report”, it’s your evaluation of the quality of the product, sometimes it’s the only proof of your input to the product. So, preparing a high quality bug report is crucial for everyone in the team and the product itself.

The more you develop that skill, the more credibility you and your job gain in the eyes of development, analysts, management and so on. And believe me, no matter how much time and effort you’ve put into building it, you can ruin it within a day, by letting negligence crawl into your bug reporting language, so it is very important to be careful, professional and specific in what we report as a bug. Don’t forget there are people depending on our judgement.

Describe your strategy and tell it in a compelling way

I think I wrote about this for probably a thousand times and a thousand times more won’t be enough to put the accent on how important it is to be able to explain what you do in a compelling and professional way.

Turns out in my experience, that many testers, even experienced ones have no issues performing their job as testers, including all their routine tasks, all but one – being able to explain what do they do. In other words, in the words of Harry Collins in fact, they hold a lot of “tacit” knowledge and they have problems turning it into explicit, in fact sometimes we even fail to explain our explicit knowledge in a good way, without oversimplifications.

This is why linguistic knowledge and the rich vocabulary and the ability to be precise in what you do are so important. Yes, we do report bugs, but no one is looking for bug reporters, after all, they are looking for testing professionals.  In order to be professional tester you have to know testing, to be able to do it well and to be able to explain and defend the approach that you’ve chosen and sound like a professional while you do it.

Philosophy

Philosophy has so many branches and so many great thinkers involved in it, that I can only wish I can make a profound analysis how it is involved in testing. Also, I am probably not going to dive too deep in it, as I plan to write a separate blog post on how software testing is related to epistemology.

Here’s just some quick thoughts on how philosophy is related to testing. What does philosophy means? From ancient Greek, the literal translation is – “love to wisdom”. Well, that’s interesting, I think I’ve seen this somewhere in IT as well, remember these guys that are always ready to learn more and more about the product, that claim software testing is perpetual learning ?

And there’s actually more, but hey, I don’t want to spoil all of it all at once.

And I think this topic became waaay much longer than planned. I hope with the above I gave some more ideas to think on next time you speak with someone on how “technical” testing is and I hope you consider the big part of social science knowledge that we use, in order to be valuable testers.

Of course, I would love to read your thoughts on the topic, that’s what comments are for. Like, share and retweets are always appreciated. Good luck. 🙂

What is software testing according to the “others”

I took a long break from blogging and the series of “what software testing is”, so before I go back to them, I would like to share my personal observations on some interesting facts. It’s actually related to what software testing is, but it’s sort of outsider’s view. It’s actually part of the reasons why I started the series what software testing is/not- the fact that opinions about testing are often shaped by people outside of testing.

So, have you ever been wondering what are others in your company thinking you’re doing? What is their perception of your job? You don’t know? Well it’s fairly easy to test this, simply read through the job description that is posted when hiring new employees. Because that’s what it is – job descriptions are often prepared by HR, leads and managers sometimes development managers, so I consider this a really good measure of what other people think testing is.

I read job postings very often – not because I am constantly looking for new job, but because I am constantly looking for personal improvement and I want to make sure my skills are still relevant to the market. I believe that’s something every good professional should do and also something that might give us perspective over the craft. Reading job posting always leaves me with the impression that people posting job offers for testers are either non-testers, copy-pasting job descriptions, without putting any sort of analytical thinking in it or they are drunk. Here are some of my favorites that I found while I was making research for this blog post(these are all requirements from real job postings):

Analyse automation and other failures and:

  • Accurately report and track defects
  • Identify areas of improvement

“Automation and other failures…” seems to me like the company is telling you, pretty straight-forward, that their automation testing strategy/tool/thing is a total failure and they want you to analyze it and probably fix it. Joke aside, I bet the author meant something way different, but didn’t pay attention close enough.

“Keeping track of developer changes and doing the appropriate tests when needed”

“Developer changes”?! Does this mean the tester should pay attention if developers switch places or if a developer changes his clothes, haircut, political views, what kind of change do they imply?

Not to mention ridiculous role headings like “We are hiring an automated tester“. Wtf do you mean by “automated tester”? Who are you going to hire T 1000? The Iron man? Wall-E? Testing is a human activity, performed by humans, which occasionally use automated tools, but that doesn’t turn them into automated testers.

Besides that job descriptions normally contain the following list of activities that are sort of “what are you going to do” when you get hired explanation. So, here’s how this looks like – it’s a raw material that I practically copy-pasted from several offers:

  • Analyse business and technical requirements, identify potential software issues
  • Design and execute test plans and test strategies
  • Execute functional, regression, integration and performance tests
  • Create test reports, defect analysis and troubleshooting
  • Maintain and regularly update QA related documentation
  • Prepare, monitor and maintain test environments and systems
  • Execute manual tests;-
  • Create detailed test plans and high fidelity test suites with test cases;-
  • Test suite execution, result analysis and reporting;- Estimate testing complexity and tasks for User Stories;- Coach other team members.
  • All applicants should be familiar with industry best practices for testing products including:
    • Defect classification and issue severity rating
    • White box, black box, and gray box testing
    • Usability testing
    • Code coverage
    • Unit, integration, system, and regression testing
    • Security and performance testing
    • Common automated testing tools
    • Continuous integration and continuous delivery
    • Agile methodologies e.g. SCRUM
    • Team sprint planning tools like JIRA
    • Customer use cases
    • Test documentation
  • Build tools and scripts to reduce the need for repetitive and manual tasks and tests.
  • Analyze requirements and product specifications.
  • Create, implement, and execute tests to break our software
  • Interact with engineers and managers to create good testing processes and test plans for software projects.
  • Interact with customers to understand their testing requirements and report issues.
  • enforce the acceptance criteria of features;
  • Design automated test cases, review existing such and analyze results;
  • Executing different types of black-box testing, including functional and non-functional
  • Automating test cases using various tools and languages
  • Documenting and assuring the quality of software applications across all architectural layers
  • Define and execute functional, automation and performance test plans and strategies;
    •   Prepare test environment, business scenarios and scripts, test scenarios, data and test scripts;
    •   Execute test cases, file bug reports, and report on product quality metrics;
    •   Drive testability requirements into the product;
    •   Follow good engineering practices.
  • Develop new and maintain existing automation tests;
  • Write automatic integration tests.
  • Write test documentation.
  • Work on understanding scenarios, and reviewing test cases to ensure that they meet the testing approach.
  • Ensure that the business process is respected
  • Work in collaboration with the managers responsible for the quality of the deliveries.

The problem:

I believe every person in IT, and testing in particular, is very good at seeing patterns. Very often, when I review job offers I see the following pattern:

Testing is mostly presented as:

  1. Writing documentation:
    Examples:
    “Create detailed test plans and high fidelity test suites with test cases”
    Design automated test cases
    Write test documentation
  2. Performing predefined tests:
    Examples:
    “Execute functional, regression, integration and performance tests”
    “Executing different types of black-box testing, including functional and non-functional”
    “Execute test cases, file bug reports, and report on product quality metrics;”

Somewhere  along these, not so frequently, but yet worth mentioning, appear stuff like “being a good team player”, “exchange knowledge with team members”, “analyse documentation”, “communicate with clients”, etc.

So, the problem itself is that none of the above descriptions mention some of the core activities during testing:

  • Creative thinking – test design is something mentioned very often, but nobody ever mentioned how do we design those tests, it’s like they are written on their own, or “we just figured it out already”
  • Problem solving – every job description reviews testing just like it is execution of steps and not an active process of problem analysis and resolution.
  • Exploration of the product – Few thousand times I saw “analyse documentation” and “analyse requirements” and none I saw “analyse the product”. We are shipping the product, after all, analysing the requirements is testing the requirements, but they are not the product. And we are not even reviewing the case when requirements and docs are missing or being outdated.
  • Experimental approach – Testing is all about learing new information about the product by the process of experimenting with it. If we remove this activity from testing we turn it into mindless zombie clicking.

In other words – job offers describe testing as the unsexiest activity ever

The way that job offers describe testing is like a simple set of activities that all revolve around – documenting, test execution and reporting.

This is why testing is often falsely viewed as an activity that can be fully automated – because the part of testing visible to management and individuals outside of testing is the one that has components easy to automate – execution, documentation and reporting. While the important components of testing, ones that are vital for expert testing are often omitted or taken for granted, such as:

  • Exploration
  • Experimentation
  • Problem solving
  • Modeling
  • Questioning
  • Risk assessment

And guess what – all of them are totally non-automatable!

Which leads to my personal favorite from the list above:

Build tools and scripts to reduce the need for repetitive and manual tasks and tests.

I’ve also seen this formulated as: “Perform automation to reduce boring and repetitive tasks.”

Now, let’s be honest about two things:

  1. You can reduce the “need of repetitive testing”, by not performing repetitive testing, anymore. Captain Obvious thought me this one.
  2. If you consider testing “boring”, you probably lack the proper mind-set in order to perform it.

Dear testers, testing is activity based on natural curiosity, if you are curious enough and love what you do, you will find every part of your job interesting. And by “interesting” I don’t mean “amusing”, you don’t work to entertain yourself, by “interesting” I mean challenging. If you have the mind-set of looking for challenge in every task that you perform, you will be curious enough to look for a solution. On the other hand – if you lack the proper motivation and curious mind-set, you will find every activity in testing being dull and boring.

What can we do about it?

I think I stated the problem well enough, but the more important question comes to mind: what should be our reaction to this?

At first place I think it’s our fault. We have been agreeing for too long to be told what are we doing and what our job consists of, without making any complaints about it and now we complain about the result that we see.

What we can do in order to fix things. Well it’s fairly simple – tell your testing story in compelling and scientific way. I know and you know that testing is not what we are told it is – execution of test cases and filing bug reports. Well, then tell the true story about it. Make a presentation about what problem solving approach you’ve chosen in resolving a particular problem or write a post about how traditional test cases strategy failed you, so you had to “switch gears” to something more effective.

My advice is very simple – Tell the true story of testing, don’t let others tell you what it is.

I would love to read your thoughts on the topics. Any shares and retweets are highly appreciated.

The non-manual, unautomated tester

I’ve been struggling with the division of manual and automation testing, practically since I started testing – about 3 years ago. I’ve switched sides couple of times, probably sharing all the false expectations that people have even today, claiming to be on any of the both sides. After all I decided for myself that I won’t pick a side, that testing is something that’s above the approach that we use to perform it. Occasionally, I will speak with people and they will ask whether I am manual or automation tester and I will explain that I don’t fit any of these descriptions and what my attitude towards testing is. But often I notice a problem.

Manual and automation turned from job descriptions to a way testers define themselves.

It was probably the 100-th time when I listened to somebody asking for advice “how to switch from manual to automation” when it struck me – people actually define themselves as being part of a tribe, they seem to see that glass wall that’s stopping them to use tools in effective way. Which often leads to absurd claims like: “I can’t write code, because I work as a manual tester” or even a better one – “I want to work as an automation tester, so I don’t have to do repetitive tests manually”.

Let’s see a simple mind map of what testing consists of, what actions do we perform, while testing:

test activities mind map

 

So, as we can see in the picture above there’s one area where software testing can benefit from automating in a really effective way and that’s the actual testing performance. Of course, aggregating results and creating reports can also benefit from automation, but I don’t consider a pile of data, no difference how it is arranged, a valid report about testing, unless it contains some sort of analysis of the process, some part with conclusions, with pros and cons, etc.

I think you might start to realize what I mean, if you don’t, you might want to take a look at an old article of mine called Test automation – the bitter truth.

So, let’s think of some conclusions.

Automation is not manual testing on steroids.

No matter how deep in love you are with idea that a script will perform your job instead of you, I will have to make you unhappy, it won’t. Unless… you downgrade the quality of the job you perform that bad, that it is totally replaceable by any programmatic script. I can guarantee you that, if you consider testing to be only the “test execution”, yes, it will be possible to be fully automated, guess what – it will be also full of crap.

Machines will not overtake humanity, Skynet is not coming for you.

Like it or not, in testing there’s plenty of “brain work” to be performed, if you want to be successful in testing and want to deliver high value testing services, you should do the “brain work”, no matter how loud you scream it drains all your energy out. One advice – get used to it, this happens everyday, anytime, to any professional that has a creative job, this is what creativity is supposed to do – many professions – surgeons, engineers, writers, painters, scientists, developers AND TESTERS do it every day. Yes, they might use tools to facilitate their job, but they are not defined by their tools. It is useful to think of testing tools in the way we think about any other tools – as an extension of the human body, not a replacement of it.
Imagine a blacksmith – he uses a hammer, it’s a tool, but he’s not using it because it will help him work less, it’s totally the opposite – he uses it because it will help him achieve more, work better, more effective, but the creative part of his job, the expertise, the experience still remain property of the blacksmith, not of the tool.

By saying “I am an automated tester only”

I feel that the so-called “automation only” people try to skip the responsibility of owning their testing. Owning the creative and analytical part of their testing. In exchange their prefer leaning on tools and technology, looking for the root cause of poor testing in the tools they use and blame them instead, or the technology or the environment. They seems to call that “flaky tests”. But they are not flaky tests, they are crappy tests, it’s flaky logic that stands behind them. It’s like asking a fish to climb a tree and then blaming the fish for sucking at tree climbing.
In order to be successful and productive, we should know the limitations of our tools and one of the limitations of the automated tools we have today is – they are not humans and they can not replace them.

By saying “I am manual tester only”

Saying “I am manual tester only” seems like an admission of the fact that a person lacks technical knowledge, coding skills and perspective how to develop them, just because they feel insecure. So, they are willing to take one of the two options:

  1. Stay in that small labeled box, called “manual tester” and complain their asses off how much of an dying breed they are and how “big bad automation” will wipe us all.
  2. or try to look for that “magical shortcut” that will skyrocket them into “automation testing” where they will somehow automagically remain relevant and up-to-date, without doing any effort.

There’s no shortcuts and, of course, there’s nothing stopping you to use any tool, you don’t have to go to a ninja school to be able to use them. Develop the skills and get rid of titles and labeling, focus on results and none of the above will seem relevant to you.

Final, final conclusion

I am really not aware when was that historic moment when a white bearded, magician looking, prophet of software testing, struck his staff in the ground and divided the testers into manual and automation testers. And I still don’t quite grasp on the idea why should anyone consider himself being one or the other.

We are testers, we perform testing – it’s an important job, it’s a complex job and as any complex job it will greatly benefit to use tools to facilitate it. That’s it.

So, tell me, will you remain in that small square having only role written in it and no meaning, or you will try to reach your full potential as a professional tester?

Hope you enjoyed the topic guys, will love to read your opinions on it. 🙂

Software testing is.. part 1 – exploratory.

It’s been a while since my last blog post, I took a long break for the summer, but I am still not done with figuring out what software testing is.
Just as a quick overview of what happened until now, the basic idea behind all of that was that I am still seeing people being able to recognize what software testing is not, what doesn’t belong to software testing, but when it comes to define testing or describe what software testing is, they remain silent.
So, we made a quick run through what software testing is not, if you didn’t have the chance to take a look at the series I encourage you to start here: Software testing is not…

And again, I am not trying to redefine anything, I am not trying to make it into the software testing textbooks, although there isn’t any, my sole purpose is to try to make sense of these to myself. If it works for you too or you agree with me, it is just a bonus. So here’s what I figured out about software testing and what it is, for me:

testing-is-mindmap

 

So, here we go with #1:

Software testing is… exploratory by nature.

I must admit that, since I started working as a tester, I always encountered that concept of exploratory testing, mainly in the sense of “mingling around testing” or freestyle testing or non-reliable, but still useful sort of testing, sometimes. It was year ago or year and a half when I started to dig deeper in ET and found what its true nature is, in fact.

Unfortunately the misunderstanding of exploratory testing, being a technique, being unreliable, chaotic, not organized is still present.

The truth is, exploratory testing is not a technique or a method of testing, in fact, exploration is one of the core abilities that testing uses in order to achieve its goals.

All testing is exploratory.

And it’s true, in its nature all testing is exploratory. Testing happens whenever we want to know more about the product, whenever we want to unfold its secrets in front of us. The real moment of testing creation has nothing to do with executing scripted activities – it happens in the moment we ask ourselves and the product the question “What if I do this now…?”.

If you ask a random person in testing – “What is testing for?” or “What is the purpose of testing?” you will probably get a series of answers that are mostly part of the truth, like: to discover bugs, to assure quality, to expose problems, to assess risks, which is fine, but not completely, because above all of these we perform testing in order to gain information about the product. Information that is vital in order to assess risk, find bugs or any of the above. Such information could be obtained only by exploring the product and all the assets that come with it – documentation, user guides, requirements, design plans, etc.

Problems with exploratory testing being a technique.

In different resources and according to some testers, exploratory testing is a technique of testing, and testing could be either exploratory or scripted, which I think is wrong.

Let’s start with a logical question – if we claim that exploratory testing is a technique and it can be completely replaced by scripted testing, how are we gathering the information that we will use in our testing scripts if we have never explored the application in order to make ourselves aware of it, what parts does it have, what complexity, etc?

There can not be scripted testing without the existence of exploratory testing. The claim that there could be a test script or a test case that doesn’t contain information or insight gained with the help of exploration and experimentation could mean only one thing – that we have a script or a case that has nothing to do with the real behavior and the real application, but rather with a model that we built, based on something else, but not the real product.

Yes, I know how many will try to counter this – how about documentation, user guides, online help, etc? Well, here’s a thing – having your testing based on the docs or any other artifact supporting the product is testing only the artifact or what that artifact claims for the product, not the product itself. There’s many occasions and I believe everyone experienced this where documentation is outdated or doesn’t exist at all. How do you construct your testing then, if you only depend on documentation?

The bottom line of this point is – testing’s main purpose is to interact with the product, to dig deep in it, to experiment with it and to explore it and this could only happen with the help of interacting with it, happily there’s no other efficient way.

Conclusion:

My opinion and based on what I learn and experienced so far, is that it works best if we approach testing a new product just as we would if we were conquering a new land. There’s no rules, there’s no limits, there’s no expectations – we get to know them on the fly, we adapt and figure out our strategy while we try to explore the possibilities this “new land” has to offer.

That’s it for now, see you next time. Don’t forget to share and retweet and as always I will be more than happy to read you thoughts in the comments. Good luck!

 

Being the “one man army tester” – mentoring question

This is another question that was received via the mentoring form on my blog and it goes like this:

Message Body:
Hi,
I’m reading your blog every week it helps me a lot, i have a couple of questions which i hope to answer, first one:

1- In my current job i’m the only tester, i defined the processes from planning, design, execution and test closing, i perform manual testing and performing very little bit automation using selenium webdriver with C# to just locate elements and also i’m using tools for bug tracking and test management, but i have a gap in test reporting which i’ll try to fill, my question how to know that everything is going well and i gain experience and knowledge from it?? because I still feel that i’m not a good tester anymore with every new project i start to analyse and test and fear to fail or can’t test it and being creative, still feel that i should working with testing team and leave my current job, so what do you think i should do in this point?

2- Is it a MUST to study testing everyday or it’s enough to study for example two days per week? and how?

3- About resources: What resources should I use to learn testing? i bookmarked more than 25 blogs, forums and websites to read their feeds and i can’t commit with that, and this makes me depressed sometimes because the feel of i’m not a good tester and i should read all of them,
Finally i feel that i missing something and i can’t know what is it, also enforce myself to be a good tester by doing this and this and this and this all in one makes me do nothing.

Thanks,
John

 

Hello, John!

First of all, I’d like to thank you for the kind words, for reading my blog and using the mentoring form. I really appreciate your trust in my advice.

I know your pain, John, the strive for perfection is something that we can consider “professional disease” of our craft. After all, we aim for high quality in our job, but that also turns into desire to have high quality or perfect skill set on our own.

The first thing I’d like to say is – do not despair. You are not perfect, nor am I and that is completely O.K. After all, we are human beings, not robots and as we know even robots have mistakes or bugs. To be honest, the fact that you are reading blogs and trying to improve, trying to figure out stuff on your own, asking yourself if you are good enough – that’s enough to say that you are better tester than a huge amount of testers. I can not describe in words how many educated and experienced testers I’ve encountered that are not even moving their finger in order to do something about their personal development, their career, to improve their chances to be more productive and sell their own skills better. And this is all something that they will benefit from directly. So, you can imagine the kind of laziness that exists in certain software testers. So, up to this point, you are doing perfect.

Now, related to the questions specifically:

Question #1: Here I see couple different questions, so I will answer them separately.

Question #1.1: How to improve the process, being the one-man army tester?
At first, make sure you divide your effort by the highest priority. So, it is important for you to make sure what are the highest valued areas of the software you are working on and try to ensure they get covered first. In order to identify these areas, you should speak and discuss with people to whom that software matters – stake holders, clients, developers, project managers, designers etc. Software testing includes a huge  amount of communication and investigation, so we should not omit that, you need to learn to ask questions and mostly, to ask the right questions.
Also, it is a good idea, since you don’t have a large testing team to work with, to establish a strong professional trust between you and the development team. Make sure they do understand the specifics of your job, but don’t be too aggressive in doing it, just be nice and helpful, if you see confusion in their actions, explain. Make sure that they understand, that software testers are not the enemy, that we are not trying to make them look bad or less professional, that we – devs and testers are actually  all on the same boat and we should work together in order to produce high quality product.
Also, if you are alone and you need to keep up with a lot of projects, may be doing automation is not a good idea. Don’t get me wrong, automation is useful and it matters, but good automation takes time – to produce, to maintain, to extend, to make a really valuable framework that will help you. If you are overloaded with tasks, doing automation will just be additional overhead, that will take from your precious time.
And last, but not least, and that might also be answer to your third question – if you feel overloaded, if you feel that you are not progressing or you can’t keep track of your current tasks, because the work is too much, my advice is to speak to your managers, let them know what your issues are. Let them know you have certain struggles in performing well and see what they can offer about it, don’t expect them to be the proactive side, you should look after your professional interests. If they are not ready to listen or don’t want to change in order to ease your job, well, you have your answer.
Question #1.2: How to make sure I am keeping up with good quality of my work, how to make sure I improve and learn?
Well, there’s two important things here, I’d like to mention. First one is – high quality of work, fortunately is not an universal value, there’s no shared value that you have to keep up with. So, again speak with the “people who matter” in your software project, ask them if they are satisfied with the job you perform, if they see any areas of improvement and how exactly do they see it. Next is your  own personal understanding of feeling if you progress and learn. It’s a feeling you can never mistake, when you learn, when you do what you do with passion and interest, you can’t feel the time passing. If it’s not the case with you, you are probably bored and don’t do what you really want to.
Second thing is, in order to make sure you learn and improve, don’t forget that the competition is not with the others, but with your former self. So, in order to determine this, try to look at yourself from the perspective of time and try to compare your current self with your former self 1 year ago or 1 month ago, it doesn’t matter, you set the period. While comparing, try to answer the question “Did I gain something in that period?” – experience, knowledge, confidence, anything. This way, you will be able to determine if you are learning or stuck at one spot.
Question #1.3: Should I switch my current job and move to a company with big testing team?
Unfortunately, this is a question that I can’t and won’t answer. You are the one to make career decisions about your career and you are the one to evaluate when is the right time for a change. Just a few quick tips, before you make a decision:

  • Being in a team doesn’t necessarily guarantee you personal development opportunities. In fact, many software testing teams have very different dynamics and people sometimes join companies with big testing teams in order to “hide in the crowd”, so they could under perform and nobody would notice.
  • Being alone teaches you a lot, when you switch to a team, the drive that you have to study and improve might not get the response you expect.
  • Running doesn’t solve the problem. My suggestion is, ask and negotiate your opportunities in your current company, if you don’t feel happy there, if there’s no conversation, at least you tried your best.

Question 2: Is it a MUST to study testing everyday or it’s enough to study for example two days per week? and how?

I can not tell you how to learn, as learning is diverse process and it is different for any of us. Some people learn easier by reading (visually), some are learning better by discussing abstract topics, some by trying stuff and failing (kinesthetically or empirically), some combine two or more of these. So, it’s up to you, to try and find what works best for you.

How often? Well, of course not everyday, you have a personal life, man, go have a beer with friends, take a vacation, go to a party, testing is not a religious cult, it’s a passion. So, how often should you do it, as often as you feel happy to do it. If it’s interesting for you to read and learn new stuff everyday – go for it, but try not to burn out. Remember, Rome was not built within a night, therefore, some stuff take time and patience, not just hard work 24/7.

Question 3: Resources and how to keep up with reading them?

Well, if you think your story is bad, hear mine – last time I checked my feed I had 160 blogs in it, I am reading twitter and couple of other feeds with topics on testing, I try to read the new edition of every testing magazine that’s out and guess what – in the last 2-3 months I wasn’t able to read a single word, besides books I read before I go to bed. Do I feel bad? Absolutely not. We are human, we are not perfect and for sure we don’t have all the time in the world, therefore we can not read any piece of information that’s out  there and there’s no reason for that to make you unhappy.

What can I offer you in order to organize stuff? Well, start building some taste for the material you read, read what’s interesting for you. Not everything that’s out there is valuable, some blog posts are out dated, biased or simply full of shit, some of my blog posts are full of shit, perhaps or just not interesting to everyone, that’s O.K. . Learning to find materials that suit your needs, will greatly reduce the load you currently have, I recently heard that by John Sonmez and he called it – “learning to chew, but spit the bones, which means – consume only the information that is valuable and beneficial for you specifically.

Also, I would recommend you to start using Pocket it’s a neat little free app, has a web and mobile version, that help you organize web pages and blog posts, so anytime you have spare time, you will have a queue of interesting blog posts to read through.

Also, some good resources – all the books by Jerry Weinberg start with the Quality series and you will see why I recommend them to you.

Also, Lessons learned in software testing by Kaner, Bach, Bolton and Pettichord – very educational, very useful and practical read.

Follow on-line communities like Ministry of testing and utest – they  have a lot of resources on their own and you will be able to enter discussions and discover more interesting sources of information.

Read online testing magazines, here are 3 I can recommend:

Testing circus

Women testers

Tea time with testers

Well I think that’s it. Thanks once again for reading my blog and trusting my opinion, hope my answers will provide you with some insight, if not you can always ask more.
Thank you, John.

Best regards,
Viktor Slavchev

If you like to ask a question and get it answered by me, please don’t be shy, you can do it in my mentoring form here

Thanks for reading! Good luck! 🙂 

Test automation – the bitter truth

Recently, I came across this awesome blog post by Mike Talks where he tells the story of his personal experience with automation and all the false expectations that he and his team had about it, and it is really an awesome post, if you hadn’t read it, go do it now.

And I really got inspired by it, I recently write a little post concerning automation and I figured out I had much more to say. I was really busy doing the “Software testing is not….” series, so I didn’t really feel like going off topic. But now I feel the time came to tell each other some bitter truth about “test automation”. It is bitter, because people don’t like it, they don’t like talking about it, they don’t like promoting automation like this, but, yet it’s the truth.

It all starts with love.

I have to say I love automation, I really do. And the reason why I write this post is, because I love testing and automation and I want to give the best perspective how they can supplement each other. Yes, it is related to saying some uncomfortable truths about it, but it has to be done.

So, I said I love test automation, not because I believe it is the one and only solution to all my testing problems, yet I am sure when I was new I had that delusion about it, probably. The reason I like it, is because I like to code. I like writing cryptic “spells” and see them turn into magic and I really don’t care if it’s a source file, an interpreter or a console, I just love it.

I see a lot of people being excited about automation, but for a different reason – because they see it as the ultimate solution to all testing problems, the silver bullet, the philosopher’s stone or you name it, it seems to them that human testing might be  replaced with automated scripts and everything will be great, we will ship the product in no time and the world will be a better place.

I have to object this. And trust me, this is not a hate post against automation, in fact it is post aiming to justify why automation is useful, but useful in the right way, with the right expectations, not the way that we want it to be. So, in order to achieve this, I will have to write for some bitter truths, about automation.

Bitter truth # 1: It’s not test automation, it is tool assisted testing.

James Bach and Michael Bolton did an awesome job, making the difference between “testing” and “checking” , so I think it will be useless for me to repeat what they already said. If you are  not familiar with their work, I strongly recommend you do check “Testing and checking refined” and their white paper on automation in testing – A Context-Driven Approach to Automation in Testing.

In a nutshell, the testing that we perform as humans is not easy to translate in machine language, we can not expect from a machine to act like a human. So, they make the difference between “checking” – shallow form of testing that could be translated into set of instructions and  expected conditions and “testing” – describing the process of human testing with all its complexity and depth. Therefore, the term “test automation” carries a false meaning – we can not automate testing, we can automate checking and it is more correct to say “automated checking”. Yet, I prefer another  term they used, for the reason it sounds more elegant and provides more information – “tool assisted testing”. Because that’s what we do, when we automate something – we use tools to assist our testing, that’s it, we don’t want to replace it, or get rid of it. We want to use tools to enable us, to do more, rather than to replace the human effort in testing.

Bitter truth # 2: Automation doesn’t decrease cost of testing.

I am really interested how does that equation work?. People claim automation reduce cost of testing. Let’s see.

  • You hire an additional person to deal with building the framework and write the tests. (If you think your existing testers can do it part-time, while doing their day-to-day tasks, my answer is … yeah, right ) so that’s additional cost
  • You probably will spend additional money for the tool licence, or otherwise you will use open source tools which means that guy will have to spend additional man hours in order to make that framework work for your specific testing needs.
  • You will have to pay that guy to write the actual tests, not only the framework, so that’s additional cost.
  • The code that you write isn’t “pixie dust”, it’s not perfect, it turns into one additional code base that has to be taken care of, just as every other production code – maintenance, refactoring, debugging, adding new “features” to it, keep it updated. Guess what, it all will cost you money.
  • And of course, let’s not forget about all the moments of “oh” and “ah” and “what was the guy writing this fucking thinking” that you will run into, related to the framework that you use and its own little bugs and specifics, that will also cost you additional money and time investment.

I think that’s enough. The main reason that people give for the “low-cost” of automation is – it pays back in long-term. Now that’s awesome. And it would be true, if we accept that once written a check is a constant that never changes, that it works perfectly and our application never changes until the end of the world. Well, that would be nice, but there’s two problems:

  1. We all know from our testing experience that things written in code don’t actually work perfect, out of the box. In fact it takes a lot of effort to make them work.
  2. If your code base never changes, if it stays the same, you don’t add anything, you don’t change anything, you don’t redesign anything, it is pretty possible that you are going out of business. We are working in a fast evolving environment and we need to be flexible to changes.

So, again. I don’t get it, how having all of the above leads to cost reduction.

Bitter truth # 3: Testing code isn’t perfect.

I see that very often at conferences and in talks among colleagues. It seems that test automation scripts are some kind of miracle of the nature that never breaks, never has bugs, once you write it and it’s a fairy tale. Well, unfortunately it is not.

As stated above already, it is code, normal code just like any other, it has “its needs” – to be maintained, to be refactored and guess what … to be tested :O Why is nobody talking about testing the automated checks, I mean are these guys testers, did they left their brains in the fridge or something?! It’s code, it has bugs, it has to be tested, so we make sure it checks what we want it to check, wake up! Not only that, I bet you, that doing your automation, you will be dealing with bugs introduced by your checks, much more than bugs introduced by the code of the app that you test.

Bitter truth # 4: Automation in testing is more reliable… it’s also dumb as fuck

One of the many reasons why testers praise automation is – it is more reliable than human testing. It can insure us that every time we execute exactly the same check, with exactly the same start conditions, which is good, at least it sounds good. The truth is, it’s simply dumb and I mean machine-like dumb. It’s an instruction, it simply does what it is told, nothing more, nothing less. Yes, it does make the same thing every time over and over, until you don’t want it do that thing any more. Then it turns into huge pain in the ass, because someone will have to spend the time to go and update the check and make sure it checks the new condition correctly. In other words, automated checks are not very flexible, when it comes to changes and changes are something we see pretty often in our  industry.

Bitter truth # 5: Automated checks don’t eliminate human error

For reasons that I stated above I claim that automated checks can not eliminate human errors. Yes, they eliminate natural variety that human actions have, but that doesn’t mean we don’t introduce many more different new human errors while writing the code. The conclusion is simple, while the code is produced by a human being, it is possible it has errors in it, it’s just life.

Not only that, but having our checks automated, we introduce the possibility for machine error. Yes, our code has all the small bugs and weird behaviors that Java, C#, Python, Php or any other language have. The framework that we use also might have these, the interaction that it has with the infrastructure it runs on, might also introduce errors. So, we must be aware of that.

Bitter truth # 6: Automated checks are not human testing automatically

I see and hear that pretty often, everyone saying – “we know we can’t automate everything” and yet they continue to talk as if they could. Not only that, they talk like if they could mimic exactly the same process that a human is doing. And it is not possible. There is no human activity that is cognitively driven, that includes analysis, experience and experimentation to work together in order to achieve a goal, that could be automated, not now, not with the current technology. Yes, AI and robotics are constantly moving forward, if in some beautiful day that happens, I would love to see it. Until then, human testing can not be automated, at least not the one I understand as high quality human testing, what we can automate, though, is shallow testing and act like it will do the job, just right.

Conclusion

Again, this is not a hate post. It is informative post, informative of the risks that automation has. Clever testers know these risks and base their testing on that knowledge. And yet there’s a lot of people running excited from conference to conference, explaining how automation is the silver bullet in software testing and it will do magic for your testing. This is wrong.

Tool assisted testing is useful, it is very useful. And it is fun to do, but we have to use it in the right way – as a tool, as an extension to our testing abilities, not like if it’s their replacement. We should know why and when it works, how it works and what traps it might have for us. In other words, when we want to use it, we should know why and if we are using it in the right way.

And most important, human and machine testing, simply don’t mix. One can assist the other, but they are not interchangeable.

Hope you liked the post. If you did, I would appreciate your shares and retweets. If you didn’t, I would love to see your opinion in the comments. Thanks for reading 🙂 Good luck.

Software testing is not … easy to explain. Last part.

Software testing is not…

Software testing is not … part 2.

Software testing is not … part 3.

Software testing is not… finding bugs part 4

Software testing is not … playing. Part 5

There it goes, finally I come to the final chapter of the series of what software testing is not. The topics that I choose to cover in it are:

Software testing is not … set of predefined actions.

I bet you and I am pretty sure I would win, that if you ask 100 people related to testing and involved in software development, what software testing is about, you will mostly get answers like – finding bugs, following testing scripts and steps and scenarios, in order to discover bugs. So, why is that? What happened?

At first it is the enormous effort of companies and testing certification academies to “formalize” testing and represent it as something easy to do, easy to learn, easy to transfer as knowledge, so they easily sell it or replace one specialist with another.

And it’s totally understandable why people fall for that lie, why they prefer taking the short cut of the formalized version of testing, instead of the more analytical and more complex version of it. Imagine the following situation, just like Morpheus in Matrix with the red the blue pill in his hands. You are the new tester in our community and you have that choice:

  • You take the red pill, you go to certify yourself, you get to write test scripts in a very document or requirement driven way, you follow the development methodology guidelines, you follow the “best practices”, you act like tools are silver bullet that could solve all problems and that would ease the testing effort so much that you will practically have to do nothing, just hit run and the magic will happen.
    In general, to believe that testing is easy, once it is documented well, if you have the proper functional documentation, design documentation, if your testing scripts are written in a detailed way, so everyone could read and repeat them. Or…
  • You take the blue pill, you learn that software testing is complex cognitive activity, that its roots lie not only in the area of technology and programming, but much more in epistemology, psychology, sociology, logic, philosophy in general. You learn that certification means only, that you showed knowledge to someone’s understanding for software testing and that sometimes that “someone’s” understanding might have nothing to do with reality. You learn that your path as a software tester will be long and rough and you will have to put a lot of effort in order to stay relevant and make a difference. You will have to learn that no practice, methodology or tool is the ultimate solution to a problem, and every time you approach a problem you will have to investigate it and build your solution on the fly, rather than following someones “instructions”.
    You will learn that automation is in fact tool assisted testing, that its purpose is not to replace human testing, but to modify it, to enable it, to achieve more. You will learn that tools don’t do miracles, they only do what they are instructed to do, that automated checking is in fact one more responsibility in your testing practice and you will have to spend additional time taking care of it, extending it, debugging it, keeping it up to date and accurate.
    You will learn that testing is not about requirements and documentation, and that tester’s purpose is not to find bugs only, but to asses risks and provide information, not even to prevent issues from happening, just to provide accurate information about the product/project and the risks that might make the difference between successful product and a total failure.
    And last, but not least, you will learn that testing is about learning, it is organic, it is constantly evolving, it’s analytical, it’s experimental and is extremely hard to do it well, not only, it is also extremely responsible activity. It will probably take you decades to learn to do it well, to explain it well and to teach others to do it well.

So, given these two options, as a novice tester, which one will you choose?

matrix-morpheus-red-pill-blue-pill
Source:http://counterinception.com/sites/default/files/pictures/MatrixBluePillRedPill.jpg

… easy to explain

Unfortunately, in the above situation many novice testers will get the easy way, the short cut. And we shouldn’t blame them for that. There is one major reason for that and it is, testers don’t talk enough about testing, we don’t explain what testing is and what it is not and one of the reasons for that is – explaining testing is not easy.

Everyday in our careers we are told by other people what testing is and how we should perform it, but think about it, how many times did you take the responsibility to say: “No, this is wrong. Let me explain what testing is and why I do it that way”. Conforming with other people’s false opinion about testing is harmful and toxic for the testing craft, we should take care and responsibility to educate our team members or other members involved in software development, what testing is and how we do it, to achieve high quality.

And by this I don’t mean confront their point of view. We should approach this with a lot of understanding, because the effort to downgrade testing, to present it as formalized activity, to strip its organic and analytic nature was huge. That’s why we should approach such opinions in an educational way, just like a teacher. As in school, when a student has wrong opinion about some problem, if we just say “no, you are wrong, you should listen to me, because I am the one who knows”, we won’t achieve anything, in fact the chance to have the student rebel against our position gets even higher. Instead we should focus on providing the other perspective, explain why do we think that this is wrong, provide information, provide our personal experiences in support to our claim. We should drive the conversation or the argument in the educational domain, where both sides will have the opportunity to test their view points and rethink them, in order to gain knowledge.

And this is not easy, it’s not easy to speak about testing and do it in a structured way, making logical conclusions about your positions. That’s why we prefer to say what testing is not. And we are comfortable doing it, that was the reason why I started the series with “What software testing is not…”, but we should take that effort and move to the other side and start telling the story of what software testing is, what is its nature, how it is beneficial for the product and so on. And that’s what I intend to do.

So, as last words from these series, I’ve said about a hundred times, but I will again, testers, remain active – blog, participate in conferences, discussions, forums, webinars, write and read blogs, comment, make a fucking difference, no one will do that for you. 🙂

I hope you enjoyed the series, if you liked this post, I would love to read your comments, I will appreciate your shares and retweets.

Also, I have a challenge for you. Do you think I missed something? (I sure did) Were you ever pissed off by a ridiculous claim about software testing that you don’t see in this list, well tell me, extend my list, in the comments below. I would greatly appreciate your input.

Thanks for reading and I hope I will see you again guys in the second part of the series – what software testing is. 🙂 Good luck.

Automation or manual testing – mentoring question.

I am happy to say, that this is the first question that came through the mentoring form on my blog. It has been a while before someone “takes the courage” to ask a question 🙂 So, once again people, if you have any questions – ask, I don’t bite, I am only trying to help. As I promised, I won’t use real names, so I will replace them with any that I made up. Here it is:

Hello,
I have some questions for you. Manual or automation tester? And if I want to change manual testing with automation, how to do that. Is there good practise for that.

Thanks in advance
Lisa

Hi, Lisa!
Thanks for asking a question via the mentoring form on my blog. I greatly appreciate the fact that you ask for my opinion in your question.
Now, from the way that  your question is structured I can assume, that you are considering to move from “manual”  to “automation” testing. Why I put these in quotes, well it’s a long story… Anyway, I don’t believe in the devision of automation and manual, but many  people do.
So, back to your question, which one of both?
First, it’s a good idea to mention what do these two mean to non-testers involved in SDLC and, unfortunately, sometimes even to testers. “Manual” tester normally is translated as “someone banging on keys mindlessly”, no expertise is taken into account, no technical knowledge, he or she is just someone “looking for bugs”.
On the other hand, “automation tester” means someone smart enough to automate the “boring job” that “clicking monkeys” will normally do by hand. Now, I don’t agree with both these claims, but if you listen to people while they talk of testing, they mention these, implicitly or explicitly. That’s why automation testing seems to be that dream career that everyone looks after and manual testing is considered deprecated and everyone tries to get rid of it.
The facts – you are more likely to get better salary as automation tester, rather than as manual, it’s just the market. Also, people get more excited when you say you are doing automation, without any regards to how actually you are doing it, you just say “I am an automation tester” and  you are considered a demi-God or something.
How to move from manual to automation?
You have to be comfortable with programming. If you are not, get comfortable with it. If you can’t, forget about it. Yes, I know all these recorders and other tools that produce tests, but they are silly, if you want to do some real stuff you need to know at least one Object oriented language like C#, Java, Python, Ruby, Perl etc.
There’s plenty of courses to do that, a lot of tutorials, books and webinars, you just have to go there and practice. Also, you need to be familiar with the automation tools and frameworks there  is, in order to show some relevant knowledge in the domain. In general, this is Selenium, but it wont be a bad idea to add some variety to your skill set by looking at tools for mobile automation like Appium and Robotioum or some BDD tools like Calabash. Also, it is strongly recommended to be familiar with certain design patterns in automation, that will help you write more concise code, avoid repetition of logic and will make your job easier in general. You will have to also be aware, that moving in automation, what you will be doing 8 to 6 is development. This is what it is, you will be writing code that tests, doesn’t matter if it tests UI, service or database. So, if you find this boring in a way, you might want to think about it.
If you are asking how to do that from a career shift point of view, I’d say, just make some good looking examples of automated testing in GutHub and start going to interviews. This will be the best way to make yourself aware of what companies are looking for, what skills and frameworks and also, to benchmark you skills.
My personal opinion on testing manually vs. automatically is – look for a company that can offer you both and let you choose how to approach testing. Getting in any of the directions too deep, in my humble opinion, downgrades your testing expertise. Testing, after all, is something we do with our brains, our experience and our personal perspective, not something that depends on tools or manual techniques.

Thanks for your time!

Good luck,
Viktor Slavchev

If you like to ask a question and get it answered by me, please don’t be shy, you can do it in my mentoring form here

Thanks for reading! Good luck! 🙂 

Software testing is not … playing. Part 5

Software testing is not…

Software testing is not … part 2.

Software testing is not … part 3.

Software testing is not… finding bugs part 4.

Software testing is not … meant to add value

I know this is controversial claim and many testers might object it. So, let me explain.

This is my position on a silly claim that was trying to prove that human testing is not useful, because it doesn’t add value to the product, the way that software development does, therefore, we don’t need it and we can absolutely replace it with machine driven checks.

Now, I want to make an important distinction here. Testing is being valuable as being part of the development process, but that doesn’t mean our testing performance and expertise adds monetary value to the product itself. On the opposite, I think our job as investigators and analysts, providing information, assessing risk and quality is actually to help the product to keep its value and help the team to prevent devaluation.

In other words, testing is not meant to add monetary value from which our product or company can benefit. It is meant to provide information and insight about the product, about the processes, about the risks that can harm the value or compromise it and/or our credibility or our client’s as provider of a software solution. And let’s not forget, software is a solution to a problem our clients have, we as testers are not the solution itself, but we are involved the solution as far as we have to make sure whether or not the solution we provide solves the right problem, if it solves it at all and if it solves it in the most optimal way we can provide as a team, given the time, resources and constraints that we have. And this is not an easy task.

Therefore, saying that testing isn’t valuable because it doesn’t provide value is like saying breaks in the car are not useful, because they don’t add horse power. Also person claiming this must have very limited or shallow view on the software testing profession and its expertise.

… playing with the product

Many of the things I can add to this will overlap with the ones I mentioned in the first part of these series, that’s why I want to add some different point of view.

One of the reasons why people see testing as an easy activity that everyone can do is, because it looks easy, in fact to some non-testers it looks like playing with the product. Tell me how many times did you hear that phrase from a developer: “Just play with it a little bit, to see if it works” or “just check if it works”.

Pretty recently Ministry of testing made a list of the icky words in software testing and “checking” was one of them. Michael Bolton stated his disagreement with the presence of “checking” in the list, then he clarified he meant checking in the RST context.

Yet, I think words like “checking” do belong to the icky list and the reason why they belong is the reason why Michael Bolton and James Bach did so much work on checking and testing series – and the reason is – we, as professional testers, don’t like someone to downgrade our effort and expertise to simply checking. In other words testing is not limited to checking simple facts that can be formalized to the pattern:

if (condition) check that result == something.

Same thing applies to the term “playing”. We might look like playing with the product, but we don’t do it to waste our time or to entertain ourselves. Anytime you see a tester or a QA specialist “playing” with the product, you should be aware that there’s way much more analytical thinking, planing and structured actions behind what looks to you as “playing”.

In fact, every educated approach towards an activity looks like playing to a viewer non-familiar to the area of the expert. Have you ever seen a master chef preparing a meal or a professor in the university explaining on a subject? They always look like they are doing minor effort doing what they do, almost like playing. Yet, what they do is the result of years of experience, improvement, planned actions and most important – errors. In fact, this is one way to make sure you are actually becoming skillful in an area, if I have to rephrase Einstein’s words, is to make your field of expertise look like you are playing.

This is it for this part. Any comments and shares are welcome, thanks 🙂

Software testing is not… finding bugs part 4.

Software testing is not…

Software testing is not … part 2.

Software testing is not … part 3.

Software testing is not … finding bugs

This is one of the first things that I thought in software testing – that our job is to find bugs or defects. It’s true, but it took me some time, couple of books and a lot of hours reading different resources in order to figure out it’s not just that. To claim the above is like claiming that software development is just about writing code and it’s not.

At first, it is good to mention that we don’t automagically find bugs just like that. We have to learn the system and build our oracles based on our knowledge of it, previous experience, knowledge of the domain, technology knowledge and so on. So, no bugs included here.

How about documenting our process – writing models of the system, mind mapping it, write check lists or design test cases (in case that works in our context, of course). Is that finding bugs? No.

In other words, if we say that testing is about finding defects, we probably never had a more profound thought on what testing is and what is its role in the big picture.

Let’s talk about defects. What is a defect? To us, as testers, normally it’s an error in the code, may be a typo, may be a logical misunderstanding, but might be also bad  interpretation of requirements, or due to missing requirements, it might be “taking the easy short way” due to laziness, might be bad design or unexpected dependency. It might be a lot of things. What it is for our management? It is a way to lose money, a way to lose credibility, a way to lose clients, or combination of all of them, or even go out of business. What does a defect means to our clients? It means they can’t use the service or product they paid for, or have a risk to lose their clients and their money due to a mistake that we made, sometimes, defect on our side might cost someone’s life. So, here you see three different perspectives of what a defect is and I bet you can think of more.

So, what’s the point here? Testing is not about finding bugs, it’s about examining the risks that defects in our software might introduce. Who is impacted by these risks and how serious will be the impact. I believe this is a better way in determining what our job is, not just in our day-to-day tasks, but in the context of our business, how our job is valuable for what we are producing as end product. In other words, it helps us think of our work holistically.

… meant to add value

This reminds me a silly statement somewhere over the internet, that you see, testing is not necessary or important, because it doesn’t add value to the product. Well, that’s almost as stupid as saying that breaks are not necessary in a car, because they don’t add horse power.

From my point of view, testing was never meant to add value to the product, in fact one of the main goals of testing is to help the product and project from devaluation or in other words, loosing quality and our clients’ credibility due to defects that we might have missed.

So, to frame this in another different way – testing won’t help you to make a valuable product – that is something your ideas and strategy should take care of, but testing will help you not to fail miserably due to low quality and bugs that might have serious impact for your or your clients.

That’s for this week. Thanks for reading and if you found it interesting, please share in social medias and/or share your opinion in comments! 🙂