Software testing is… part 2 – rooted in social science

This is the second part of the series software testing is, based on the mind map I provided in the initial post, you can take a look here: http://mrslavchev.com/2016/11/04/software-testing-is-part-1/

I bet that big part of the readers of this blog will be puzzled by the presence of the words software and social science in the same sentence and that’s OK. Here’s how it goes.

The current state of thinking about software testing.

When I was a newbie, not that I consider myself experienced now, I am just an old newbie, so when I was a newbie and first started studying the software testing craft I was presented with the following picture of software testing.

Software testing is a technical activity that relies on understanding of the technical side of the product, such as:

  • How the product is built – structure and frameworks
  • How specific functionalities are implemented?
  • What protocols are used for communication?
  • How does an application works – databases, interfaces, memory, processor usage?
  • How the internet operates – basic networking knowledge?
  • How is the application dependent on its environment – OS, hardware, 3-rd party libraries?
  • and many more

All of this is correct, but far from enough. It would be enough, if we only accept the definition that an application is just a series of instructions executed following a specific logic.

I like the example that Cem Kaner gives in the BBST foundation course, saying the above is like saying a house is just a bricks and materials ordered in a specific way, which is not true. A house is a place where someone can live and feel at comfortable, so there’s something more to a house than just the materials. Same with a software product – it’s not just the code, but it comes to partially replace a human interaction, but I will get into more depth in this a bit later.

The soft skills people.

Suddenly, when speaking of testing a person or two will show up saying that testing is not just technical, but incorporates some so-called “soft skills”. By “soft skills” they normally mean some part of negotiation skills, some part of team work abilities, some part of leadership, presentation skills, communication, etc. I say “some part”, because, so far, I didn’t hear or read any credible explanation what “are soft skills made of”, exactly, normally they are a small amount of this and that, etc.

I also agree, that this is important, too, but too general, in my opinion. It’s expected by a good professional in almost any area, interacting with clients/colleagues or any other human beings, to be a good communicator or a good negotiator and a good team player. So, I personally consider these – basic corporate survival skills, they are not something that’s strictly specific to software testing. So, what is it?

Software testing is rooted in social science.

Social science cloud
Source: http://www.proprofs.com/flashcards/topic/social-science

I mentioned earlier that a software product is not just a bunch of computer instructions ordered following specific logic. Software applications came to existence in order to replace, although partially, a human interaction, therefore, software applications are not only there to do what they are supposed to do, but they also have to bring us comfort and satisfaction. To understand how an application comforts our clients and how we can measure this in a reliable way, we need to understand the human nature and this is where social sciences come to help us. Or, if I have to use the house analogy Kaner made, it will be more useful to know how to evaluate a comfortable house, rather than just knowing how it’s built.

So, many times, under the influence of Kaner, Bach, Bolton and the CDT community in general, I have claimed that software testing is a science, or may be not a science, but uses many scientific methods and approaches and it corresponds with other sciences, not only but we can apply knowledge from other sciences in testing, so our testing can evolve and be more diverse. The aim of this topic is to give some perspective how testing corresponds with social science and which branches of it, I believe were useful for me until now. It’s important to mention that I am not considering myself an expert in any of the following, so all of the conclusions are made by my personal, non-expert, observations.

Psychology

In fact psychology is a science that can help any aspect of life and it has so many branches and sub-branches, that it will be practically impossible to cover all the possible applications it can have on software testing.

Psychology is science about the mind and behavior and that’s something that we rely on very heavily in testing, in two aspects:

Clients

In order to satisfy our clients’ needs, producing specific software product, we want to make them happy, we want to influence their emotions. And in no way, we want to influence their emotions in a negative way. That’s why, psychological knowledge can be useful in software testing, specifically in areas like usability evaluation. If you want more information on this I recommend you the book “The Design of Everyday Things” by Don Norman where you can learn how design of some utensils we use in our daily life has actually the ability to call for specific action.

Ourselves

The second aspect in which psychology can be very useful to software testing is sort of reflection oriented. We are professionals in assessing quality and performing experiments, but how can we say if we make the right experiments, if we have the correct judgement or in other words – how do we test a tester ?

Well, there’s no guideline actually, how to be a better tester or how to have the proper judgement, but we can be aware of what our own weak sides are or what known “defects and features” (because most of them are actually features) our minds have. And this is where cognitive psychology can help a lot.  We need to know how our minds can trick us, how our perception might be distorted, absolutely naturally and how our judgement can become biased. Another interesting topic that cognitive psychology reviews is the heuristic method for solving complex problems. This is a topic that is often discussed in the CDT community and a popular approach of solving testing problems, even when people are unaware they are doing it and don’t call it a heuristic.

A book that might be of great help to those curious of you, which makes a great analysis of all of these is: “Thinking, Fast and Slow” by Daniel Kahneman

Quantitative and qualitative research

Again, without any ambition to be an expert – this is something that is widely used in other fields like statistics and marketing and believe it or not a problem that we solve in testing every day.

In testing we often have that argument on whether or not our testing process should be structured around trying to provide some specific metrics or focus on experiment that is profound and provides valuable contextual information.

Seems that in science this is a problem that existed for ages in the face of quantitative and qualitative scientific research methods. And we deal with it in testing as well. How many times have you heard the argument on whether or not we should use metrics to direct our testing and if yes what kind of metrics, how do we translate these metrics in the language of quality and risks? How are they meaningful for us and for our peers?

On the other hand, there’s this group of people saying that metrics are easily manipulatable and they give us a good statistical results, but bad human or contextual results. They advice us to more “qualitative”, human and contextual approach in testing, which is in its nature experimentation.

Another question arises on how we choose one or the other, or if we have to choose at all? Do we need to combine them? If yes, how do we combine them, which is first, which is more important.

Seems to me we have a lot to learn from the field of qualitative and quantitative scientific research, in fact while preparing this article I reviewed some materials that made me think of a separate post on this topic only, so expect a follow-up on this in the near future.

Linguistics

This might seem a little bit odd to you may be, but expressing ourselves in testing is actually a vital part of our job. I often see people having arguments on the words that we use in testing or what a specific testing term means, what is the purpose of using this term over that term, I also often see these distinctions being qualified as “just semantics”, but I don’t think people realize how vital are these semantics for the quality of our craft. And believe me, this comes from me – a person that normally communicates his thoughts in testing in a language that’s not his native, so I am used to saying something stupid or incorrect and having to deal with the result.

Here’s how proper use of the language can be crucial for testing:

Bug advocacy:

Bug reporting is not just a routine task that we do in our day-to-day activities, although it looks like one. Quality bug report is not “just a bug report”, it’s your evaluation of the quality of the product, sometimes it’s the only proof of your input to the product. So, preparing a high quality bug report is crucial for everyone in the team and the product itself.

The more you develop that skill, the more credibility you and your job gain in the eyes of development, analysts, management and so on. And believe me, no matter how much time and effort you’ve put into building it, you can ruin it within a day, by letting negligence crawl into your bug reporting language, so it is very important to be careful, professional and specific in what we report as a bug. Don’t forget there are people depending on our judgement.

Describe your strategy and tell it in a compelling way

I think I wrote about this for probably a thousand times and a thousand times more won’t be enough to put the accent on how important it is to be able to explain what you do in a compelling and professional way.

Turns out in my experience, that many testers, even experienced ones have no issues performing their job as testers, including all their routine tasks, all but one – being able to explain what do they do. In other words, in the words of Harry Collins in fact, they hold a lot of “tacit” knowledge and they have problems turning it into explicit, in fact sometimes we even fail to explain our explicit knowledge in a good way, without oversimplifications.

This is why linguistic knowledge and the rich vocabulary and the ability to be precise in what you do are so important. Yes, we do report bugs, but no one is looking for bug reporters, after all, they are looking for testing professionals.  In order to be professional tester you have to know testing, to be able to do it well and to be able to explain and defend the approach that you’ve chosen and sound like a professional while you do it.

Philosophy

Philosophy has so many branches and so many great thinkers involved in it, that I can only wish I can make a profound analysis how it is involved in testing. Also, I am probably not going to dive too deep in it, as I plan to write a separate blog post on how software testing is related to epistemology.

Here’s just some quick thoughts on how philosophy is related to testing. What does philosophy means? From ancient Greek, the literal translation is – “love to wisdom”. Well, that’s interesting, I think I’ve seen this somewhere in IT as well, remember these guys that are always ready to learn more and more about the product, that claim software testing is perpetual learning ?

And there’s actually more, but hey, I don’t want to spoil all of it all at once.

And I think this topic became waaay much longer than planned. I hope with the above I gave some more ideas to think on next time you speak with someone on how “technical” testing is and I hope you consider the big part of social science knowledge that we use, in order to be valuable testers.

Of course, I would love to read your thoughts on the topic, that’s what comments are for. Like, share and retweets are always appreciated. Good luck. 🙂

The non-manual, unautomated tester

I’ve been struggling with the division of manual and automation testing, practically since I started testing – about 3 years ago. I’ve switched sides couple of times, probably sharing all the false expectations that people have even today, claiming to be on any of the both sides. After all I decided for myself that I won’t pick a side, that testing is something that’s above the approach that we use to perform it. Occasionally, I will speak with people and they will ask whether I am manual or automation tester and I will explain that I don’t fit any of these descriptions and what my attitude towards testing is. But often I notice a problem.

Manual and automation turned from job descriptions to a way testers define themselves.

It was probably the 100-th time when I listened to somebody asking for advice “how to switch from manual to automation” when it struck me – people actually define themselves as being part of a tribe, they seem to see that glass wall that’s stopping them to use tools in effective way. Which often leads to absurd claims like: “I can’t write code, because I work as a manual tester” or even a better one – “I want to work as an automation tester, so I don’t have to do repetitive tests manually”.

Let’s see a simple mind map of what testing consists of, what actions do we perform, while testing:

test activities mind map

 

So, as we can see in the picture above there’s one area where software testing can benefit from automating in a really effective way and that’s the actual testing performance. Of course, aggregating results and creating reports can also benefit from automation, but I don’t consider a pile of data, no difference how it is arranged, a valid report about testing, unless it contains some sort of analysis of the process, some part with conclusions, with pros and cons, etc.

I think you might start to realize what I mean, if you don’t, you might want to take a look at an old article of mine called Test automation – the bitter truth.

So, let’s think of some conclusions.

Automation is not manual testing on steroids.

No matter how deep in love you are with idea that a script will perform your job instead of you, I will have to make you unhappy, it won’t. Unless… you downgrade the quality of the job you perform that bad, that it is totally replaceable by any programmatic script. I can guarantee you that, if you consider testing to be only the “test execution”, yes, it will be possible to be fully automated, guess what – it will be also full of crap.

Machines will not overtake humanity, Skynet is not coming for you.

Like it or not, in testing there’s plenty of “brain work” to be performed, if you want to be successful in testing and want to deliver high value testing services, you should do the “brain work”, no matter how loud you scream it drains all your energy out. One advice – get used to it, this happens everyday, anytime, to any professional that has a creative job, this is what creativity is supposed to do – many professions – surgeons, engineers, writers, painters, scientists, developers AND TESTERS do it every day. Yes, they might use tools to facilitate their job, but they are not defined by their tools. It is useful to think of testing tools in the way we think about any other tools – as an extension of the human body, not a replacement of it.
Imagine a blacksmith – he uses a hammer, it’s a tool, but he’s not using it because it will help him work less, it’s totally the opposite – he uses it because it will help him achieve more, work better, more effective, but the creative part of his job, the expertise, the experience still remain property of the blacksmith, not of the tool.

By saying “I am an automated tester only”

I feel that the so-called “automation only” people try to skip the responsibility of owning their testing. Owning the creative and analytical part of their testing. In exchange their prefer leaning on tools and technology, looking for the root cause of poor testing in the tools they use and blame them instead, or the technology or the environment. They seems to call that “flaky tests”. But they are not flaky tests, they are crappy tests, it’s flaky logic that stands behind them. It’s like asking a fish to climb a tree and then blaming the fish for sucking at tree climbing.
In order to be successful and productive, we should know the limitations of our tools and one of the limitations of the automated tools we have today is – they are not humans and they can not replace them.

By saying “I am manual tester only”

Saying “I am manual tester only” seems like an admission of the fact that a person lacks technical knowledge, coding skills and perspective how to develop them, just because they feel insecure. So, they are willing to take one of the two options:

  1. Stay in that small labeled box, called “manual tester” and complain their asses off how much of an dying breed they are and how “big bad automation” will wipe us all.
  2. or try to look for that “magical shortcut” that will skyrocket them into “automation testing” where they will somehow automagically remain relevant and up-to-date, without doing any effort.

There’s no shortcuts and, of course, there’s nothing stopping you to use any tool, you don’t have to go to a ninja school to be able to use them. Develop the skills and get rid of titles and labeling, focus on results and none of the above will seem relevant to you.

Final, final conclusion

I am really not aware when was that historic moment when a white bearded, magician looking, prophet of software testing, struck his staff in the ground and divided the testers into manual and automation testers. And I still don’t quite grasp on the idea why should anyone consider himself being one or the other.

We are testers, we perform testing – it’s an important job, it’s a complex job and as any complex job it will greatly benefit to use tools to facilitate it. That’s it.

So, tell me, will you remain in that small square having only role written in it and no meaning, or you will try to reach your full potential as a professional tester?

Hope you enjoyed the topic guys, will love to read your opinions on it. 🙂

Software testing is.. part 1 – exploratory.

It’s been a while since my last blog post, I took a long break for the summer, but I am still not done with figuring out what software testing is.
Just as a quick overview of what happened until now, the basic idea behind all of that was that I am still seeing people being able to recognize what software testing is not, what doesn’t belong to software testing, but when it comes to define testing or describe what software testing is, they remain silent.
So, we made a quick run through what software testing is not, if you didn’t have the chance to take a look at the series I encourage you to start here: Software testing is not…

And again, I am not trying to redefine anything, I am not trying to make it into the software testing textbooks, although there isn’t any, my sole purpose is to try to make sense of these to myself. If it works for you too or you agree with me, it is just a bonus. So here’s what I figured out about software testing and what it is, for me:

testing-is-mindmap

 

So, here we go with #1:

Software testing is… exploratory by nature.

I must admit that, since I started working as a tester, I always encountered that concept of exploratory testing, mainly in the sense of “mingling around testing” or freestyle testing or non-reliable, but still useful sort of testing, sometimes. It was year ago or year and a half when I started to dig deeper in ET and found what its true nature is, in fact.

Unfortunately the misunderstanding of exploratory testing, being a technique, being unreliable, chaotic, not organized is still present.

The truth is, exploratory testing is not a technique or a method of testing, in fact, exploration is one of the core abilities that testing uses in order to achieve its goals.

All testing is exploratory.

And it’s true, in its nature all testing is exploratory. Testing happens whenever we want to know more about the product, whenever we want to unfold its secrets in front of us. The real moment of testing creation has nothing to do with executing scripted activities – it happens in the moment we ask ourselves and the product the question “What if I do this now…?”.

If you ask a random person in testing – “What is testing for?” or “What is the purpose of testing?” you will probably get a series of answers that are mostly part of the truth, like: to discover bugs, to assure quality, to expose problems, to assess risks, which is fine, but not completely, because above all of these we perform testing in order to gain information about the product. Information that is vital in order to assess risk, find bugs or any of the above. Such information could be obtained only by exploring the product and all the assets that come with it – documentation, user guides, requirements, design plans, etc.

Problems with exploratory testing being a technique.

In different resources and according to some testers, exploratory testing is a technique of testing, and testing could be either exploratory or scripted, which I think is wrong.

Let’s start with a logical question – if we claim that exploratory testing is a technique and it can be completely replaced by scripted testing, how are we gathering the information that we will use in our testing scripts if we have never explored the application in order to make ourselves aware of it, what parts does it have, what complexity, etc?

There can not be scripted testing without the existence of exploratory testing. The claim that there could be a test script or a test case that doesn’t contain information or insight gained with the help of exploration and experimentation could mean only one thing – that we have a script or a case that has nothing to do with the real behavior and the real application, but rather with a model that we built, based on something else, but not the real product.

Yes, I know how many will try to counter this – how about documentation, user guides, online help, etc? Well, here’s a thing – having your testing based on the docs or any other artifact supporting the product is testing only the artifact or what that artifact claims for the product, not the product itself. There’s many occasions and I believe everyone experienced this where documentation is outdated or doesn’t exist at all. How do you construct your testing then, if you only depend on documentation?

The bottom line of this point is – testing’s main purpose is to interact with the product, to dig deep in it, to experiment with it and to explore it and this could only happen with the help of interacting with it, happily there’s no other efficient way.

Conclusion:

My opinion and based on what I learn and experienced so far, is that it works best if we approach testing a new product just as we would if we were conquering a new land. There’s no rules, there’s no limits, there’s no expectations – we get to know them on the fly, we adapt and figure out our strategy while we try to explore the possibilities this “new land” has to offer.

That’s it for now, see you next time. Don’t forget to share and retweet and as always I will be more than happy to read you thoughts in the comments. Good luck!

 

Being the “one man army tester” – mentoring question

This is another question that was received via the mentoring form on my blog and it goes like this:

Message Body:
Hi,
I’m reading your blog every week it helps me a lot, i have a couple of questions which i hope to answer, first one:

1- In my current job i’m the only tester, i defined the processes from planning, design, execution and test closing, i perform manual testing and performing very little bit automation using selenium webdriver with C# to just locate elements and also i’m using tools for bug tracking and test management, but i have a gap in test reporting which i’ll try to fill, my question how to know that everything is going well and i gain experience and knowledge from it?? because I still feel that i’m not a good tester anymore with every new project i start to analyse and test and fear to fail or can’t test it and being creative, still feel that i should working with testing team and leave my current job, so what do you think i should do in this point?

2- Is it a MUST to study testing everyday or it’s enough to study for example two days per week? and how?

3- About resources: What resources should I use to learn testing? i bookmarked more than 25 blogs, forums and websites to read their feeds and i can’t commit with that, and this makes me depressed sometimes because the feel of i’m not a good tester and i should read all of them,
Finally i feel that i missing something and i can’t know what is it, also enforce myself to be a good tester by doing this and this and this and this all in one makes me do nothing.

Thanks,
John

 

Hello, John!

First of all, I’d like to thank you for the kind words, for reading my blog and using the mentoring form. I really appreciate your trust in my advice.

I know your pain, John, the strive for perfection is something that we can consider “professional disease” of our craft. After all, we aim for high quality in our job, but that also turns into desire to have high quality or perfect skill set on our own.

The first thing I’d like to say is – do not despair. You are not perfect, nor am I and that is completely O.K. After all, we are human beings, not robots and as we know even robots have mistakes or bugs. To be honest, the fact that you are reading blogs and trying to improve, trying to figure out stuff on your own, asking yourself if you are good enough – that’s enough to say that you are better tester than a huge amount of testers. I can not describe in words how many educated and experienced testers I’ve encountered that are not even moving their finger in order to do something about their personal development, their career, to improve their chances to be more productive and sell their own skills better. And this is all something that they will benefit from directly. So, you can imagine the kind of laziness that exists in certain software testers. So, up to this point, you are doing perfect.

Now, related to the questions specifically:

Question #1: Here I see couple different questions, so I will answer them separately.

Question #1.1: How to improve the process, being the one-man army tester?
At first, make sure you divide your effort by the highest priority. So, it is important for you to make sure what are the highest valued areas of the software you are working on and try to ensure they get covered first. In order to identify these areas, you should speak and discuss with people to whom that software matters – stake holders, clients, developers, project managers, designers etc. Software testing includes a huge  amount of communication and investigation, so we should not omit that, you need to learn to ask questions and mostly, to ask the right questions.
Also, it is a good idea, since you don’t have a large testing team to work with, to establish a strong professional trust between you and the development team. Make sure they do understand the specifics of your job, but don’t be too aggressive in doing it, just be nice and helpful, if you see confusion in their actions, explain. Make sure that they understand, that software testers are not the enemy, that we are not trying to make them look bad or less professional, that we – devs and testers are actually  all on the same boat and we should work together in order to produce high quality product.
Also, if you are alone and you need to keep up with a lot of projects, may be doing automation is not a good idea. Don’t get me wrong, automation is useful and it matters, but good automation takes time – to produce, to maintain, to extend, to make a really valuable framework that will help you. If you are overloaded with tasks, doing automation will just be additional overhead, that will take from your precious time.
And last, but not least, and that might also be answer to your third question – if you feel overloaded, if you feel that you are not progressing or you can’t keep track of your current tasks, because the work is too much, my advice is to speak to your managers, let them know what your issues are. Let them know you have certain struggles in performing well and see what they can offer about it, don’t expect them to be the proactive side, you should look after your professional interests. If they are not ready to listen or don’t want to change in order to ease your job, well, you have your answer.
Question #1.2: How to make sure I am keeping up with good quality of my work, how to make sure I improve and learn?
Well, there’s two important things here, I’d like to mention. First one is – high quality of work, fortunately is not an universal value, there’s no shared value that you have to keep up with. So, again speak with the “people who matter” in your software project, ask them if they are satisfied with the job you perform, if they see any areas of improvement and how exactly do they see it. Next is your  own personal understanding of feeling if you progress and learn. It’s a feeling you can never mistake, when you learn, when you do what you do with passion and interest, you can’t feel the time passing. If it’s not the case with you, you are probably bored and don’t do what you really want to.
Second thing is, in order to make sure you learn and improve, don’t forget that the competition is not with the others, but with your former self. So, in order to determine this, try to look at yourself from the perspective of time and try to compare your current self with your former self 1 year ago or 1 month ago, it doesn’t matter, you set the period. While comparing, try to answer the question “Did I gain something in that period?” – experience, knowledge, confidence, anything. This way, you will be able to determine if you are learning or stuck at one spot.
Question #1.3: Should I switch my current job and move to a company with big testing team?
Unfortunately, this is a question that I can’t and won’t answer. You are the one to make career decisions about your career and you are the one to evaluate when is the right time for a change. Just a few quick tips, before you make a decision:

  • Being in a team doesn’t necessarily guarantee you personal development opportunities. In fact, many software testing teams have very different dynamics and people sometimes join companies with big testing teams in order to “hide in the crowd”, so they could under perform and nobody would notice.
  • Being alone teaches you a lot, when you switch to a team, the drive that you have to study and improve might not get the response you expect.
  • Running doesn’t solve the problem. My suggestion is, ask and negotiate your opportunities in your current company, if you don’t feel happy there, if there’s no conversation, at least you tried your best.

Question 2: Is it a MUST to study testing everyday or it’s enough to study for example two days per week? and how?

I can not tell you how to learn, as learning is diverse process and it is different for any of us. Some people learn easier by reading (visually), some are learning better by discussing abstract topics, some by trying stuff and failing (kinesthetically or empirically), some combine two or more of these. So, it’s up to you, to try and find what works best for you.

How often? Well, of course not everyday, you have a personal life, man, go have a beer with friends, take a vacation, go to a party, testing is not a religious cult, it’s a passion. So, how often should you do it, as often as you feel happy to do it. If it’s interesting for you to read and learn new stuff everyday – go for it, but try not to burn out. Remember, Rome was not built within a night, therefore, some stuff take time and patience, not just hard work 24/7.

Question 3: Resources and how to keep up with reading them?

Well, if you think your story is bad, hear mine – last time I checked my feed I had 160 blogs in it, I am reading twitter and couple of other feeds with topics on testing, I try to read the new edition of every testing magazine that’s out and guess what – in the last 2-3 months I wasn’t able to read a single word, besides books I read before I go to bed. Do I feel bad? Absolutely not. We are human, we are not perfect and for sure we don’t have all the time in the world, therefore we can not read any piece of information that’s out  there and there’s no reason for that to make you unhappy.

What can I offer you in order to organize stuff? Well, start building some taste for the material you read, read what’s interesting for you. Not everything that’s out there is valuable, some blog posts are out dated, biased or simply full of shit, some of my blog posts are full of shit, perhaps or just not interesting to everyone, that’s O.K. . Learning to find materials that suit your needs, will greatly reduce the load you currently have, I recently heard that by John Sonmez and he called it – “learning to chew, but spit the bones, which means – consume only the information that is valuable and beneficial for you specifically.

Also, I would recommend you to start using Pocket it’s a neat little free app, has a web and mobile version, that help you organize web pages and blog posts, so anytime you have spare time, you will have a queue of interesting blog posts to read through.

Also, some good resources – all the books by Jerry Weinberg start with the Quality series and you will see why I recommend them to you.

Also, Lessons learned in software testing by Kaner, Bach, Bolton and Pettichord – very educational, very useful and practical read.

Follow on-line communities like Ministry of testing and utest – they  have a lot of resources on their own and you will be able to enter discussions and discover more interesting sources of information.

Read online testing magazines, here are 3 I can recommend:

Testing circus

Women testers

Tea time with testers

Well I think that’s it. Thanks once again for reading my blog and trusting my opinion, hope my answers will provide you with some insight, if not you can always ask more.
Thank you, John.

Best regards,
Viktor Slavchev

If you like to ask a question and get it answered by me, please don’t be shy, you can do it in my mentoring form here

Thanks for reading! Good luck! 🙂 

Organic testing

Note: Before I start this, I want to share the story how it came to my mind. I was working on the Lightning talk – “The hidden wisdom of the testing troll” and I actually made it as a joke, using all the public buzz that all organic food offer. When I thought about it, testing is organic in it’s nature. So, I decided it’s worth it to split that topic in a separate post.

It’s also worth it to mention I recently found out that James Bach once made a talk for Apple called “Organic QA”, I saw it in his archives, but I am not aware what he was talking about, since it was internal talk for Apple employees only. So, I am in no way adding anything to his point, yet I will be interested to know what he was talking about, if at some point he decides to share it.

How “organic” came as term in first place.

I mentioned a couple of times a talk about education by Sir Ken Robinson, where he compares the current state of education to a “fast food” model – it serves high demand of production, it aims to produce the same product, no matter which school or university the graduate is from and mostly it applies the “one size fits all” strategy, using formalized tests in order to measure and assess knowledge. What Sir Ken Robinson offers an alternative is a more “agricultural” or “organic” approach to education, where every student is given personal care, based on his needs and his “talents”, because human creativity and intelligence are diverse and unique and we can’t try to “equalize” it by applying same principles to different personalities. It is a very interesting talk and worth watching it, if you didn’t I strongly recommend it to you.

Seems we have a similar problem in testing, too…

While I was talking of education, do you felt we can relate some of these things to testing, too? Me, either.

In the large-scale. Testing is thought as a process that could be “mass produced” just as in a fast food restaurant, that testing skills are something that could be trained just as easy as making hamburgers.

People pretend testing is a skill that is easily transferred by repeating monotonous, meaningless actions that are meant to be based on some script or scenario, pretend testing is easy to learn, easy to master, as long as you follow the steps carefully.

Not long ago, I listened to this wonderful interview by Joe Colantonio with Gerald Weinberg. In it Jerry mentioned how testing appeared in first place, as a job that developers didn’t wanted to do themselves – testing, so management decided to give the task to a specific role, to someone they will pay less than a developer to do.

And like it or not, I still see signs of that attitude towards testing nowadays. Testing is always considered less technical, less creative than development, less experimental than development, if you like even less intelligent than development. Tester is commonly thought as a person that is just “looking for bugs”, “checking if it’s O.K.”, “playing with the product”, “responsible for quality”(keeping in mind nobody normally speaks what quality is, to whom the product is considered of high or low quality and why are we evaluating it in first place).

And that all “mythology” around testing is false, bad testing might be performed like this, lazy testing might be performed like this, but not what we aim for – expert testing. I spoke a lot about this in my series “Outdated testing concepts” and “Software testing is not…”, about the falsified mythology of testing. So, here we get to the bottom of it, what is testing? Well, I might for sure say that testing is organic.

The organic nature of testing.

And of course we are not talking about any testing – we are talking about testing performed by qualified, educated, self-driven and ready to develop person(big emphasis on the word person). Yes, we are looking for a personal approach – as it was mentioned above, intelligence is diverse, humanity is diverse, and this diversity is actually something we are looking for, something we are interested in. The way that one tester will look into a problem, the way one tester will perform testing is unique to him only, and  this kind of diversity is something we are interested in, not mass-producing testing minions that think alike, act-alike and test-alike.

 

Why is testing organic?

  • Diversity as opposition  to conformity – as mentioned above, organic as opposition to the “one size fits all model” is what we are looking for. We are looking for every tester’s personal view on the testing problem, we want variety of solutions, not just any solution, we want many paths in which we can obtain information about the product, the risks that it has, the needs that it will satisfy, not just one solution.
  • Testing is constantly evolving
    It’s a fact that  many people try to make us testers believe, that testing is the same as it was in the 70s or the 80s that nothing has changed, that it lies on the same principles. Well, news flash – if it was like this, if testing wasn’t evolving, it would have died, simple as that. This is another proof for the organic nature of testing, it is constantly evolving, and all of us involved in it, contributing to it in any matter – surveys, blogs, discussions, magazines, podcasts, webinars, workshops, courses, books, anything helps develop and refine testing as a craft or in other words, help it evolve as an organism.
  • It has its own life cycle.
    I know many of us were forced to think testing is part of another cycle – the Software development life cycle, but testing as a process has life cycle on its own. And we are supposed to be the ones who have to be aware of that life cycle – when is the early pre-mature period, when it is too early to start testing, when testing will lead to no results or such of low importance, when is the peak, when is the period when we are able to explore the full potential of the deliverable we are testing and when is it to late to do it. All these, again resemble to me, organic nature in the process of software testing.
  • It can not be  fully automated.
    And the reason, as mentioned above, is diversity. We are interested in the different things that human testers can do, not the repetitive ones that our scripts can perform over and over again. Because where there’s unpredictability, that’s where the interesting stuff happens. Of course, automation has its honorable place in the testing process, but it’s in the tool domain, not in the organic part. If you are interested in my more profound opinion on test automation, you might like to take a look at Test automation – the bitter truth.
  • It needs specialized, constant, adaptable care, in order to produce something meaningful.
    Just as any other organic thing, no matter if it’s a carrot or a dog, testing requires specialized care in any case where we approach something to test it. Yes, we might rely on same principles, but we have to adapt the specific contextual needs of the situation in order to produce satisfying results with our testing. That’s why we don’t rely on algorithms, golden, rules, best practices or any other magic formulas, that seem to offer solution, but are somehow not following that same principle.

That’s it for this post. I hope you don’t mind my metaphorical style in it, it was all meant to be  for fun and interesting, I said I will write about it, so it was an experiment I had to do. In case the topic was  interesting, as always I will appreciate your opinion in the comments, tweets and shares in the social medias. Thanks for reading and good luck. 🙂

 

Software testing is not … easy to explain. Last part.

Software testing is not…

Software testing is not … part 2.

Software testing is not … part 3.

Software testing is not… finding bugs part 4

Software testing is not … playing. Part 5

There it goes, finally I come to the final chapter of the series of what software testing is not. The topics that I choose to cover in it are:

Software testing is not … set of predefined actions.

I bet you and I am pretty sure I would win, that if you ask 100 people related to testing and involved in software development, what software testing is about, you will mostly get answers like – finding bugs, following testing scripts and steps and scenarios, in order to discover bugs. So, why is that? What happened?

At first it is the enormous effort of companies and testing certification academies to “formalize” testing and represent it as something easy to do, easy to learn, easy to transfer as knowledge, so they easily sell it or replace one specialist with another.

And it’s totally understandable why people fall for that lie, why they prefer taking the short cut of the formalized version of testing, instead of the more analytical and more complex version of it. Imagine the following situation, just like Morpheus in Matrix with the red the blue pill in his hands. You are the new tester in our community and you have that choice:

  • You take the red pill, you go to certify yourself, you get to write test scripts in a very document or requirement driven way, you follow the development methodology guidelines, you follow the “best practices”, you act like tools are silver bullet that could solve all problems and that would ease the testing effort so much that you will practically have to do nothing, just hit run and the magic will happen.
    In general, to believe that testing is easy, once it is documented well, if you have the proper functional documentation, design documentation, if your testing scripts are written in a detailed way, so everyone could read and repeat them. Or…
  • You take the blue pill, you learn that software testing is complex cognitive activity, that its roots lie not only in the area of technology and programming, but much more in epistemology, psychology, sociology, logic, philosophy in general. You learn that certification means only, that you showed knowledge to someone’s understanding for software testing and that sometimes that “someone’s” understanding might have nothing to do with reality. You learn that your path as a software tester will be long and rough and you will have to put a lot of effort in order to stay relevant and make a difference. You will have to learn that no practice, methodology or tool is the ultimate solution to a problem, and every time you approach a problem you will have to investigate it and build your solution on the fly, rather than following someones “instructions”.
    You will learn that automation is in fact tool assisted testing, that its purpose is not to replace human testing, but to modify it, to enable it, to achieve more. You will learn that tools don’t do miracles, they only do what they are instructed to do, that automated checking is in fact one more responsibility in your testing practice and you will have to spend additional time taking care of it, extending it, debugging it, keeping it up to date and accurate.
    You will learn that testing is not about requirements and documentation, and that tester’s purpose is not to find bugs only, but to asses risks and provide information, not even to prevent issues from happening, just to provide accurate information about the product/project and the risks that might make the difference between successful product and a total failure.
    And last, but not least, you will learn that testing is about learning, it is organic, it is constantly evolving, it’s analytical, it’s experimental and is extremely hard to do it well, not only, it is also extremely responsible activity. It will probably take you decades to learn to do it well, to explain it well and to teach others to do it well.

So, given these two options, as a novice tester, which one will you choose?

matrix-morpheus-red-pill-blue-pill
Source:http://counterinception.com/sites/default/files/pictures/MatrixBluePillRedPill.jpg

… easy to explain

Unfortunately, in the above situation many novice testers will get the easy way, the short cut. And we shouldn’t blame them for that. There is one major reason for that and it is, testers don’t talk enough about testing, we don’t explain what testing is and what it is not and one of the reasons for that is – explaining testing is not easy.

Everyday in our careers we are told by other people what testing is and how we should perform it, but think about it, how many times did you take the responsibility to say: “No, this is wrong. Let me explain what testing is and why I do it that way”. Conforming with other people’s false opinion about testing is harmful and toxic for the testing craft, we should take care and responsibility to educate our team members or other members involved in software development, what testing is and how we do it, to achieve high quality.

And by this I don’t mean confront their point of view. We should approach this with a lot of understanding, because the effort to downgrade testing, to present it as formalized activity, to strip its organic and analytic nature was huge. That’s why we should approach such opinions in an educational way, just like a teacher. As in school, when a student has wrong opinion about some problem, if we just say “no, you are wrong, you should listen to me, because I am the one who knows”, we won’t achieve anything, in fact the chance to have the student rebel against our position gets even higher. Instead we should focus on providing the other perspective, explain why do we think that this is wrong, provide information, provide our personal experiences in support to our claim. We should drive the conversation or the argument in the educational domain, where both sides will have the opportunity to test their view points and rethink them, in order to gain knowledge.

And this is not easy, it’s not easy to speak about testing and do it in a structured way, making logical conclusions about your positions. That’s why we prefer to say what testing is not. And we are comfortable doing it, that was the reason why I started the series with “What software testing is not…”, but we should take that effort and move to the other side and start telling the story of what software testing is, what is its nature, how it is beneficial for the product and so on. And that’s what I intend to do.

So, as last words from these series, I’ve said about a hundred times, but I will again, testers, remain active – blog, participate in conferences, discussions, forums, webinars, write and read blogs, comment, make a fucking difference, no one will do that for you. 🙂

I hope you enjoyed the series, if you liked this post, I would love to read your comments, I will appreciate your shares and retweets.

Also, I have a challenge for you. Do you think I missed something? (I sure did) Were you ever pissed off by a ridiculous claim about software testing that you don’t see in this list, well tell me, extend my list, in the comments below. I would greatly appreciate your input.

Thanks for reading and I hope I will see you again guys in the second part of the series – what software testing is. 🙂 Good luck.

Automation or manual testing – mentoring question.

I am happy to say, that this is the first question that came through the mentoring form on my blog. It has been a while before someone “takes the courage” to ask a question 🙂 So, once again people, if you have any questions – ask, I don’t bite, I am only trying to help. As I promised, I won’t use real names, so I will replace them with any that I made up. Here it is:

Hello,
I have some questions for you. Manual or automation tester? And if I want to change manual testing with automation, how to do that. Is there good practise for that.

Thanks in advance
Lisa

Hi, Lisa!
Thanks for asking a question via the mentoring form on my blog. I greatly appreciate the fact that you ask for my opinion in your question.
Now, from the way that  your question is structured I can assume, that you are considering to move from “manual”  to “automation” testing. Why I put these in quotes, well it’s a long story… Anyway, I don’t believe in the devision of automation and manual, but many  people do.
So, back to your question, which one of both?
First, it’s a good idea to mention what do these two mean to non-testers involved in SDLC and, unfortunately, sometimes even to testers. “Manual” tester normally is translated as “someone banging on keys mindlessly”, no expertise is taken into account, no technical knowledge, he or she is just someone “looking for bugs”.
On the other hand, “automation tester” means someone smart enough to automate the “boring job” that “clicking monkeys” will normally do by hand. Now, I don’t agree with both these claims, but if you listen to people while they talk of testing, they mention these, implicitly or explicitly. That’s why automation testing seems to be that dream career that everyone looks after and manual testing is considered deprecated and everyone tries to get rid of it.
The facts – you are more likely to get better salary as automation tester, rather than as manual, it’s just the market. Also, people get more excited when you say you are doing automation, without any regards to how actually you are doing it, you just say “I am an automation tester” and  you are considered a demi-God or something.
How to move from manual to automation?
You have to be comfortable with programming. If you are not, get comfortable with it. If you can’t, forget about it. Yes, I know all these recorders and other tools that produce tests, but they are silly, if you want to do some real stuff you need to know at least one Object oriented language like C#, Java, Python, Ruby, Perl etc.
There’s plenty of courses to do that, a lot of tutorials, books and webinars, you just have to go there and practice. Also, you need to be familiar with the automation tools and frameworks there  is, in order to show some relevant knowledge in the domain. In general, this is Selenium, but it wont be a bad idea to add some variety to your skill set by looking at tools for mobile automation like Appium and Robotioum or some BDD tools like Calabash. Also, it is strongly recommended to be familiar with certain design patterns in automation, that will help you write more concise code, avoid repetition of logic and will make your job easier in general. You will have to also be aware, that moving in automation, what you will be doing 8 to 6 is development. This is what it is, you will be writing code that tests, doesn’t matter if it tests UI, service or database. So, if you find this boring in a way, you might want to think about it.
If you are asking how to do that from a career shift point of view, I’d say, just make some good looking examples of automated testing in GutHub and start going to interviews. This will be the best way to make yourself aware of what companies are looking for, what skills and frameworks and also, to benchmark you skills.
My personal opinion on testing manually vs. automatically is – look for a company that can offer you both and let you choose how to approach testing. Getting in any of the directions too deep, in my humble opinion, downgrades your testing expertise. Testing, after all, is something we do with our brains, our experience and our personal perspective, not something that depends on tools or manual techniques.

Thanks for your time!

Good luck,
Viktor Slavchev

If you like to ask a question and get it answered by me, please don’t be shy, you can do it in my mentoring form here

Thanks for reading! Good luck! 🙂 

Software testing is not … part 3.

Links to the previous two parts:

Software testing is not…

Software testing is not … part 2.

Software testing is not activity that could be performed by irreplaceable robots.

I can say for sure that this is one of the most incorrect statements in testing and also one of the most toxic ones. Of course no body is saying that, no body is that dumb, but it is based on a false assumptions and it produces a lot of other false assumptions which become popular for the only reason that someone trusts them blindly, without asking questions. So, I split these assumptions in two groups:

  1. Everyone can test, testing is easy. Your manager can test, the dev can test, the designer can test, the client can test, your managers dog can test, etc, etc… But, they can’t do it, because they are busy doing their important stuff, so you seem to be the guy/girl who has to do it.
  2. Testing is easy, because a well written test is described in a finite number of steps or instructions, and if these instructions are followed exactly you can test the problem without any impediments.

Now, what I want you to do is – write down these on a piece of paper, bring it outside and burn it, while chanting some evil spell. 😀

Believe it or not, the above statements somehow made it in our craft and are even institutionalized, meaning a lot of people take them for granted. Which leads to a whole  bunch of false assumptions that derive from these, such as:

  • Every step in testing needs to be documented in a script or a test case.
  • If you don’t have test cases, you are doing testing wrong.
  • If you don’t have test cases, you can’t prove that you did any job on the project.
  • Testing is expensive.
  • Testing is unnecessary.
  • Testing can be performed automatically, so we reduce cost.

It’s easy to understand why these myths made it into our craft – one reason is, the senior management doesn’t really care about testing, they care about budgets and efficiency. Another reason is – we fail to provide the true story behind testing, how it is efficient and beneficial for the product/project, if we fail to do that, it’s our fault. And third reason – there are way too many vendors of tools and software testing courses and academies that have interest to sell you “snake oil” – the ultimate solution that will solve all your testing problems, whether it is a tool or methodology, it doesn’t matter.

Why testing can’t be performed by irreplaceable robots, and here I mean both human robot-like acting specialist and actual automation? One good reason for that is the fact that testing isn’t based on instructions, it’s based on interaction, simultaneous information gathering, evaluation and action according to that information. Testing is experiment, investigation and decomposition of statements that the others (development, management, you name it) believe are true(paraphrasing James Bach). Using all this, we have to provide relevant information, about the risk and the product itself. So, how all that aligns with the “concept” of following instructions?

And there’s one more general reason why testing can not be represented as a limited set of connected steps or instructions, which causes also the impossibility to fully automate testing – converting these instructions to machine instructions. The reason is the way knowledge works. More specifically the impossibility to convert tacit knowledge into explicit or even explicable knowledge(Collins, “Tacit and Explicit knowledge”).

This might be explained very simple – we all done cooking, sometimes, right? Well, we all know a cooking recipes are sort of algorithmic, step-by-step guide how to make a dish X. And they could be written by a really good expert, like a master-chef, for example. Yet, following his advice, even to the smallest detail, you might be able to make a decent dish, but not one made by the expert. You wouldn’t notice the small tweaks that the master chef will make, depending on his environment or the product that he/she uses, or the freshness of the spices, all based on a “gut feeling”. Why is this so? We will call that experience or routine, what it really is –  the huge underwater part of iceberg of knowledge, called tacit knowledge.

Same thing is applicable to any area, testing as well. “We can know more than we can tell” (Polanyi), therefore in testing we do much more than we can actually put in words, scenarios, cases, steps, scripts or you name it. Testing is an intellectual process and it has organic nature, it is normal to be inexplicable and for us to have trouble describing it or documenting it. That’s how knowledge and science work.

The reason why this is not a shared belief in the mainstream testing is the fact that many consultants and certification programs like to give a short and exciting story on “how can you become great tester by following this simple set of rules”. And it  is “sold” to us under the cover of the words “best practices”, meaning “I have nothing to prove to you, this is what everyone is using”. Well, newsflash, this is the testing community, you do a statement here, you have to be ready to defend your position, no body cares how much of a guru or an expert you are.

That’s it for this part, would like to read you comments and reactions on the social media. Thanks for reading. 🙂

Software testing is not … part 2.

Software testing is not … part 1.

I am moving towards next couple of branches from the mind map that I posted in first post of the series of “Software testing is not…”. You can find the mind map following the link up here.

Software testing is not … quality assurance.

I wrote about it in the “Quality guardian” part of Outdated testing concepts, so I will really try not to repeat, but build on the statements made there.

I believe it is more correct to say that quality assurance is not limited to the testing role alone and it’s something that has to be a philosophy and strategy shared by the whole development team, no matter what role. The opposition that I want to distinct clearly here is the old concept of quality guardian, having the right of veto to the release versus a modern approach of team responsibility towards quality. It is obvious that the former model doesn’t work even though many organisations still support the myth.

I’d rather like to think of quality as collective responsibility and I want to emphasize on the term collective responsibility, not shared responsibility. Why do I want to make that important distinction? If we share the responsibility, we may intentionally or not, give bigger part of it to a single member or make another less responsible. Collective responsibility require a collection of all team members’ personal responsibilities on the project. This way, if one person fails to be quality driven and delivers a crappy work, his part of the collection will be missing, therefore not all criteria for collective quality will be present. Also, we are able to determine quickly where things go wrong.

To summarize, we are really not in the position to give a god like decisions on whether or not the product should go live, whether or not a bug is critical enough to hold a release etc. But that isn’t necessarily a bad thing, because we can do so much more. Testing is about being aware of the products risks and providing correct information about it. Therefore, we can be quality advocates, but we are not actually assuring it by ourselves. And I’d like to present to you a quote from James Bach’s paper the Omega tester:
You are part of the process of making a good product, but you are not the one who controls quality. The chief executive of the project is the only one who can claim to have that power (and he doesn’t really control quality either, but don’t tell him that).”

And if you are interested in that topic, Richard Bradshaw made an awesome video in his series “Whiteboard testing”, where he explains his view-point on that same issue, you can watch it here: I’m QA. It’s in QA. It’s being QA’ed. You sure about that?

Software testing is not … breaking the product.

I will never get tired repeating what Michael Bolton has to say on that same topic and it is: “I didn’t break the software. It was already broken when I got it.”

It is funny, I know most people say this in ironical way, but yet that sort of speaking is one of that “toxic” phrases that actually do us no good and harm our craft along with stiff like “playing with the proiduct”, “just checking if it works”, “finding bugs”, but I will cover those in the next branches.

We are not actually breaking the product in the meaning of – causing damage that was never there and changing the consistency of the logic of the SUT. Our job is to expose inconsistencies in the logic, misbehaviour, false assumptions in logic or more generally – problems that occur while the SUT is performing its job. So, as you see, there’s no demolition squad, no bombing, no gunfire, we are not that bad, what is our job is – investigation, we have to investigate such behaviour, document the process that causes it and determine what are the reasons why we believe it is actually incorrect and raise the awareness of people who are interested in fixing it or the ones that will be affected if it continues to exist.

That’s it for these two branches of the mind map, please feel free to add your opinion in the comments and share it with your followers if you liked it. Thanks for reading. 🙂

Some kick ass blog posts from last week #12

And we are going back to some kick ass blog posts from the last week:

Another round-up posts:

Automate the planet’s Compelling Sunday by Anton Angelov.

That’s it for now, see you next week. 😉