The state of testing 2013 – survey

This is a great initiative of the guys from  Tea Time with Testers and  QA Intelligence to start a survey and gather all the software testers form around the world and who would like to participate to give their opinion on the topic of software testing and some important questions that all of us have ot deal with everyday. Some of them are:

  • the challenges of software testing 
  • the right methodology
  • the right attitude
  • how do salaries of software testers vary around the world

I think this is a great idea, for everyone of us to support the community and show how much does he really care. If you are enthusiastic about this too, you could read the full post here http://qablog.practitest.com/state-of-testing/ and subscribe if you want to participate. 

Good luck

Quality means bugs

No, not that little fellow, I mean software bugs 🙂Scared bug

Introduction

I’ve had a lot of conversations with co-workers on whether a bug should be logged or not, and I really enjoyed some very creative excuses on skipping the defect tracking part of the debugging process. I don’t know if it’s ego, or simply attempting to make the work less formal or fear but in my honest opinion it’s a huge mistake and it could pretty soon stab you in the back.

Insectophobia or defectophobia

I mean, there should be a proper name for “fear of software bugs” or may be “fear to admit you are a human and you make mistakes”. I can’t say for sure it is exactly fear, but it’s in most cases some kind of discomfort when a person doesn’t matter if QA,  developer or even a project manager, has to report an issue. And this is where the silly excuses start:

“I’m wondering if it’s an issue or it’s the way it works” – well newsflash – wondering wont make it any better so read the requirements and if they are not clear enough, just ask, developers don’t bite… normally.

“If it’s not an issue I will look like an idiot” – but if it happens to be an issue you will be an idiot for sure. You will never learn anything if you don’t try. We as testers make mistakes, too. And this is how we learn.

“It’s a minor bug, we can skip it for now and log it later” – this is kind of tricky, because what’s a minor bug? Google says nothing, because there’s no theoretical definition on what’s minor… well of course it’s a bug with low importance but what’s low importance, then. If you are making a video game some of the NPCs might behave a little bit moron-ish and this might be completely fine if it doesn’t affect the overall experience – Assassin’s creed 3 had a lot of these, but let’s say you work on a medical software for brain surgery or autopilot on an airplane – what’s minor bug there? Just a small crash, I don’t think so…

“Don’t log this one it’s an easy fix” – You will hear this a lot during your QA career from the developers and you will know that they do it because they want to reduce the formal stuff of reading and writing solutions in bugs, because it’s really boring, but hey we are all humans, are we … and we all forget. And if you forget to log it and they forget to fix it, it will be all your fault when the client finds it and asks why. And one more thing, what will happen if you keep that work flow but you work in a huge software project with a lot of issues? You will find a developer with incredible memory or you will just log the issue so you are sure it’s there waiting for solution ?

“I don’t want this to affect my defect trend” – this is a good one, well to have a defect trend you need defects, right? And it’s purpose is to actually measure how well you are going on fixing them. So if you don’t log any or if you log only the “important ones” you will corrupt your results, which will make the trend totally useless.

No bugs doesn’t mean high quality product

If you use more man to man approach on reporting your bugs, let’s say via skype or by personal meetings, that’s also good, this way you improve your skills as a team, but this isn’t always the optimal solution to achieve high quality product.
Once I asked a co-worker “Why aren’t you logging any bugs, but reporting them via skype?” the answer was “Because it’s faster”. So I asked “How are we going to prove we’ve done any work on this if there’s no hard evidence for it, like bugs found?” the answer was “By a high quality product”. Now, this is misunderstanding – I wouldn’t consider a product that has 0 bugs in it’s bug tracking system a  high quality one, but rather a timer bomb waiting to explode. It’s very important for the QAs to realize that from the perspective of a non-technical user/client if a software product goes well and has no bugs, it’s always due to the developers’ amazing coding skills, but if there’s some bugs after the release – it’s always QA’s fault. It’s life, you may disagree, but you can’t change it. So, save yourself some pressure and log your bugs where they belong – in the defect tracking system.
Yes I know what many of you will say: “What if I work with amazingly skillful hyper-ultra-uber-senior-master developer that never makes mistakes?”. Well, I’d say, I will really like to meet this Obi-Wan Kenobi of software development myself, but until than, they are all under suspicion. It’s simple, until there’s people writing the code and people testing it, there will be mistakes, it’s up to us to find ’em and fix ’em, or leave them and look like fools.

Don’t get me wrong, not that we have to be proud of how many defects we’ve fixed, that wont make us look professional, too, but having bugs is part of the developments process, don’t be afraid to deal with it.

29169214 And that’s true, too. You will never be able to find all the defects and fix them, but at least give it a good try and make sure you are releasing a good quality product.

Conclusion

“There is no war without victims” they say and I will add “and there’s no software without bugs” and during your career as a software quality assurance you will have to find them and report them, this how it works. There’s no silver bullet for it, but try to follow the workflow, people didn’t invented the bug tracking tools just because they are cool, but because they are useful. And after all we are all humans and we make mistakes and by finding them and fixing them, we improve and go forward. Don’t be afraid to face issues and fix them, this will make you only a better professional.

Software testing – Mythbusters style

mythbustersThis idea came to my mind as a joke, I thought to myself “What if Mythbusters were software testers” and to be honest, the more I think of it, the more similarities I find between software QA job and the Mythbuster’s job. Of course not that QA’s are always trying to blow up and brake things, but that too. The main reason I find so many similarities is the process, the approach that Mythbusters use to attack the so called “myths”, and all of them look so much like a test plan. Here’s how it can be compared in a few simple examples.

Myth vs. test case

To give some visual representation on what I am talking about, let’s first take a look and this short video, a really typical example of mythbusting:

So, long story short – the myth is: if you mishandle a revolver, you may get your thumb ripped off, pretty simple.  And there’s our first similarity – this sounds pretty much like a test case – we have the user story – “as a user I should be able to rip off my thumb if I mishandle the revolver” and the exit states – “Pass” thumb lost – myth confirmed and “Fail” all fingers in place and myth busted.

Setting up “busting” environment:

As we could see from the video, Jamie and Adam started by first setting their testing environment – get a gun, get the high speed camera and of course – the artificial hands.

There’s one important thing not to be missed here – poorly designed testing environment leads to a corrupted result. In this specific case – the “hand” that Jamie created was not realistic enough and the damage it took wasn’t as much as expected.  So one important conclusion to make out of this is – always design your test cases to mirror the real world and to be objective. Non realistic test cases will just compromise the test results.

Time for myth testing

As you could already guess the attempts to “bust” the myth are the tests executed themselves. In this specific case it was a mini myth and I deliberately have chosen it but if you review any other myth you would see the exact same pattern.

And one last thing…

Good+Guy+Mythbusters_c8ea03_4151376

This was actually the main idea behind this whole article, if it isn’t blowing up, blow it up anyway. You could probably observe this in every bigger myth they deal with – they set it up, they test the so called “happy path” and if it doesn’t bust the myth, they repeat it but this time under extreme conditions.

Well in software testing this is called stress testing – it consists of testing your system, application or functionality under extreme conditions to check if it’s stable enough. This is a very important thing to learn from thеsе guys, you have to know how to break it, even if it’s in a ridiculously weird manner. The important part here is – better know where it crashes and decide if it’s “Ok” to go live with it, instead of letting your client find it and have yourself in the role of the “bad guy”. It’s vital to remember to test all positive paths first, before starting to hit the application with everything you have. Other wise you may fall in let’s call it “negative testing trap” which means getting so focused on “breaking” the app, that you may actually miss all defects that appear in it’s normal flow, and you really don’t want this to happen.

So, myth testing or software busting, what we should always aim as software testers is in first place – knowing the product we are testing and all it’s capabilities and of course test all the possible ways to destroy it, positive or negative ones.

Error, fault and failure – what’s it all about?

Since I recently started my career as a QA I am still learningQuality Assurance badge lots of new things. I wont lie if I say all the definitions from textbooks and ISTQB books still look a bit blurry to me. So every new breakthrough I achieve matters. Same case was with the definitions of “error”, “fault” and “failure”. At the beginning all these terms looked exactly the same to me, I thought “Why would anyone give a f**k what’s the definition, it has to be fixed, does it?” Well that’s what we are about to figure out.
First to mention is – in software business it’s all about credibility and every poor quality software product is kicked out with the speed of light. There’s a huge variety of software companies and if you don’t keep up top standards, you suck. You need to provide the best quality of software. Software without bugs is utopia but good software company doesn’t release a product with major issues, because they might cost a lot.

So let’s dive into definitions. Error – fault – failure might look familiar, but they actually have different impact.
Lets say we have a scenario like this:

A bank asks us to write a web based software which has only one job, to transfer money from client’s bank account to some cash desk where the client can withdraw. It only has log in form and one input field for the amount of money they want to transfer. They push one button and it’s done. Lets say we forgot to test if the input takes negative value. And we release … now if a client inputs negative value he’ll actually gain money. This is a bug we all know this, and it goes live you will cry.

Now, I know that’s a dumb example but it’s simple enough to notice few things:
What caused the problem is a simple “Error”, a human mistake, probably one or two lines of code.
In this case may be a simple check like this will do it:

[code]if (input < 0) {
return //some error here
};[/code]

But it actually doesn’t matter, what does is “How it costs for the developer to fix it?” Well as we see it’s just a matter of review some code find the error and fix it.

If he misses it, it goes to the QA, if he finds it there we have the second definition – a “fault”, meaning the error, led to some incorrect behavior of the app – user is able to input invalid value. We log a bug, we fix it, it’s still cool and safe for the client.

Bet let’s say the QA missed this and it went live – that would be a disaster, this is a “failure” – we failed to provide quality piece of software, our client might have lost millions because of this. What if it was a software for a space shuttle, or autopilot for an airplane, or a laser brain surgery tool – failures like these might cost even human lives. So having that in mind, we can describe the error – fault – failure dependency with this scheme.

ErrorScheme

So the conclusion is simple – the closer a problem gets to the client, the more effort it takes to get fixed, more money, more time and most of all – the highest price we pay is our good reputation – once  lost, it might take forever to build it again.

So be careful, code wise, test aggressive and clever, don’t be simply a developer or QA engineer, be a craftsman.