The Rob Lambert

You need to “Test It Better”

I’ve heard a lot of people recently talking about how they have been asked to “test better” or “test more” after bugs have been found in live. This is a question/statement often asked by people who assume (or expect) that testing guarantees no bugs in live. And here is the main problem. In a sense testing could continue forever on certain projects and you’d still potentially have bugs in the code. Most testers know that, but expectations and assumptions of others might not align.

 

There is also a misunderstanding of what testing is and how it is done. In my experience, the most experienced testers are those people who have never done testing, at least it appears that way when they talk about “testing better” or “testing more”. Testing is seen as the poor alternative to programming and an easy job to do. It’s often seen as someone running pre-defined test cases and ticking boxes.

Many people ignore the human element; skill, culture, outlook, motivation, experience, awarenes, understanding of features and requirements, competency levels of using certain technologies etc. And this ignorance of the human element is why many people assume one tester is the same as another and that testing can be done by anyone.

 

So statements like “test it better”, “test it more effectively” and the general expectation of completely bug free features are formed, in large, from a misunderstanding of what testing is, how skilled it can (and should) be and how relatively easy it is percieved.

 

But more worrying, in my experience, is the assumption in statements like “test it better” that testers are the only ones responsible for quality. That the buck stops with testing to ensure our products are good and our customers are kept happy.

 

I think we need to challenge statements like this and make people aware of other factors that can, and should, be considered when we find a bug in live. I’m not advocating pushing blame away or adopting sloping shoulders. Far from it. There are times when a tester needs to hold their hand up and say, “Yep. Missed that one which I really should have caught”, learn from it and move forward.

 

But there are times when in addition to saying “we should test it better” we should add in statements like:

 

  • We should specify it better
  • We should gather requirements better
  • We should budget better
  • We should resource it better
  • We should focus better
  • We should project manage better
  • We should code it better
  • We should unit test it better
  • We should integration test it better
  • We should UAT test it better
  • We should dogfood it better
  • We should support it better
  • We should train our end users better
  • We should communicate better
  • We should prioritise our work better

What we absolutely should not be doing though, is settling for anything less than a fair statement about testing. If we messed up, fair enough. If we did our job with pride and resolution then it’s time to step up and say so.

11 thoughts on “You need to “Test It Better””

  1. Our whole team is committed to quality and everyone engages in testing activities. Still, on rare occasions when a bad bug gets into prod, I’ve had someone say, “The testers should get together and figure out how this bug was missed”. I respond with “Good idea, except everyone who worked on the story including the programmers should get together and figure out how we can prevent a similar bug from being missed in the future”.

  2. Great post, Rob. Thanks for sharing.Strangely enough I was discussing this with a colleague of mine this afternoon as we were both feeling a bit down heartened about our individual assignments. We both agreed that when assignments are being planned out, it always seems to us like project managers and Developers think testing is, like you said, running pre-defined test cases and ticking boxes. That doesn’t take long, right? So testing tends to get under estimated and time lines get squeezed a bit because of a misunderstanding of what we actually do.The last paragraph was ace! “What we absolutely should not be doing though, is settling for anything less than a fair statement about testing. If we messed up, fair enough. If we did our job with pride and resolution then it’s time to step up and say so.” Very good point!Regards, Adam Brownhttp://testing.gobanana.co.uk

  3. A word of caution with words like “should”, “ought”, and the like. While Alistair Cockburn translates “should” with not, Jerry Weinberg calls these words lullaby language. What can we do to avoid that slipage the next time around? leaves room for many ways to deal with the problem. Or you might want to reframe that statement to “better compared to what?” or ask “we could have run that test, please tell which of the following tests we should have left out for tbis one.” Matt Heusser had a series of articles in 2009 on how to answer these impossible to answer questions in different contexts. I think we should understand should as a starting point for a conversation, not as blame.

  4. Hi Markus,Nice point. I have a goal this year to avoid confirmationary and dogmatic points of views or statements. I’ve failed miserably in this post I know, but sometimes you’ve got to write what feels right.CheersRob

  5. Hi Adam,Thanks for commenting. Glad you enjoyed the post. The squeeze and expectation that testing is simple is really frustrating but it’s something that we should be challenging, not only to make our jobs more enjoyable and rewarding but also to challenge underlying stereotypes and images of testing.ThanksRob

  6. Hi Lisa,Thanks for that story. I think that is absolutely the approach many more testers need to take. Sadly though I see a lot of environments with gated phases and clear departmental handovers which mean blaming people becomes easier. Thanks for commentingRob..

  7. Good thoughts! Just one question, what does “dogfood” mean in the context above?

  8. Hi Martin,Dogfooding is when a company (if they can) use their own product. So if you are a provider of financial accounting software, it would be prudent to dogfood that software rather than use a product from the competition.Dogfooding often occurs before the release, but it can happen at the same time and more often happens after the current release. For many companies dogfooding their own product can be incredibly powerful at finding bugs as a wider range of people are using the system, in everyday conditions.Thanks for commentingRob..

  9. Rob,I basically tell them to come live in my world for a while to see what testing is “really all about”. That usually wakes them up. Once they see all that is involved they get a new appreciation for the work I do. But there are always the issues with misperceptions, misconceptions and plain old misunderstandings. Educating people is a non-stop thing. And when you do get into a situation where people ‘get it’ then life is kinda good.Regards,Jim

  10. Hi Jim,That sounds like the best way to address any misconceptions. Get them working with you and see what it’s really like. I like it.Nice point about continuous education. That’s something I think many people lose sight of. Thanks for commentingCheersRob

Comments are closed.