You need to “Test It Better”

I’ve heard a lot of people recently talking about how they have been asked to “test better” or “test more” after bugs have been found in live. This is a question/statement often asked by people who assume (or expect) that testing guarantees no bugs in live. And here is the main problem. In a sense testing could continue forever on certain projects and you’d still potentially have bugs in the code. Most testers know that, but expectations and assumptions of others might not align.

 

There is also a misunderstanding of what testing is and how it is done. In my experience, the most experienced testers are those people who have never done testing, at least it appears that way when they talk about “testing better” or “testing more”. Testing is seen as the poor alternative to programming and an easy job to do. It’s often seen as someone running pre-defined test cases and ticking boxes.

Many people ignore the human element; skill, culture, outlook, motivation, experience, awarenes, understanding of features and requirements, competency levels of using certain technologies etc. And this ignorance of the human element is why many people assume one tester is the same as another and that testing can be done by anyone.

 

So statements like “test it better”, “test it more effectively” and the general expectation of completely bug free features are formed, in large, from a misunderstanding of what testing is, how skilled it can (and should) be and how relatively easy it is percieved.

 

But more worrying, in my experience, is the assumption in statements like “test it better” that testers are the only ones responsible for quality. That the buck stops with testing to ensure our products are good and our customers are kept happy.

 

I think we need to challenge statements like this and make people aware of other factors that can, and should, be considered when we find a bug in live. I’m not advocating pushing blame away or adopting sloping shoulders. Far from it. There are times when a tester needs to hold their hand up and say, “Yep. Missed that one which I really should have caught”, learn from it and move forward.

 

But there are times when in addition to saying “we should test it better” we should add in statements like:

 

  • We should specify it better
  • We should gather requirements better
  • We should budget better
  • We should resource it better
  • We should focus better
  • We should project manage better
  • We should code it better
  • We should unit test it better
  • We should integration test it better
  • We should UAT test it better
  • We should dogfood it better
  • We should support it better
  • We should train our end users better
  • We should communicate better
  • We should prioritise our work better

What we absolutely should not be doing though, is settling for anything less than a fair statement about testing. If we messed up, fair enough. If we did our job with pride and resolution then it’s time to step up and say so.

Rapid Reporting

After meeting Shmuel Gershon, probably the most energetic and enthusiastic tester on the Planet, at EuroSTAR 2010, I was introduced to a man of many talents. Not only does he like to read, write and talk about testing but he’s also adding back to the community in other ways with probably the most useful tool I’ve ever used whilst testing.

 

I introduce to you…….Rapid Reporter.

 

In essence, it is a reporting tool that allows you to jot down your thoughts, bugs, tests, notes, configs and setups as you test. Things I would have normally used Notepad++ for or a simple writing pad. But the tool is more than that really because it not only allows you to write down your testing notes, but it frees you up from breaking your lines of enquiry/testing by being so simple and easy to use.

 

The interface is incredible lean and minimalist and everything is pretty much controlled from the keyboard. It allows screenshots and rich text and pushes it all out in CSV format afterwards meaning it’s fairly generic.

 

It’s got some neat little features like the transparancy setting which I find myself using a lot whilst running multiple desktops, each one cluttered with various heads-up feeds or information radiators or browsers. Another neat feature is the countdown of session time and warning when you exceed the session time set. Nice.

 

Rapid Reporter has become my favourite little tool of choice now and no testing session starts without it running in the background. It’s proven invaluable when troubleshooting, retrospectively analysing sessions and tracking what sequences/patterns show bugs. I find myself typing more notes than I would have traditional entered and I also list out a lot more data than before which means more data when logging defects or replicating bugs.

 

So why not give it a download and see how you get on with it. And don’t forget to check out Shmuel Gershon’s blog too.

Cannot Reproduce

Image from : viZZZual

Call me old fashioned, but in all of my years working nothing has beaten a good old face to face conversation with a developer about an issue or bug in the software. Using a bug tracking system as the primary means of communication is a “no no” for me.

Yet, last week I was chatting with a seasoned test manager who was extolling the virtues of bug reports. Now, I’m a fan of bug reports (in moderation) and I’m a fan of tracking issues (though not necessarily in a bug report) and I’m also a major fan of communicating clearly, but I also like to get things done with the least amount of friction. But this test manager was suggesting that bug reports are the single best form of communication between developers and testers and therefore every single bug should be logged in a defect tracking system mainly for no other purpose than communication. Communicating through bug reports is “fundamental” apparently. 

 

In my view, communicating through bug reports is a sign of some deeper issue within that organisation. If you are using a lifeless remote system to communicate between two humans as your primary medium of communication then you are destined to have lost meaning, wasted time and rework. You have a problem.

 

When we write things down we often (not always) lose information and more often than not, in the case of defects, waste lost of time bouncing bugs backwards and forwards when a simple conversation might have cleared the whole thing up. It often means the bug reports themselves are more thorough and complete when the developer and tester have chatted about the issue in advance. Now I’m not adovcating badgering a dev each time we find something, but I am suggesting we stop and think about how best to communicate the issue. Think about The Purpose, The Audience and The Context.

Would it be easier and quicker to show the dev the issue and talk it through before raising a ticket for it?

Or would it be easier to raise the ticket and then talk it through?

Or would the ticket clearly explain the issue without talking it through?

Or should I raise the ticket and then sit back and wait for the inevitable bounce back with “cannot reproduce” or “as designed”?

Or do we even need a ticket in the first place?

 

Here’s an over the top example, but scarily it’s not far from the reality I’ve seen in the past,.

Read More

Translated in to Russian – I’m honored

I do feel honoured indeed to have had a blog posts translated in to Russian for the very excellent http://software-testing.ru community. Very honoured indeed and it was great to be in touch with Alexei Barantsev from the http://software-testing.ru community. I love what they are doing in the community. It also looks like the Russian testing community has some very exciting training and learning initiatives, so if you’re in Russia and want to learn more then check them out. I get a sense I may need to brush up my Russian language skills 🙂

Less is more

As a keen writer and communications fan I spend a lot of time looking at how I (or we) can improve our communication and one of the best techniques is to remove the bits of communication that people skip.

 

In the world of writing this means removing the sentences, words or entire chapters that you *think* or *know* people will skip. With my own writing this is a very subjective process. I don’t know which bits to remove but I do know I self edit my work quite heavily.

 

It’s like using twitter. It’s amazing how you can still get your point across in a few characters. It’s a good practice to get in to. Brevity of writing is a good thing.

 

Much of my writing gets drastically chopped down but no doubt there are still huge amounts of it that are what I call “fluff”.

 

And it’s important to realise that this idea of fine tuning can be incredibly valuable in our testing world too.

 

Let’s take a test case. One of the typical ones with plenty of steps and lots of detail. This is not uncommon, there are loads of teams still using highly scripted tests. In fact, I’d go so far as to say it’s the “norm”

 

There are several audiences for these test cases and if you are dishing these out to people simply ticking the boxes then the more detail the better. The fact that this approach is insane, wasteful and insulting to these testers is something best left to another blog post.

 

But give your scripted tests to a software tester and observe. Ask them for feedback too on the overall effectiveness of the test case. Here is what I believe you will find:

 

  • They will not follow the test case step by step
  • They will read the whole test case in advance to get the gist of it, to formulate an idea about the intentions of the test case
  • They will often make extra notes or areas to concentrate/focus on
  • They will often re-read this test case a few times to truly understand the intentions or expectations and to double check any missing prerequisites or knowledge
  • They will draw on all of their domain knowledge and experience
  • They will perform the testing but not as a step by step process. Instead they will tend to complete the whole test and revisit the test case to tick all of the boxes after, just to check they got everything asked for in the right way.
  • They will probably explore or diverge slightly from the test, but no so far that they don’t fulfill the intentions
  • They will mark the test case with edits, suggestions, ideas and improvements.
  • They will probably suggest several new test ideas too
  • They will note down observations which in turn often get fedback to the test case
  • But most importantly they will not be guided by a step by step process

These are just my own observations but I encourage you to study your own work and that of your colleagues. I suspect you will see the same thing.

 

And here’s the point. Don’t include the unneccessary, the verbose, the irrelevant, the obvious, the waste or the *skipped* detail. In ommiting this “fluff” you will probably end up with what I call “Guidance Level Tests” (it’s an old post on my old blog but the sentiment remains the same).

 

If testers are skipping sentences, ideas or sections because they are not needed or irrelevant or verbose – get rid of them. In the end you’ll end up with a checklist to guide, not a document to steer. And this moves the thinking to the tester running the test. And this is a powerful thing.

Lost in translation – a lesson learned at a hot dog stand – #agiletd

Image from : http://www.flickr.com/photos/bunchofpants/2946843497/sizes/z/in/photostream/

It seems there is inspiration for learning all around us. I’ve just got back to my room after a walk around the local area here at Agile Testing Days in Berlin. The location is excellent, despite being a distance from the airport, and the local area is just lovely. Lots of fabulous buildings and shops and tree lined streets. I’d left my memory card back in the laptop otherwise I’d have included some photos.

I’m one of these tourists that loves to sample the local food. I don’t see the point in being abroad and eating English food. It’s not like we’re known for our culinary delights anyway. So I decided to stop off at a wiener stand by the side of the road.

What followed can only be described as a “Lost in Translation” moment.

Me: Hallo

Vendor : Hallo. (Something else which I suspect was “what can I get you”, but cannot be sure as my German is very very bad)

Me: I’d like what the locals eat please.

Vendor: Ok.

He then proceeded to create the most creative looking “hot-dog” I’ve ever seen. His face told a thousand horrors as he randomly put bits of various stuff from the fridge in a tiny bread roll. He looked like a scientist trying different flavour combinations, each one accompanied by a frown and a suggestion that this might not taste so good.

With a look of sheer panic he placed the largest weiner I’ve ever seen in the tiny roll. Then added Ketchup and Mustard. Some more stuff from the fridge. He rolled his eyes, raised his eye brows and gave off more non-verbal clues that he really wasn’t sure about this thing.

 

It seems my request for something authentic and local had been Lost in Translation and since my German was so embarassingly bad neither of us had really got a good deal out of this.

He leaned over the counter and said:

“hmmm. I guess this is a German Hotdog” His eyes told a chilling truth though. Here was a man who’d created something he’d no doubtedly have nightmares about. I could see in his eyes that he was gutted. I too was apprehensive of this “German hot-dog”.

 

As it happens it was delicious but I suspect, not very authentic. The locals in the cafe were all incredibly amused by my lunch too. I think one or two took pity on me as they offered me kitchen roll and laughed heartily with (at?) me. They’d seen this epic failure on my behalf to communicate my requirements clearly and seen the resulting meal. I’d done everything I preach against doing. I’d failed to communicate my intentions clearly.

If only I had have brought a German Phrase book with me things could have been very different.

 

And so I learned a valuable lesson here today. Even with the best intentions in the world, communication breakdown is a reality and both parties can be dissatisfied with the final product. I should have learned some German for my trip or at least given the vendor something more substantial to go on.

 

This happens in the work place too, even when there is a common cultural language in use. Ideas and concepts get lost in a fog of misunderstanding and assumptions.

 

I think next time I venture out to taste the local cuisine I shall take a phrase book with me and try to clearly articulate my intentions in more detail. I can still see the vendors face now (who I learned was called Alios….I think). But I also tried to let him know how delicious it was, because it truly was amazing. And despite the fact his non-verbal communication suggested otherwise, I can’t help wonder whether he’ll add it to his menu as “The German Hotdog”.

What does the future hold?

 

One of the most amazing things about working in the technology and software world is that this industry advances so fast that I have no real idea what software/technology I’ll be testing on and writing about in 10 years time.

 

10 years ago I hadn’t really heard of cloud computing, now I’m testing call centres hosted in the cloud. 10 years ago I was just starting to test on web apps, now it seems everything is available online.

Almost every article I read is about testing web apps, using web testing tools or some form of web based testing service. Combine this with the general mobile phone uptake trends you have a very mobile, very web, very technology heavy future. Add to this the lowering of barriers of entry (cost and availability of technology) and it wont be long before we are facing option paralysis when deciding which configuration to start testing on first.

As more people adopt modern technology, whether willingly or not, it’s clear a wider array of technology and applications will be moving online and mobile providing us with numerous challenges of not only testing the functionality but also the human factors (or ergonomics) side of our apps and the devices they run on.

Self service (by end users) is also becoming a big selling feature and with mobile devices being just as powerful as many desktop devices it’s clear we have a future dominated by mega massive selections of devices to test against, all running on varying degrees of network capability used in a mind melting variety of contexts. Add to that the seemingly natural way in which the future generations have perfected the art of managing multiple devices and we can quickly see how testing may need to evolve to keep up.

 

So my domain knowledge of the product and industry will need to evolve to keep up with technology and trends, this hasn’t changed much. It’s always been the case.

But also my approaches, test tools and ideas about testing will need to grow, shift and evolve too. But also too my test environments will need to evolve to keep up with the relentless progress of technology and communications. In fact, some test environments may just need ripping apart and rebuilding. And for some, this could be the biggest testing challenge they will face.

So how do you see your test lab or your test “approach” evolving over the next 10 years?

Image courtesy of : http://www.flickr.com/photos/levitateme

I’ll show you mine…….

Inspired by a post by Michelle Smith about what’s on her walls when she is working I thought I’d open something up to the testing community and see if we can’t get some insights in to testers workspaces going. To continue the mantle from Michelle I’ve posted some pictures of my walls and home office. I work from an office at New Voice Media but when I’m at home writing blogs yada yada this is my inspiration.

Mine, it would appear, is not quite as neatly laid out or as decorative as Michelle’s and I couldn’t photograph the whole study as I have a bath side panel, some tiles and my DIY tool kits laid out…I must find some time to fix that bathroom.

I’m a big fan of the NBA. I’m not entirely sure how I got in to it, because here in the UK it’s by no means a popular sport but I’m a fan none-the-less. I was also lucky enough to get to see a NBA match when I was in New York a few years back. It was excellent. The game itself was poor but the atmosphere was fantastic. Nicks versus Golden State Warriors……

Here in Britain it’s got some core followers but it barely registers in the newspaper and you’d be hard pushed to find someone to talk to about basketball. Although you can’t see in this picture, but the NBA image is made up of lots of little images of basketball players from the past. Also some modern art from my son 🙂

P1040375-900

 

A map of the British Isles. I find it intriguing that many people in Britain are striving to explore the world and dismiss Great Britain as boring and dull. One look at this map and it’s clear I haven’t explored 99% of the British Isles so I can’t join them in saying it’s boring and dull.

P1040378-900

And some books.

P1040380-900

So why not upload your photos to flickr and tag them with “softwaretestingclub” so we can collate them together, maybe for a funky ebook. Maybe just as an experiment.

There’s method in the madness

There’s a growing number of people in the testing world doing something I’ve been calling Method Testing, although that’s probably not the right term. Named after Method Acting (Wikipedia : Method actors are often characterized as immersing themselves in their characters to the extent that they continue to portray them even offstage or off-camera for the duration of a project.) I’m using this term to describe the testers who emerse themselves in their end users environment to learn how their customers use their product and in what way and on what devices. They then bring this back to the lab and test against it, often in situ, often with real end users.

 

In my opinion this is taking testing to a higher level than many can realistically go, but it all comes down to your goals, budgets and aims. Unfortunately, those who do Method Testing don’t seem to write much about it….. In the examples I hear about it’s not uncommon for the test team to prepare a “real life” environment (not just a real life computer configuration) in which to test, often using real end users, sometimes using actors and often using props.

 

This differs from UAT in that they are doing this as part of their day to day testing, not just at the end, when to be honest..it’s often far too late.

 

A friend of mine tests medical equipment and it’s associated software. He has a fake emergency room kitted out with dummies (in various medical emergency positions), medical equipment and costumes, he even uses fake blood. Proving the software works (or doesn’t) is one thing but making sure it works under pressure, in situ and is usable is something completely different. Usability testing at it’s best. He even employs actors once a week to act as patients adding extra drama and realism. They often bring in real surgical teams to run through a typical procedure. The whole development team observe the behaviour and adjust accordingly. He films the sessions to learn more about the actions performed in surgery and potential flaws with design. Sure it’s expensive…very expensive, but it’s invaluable for him. The software industry he is in demands working kit..all the time. I’m trying desperately to get him to write about it.

 

At the almost completely free end of the spectrum, another friend of mine tests call centre software and he has a small call centre environment set up with a seating plan in the corner of the office, so he and his team can role play a typical call centre scenario. He’s not only testing the software, but testing in sudo-situ, with terminals, seats, wallboards and background chatter.

He suggests that it “tricks the mind” in to thinking differently about the software. I think he’s on to something.

 

I know of several testers who test their software whilst out and about on trams, trains, at the beach, from a cafe etc on a variety of mobile devices using free wi-fi or shared connections. That’s because they know roughly how and where their software is to be used. They are testing in situ. They are exploring, learning, feeding back. They are immersing themselves in the environments their customers will use their product in. They are doing method testing.  They are also extremely lucky 🙂

 

But what ties all of the above is that they are all reporting remarkable results in customer satisfaction <– the ultimate feedback on product acceptability?

 

They are making an effort to emulate the end users real environment, to make sure the softwares original purpose is right, for the right people and under the right contexts.

 

But it’s too tricky to find out who uses our software and how they use it..isn’t it?

 

Not always. For a small few this might be true but in reality the majority of us just don’t bother to seek out this information or we look in the wrong place. Why not try one or all of the following:

 

1. Speak to support desk or customer services. They are on the front line of complaints and feedback and can give you invaluable insight. They often maintain lots of records on system configurations and usage patterns. They don’t stash it away from you to be mean, they probably just don’t know you want it.

 

2. Speak to sales or accounts teams. They sell the stuff and maintain relationships with the customers. They will have more information than you would imagine.

 

3. Attend a sales pitch. You will learn loads, trust me. You’ll also see the types of questions your customers ask about the product.

 

4. Attend (or give) a training session on the softwared to real users. You can observe them first hand and see which bits trip them up or confuse them.

 

5. For webapps or websites use Google Analytics or other website traffic monitoring tools or build in some performance/usage measuring. The amount of information Google Analytics gives you is insane. Browsers, screen resolutions, locales, referal sites, devices. You can find out bounce rates, pages navigated to/from, time on page, links hit etc etc etc etc Mind blowing stuff that really will make a difference to your test approach.

 

6. Talk to the customers….(might be worth checking with key people in your company first though 🙂 )The majority of customers are more than happy to show you around their offices and show you the software in use, especially if it’s not working too well!

 

7. Give customers easy and simple ways of reporting back problems from your software. Encourage them to be clear about the issues and log pertinent information.

 

8. Actively encourage customer involvement by hosting forums and chat rooms where they can discuss your product. Encourage them to be truthful and honest. Engage with them. You’ll be amazed at how receptive customers are to honest and upfront conversations. Ask for feedback on what environments your product gets used in. Ask for the truth about your product. You can’t blindly push on releasing if people are hating on your product. Or you can, but it won’t be for long 🙂

 

The information is out there..everywhere. It’s a case of widening your awareness fields and grabbing it. Analysing it. Using it. Reacting to it. Learn how your users use your product and you are gaining invaluable feedback on how to test it. It’s never a replacement for real end users but it’s a great addition to testing your software in a test lab with controlled environments and no idea about how your users use your software.

It might be time to step outside of this test lab comfort zone and do some in situ testing. Or should that be Method testing? Customer emulation testing? End user situ testing? No doubt someone will have a good name for it. I personally like method testing. And although your managers may think your mad for wanting to test in the same environments as your users, they’ll appreciate it in the long run. There is, after all, some method in your madness.

What do you want from me?

You’ve surely been in the same situation? You’re sat using an app or site that requires some form of data entry from you, the problem is, you have no idea what it wants. There is no advanced information, communication or guidelines on what rules are going to be applied to your data. There’s sometimes an ambiguous or tricky to find error message. Sometimes there’s nothing.

 

As a software tester these types of bugs in our apps are low hanging fruit. They are quick wins, easy gains and an incredible booster of usability. I refer to them as “clarity bugs“. Features, functions or capabilities that just need a little bit of extra “clarity” to understand them or make them work. This may come in the form of on-screen prompts or help files or documentation or maybe through training, but sometimes there just needs to be that little bit of something extra.

 

 

 

Help text that isn’t helpful is pointless and confusing. None existent help is assuming a level of knowledge by the end user that might not be right. Errors that aren’t descriptive are frustrating and time wasting.

To only explain what the user did wrong after they’ve actually got something wrong is wasteful and irritating. Communication is key.

When there is no explanation about what criteria the data and subsequent validation must meet it’s is an easy usability defect to grab. (think about max and min lengths on fields, or password validations or date formats etc) Don’t become blind by your pre-existing knowledge and sail straight past the fact it’s not explained anywhere. Are we sure our end users know about our specific data validation?

 

It can end up with the user sat there screaming “what do you want from me?” whilst hammering on the keyboard hoping they submit the perfect data.

 

These usability bugs are bugs. Plain and simple. Even if the application works, if it’s not usable then the chances are your customers will simply stop using it 🙁

 

Want to know some more about good form design and general usability?

Check out :

 

Double check your audiences values

The other night I tweeted that I thought I’d lost my testing mojo. I didn’t feel inspired to write about testing. Some good friends of mine suggested I get some rest, that it would indeed return and to be patient. So I kicked back and chilled out. As it turns out it didn’t return today so I decided to do the second best thing and re-post a fairly old post from a now defunct blog.

I sense my mojo is returning as my mind is racing around thinking about some very interest..and far out…ideas about megafactories, plastic testers, ghosts, snot (bear with me), testing therapy and idleness. But for now….a golden oldie (apologise to all who’ve read it already). For those that are interested my old blog is here : http://pac-testing.blogspot.com/

 

Double Check Your Audiences Values

I’ve been wondering a lot about phrases mentioned in the test world like ‘quality is the value to someone who matters’. I completely subscribe to this notion that what a tester may perceive to be a high priority bug, the customer (or product owner) may actually not. It is a sound theory and something that I try to remember when testing. Its something a tester must appreciate so that they don’t get all gnarled up over a defect they think should have been fixed.

 

So day one, you find out what your ‘person that matters’ deems to be important and not important, so you raise defects and only highlight (add extra emphasis on communicating in a rapid way) defects in areas your audience has told you have more value. You keep doing this until one day you get asked why you let a high priority defect go in the pot out without making more of a fuss. ‘But you didn’t see the value in fixing that bug last week…’ is your response.

 

What’s happened is that some dimension of this persons perception of value has shifted and you’ve been left out in the cold. The ‘person that matters’ now perceives this once low value area to be a high value area because of new information they now have at this moment in time. You feel annoyed, frustrated and betrayed.

 

So when we say that quality is the value to someone that matters I feel we need to add on to that ‘at that moment in time’, in fact I’m pretty sure someone, somewhere will have already done that….. (as it happens it was none other than Mr Bolton and Mr Bach (There’s a line in the Rapid Software Testing course: “Your context guides your choices, both of which evolve over time”. – Michael B))

 

I’ve seen this happen more times than I can remember where a defect in an area or a defect of a type (i.e. tab order) was not important at point X, but now a shift in focus has happened and it’s suddenly the top priority now at point Y. Whether you get told about this shift in focus or not is irrelevant – I see it as a testers job to find out what the focus is, from the ‘person that matters’. You can report defects until the cows come home but what really adds value for your ‘person that matters’ is when you report the defects with an emphasis on the ones of value to them.

 

I’m not saying don’t raise every defect you encounter, far from it, but from my experience, testers have a direct influence on how well communicated these issues are to your audience. So spend the time making the ones important to your ‘person that matters’ read like a horror story and also make them aware of these issues faster.

 

So find out what is important at that time to whoever may matter and work with them to add ‘quality’. Don’t forget that the person who matters will have more dimensions of information at their disposal than you do. They may not care about 100 defects in module X, they may care about 1 defect in module Y due to commercial issue Z. Work with them, let them know they can trust you, let them know you are pro-active and let them know about the bugs that matter to them. But most importantly, keep checking your audiences values as they can, sorry, they will, change over time.

Enthusiasm is catching

Have you ever noticed how you suddeny become fired up when working/training/observing someone whose enthusiastic about their topic or subject of interest? Even if you don’t enjoy the subject, their enthusiasm still rubs off on you. You feel fired up to get out and do something you enjoy. You feel motivated.

This last weekend I was lucky enough to be at Goodwood Festival of Speed. Probably the greatest car show on Earth. It really was awesome.

 

I got to see Sir Jackie Stewart, Richard Attwood, Nico Rosberg and many others all being interviewed as well as discussions and chit chat over the show radio and there was one major linking factor (other than cars) : Enthusiasm. Bags of it.

Passion, enthusiasm, desire, drive, attitude, engagement…whatever you want to call it. They all had barrel loads of it. Just like the many people in our testing community who live, breathe, talk, write and banter about software testing. It’s what floats their boat. And their enthusiasm is catching.

 

So if you’re new to the game of software testing or you want your spark re-igniting then I’d suggest you seek out some of these passionate and enthusiastic testers and soak in as much as you can from them. Their enthusiasm will rub off on you. You’ll feel motivated.

And if you are in the UK in July 2011 and have an interest in motorsport then I’d suggest you seek out a ticket for the Goodwood Festival of Speed – I’ll see you there 🙂

And by the way. I think I’ve found “The Social Tester Transporter”……..

 

Car

More photos here: http://britainsroadsrevealed.posterous.com/

Tool Time Part 1

I’ve tended to keep this blog more distanced from my day to day testing but this in itself has drawn some comments that I don’t talk about day to day “testing”. Well, I’ll hopefully address that with the following few posts about some of the tools and processes I use when testing.

Most of my testing is very lightly scripted and consists of lots of exploration, mainly because this way of working suits me better. I’m not one for re-writing manual tests, ammending manual tests when little details change and being too reliant on a test I wrote well in advance of my testing. Information is constantly coming in and to re-adjust to this with static tests is something I’m not so keen on.

One of the most important things for me when testing is to have as much information as I can at my fingertips, so that when I find a bug I don’t have to spend ages setting up some tracing, reproducing loads of times and then constantly adding more details to the bug ticket. All at once. As much information as possible is my moto. I can then whittle it down and include only the necessary. It also helps when communicating issues as the more information I have available the more concise and accurate I can be when discussing bugs.

What I list here is MY way of working. It suites ME and the way I think. For many testers this may be too over the top, for others it may be too little, for some it might prove useful. I do hope you get something from it or gain some inspiration to check out new tools that might add some value to you. But more importantly I hope it inspires you to tell me (and the test community) how you do your testing. What tools do you use? What approaches make you more effective? I’m always intrigued to see how testers work. Testing is very personal and we each have a special way of utilising our minds to the fullest. This here, is mine, but I would so love to hear more about yours.

 

Logging, tracking, effectiveness and monitoring

The following tools and techniques are all free and could offer some effectiveness gains when testing.

 

Fast Access to tools

Being able to fire off tools and apps quickly is a key point to becoming super effective for me. For all of my regular applications I have a batch file on the desktop which opens the tool of choice with the right settings first time. This means I’m never more than two clicks away from launching an app. I could use the quick launch bar or other docking stations too.

Separation of work and play

The first thing I recommend to anyone is to get some separation between work and play. In my view there is a huge effectiveness gain to be made by focussing on the task at hand without the distraction of Email, web, instant chat and twitter etc. Sure, you might need them for collaborative projects but when testing alone I always recommend separation. Time boxed sessions of testing where distractions are limited lead to wonderful testing.

To keep work and distraction seperate I use “desktops” from sysinternals. Desktops allows me to have four different desktops available. So I keep work in one, and communications/distractions in the other. I often work on VM’s so I keep these sessions on another desktop. One physical desktop, four seperate containers to split them out.

 

ALT + 1,2,3 or 4 switches between the desktops and the usual copy/paste stuff works too.

I also have a VM created for my different browser tests. The VMs have nothing but testing tools on them meaning limited distractions.

I use the various bookmark synching tools to get bookmarks on all machines. An example of which is XMarks  for Firefox. I can install the add on, download my favourites, then disable XMarks and remove all none-work links. This means I won’t lose the server stored bookmarks but I can remove temptation to surf the web for the latest NBA results or that nice shiny car I want.

 

Browsers

Having every browser you test on already set and ready to go is essential for any web tester..right?. I also have VMs with older versions of browsers too meaning I can quickly spin up a version and repro a bug or do some new testing. A good source of old browsers is here

Firebug and Fiddler

When I test I like to use Firebug and Fiddler as a base set of traffic/post sniffing tools. Firebug not only captures errors (that you have configured it to catch) but it also enables you to inspect the page elements too. This is great for seeing where specific errors might lie in the code. It’s also a good way of identifying elements for automation.

They are both super little tools for finding out what messages are being sent between browser and server. You can intercept, change and remove these request too, which is great for failover and security testing. Others are available though….I also quite like CharlesProxy.

Accessibility Toolbars/Sites

When testing websites I always like to run a quick accessibility test just to give me some feedback on the structure of the html and CSS. Even though it is looking for accessibility issues it often reveals some surprising information about the structure and quality of the code. I did a whole post on accessibility testing here.

Samurize

Big thanks go to Alan Richardson for his inspirational blog post regarding Samurize.

I copied Alan’s setup and then changed mine slightly to also include the last 50 records of the main activity server log which basically shows me instant reports on what the server is doing and what traffic is moving within the system. This is excellent when you encounter a bug as you can often see straight away what got logged out. Extra information but also rapid information for raising bugs.

 

I run Samurize in a VM as it’s not supported on Windows 7 64 bit. I find this keeps it clean also and doesn’t annoy me when I’m not testing. It’s also possible to create many different config files meaning I can easily switch between log reports or other heads up information. It really is a super little tool.

Notepad++

Your not using a notepad on steroids? Notepad++ is my favourite with it’s massive array of add-ons like file comparison, file monitoring (tracks the last x rows in a log file), coding intelligence and a huge number of other features. Superb.

 

BGInfo

BGInfo is a new tool to me that got mentioned in a meeting the other week. It basically adds to your desktop background a whole load of information about your machine. This not only aids in reporting bugs about specific machine make up but it also stops confusion when using multiple test machines. It’s now nice and clear which test machine I’m testing on.

 

I’ve blanked out my computer settings btw.

Xenu Link Checker

A quick run through of your site with Xenu Link Checker will give you a good idea of all the dead links and missing content. This is  a really fast way of seeing broken content on websites. I tend to run this first off as it gives me clues and ideas of where to look for more serious bugs.

Process Explorer

I tend to have process explorer on standby ready for those times when i need to find out more about an application and further details of it’s process. Think of this tool as Task Manager on Steroids. I particularly find this useful for searching out detailed DLL and memory usage.

Process Monitor

Process Monitor is a super useful way of seeing what processes are being used when. I use this rarely but it’s a great tool for finding out what processes are being used when.

Wireshark

Wireshark is a new one to me [Wireshark is a packet trace tool that allows the user to capture network traffic going in or out of a specific device]. I’ve been using it recently to track SIP protocol messages whilst investigating a loss in connectivity between my SIP phones. It showed that no messages were going one way which highlighted a dodgy config of SJ Phone. That would have taken me a while to find this fault without a useful tool like WireShark. In some instances that could have resulted in a bug ticket using up resource to investigate?

Komodo Edit

My IDE of choice is Komodo edit. I’m no coding expert but I dabble with really basic Python and Ruby scripts. I use them mainly to do basic checking like scanning folders, extracting text from websites and launching basic ui tests.

I also do internal reporting pages in HTML etc so use it for those tasks too.

Burp Suite

Where would I be without trusty Burpsuite? I’ve only really dabbled with Burpsuite sporadically in the past but since starrting here at NewVoiceMedia I’ve been cracking on with Burpsuite in gusto. Burpsuite is a security testing tool. It’s so rich in features that I could spend an entire blog post on it, maybe I will do. I’ll let the burpsuite site explain the basics instead.

There’s some more links here on my delicious feed: http://delicious.com/maximumbobuk/security

Selenium IDE

For basic data loading through the UI or observational testing I find Selenium IDE invaluable. On a project a while back I needed to constantly log in and out to get access to the feature under test. So I used a UI automation test to do the donkey work for me. Don’t be put off by people saying NO UI TESTING!. There are levels of UI testing. If you are basing your entire automation suite on a set of record and playback tests then I’d suggest you need a new strategy. If you are using Recorded tests to speed up repetitive jobs, to get you to known states, to do blink testing or to load sample data then there’s not much wrong with that.

 

Notetaking, Reporting and Test Management KeePass password manager

With a huge array of networks, tools, web logons and environments it takes a secure password manager to keep them all safe. Step up KeePass. Simple. Safe. Quick.

Treepad

Treepad is my uber multitask tool. I use Treepad for all of my work. I’m writing this blog in Treepad. I have my books, eBooks, scripts, articles, presentation outlines and learning material in treepad. I use Windows Live Mesh to sync between machines. What this means is that it has also become the defacto place for me to write out my notes and other pertitnent test information.

I jot down observations and insights, ideas for discussions and snippets of information.

It’s easy and quick to open. Saves using CTRL+S. Outputs to txt file. Allows copy and paste. Allows hyperlinks. And that’s about all. Not a lot of gloss. just the basics. Perfect.

 

Freemind and Mindmeister

My mind flows in diagrams and visualisations. My mind works best when I try to visualise things. I like simple diagrams to explain logical flows. I like screen mockups rather than words. I also prefer mindmaps to descriptions and notepads. A good mindmap tool allows me to jot down ideas and thoughts and move them around to form a logical pattern or flow.

I always use mindmaps to brainstorm test ideas. Some of these ideas become test cases, some become areas to explore, some become nothing but information and many often get deleted. Freemind is a good free and open source tool. I prefer mindmeister because it feels quicker.

Spreed

When we read we often read at a much slower speed than possible. To speed up our reading process we can use the tried and tested finger moving across the page faster or hop on to the web and use a tool like spreeder

I started using Spreeder about a year ago and absolutely love it. Got a long spec to read, some emails, blog posts? Not a problem, load up spreeder copy the text in to the window and then get reading. Spreeder works by flashing the words on the screen one at a time at a speed you can define and change on the fly.

At first I was skeptical but I actually ended up reading at a much faster rate and accomodating this information at the same time.

There’s also a handy Bookmarklet which opens up any selected text from your webpage in Spreeder ready to go. It really does make reading text quicker, but don’t just take my word for it. Have a go.

Let me know what tools you use in your day to day testing.

That tester’s incompetent….hide them

Last year I attended an excellent agile testing conference but came away frustrated at one of the keynote presentations.

One segment of the talk mentioned the often painful transition to agile. The presenter made reference to a British TV show character who is quite nice, but quite dim. In other words, an amiable kind of person but incompetent. Commonly known as an idiot.

The presenter then suggested that in an agile environment there is no room for an incompetent tester. Agreed, but surely the same is true for any environment? IS there a room for an incompetent tester in waterfall?

But what really got my goat was when the presenter suggested that we simply hide this person away from management.

Photo courtesy of http://www.flickr.com/photos/gunnisal/

The first reaction by most people seemed to be that this was a complete farce. This person is a part of the team and hence contributing to the bottom line. Paying a salary for someone who remains hidden? Really? Why would you hide someone away from management? The reasons given were that it was humane when transitioning to agile; to hide them away rather than expose them, then get rid of them. For real?

This spurned some furious discussions afterwards where I was intrigued to hear that many of the attendees thought the incompetent tester should be sacked. After all they are contributing to the bottom line. Get rid of them. And that’s often a sentiment held with many in the agile community. Incompetent testers (any team members too!) can ruin an entire agile project. No room for the weak. Get rid of them. Maybe this is why many testers are worried and negative about moving to agile?

But even this missed the point entirely for me. I found myself asking the question “Is this tester really incompetent or are we just not trying hard enough to integrate them?”

Are we asking the wrong questions, taking the wrong actions, avoiding the hard work, wanting instant results like we are often led to believe is the norm? Sure, on the surface you may think this tester IS incompetent but is this a result of their skills or their environment and lack of motivation?

Maybe this incompetent tester isn’t being given the tasks and challenges that make them enthused. Have we spent time finding out what makes them tick? Have we sat down and tried to get to know them? To understand them? To challenge them? To motivate them? Have we asked them what they think? Have we even trained them or coached them in what agile is?

Have we invested in them with time and energy? Encouraged them to attend inspiring testing conferences? Training Sessions? Local user groups? Pointed them at online communities like The Software Testing Club?

Have we done enough to really come to the conclusion that they are an idiot? Have we explored all the avenues?

If we have, then maybe that’s fair enough. Let them go, but don’t hide them away. However, I would bet a whole £1 that we haven’t invested in this person as much as we could have. And if we don’t do that, then we may never know that this person is a whizz at building automation frameworks, is an absolute star in front of the customer, can articulate complex ideas to non-tech audience or comes alive when performing exploratory testing.

You’ve only got to take a glance around the testing communities to see that some testers are real customer advocates, some love automation, some thrive on efficiency improvements, others love lean and agile, others teach, some berate, some challenge, some build social networks and communities, some are socialisers, some live and breathe quality. Some are funny, others are sad, some are arrogant, some are happy. Some write about testing, some present at conferences and some are super thorough and live for nothing but quality, quality, quality.

It’s this mixed bag of skills, talents and attitudes that make the testing community so interesting. Surely there’s an element of this in your test teams too? Isn’t it just a case of finding out who’s interested in what and giving them work to inspire, challenge and enthuse?

So instead of saying “That tester’s incompetent. Hide them” we should instead be saying “That tester’s got hidden skills, experience and ability. Let’s find out what it is…..and if they are still incompetent at the end of that process then….well, we’ll sort something out” or something like that.

But this is a difficult concept to grasp for many people who see testers as a quantifiable, measurable, certifiable and replaceable person. Hiding them away ignores the fact that testers are complex and diverse. It’s a simple way of ignoring the truth. It’s an easy way of avoiding hard work. It’s an easy way of applying testers to a project as a resource rather than a person. Resources take little maintenance; people have ups and downs, goals and ambitions. Resources act the same each and every week. Testers fluctuate. They need motivation and inspiration.

Leaving the incompetent tester to fall even further behind, hiding them from management and giving them ridiculous job after ridiculous job is not humane. So to suggest that doing so is a humane action by middle management is misguided and painfully difficult for many to comprehend. The humane thing to do would be to invest some time in them, step up and inspire them, get them all fired up and find out what makes them tick. Take responsibility for motivating and inspiring them and their work. Or could it be that I live in a dream world where we all share some responsibility for making our working environments great places to be?

So if someone said to you: “Bob over there is incompetent. Hide him away” …………………………..What would you do?

Social Media meets Testing………Delicious

Those who know the history of The Social Tester will know there are two reasons for choosing the “Social” alias.

One: I like to socialise. Two: I’m a fanboy of social media.

 

So here’s the first in a series of posts about how we can use social media tools in our testing world. This post is about social bookmarking and website tagging. It may seem miles away from your testing world but the fact you are reading this is evidence you use the web. Many people test on webapps – how do they manage their websites? What about all of those reference lists, solutions to problems, cheat sheets, blogs, learning source? Saved in your browsers bookmarks? Shared with the community?

 

For this post I’m going to explain how using a Social Bookmark like Delicious could really help your organisation of bookmarks but also help you share them.

Social Bookmarking is a way of “tagging” (Tagging is a system of classification, often referred to as Folksonomy) content and sharing it with others. Delicious lets you tag web pages and keep them private too, but generally the idea is to mark content and share it with others via rss feeds or the Delicious main page.

 

I did the same thing with my accessibility links on a previous post here: http://thesocialtester.posterous.com/accessibility-testing-and-ownership At the end of the article I included a link to my Delicious bookmarks as a way of sharing the content I’ve found. If people have already gone to the trouble of searching out information on specific testing topics then it’s nice to share that with others.

 

If you check out the Software Testing Club homepage there’s a little section at the bottom right which aggregates all Delicious content tagged with the tag : softwaretestingclub. So this is other users, like myself, who find something we think might be interesting to the Software Testing Club audience and we tag it with “softwaretestingclub” – it then shows in the feed.

But how else is it useful for me?

 

Well, every day I find new information that I might not want to digest there and then. I also find stuff that is essential for my learning or stuff that I use as a quick reference. So I tag it and I can quickly find that new information at a later date.. fast.

 

But why not print it out or save it as a Standard Bookmark?

 

I still use standard browser based bookmarking for my work links and every day browsing links. I use browser “sync” tools to sync these bookmarks between machines. But I find it gets overwhelming to manage when I add lots of webpages. I often used to find myself in a dilema over which sub folder to include the link in. Is it an agile link? Or a testing link?

I also used to spend hours sorting through them and re-organising them so they made sense. I could still spend minutes trying to find the right link. Delicious means I actually now don’t care about structure of bookmarks. I just tag something. It also takes seconds to find the link (assuming you can find the right tag)

I also used to print out cheat sheets and reference lists but I felt guilty about killing so many trees and as is always the case, I could guarantee that when I needed the sheet of paper I was either not at my desk or I simply couldn’t find it.

 

It seems that more and more of my life is moving online so why not embrace the tools to make it easier to manage this process?

 

So how do I get started?

 

Simple. Head over to Delicious and sign up.

 

How do you access your Bookmarks?

It’s a simple no brainer for me. I use Firefox web browser for my everyday surfing so I use the very excellent Delicious plug in. But I could just use the Delicious website. Other browsers like Chrome and Opera also have Delicious plugins/addons. Best way to see if it works for you is the trial and error approach. I was lucky, the Firefox one just worked nicely first time.

 

Can you give me an example of how to use it?

 

Sure. The example here is using the Firefox add-on. The example assumes that you have added the two toolbar links to the browser (one is Tag and one is Bookmark). As shown here:

The example also assumes you are signed in to Delicious. The techniques shown here are pretty much the same no matter which plug-in and add-on you use.

 

Let’s Search

 

Let’s choose a topic of learning. How about….Exploratory Testing? I want to find out more about Exploratory Testing, so I’m going to do some searching and then social bookmark each resource for future reference.

 

First I search for Exploratory Testing using Google.

 

First of all I’ll check out wikipedia. So I open it in a new browser tab. Seems interesting. Seems these people called James Bach and Cem Kaner are responsible for much of what’s happening in the world of Exploratory Testing. Interesting article with loads of links in it.

 

I’ll open in a new tab a few of the links:

 

  • Session Based Testing
  • An external link “James Bach, Exploratory Testing Explained”

I’m also now intrigued in James Bach and Cem Kaner so I’ll google them and open their home pages.

 

Excellent. Back to my original Google search. I’m now going to open the following return:

 

NOTE: your return results may differ depending on country searches etc.

 

I’ve now got 6 tabs open.

At this point it’s time to bookmark them for future reference. So here’s how to do it using delicious.

 

First page would be the original wikipedia Exploratory Testing Page. When the page is open either hit the “tag” button on the browser toolbar or use the shortcut combination of CTRL+D.

This pops open the following dialogue:

 

  • Number 1 – This is the URL of the webpage you are bookmarking
  • Number 2 – This is the page title (you can change this if you like)
  • Number 3 – These are the tags you are giving the page. Use whatever comes to mind and use as many as possible. It means searching for it later is easier and more effective. Any tags you have used before will appear in the drop down as you type.
  • Number 4 – This is the checkbox for keeping this bookmark private. Check this box and it will not appear on your public bookmark list.
  • Number 5 – This is who you are signed in as

Then hit the save button.

 

I went through the remaining pages and gave them all suitable tags. Out of the tags I gave them they all shared the following three tags in common:

  • Testing
  • Exploratory
  • Blogsample

After closing all open tabs I then hit the “bookmark” icon on the firefox toolbar or use the shortcut combination of CTRL+B. It opens the following dialogue:

 

  • Number 1 – This is the search box for entering keywords that may match your tags. It also searches on keywords in the title and description too – which is super useful.
  • Number 2 – This is the tag sort list. You can sort this list in a number of ways
  • Number 3 – This is the actual tag list. Clicking on a tag in here will show all pages that match it
  • Number 4 – This is the list of all matches. So when you select a tag it will show all pages that match it. If you do not select any then it will show all.
  • Number 5 – This is the sort filter for the webpages

As soon as you start typing in to the search box the tag list will start filtering.

Right clicking on a tag gives you the option of opening all of the matching pages in tabs.

 

Right clicking on a webpage result gives you a variety of options.

 

Clicking on any one of the matches will open the page in the currently selected tab.

 

 

In our example we used the three tags Testing, Exploratory and blogsample

 

  • Number 1 – This is the tag testing
  • Number 2 – The tags that match the word testing and the subsequent highlighting of the exact tag of “testing”
  • Number 3 – The list of pages that match. As this is my real account there are already a number of pages that match. The keen eyed amongst you may have noticed that James Bach’s homepage is not in the list of matching pages. This is because I had already bookmarked this page a few years back and so it already had the tag.

Next one: Exploratory

Next one: blogsample

 

So now we have all of the pages we tagged showing in the delicious side bar within the browser. Type the word > Find the tag > Click the link > Web page opens. As easy as that.

 

What shows on Delicious?

If we now go to the Delicious home page and log in I can now see all of my tags on the web:

  • Number 1 – Tag search box
  • Number 2 – Pages that I’ve bookmarked
  • Number 3 – The tags applied to each bookmark
  • Number 4 – The controls for adding new bookmarks and editing existing ones
  • Number 5 – My top ten tags (notice the tag “bookmarks”. This is because Delicious very cleverly bookmarks ALL of your existing browser bookmarks. Best to go through afterwards and delete any you don’t want public)
  • Number 6 – The expansion box for opening all tags

So all of our tags are now on Delicious and all of the pages that match the tag are available for all to use. We now just need to share that tag url (http://delicious.com/maximumbobuk/blogsample) and let people see what we’ve found (unless it’s marked as private). Or alternatively just tag everything with softwaretestingclub and go to the Software Testing Club homepage to get your testing feeds. 🙂

 

Why do it this way?

 

The next time I quickly need to get the Robot Framework user guide I can type in robot and I’m there : http://code.google.com/p/robotframework/wiki/UserGuide

 

Or I might want a good laugh and re-read QAHatesYou’s improvisational agile. I’ll just type hate and I’m there : http://qahatesyou.com/wordpress/2009/02/you-want-good-improvisational-agile-training/

 

Or I may want to find out more about Dark Metrics and how to lie with them, so I type Dark and I’m there : http://www.softwarequalitymethods.com/Papers/DarkMets%20Paper.pdf

 

How about using the standard Python libraries with Sharp Develop and Iron Python. Type python and I’m there : http://community.sharpdevelop.net/blogs/mattward/archive/2009/03/01/UsingPythonStandardLibrary.aspx

 

You get the idea. I’ve gone from bookmarks in the browser which got messy, to searching using keywords which just made my whole learning and web content bookmarking a whole lot easier.

 

I’m not saying this will work for you, maybe give it a go and see. And as usual. Don’t forget to let me know how you get on.

Rob..