July UKTMF

A big thanks to Paul Gerrard and Susan Windsor for organising yesterdays UK Test Management Forum in London. Also a big thanks to the sponsors for the event; SQS UK, Original Software and Tricentis Technology & Consulting

It was a cracking turn out with about 100 people there. Great to see so many familiar faces but also lots of new faces too.

Paul started the afternoon with news of the merger between his company, Gerrard Consulting, and Susan Windsor’s company, WMHL. They are keeping the Gerrard Consulting name and are looking to expand in to the main stage of the consulting world. They both talked about a new tool called Testela which sounded interesting. Looking forward to finding out more when it’s released.

BTW – Stephen Hill’s blog is an excellent write up of the afternoon…here, so I wont do any more other than to include some photos.

There was a lot to agree with and a lot to disagree with about Jonathan Pearson’s talk on “The Dark Side of Application Quality Management Ten Black Holes to Avoid For Successful Application Delivery” Some lively discussions took place around metrics and agile and many other topics. Jonathon also remained dressed as Obi Wan for the entire talk. Impressive. I must admit that their tool is looking increasingly like a tool capable of satisfying many needs within the testing community. It appears to be flexible, incredibly extensible and doesn’t empose restrictions on how and what methodology or approach you can take. It’s also got support for exploratory testing, which sounds very interesting indeed.

Jonathon

Second up for me was James Wilson. I worked with James many years ago and his topic of choice was very interesting. Why is testing in an Agile with Scrum environment hard? (And what can we do about it?)

It was one of the best discussions I’ve had for a long time. The presentation essentially got hijacked but there were some interesting ideas put forward about how to run soak tests in short sprints and how to manage the quality of your release, when time and cost are fixed. Good stuff.

P1040396-1000

As usual, some after conference drinks ๐Ÿ™‚

ย 

ย 

I’ll show you mine…….

Inspired by a post by Michelle Smith about what’s on her walls when she is working I thought I’d open something up to the testing community and see if we can’t get some insights in to testers workspaces going. To continue the mantle from Michelle I’ve posted some pictures of my walls and home office. I work from an office at New Voice Media but when I’m at home writing blogs yada yada this is my inspiration.

Mine, it would appear, is not quite as neatly laid out or as decorative as Michelle’s and I couldn’t photograph the whole study as I have a bath side panel, some tiles and my DIY tool kits laid out…I must find some time to fix that bathroom.

I’m a big fan of the NBA. I’m not entirely sure how I got in to it, because here in the UK it’s by no means a popular sport but I’m a fan none-the-less. I was also lucky enough to get to see a NBA match when I was in New York a few years back. It was excellent. The game itself was poor but the atmosphere was fantastic. Nicks versus Golden State Warriors……

Here in Britain it’s got some core followers but it barely registers in the newspaper and you’d be hard pushed to find someone to talk to about basketball. Although you can’t see in this picture, but the NBA image is made up of lots of little images of basketball players from the past. Also some modern art from my son ๐Ÿ™‚

P1040375-900

ย 

A map of the British Isles. I find it intriguing that many people in Britain are striving to explore the world and dismiss Great Britain as boring and dull. One look at this map and it’s clear I haven’t explored 99% of the British Isles so I can’t join them in saying it’s boring and dull.

P1040378-900

And some books.

P1040380-900

So why not upload your photos to flickr and tag them with “softwaretestingclub” so we can collate them together, maybe for a funky ebook. Maybe just as an experiment.

There’s method in the madness

There’s a growing number of people in the testing world doing something I’ve been calling Method Testing, although that’s probably not the right term. Named after Method Acting (Wikipedia : Method actors are often characterized as immersing themselves in their characters to the extent that they continue to portray them even offstage or off-camera for the duration of a project.) I’m using this term to describe the testers who emerse themselves in their end users environment to learn how their customers use their product and in what way and on what devices. They then bring this back to the lab and test against it, often in situ, often with real end users.

In my opinion this is taking testing to a higher level than many can realistically go, but it all comes down to your goals, budgets and aims. Unfortunately, those who do Method Testing don’t seem to write much about it….. In the examples I hear about it’s not uncommon for the test team to prepare a “real life” environment (not just a real life computer configuration) in which to test, often using real end users, sometimes using actors and often using props.

This differs from UAT in that they are doing this as part of their day to day testing, not just at the end, when to be honest..it’s often far too late.

A friend of mine tests medical equipment and it’s associated software. He has a fake emergency room kitted out with dummies (in various medical emergency positions), medical equipment and costumes, he even uses fake blood. Proving the software works (or doesn’t) is one thing but making sure it works under pressure, in situ and is usable is something completely different. Usability testing at it’s best. He even employs actors once a week to act as patients adding extra drama and realism. They often bring in real surgical teams to run through a typical procedure. The whole development team observe the behaviour and adjust accordingly. He films the sessions to learn more about the actions performed in surgery and potential flaws with design. Sure it’s expensive…very expensive, but it’s invaluable for him. The software industry he is in demands working kit..all the time. I’m trying desperately to get him to write about it.

At the almost completely free end of the spectrum, another friend of mine tests call centre software and he has a small call centre environment set up with a seating plan in the corner of the office, so he and his team can role play a typical call centre scenario. He’s not only testing the software, but testing in sudo-situ, with terminals, seats, wallboards and background chatter.

He suggests that it “tricks the mind” in to thinking differently about the software. I think he’s on to something.

I know of several testers who test their software whilst out and about on trams, trains, at the beach, from a cafe etc on a variety of mobile devices using free wi-fi or shared connections. That’s because they know roughly how and where their software is to be used. They are testing in situ. They are exploring, learning, feeding back. They are immersing themselves in the environments their customers will use their product in. They are doing method testing.ย  They are also extremely lucky ๐Ÿ™‚

But what ties all of the above is that they are all reporting remarkable results in customer satisfaction <– the ultimate feedback on product acceptability?

They are making an effort to emulate the end users real environment, to make sure the softwares original purpose is right, for the right people and under the right contexts.

But it’s too tricky to find out who uses our software and how they use it..isn’t it?

Not always. For a small few this might be true but in reality the majority of us just don’t bother to seek out this information or we look in the wrong place. Why not try one or all of the following:

1. Speak to support desk or customer services. They are on the front line of complaints and feedback and can give you invaluable insight. They often maintain lots of records on system configurations and usage patterns. They don’t stash it away from you to be mean, they probably just don’t know you want it.

2. Speak to sales or accounts teams. They sell the stuff and maintain relationships with the customers. They will have more information than you would imagine.

3. Attend a sales pitch. You will learn loads, trust me. You’ll also see the types of questions your customers ask about the product.

4. Attend (or give) a training session on the softwared to real users. You can observe them first hand and see which bits trip them up or confuse them.

5. For webapps or websites use Google Analytics or other website traffic monitoring tools or build in some performance/usage measuring. The amount of information Google Analytics gives you is insane. Browsers, screen resolutions, locales, referal sites, devices. You can find out bounce rates, pages navigated to/from, time on page, links hit etc etc etc etc Mind blowing stuff that really will make a difference to your test approach.

6. Talk to the customers….(might be worth checking with key people in your company first though ๐Ÿ™‚ )The majority of customers are more than happy to show you around their offices and show you the software in use, especially if it’s not working too well!

7. Give customers easy and simple ways of reporting back problems from your software. Encourage them to be clear about the issues and log pertinent information.

8. Actively encourage customer involvement by hosting forums and chat rooms where they can discuss your product. Encourage them to be truthful and honest. Engage with them. You’ll be amazed at how receptive customers are to honest and upfront conversations. Ask for feedback on what environments your product gets used in. Ask for the truth about your product. You can’t blindly push on releasing if people are hating on your product. Or you can, but it won’t be for long ๐Ÿ™‚

The information is out there..everywhere. It’s a case of widening your awareness fields and grabbing it. Analysing it. Using it. Reacting to it. Learn how your users use your product and you are gaining invaluable feedback on how to test it. It’s never a replacement for real end users but it’s a great addition to testing your software in a test lab with controlled environments and no idea about how your users use your software.

It might be time to step outside of this test lab comfort zone and do some in situ testing. Or should that be Method testing? Customer emulation testing? End user situ testing? No doubt someone will have a good name for it. I personally like method testing. And although your managers may think your mad for wanting to test in the same environments as your users, they’ll appreciate it in the long run. There is, after all, some method in your madness.

A slight return

I’ve been a bit too busy on various side projects recently and realise I’ve not given this blog the attention I used to. Unfortunately the rest of the year is going to be mad too. So this blog will see fewer posts than normal..but I am still working on some new content for it. Here’s why it’s taking a back seat though:

ย 

Firstly I’m going to be taking a short break to get my Agile Testing Days presentation finished. I’ve got lots of research that needs collating and some more interviews to do with peeps in the testing community. The super cool organisers of Agile Testing Days also sent me a special speakers logo. How neat is that?

Speaker_at_agiletestingdays

Secondly, I’ll also be starting to ramp up my contribution to The Software Testing Club blog, which, by the way, you can all do too here.

Thirdly, I need some sleep ๐Ÿ™‚

ย 

And for those who have not been in the loop with all things Software Testing Club then I should let you know we have officially disapproved this cheeky little ebook.

http://blog.softwaretestingclub.com/2010/07/ebook-by-cbpbos-building-a-test-team/

Quality

And we had nothing to do with itย  ๐Ÿ™‚

We’ve also started posting out some printed copies of the forthcoming Testing Planet (the Software Testing Clubs very own newspaper)

Some people will have already received the printed version and we are releasing the digital version very soon indeed. It’s slightly later than expected, but hey….it will be worth the wait.

I must admit it looks amazing (I am biased though). Don’t forget you can help support the work that the peeps here at The Software Testing Club do by purchasing your printed copy. Wink Wink.

Oh yeah. I’ll also be at Eurostar later this year. Hopefully I’ll get to see some of you there.

Rob

What do you want from me?

You’ve surely been in the same situation? You’re sat using an app or site that requires some form of data entry from you, the problem is, you have no idea what it wants. There is no advanced information, communication or guidelines on what rules are going to be applied to your data. There’s sometimes an ambiguous or tricky to find error message. Sometimes there’s nothing.

As a software tester these types of bugs in our apps are low hanging fruit. They are quick wins, easy gains and an incredible booster of usability. I refer to them as “clarity bugs“. Features, functions or capabilities that just need a little bit of extra “clarity” to understand them or make them work. This may come in the form of on-screen prompts or help files or documentation or maybe through training, but sometimes there just needs to be that little bit of something extra.

ย 

Aaaa306_21ac555261_b

Image courtesy of : http://www.flickr.com/photos/rdolishny/

ย 

Help text that isn’t helpful is pointless and confusing. None existent help is assuming a level of knowledge by the end user that might not be right. Errors that aren’t descriptive are frustrating and time wasting.

To only explain what the user did wrong after they’ve actually got something wrong is wasteful and irritating. Communication is key.

When there is no explanation about what criteria the data and subsequent validation must meet it’s is an easy usability defect to grab. (think about max and min lengths on fields, or password validations or date formats etc) Don’t become blind by your pre-existing knowledge and sail straight past the fact it’s not explained anywhere. Are we sure our end users know about our specific data validation?

It can end up with the user sat there screaming “what do you want from me?” whilst hammering on the keyboard hoping they submit the perfect data.

These usability bugs are bugs. Plain and simple. Even if the application works, if it’s not usable then the chances are your customers will simply stop using it ๐Ÿ™

Want to know some more about good form design and general usability?

Check out :

ย 

ย 

Test Data or Shady Re-Ordering Technique?

I don’t normally post these types of blog where I include a screen shot of an error I’ve found in the wild on someone else’s site. There’s plenty of others doing that and with great effect. But this one really made me think.

On the Hampshire Library Search page there appears to be either:

  • Left over test data

Or

  • A lazy data ordering technique where everything no longer needed is prefixed with a Z to shift it to the end of the list. (why not delete it?)

Untitled

ย 

Sadly though, some languages seem to have been discontinued (do the people who speak them know?) yet have double entries making the list somewhat confusing. To make matters worse the search form itself didn’t appear to work. When searching for books in Winchester I get books from everywhere but Winchester. It’s a reverse search feature.

How many of us go through the software once it’s released to the wild? Would we have drilled down this low and spotted these issues? Do you think these issues were present in testing?

ย 

London Testers Gathering

P1040346-900

I had an absolute blast at the London Testers Gathering organised by Tony Bruce. Great bunch of peeps who love to talk testing. Enjoyed The Testing Geeks presentation on iCheckWebsite.com which looks really interesting. I was scooting around the site this evening and it looks very interesting indeed. Head on over to check it out : http://www.icheckwebsite.com/

It was also the first time that I got to see the Testing Planet printed version and I am really stoked with it. It looks fantastic printed out in newspaper style. It really is something very different. Love it. Digital version out soon.

If you want to get your hands on a copy then head over to : http://blog.softwaretestingclub.com/thetestingplanet/

Or you can check out issue 1 of The Software Testing Club magazine : http://wiki.softwaretestingclub.com/The+Software+Testing+Club+Magazine+-+No+1

I was also fielding a lot of questions about the Tester Types and who/what they are. Well, you’ll find them here with a load of other cool content from the STC : http://wiki.softwaretestingclub.com/Software-Testing-Resources

I’d suggest that if you get the chance to get along to one of these events you jump at it. Drinks, food and testing.

Enjoy:

On Fire

Fire

I recently tweeted that I'd lost my testing mojo. I didn't feel inspired to write. It seems some other people feel the same way. Yet others are still cranking out superb posts each day, week, month. Others have been quiet recently, but that might have nothing to do with losing their mojo.

I think my mojo has returned though. I think I'm back on fire. I've certainly had little problem writing this post. I also jotted down a few hundred lines this morning at about 5:30 am when my son kindly woke me and insisted I prepare him some breakfast,. Whilst he rapidly consumed his shreddies I blasted down some notes. It didn't take long and it felt right. Mojo. Back on track. Then we played Thomas the Tank Engine and he used me as a human bouncy castle.

I know why I'd lost my mojo, which in turn prompted this blog post. I'd been ignoring my (RSS) reading list.

It had become so large that I'd had a mild form of information overload. So I stopped reading it for a few days. Since ignoring it I'd done little reading about anything. Just a couple of books on hip-hop.

But in ignoring the feed I'd actually not addressed the underlying problem. Information Overload. So I've started dropping some subscriptions instead of ignoring my feeds. It's been difficult but I've managed to get the feed down low. And in reading a manageable amount of posts I'm feeling inspired and fired up to write again.

It's interesting how many people come up with ideas about testing and even more interesting, to me anyway, is how they actually get these ideas online. Some of the conversations that happened off the back of my mojo tweet indicated that some people just sit down and write, then publish;
some people sit down, write a little, come back to it and build on it over time;
some people, like me, write almost all of it, but then come back to fine tune and amend it, usually after a good nights sleep.

No doubt there are countless other ways of coming up with ideas and then getting the writing down. I know this topic has been discussed a few times on the writing about testing community (http://groups.google.com/group/writing-about-testing).

So go on, tell me why you sometimes lose your testing mojo (if you ever do), what you do to get it back (if you ever do) and how you manage your writing.

And by the way, if you are a blogger then you might be interested in this competion from Eurostar Conferences

Photo Courtesy of http://www.flickr.com/photos/spiritual_marketplace/

Double check your audiences values

The other night I tweeted that I thought I’d lost my testing mojo. I didn’t feel inspired to write about testing. Some good friends of mine suggested I get some rest, that it would indeed return and to be patient. So I kicked back and chilled out. As it turns out it didn’t return today so I decided to do the second best thing and re-post a fairly old post from a now defunct blog.

I sense my mojo is returning as my mind is racing around thinking about some very interest..and far out…ideas about megafactories, plastic testers, ghosts, snot (bear with me), testing therapy and idleness. But for now….a golden oldie (apologise to all who’ve read it already). For those that are interested my old blog is here : http://pac-testing.blogspot.com/

ย 

Double Check Your Audiences Values

I’ve been wondering a lot about phrases mentioned in the test world like ‘quality is the value to someone who matters’. I completely subscribe to this notion that what a tester may perceive to be a high priority bug, the customer (or product owner) may actually not. It is a sound theory and something that I try to remember when testing. Its something a tester must appreciate so that they don’t get all gnarled up over a defect they think should have been fixed.

So day one, you find out what your ‘person that matters’ deems to be important and not important, so you raise defects and only highlight (add extra emphasis on communicating in a rapid way) defects in areas your audience has told you have more value. You keep doing this until one day you get asked why you let a high priority defect go in the pot out without making more of a fuss. ‘But you didn’t see the value in fixing that bug last week…’ is your response.

What’s happened is that some dimension of this persons perception of value has shifted and you’ve been left out in the cold. The ‘person that matters’ now perceives this once low value area to be a high value area because of new information they now have at this moment in time. You feel annoyed, frustrated and betrayed.

So when we say that quality is the value to someone that matters I feel we need to add on to that ‘at that moment in time’, in fact I’m pretty sure someone, somewhere will have already done that….. (as it happens it was none other than Mr Bolton and Mr Bach (There’s a line in the Rapid Software Testing course: “Your context guides your choices, both of which evolve over time”. – Michael B))

I’ve seen this happen more times than I can remember where a defect in an area or a defect of a type (i.e. tab order) was not important at point X, but now a shift in focus has happened and it’s suddenly the top priority now at point Y. Whether you get told about this shift in focus or not is irrelevant – I see it as a testers job to find out what the focus is, from the ‘person that matters’. You can report defects until the cows come home but what really adds value for your ‘person that matters’ is when you report the defects with an emphasis on the ones of value to them.

I’m not saying don’t raise every defect you encounter, far from it, but from my experience, testers have a direct influence on how well communicated these issues are to your audience. So spend the time making the ones important to your ‘person that matters’ read like a horror story and also make them aware of these issues faster.

So find out what is important at that time to whoever may matter and work with them to add ‘quality’. Don’t forget that the person who matters will have more dimensions of information at their disposal than you do. They may not care about 100 defects in module X, they may care about 1 defect in module Y due to commercial issue Z. Work with them, let them know they can trust you, let them know you are pro-active and let them know about the bugs that matter to them. But most importantly, keep checking your audiences values as they can, sorry, they will, change over time.

A quick survey on testing #testing #qa

I’m trying to gauge some feedback on a specific element of creativity in the testing arena and as such have the first of many surveys open to gather some basic ground information.

ย 

I would really appreciate it if you could spend a few minutes (that’s all it will take for most people) to complete the survey. Please feel free to send this survey to other testers you know also. I have a limit of 100 responses so please don’t be too disappointed if you don’t get to take part.

ย 

The Survey

http://www.surveymonkey.com/s/S38KNJP

ย 

Thanks in advance to all those who take part. I’ll be sure to publish the anonymous results after Agile Testing Days Conference in October.

ย 

Note: The survey is not very scientific and is being used to gather some base numbers only. I am working on a more in-depth quantitative survey to explore a few more avenues of creativity. Hope you’ll be able to take part in that too. ๐Ÿ™‚

Rob..

Enthusiasm is catching

Have you ever noticed how you suddeny become fired up when working/training/observing someone whose enthusiastic about their topic or subject of interest? Even if you don’t enjoy the subject, their enthusiasm still rubs off on you. You feel fired up to get out and do something you enjoy. You feel motivated.

This last weekend I was lucky enough to be at Goodwood Festival of Speed. Probably the greatest car show on Earth. It really was awesome.

I got to see Sir Jackie Stewart, Richard Attwood, Nico Rosberg and many others all being interviewed as well as discussions and chit chat over the show radio and there was one major linking factor (other than cars) : Enthusiasm. Bags of it.

Passion, enthusiasm, desire, drive, attitude, engagement…whatever you want to call it. They all had barrel loads of it. Just like the many people in our testing community who live, breathe, talk, write and banter about software testing. It’s what floats their boat. And their enthusiasm is catching.

So if you’re new to the game of software testing or you want your spark re-igniting then I’d suggest you seek out some of these passionate and enthusiastic testers and soak in as much as you can from them. Their enthusiasm will rub off on you. You’ll feel motivated.

And if you are in the UK in July 2011 and have an interest in motorsport then I’d suggest you seek out a ticket for the Goodwood Festival of Speed – I’ll see you there ๐Ÿ™‚

And by the way. I think I’ve found “The Social Tester Transporter”……..

ย 

Car

More photos here: http://britainsroadsrevealed.posterous.com/