How to gain insights from customers to help your testing

In the past I have talked about the need to find out more about our end users so that we can help to focus our Testing on genuine use cases and problem areas for customers.

One of the ways I believe we, as Testers and Managers, can do this is by doing on-site visits to our customers.

However, I’ve had a number of responses stating that it is not possible or feasible. I agree it’s hard work and not always possible at all work places.

I believe a firm trait for Testers is pro-activity and persistence. I think Testers can explore other avenues to seek out information on products, customer usage and customer expectations.

Understanding end user behaviour doesn’t always rely on field research and observation.

The web is making it much easier to connect and find people, and that also includes people who use the products you work on.

You can connect to people, study the behaviours of people in your domain, or read about experiences through blogs and articles.

The web allows you to seek out deeper understandings of the types of roles, responsibilities and daily frustrations your customers (or potential customers) may face.

We aren’t just Testers, we can also be the researchers who find out more about how our products are used.


When studying data it’s important to look for patterns you can make inferences from, but it’s also important to ensure you balance the research. Don’t just find a good source and use that as your only source of insights.

For example, a power user blogging about how they use a product will give you a different perspective than a novice user.

Always try to find a counter view, or objection to the view/statement/observation made. The more dimensions you can research and explore, the stronger your assumptions and inferences from that data will be.

One of the greatest things to seek out are stories. Stories from people who use the software. Stories from people who don’t use the software. Stories from people who work in the domain.

Stories give you deep insights. Stories are basic human forms of communication. Stories are part of how we make sense of the world. However – accept that stories are often deeply one-sided. Balance is required.


How to do it

The web (social web in particular) is daunting to many people, yet others have flocked to it and adopted it with ease. Generations are growing up with the social web as an extension of themselves.

People are sharing more information than ever before.

Narrative of life is the new focus for many people. The Social Web is a fascinating place to research more about people, tech and culture.

What follows is a very short list of some of the sources of learning and insight I have used.


Research Papers

There are literally thousands of research papers available free on the web. Many of these papers are directly related to the industries we work in. They are often deeply insightful, but often very dry in their delivery.


Twitter/Facebook/Quora etc

Seek out the industry you are working in using your favourite social channel. Search using the relevant search option (for example: hashtags on Twitter are a useful way of filtering the noise) and then start observing the discussions, problems and concerns being raised.

This could be useful information for understanding how people using products like yours perform their day jobs. It can help to build persona details which might give you a richer understanding of what it’s like to work in this domain.

Each social channel comes with ways to filter the noise and ways to search for content specific to the tribe, trend or topic you are interested in. A quick google search will provide a wealth of tools for each of the social channels.



LinkedIn has forums which are one of the best sources of information about a domain or the products being used. Ask a question or sit back an observe. Just be careful of Best Practice pushers (i.e. people who see no way other than their way) – LinkedIn is rife with these peeps.



A quick google search for a blog related to your industry could soon lead you down a path of amazing information. Follow links and click through to other blogs. Read the comments – they are often the really insightful content.

Testing is about information gathering as well as bug finding. It’s too easy to put your hands up and say “can’t be done” when trying to find more information.


It’s a lot harder (but more fruitful) to be pro-active and seek alternative ways of gathering information.


How do you find out more about how your users use your product?

Exploratory Regression Testing

I was chatting through some ideas for “firing” up our regression testing process with one of my guys, Andy.

We already have a good set of the regression tests automated, but there are always some tests that need (or would benefit from) human interaction, especially when phone calls are involved.

We have a regression “pack” of standard tests but as Andy is running these each week (weekly release frequency) you can understand how it might get repetitive…and a tad boring.

So we talked about a few strategies. He and another one of the team, Simon, had decided to switch products every so often to break up the repetitiveness also.

This is a great plan. It means each of them get to see another part of the system and learn about it, it breaks up the repetitiveness and it also has an added benefit of a “fresh set of eyes”.

I won’t bore you with further details on this but one thing blurted our of my mouth whilst we were talking and at the time it made sense.

Only upon reflection later did I start to dig deeper and wonder what I was talking about.

I suggested to Andy that maybe we do some “exploratory regression testing“.

Sounds good right?

But can Exploratory Testing really be used to do regression testing?

  • Isn’t regression testing “supposed” to be about running already executed tests (that subsequently become checks)?
  • Should regression testing be about repeating already executed tests/checks? Is there value in this activity?
  • Can exploratory testing really be performed to give confidence that nothing has regressed?

The more I explored some of these thoughts the more I realised that I actually quite like the term and concept of exploratory regression testing.

No doubt there are already people using the technique and phrase, and no doubt there will be many people who don’t believe exploratory testing is regression testing, but for now I’m going to keep using it.

Here we all seemed to understand the term and idea and it’s giving us great value.

I just thought I’d get your feedback on it.


Are certifications relevant?

I know many readers of my blog would suggest I’m a hater of certifications for Testers.

That’s simply not true. Despite my ardent fight against them I am pragmatic enough to realise that getting a job often requires getting a certification. And putting a roof over your head often trumps principles and ideals.

I also believe that a certification course, delivered by a competent tutor who has bucket loads of skills and experience, can be very valuable.

I just don’t like what they have come to symbolise in the market place. I don’t like how you DO NEED A CERTIFICATE to get a job (in most cases).

Where did it all go wrong?

I’m not here to bash Certification schemes. Use your own judgment and experience on whether you think they give you insights and learnings or not.

Instead I’m going to ask you a question:

Are certifications still relevant?

  • I don’t believe they have succeeded in making people competent Testers. This is evident from the number of certified people on forums and LinkedIn asking “What is Testing?” or “Tell me how many Tests I should have for X feature!” or “why is testing so boring”.
  • I don’t believe they have succeeded in creating a universal language with which to talk about Testing. This is evident from the fact most Testers don’t know what “action word driven testing” is or what a “Software Failure Mode and Effect Analysis” is OR the fact that I call it a Test Case you call it a Test Script. The big question here is “Do most Testers care outside of their own company and context?”.
  • I don’t believe they have succeeded in promoting the value of software testing to organisations and business. I still come in to contact with a vast array of companies who don’t test, don’t appreciate testing and don’t understand what value testing can bring.

So are they still relevant?

There was a time before the Internet when you had very few places to go to obtain Testing knowledge, training or awareness. When I started out I went to the British Computer Society, a few well known books and the ISEB foundation. The ISEB crowd certified me. I still kept Testing as I had before. I just felt slightly more hire-able.

Only when I reached out to the wider community online did I find a place to soak up information and ideas about testing. I started sharing ideas. I started to meet people who thought the same way that I did. I started to feel like Testing was actually interesting. I started to find people who didn’t talk about standards, didn’t speak in platitudes and marketing pitches and didn’t push certifications at me from all angles.

When access to information is restricted or impossible those that hold the information have the power. If you wanted that information you had to pay. If you wanted to see what the “industry” thought was a good standard, you had to pay to find out, and then pay even more to be accepted.

Social networks and the “digital revolution” has made that information (and a much broader selection of ideas too) available to the masses. Having to pay for access to information is becoming rare.

Yet we still continue to pay for certifications.

We’re no longer paying for the content; almost all of that is available online, for free.

We are no longer paying for the training as it’s possible to sit the course and pass without in person training. (There are also a vast selection of excellent paid and free courses available online and in person outside of the certification schemes.)

I believe the masses* are paying for the right to say “I have a certificate!!!!!”

In a sea of people all shouting “I have a certificate!!!!!” why would anyone pick you?


* There are some people I meet who sit the certification courses as just one part of their continued learning…not the only part of their learning.

Social Media alone will not get you a job

Judging by the swarm of testers trying to connect with me on LinkedIn recently it seems no-where is now safe from those who want a job.

99.9% of those who try to connect with me have never had a conversation with me (in the real world or digital). I see the same thing on Twitter, where I have less control over who follows me and also on this blog, where I delete a huge number of comments from people wanting a job.

It seems many testers are falling in to the simplistic trap of believing that social media will get them a job. It’s not surprising when there are a number of companies, individuals and organisations who are pushing social media (in the Testing world) as the way to land a job.

It’s easy to see why people would sign up for Twitter, follow some people in testing and then directly ask them for a job.

Social Media alone will not get you a job. Social Media, in a simplistic sense, is a communication (and publishing) platform/channel. It’s a way of networking, communicating and distributing content. Social Media may get you connected to people with jobs, or put you in front of people who are hiring, but you’ll still need the skills to see the application through (unless of course the hiring manager is not assessing you on anything). You still need your thoughts, ideas and the ability to articulate these clearly. Social Media alone won’t help you there.

(Note: I do know of some examples where simply turning up to the interview would get you a testing job, but I don’t really consider that testing, or a proper job.)

Here’s some thoughts on why I believe the message is too simplistic.

  • Some companies may see an active social media user as a drain on resource. After all, you should be working instead of tweeting…right?
  • Some companies may see an active social media user as a risk. “What are they going to say that could get us in trouble?”
  • Some companies may see active social media users as a boon. After all, their business may benefit from the thought leadership, experience or free marketing. They may even be working in the social space and demand people who understand and “get” social media.
  • Some testers wield their social presence with great skill and attention, yet aren’t great testers, but great marketeers.
  • Some testers wield their social presence with great skill and attention and are amazing testers, and great marketeers.
  • Some testers don’t wield their social presence well, and are bad marketeers.
  • Some testers sign up to social channels because they think they should and do nothing spam others.
  • Some testers sign up to social channels because they think they should and have no interactions, followers or conversations.
  • Some testers use social media as a platform to sell services and goods.
  • Some testers use social media to communicate ideas and further their skills and thinking.
  • Some testers use social media to build communities.
  • Some testers use social media to destroy communities.
  • Some testers just don’t get it and stay away.
  • Some testers are afraid of social media. Their personalities don’t gel with the public display of thoughts and ideas, yet they can be incredibly talented.
  • Some companies hire very well without social media.
  • Some companies go straight to their social channels for hiring.

I probably know someone who fits all of these, and a whole load more categories too.

I know a great tester who recently landed an amazing job and she isn’t on any social networks. Not even LinkedIn.

I know testers who are on social networks and actively connecting, but they can’t get jobs. They lose out to better candidates.

I know of some testers who have been offered a job directly through Linkedin without even being interviewed.


It’s too simplistic to say that getting a job requires you to be on a social channel and it’s too simplistic to say that being on a social channel will get you a job. It’s clearly not true in either case.

What get’s you a job (apart from “jobs for the boys” mentality) is proving to yourself, your peers and the hiring manager that you can do the job.

Social Media is no different to networking in person at Testing events, presenting at conferences and offering training services to build a wider network.


Getting in front of the hiring manager is easier with more connections, but these don’t have to be online through social channels. Many testers operate very well indeed and that’s purely from the old fashioned face to face networking.

Networking has been the key to getting a job (especially in hard times) for years, and it’s true that Social Media makes that’s easier now for the masses. But any hiring manager worth their salary should be recruiting for the person and their mind, not the number of social channels they are on.

Social channels and social media can help you amplify your thoughts, build your network, generate content and communicate with those who can get you jobs, but your skills and experience will be what ultimately separates you from others.,0,1257938.column