Really enjoyed this video on Designing For Disaster for cloud companies. There are some great ideas for dealing with issues and building great teams around it.
I’ve been digging around in the archives and found my comedy white paper on The Problems With Testing (pdf).
Even after a year or so I still enjoy reading it. Is it wrong to laugh at your own jokes?
Here’s an excerpt.
Meetings are exactly why testing is in the state it is in today. Time being wasted in meetings is on the increase and the negativity that accompanies most meetings is enough to drag you down for the rest of the day.
You pitch up for the 10am meeting eager to discuss project details with the rest of the team only to find that the previous meeting is over running.
After standing around making over enthusiastic “watch checking” gestures you decide to wander around to the kitchen to make a coffee.
You return with your coffee. The meeting room is still busy. No other attendees have turned up for your meeting.
The Project Manager from the over running meeting holds up a finger to indicate he will be no longer than one minute. You resist waving a different finger back.
The previous meeting finally finishes. Attendees leave the room scowling at you for making them end their meeting before any decisions were made.
Still no one has turned up. On a positive note though you’ve bagged the best seat at the back where you can “people watch” through the window.
With no one turning up you decide to go and make another coffee.
Whilst making a coffee you suddenly become overwhelmed with a streak of mischievousness and decide to unscrew the lids on all the sugar shakers and balance them on the top, so it looks like the lid is still on.
You return to the meeting room to find all attendees in their waiting for you. You are gobsmacked but make your apologies.
You now only have the choice of one chair.
The one next to the projector at the front.
The Business Analyst (BA) decides to make a phone call and leaves the room.
The BA now returns but the Project Manager (PM) decides she now wants to make a call so leaves the room.
The Project meeting finally kicks off. First agenda point is raised. How to eliminate waste in the project.
You bite your lip and keep your late meeting comments to yourself.
You start to have heart palpitations from all the caffeine. Your foot starts to tap sporadically. You have an urge to throw rolled up balls of paper at the tech lead.
Apparently the defect count is down.
Smiles all round
The low defect count is because the build is broken and no-one is testing.
General feeling of disappointment.
After more pointless defect stats the urge to throw paper is becoming increasingly strong. You now need the toilet. Badly.
The PM announces, after checking corporate emails on his Blackberry, that the MD has just had a sugar shaker full of sugar dumped in his coffee and demands that the child responsible for this behavior steps forward.
Your face turns grey. You feel faint.
You start humming.
You are asked to stop humming.
And to sit still.
Discussion moves to how we solve the quality problem.
Blame session erupts and heated exchanges are made.
You start to feel quite faint now after more blaming and shouting.
You’ve also noticed how the support manager has a faint smell of Angel Delight about him. Strawberry flavour with a hint of marker pen.
Someone suggests a coffee break, someone else suggest we just abandon the meeting. The PM suggests we press on.
Meeting ends after being harangued by the next meeting attendees.
The group disperse. You charge to the toilet. No decisions were made.
You turn up to your training session fifteen minutes late. Today’s topic: time management.
In the past I have talked about the need to find out more about our end users so that we can help to focus our Testing on genuine use cases and problem areas for customers.
One of the ways I believe we, as Testers and Managers, can do this is by doing on-site visits to our customers.
However, I’ve had a number of responses stating that it is not possible or feasible. I agree it’s hard work and not always possible at all work places.
I believe a firm trait for Testers is pro-activity and persistence. I think Testers can explore other avenues to seek out information on products, customer usage and customer expectations.
Understanding end user behaviour doesn’t always rely on field research and observation.
The web is making it much easier to connect and find people, and that also includes people who use the products you work on.
You can connect to people, study the behaviours of people in your domain, or read about experiences through blogs and articles.
The web allows you to seek out deeper understandings of the types of roles, responsibilities and daily frustrations your customers (or potential customers) may face.
We aren’t just Testers, we can also be the researchers who find out more about how our products are used.
When studying data it’s important to look for patterns you can make inferences from, but it’s also important to ensure you balance the research. Don’t just find a good source and use that as your only source of insights.
For example, a power user blogging about how they use a product will give you a different perspective than a novice user.
Always try to find a counter view, or objection to the view/statement/observation made. The more dimensions you can research and explore, the stronger your assumptions and inferences from that data will be.
One of the greatest things to seek out are stories. Stories from people who use the software. Stories from people who don’t use the software. Stories from people who work in the domain.
Stories give you deep insights. Stories are basic human forms of communication. Stories are part of how we make sense of the world. However – accept that stories are often deeply one-sided. Balance is required.
How to do it
The web (social web in particular) is daunting to many people, yet others have flocked to it and adopted it with ease. Generations are growing up with the social web as an extension of themselves.
People are sharing more information than ever before.
Narrative of life is the new focus for many people. The Social Web is a fascinating place to research more about people, tech and culture.
What follows is a very short list of some of the sources of learning and insight I have used.
There are literally thousands of research papers available free on the web. Many of these papers are directly related to the industries we work in. They are often deeply insightful, but often very dry in their delivery.
Seek out the industry you are working in using your favourite social channel. Search using the relevant search option (for example: hashtags on Twitter are a useful way of filtering the noise) and then start observing the discussions, problems and concerns being raised.
This could be useful information for understanding how people using products like yours perform their day jobs. It can help to build persona details which might give you a richer understanding of what it’s like to work in this domain.
Each social channel comes with ways to filter the noise and ways to search for content specific to the tribe, trend or topic you are interested in. A quick google search will provide a wealth of tools for each of the social channels.
LinkedIn has forums which are one of the best sources of information about a domain or the products being used. Ask a question or sit back an observe. Just be careful of Best Practice pushers (i.e. people who see no way other than their way) – LinkedIn is rife with these peeps.
A quick google search for a blog related to your industry could soon lead you down a path of amazing information. Follow links and click through to other blogs. Read the comments – they are often the really insightful content.
Testing is about information gathering as well as bug finding. It’s too easy to put your hands up and say “can’t be done” when trying to find more information.
It’s a lot harder (but more fruitful) to be pro-active and seek alternative ways of gathering information.
How do you find out more about how your users use your product?
I was really hoping to get to Eclipse Testing Day 2012 this year, but sadly can’t make it.
Check out the talks though! Some great ideas…and who can fault the price (especially if you’re in Germany) 🙂
It looks like there will be some good panel discussions too if the submitted questions are anything to go by:
- Some people say that testing, as we know it, is dead, because a testing phase is something that only happens in waterfall projects at the very end. Do you agree with this statement?
- When an active community of users works with a product, they are likely to give feedback in the form of bug reports and enhancement requests. How do you suggest incorporating their feedback into the process for the further development of the product?
- The community is a great source of feedback. Nevertheless, it is advantageous to release as few bugs as possible. What do you recommend to gain the balance between community feedback and minimizing bugs in production?
- A frequent claim when talking about testing is “my developers don’t make any errors, so we can save money and time by not testing”. What is your view on this?
I was chatting through some ideas for “firing” up our regression testing process with one of my guys, Andy.
We already have a good set of the regression tests automated, but there are always some tests that need (or would benefit from) human interaction, especially when phone calls are involved.
We have a regression “pack” of standard tests but as Andy is running these each week (weekly release frequency) you can understand how it might get repetitive…and a tad boring.
So we talked about a few strategies. He and another one of the team, Simon, had decided to switch products every so often to break up the repetitiveness also.
This is a great plan. It means each of them get to see another part of the system and learn about it, it breaks up the repetitiveness and it also has an added benefit of a “fresh set of eyes”.
I won’t bore you with further details on this but one thing blurted our of my mouth whilst we were talking and at the time it made sense.
Only upon reflection later did I start to dig deeper and wonder what I was talking about.
I suggested to Andy that maybe we do some “exploratory regression testing“.
Sounds good right?
But can Exploratory Testing really be used to do regression testing?
- Isn’t regression testing “supposed” to be about running already executed tests (that subsequently become checks)?
- Should regression testing be about repeating already executed tests/checks? Is there value in this activity?
- Can exploratory testing really be performed to give confidence that nothing has regressed?
The more I explored some of these thoughts the more I realised that I actually quite like the term and concept of exploratory regression testing.
No doubt there are already people using the technique and phrase, and no doubt there will be many people who don’t believe exploratory testing is regression testing, but for now I’m going to keep using it.
Here we all seemed to understand the term and idea and it’s giving us great value.
I just thought I’d get your feedback on it.
I know many readers of my blog would suggest I’m a hater of certifications for Testers.
That’s simply not true. Despite my ardent fight against them I am pragmatic enough to realise that getting a job often requires getting a certification. And putting a roof over your head often trumps principles and ideals.
I also believe that a certification course, delivered by a competent tutor who has bucket loads of skills and experience, can be very valuable.
I just don’t like what they have come to symbolise in the market place. I don’t like how you DO NEED A CERTIFICATE to get a job (in most cases).
Where did it all go wrong?
I’m not here to bash Certification schemes. Use your own judgment and experience on whether you think they give you insights and learnings or not.
Instead I’m going to ask you a question:
Are certifications still relevant?
- I don’t believe they have succeeded in making people competent Testers. This is evident from the number of certified people on forums and LinkedIn asking “What is Testing?” or “Tell me how many Tests I should have for X feature!” or “why is testing so boring”.
- I don’t believe they have succeeded in creating a universal language with which to talk about Testing. This is evident from the fact most Testers don’t know what “action word driven testing” is or what a “Software Failure Mode and Effect Analysis” is OR the fact that I call it a Test Case you call it a Test Script. The big question here is “Do most Testers care outside of their own company and context?”.
- I don’t believe they have succeeded in promoting the value of software testing to organisations and business. I still come in to contact with a vast array of companies who don’t test, don’t appreciate testing and don’t understand what value testing can bring.
So are they still relevant?
There was a time before the Internet when you had very few places to go to obtain Testing knowledge, training or awareness. When I started out I went to the British Computer Society, a few well known books and the ISEB foundation. The ISEB crowd certified me. I still kept Testing as I had before. I just felt slightly more hire-able.
Only when I reached out to the wider community online did I find a place to soak up information and ideas about testing. I started sharing ideas. I started to meet people who thought the same way that I did. I started to feel like Testing was actually interesting. I started to find people who didn’t talk about standards, didn’t speak in platitudes and marketing pitches and didn’t push certifications at me from all angles.
When access to information is restricted or impossible those that hold the information have the power. If you wanted that information you had to pay. If you wanted to see what the “industry” thought was a good standard, you had to pay to find out, and then pay even more to be accepted.
Social networks and the “digital revolution” has made that information (and a much broader selection of ideas too) available to the masses. Having to pay for access to information is becoming rare.
Yet we still continue to pay for certifications.
We’re no longer paying for the content; almost all of that is available online, for free.
We are no longer paying for the training as it’s possible to sit the course and pass without in person training. (There are also a vast selection of excellent paid and free courses available online and in person outside of the certification schemes.)
I believe the masses* are paying for the right to say “I have a certificate!!!!!”
In a sea of people all shouting “I have a certificate!!!!!” why would anyone pick you?
* There are some people I meet who sit the certification courses as just one part of their continued learning…not the only part of their learning.
A common problem for test teams looking to grow their performance testing capabilities is the lack of horse power to provide sufficient load. Well, step up Blaze Meter.
BlazeMeter is an on-demand load testing service which can take your jMeter scripts and deliver them across a number of their own machines/clients to give you the large load profile you want.
It’s a straight forward process to get scripts running through their simple and intuitive interface.
It’s got a wealth of performance metrics available for you to work through too, so you can clearly see how the tests have performed at a variety of points. There are also a number of advanced features for those more familiar with jMeter and performance testing.
It’s certainly a cost effective way of hitting high loads with your jMeter scripts, although you will need a decent test stack to support the delivered load.
We’ve had a great experience with Blaze Meter and their support was outstanding too.
And BTW – I’m not affiliated with Blaze Meter in the slightest – just a very happy customer
Judging by the swarm of testers trying to connect with me on LinkedIn recently it seems no-where is now safe from those who want a job.
99.9% of those who try to connect with me have never had a conversation with me (in the real world or digital). I see the same thing on Twitter, where I have less control over who follows me and also on this blog, where I delete a huge number of comments from people wanting a job.
It seems many testers are falling in to the simplistic trap of believing that social media will get them a job. It’s not surprising when there are a number of companies, individuals and organisations who are pushing social media (in the Testing world) as the way to land a job.
It’s easy to see why people would sign up for Twitter, follow some people in testing and then directly ask them for a job.
Social Media alone will not get you a job. Social Media, in a simplistic sense, is a communication (and publishing) platform/channel. It’s a way of networking, communicating and distributing content. Social Media may get you connected to people with jobs, or put you in front of people who are hiring, but you’ll still need the skills to see the application through (unless of course the hiring manager is not assessing you on anything). You still need your thoughts, ideas and the ability to articulate these clearly. Social Media alone won’t help you there.
(Note: I do know of some examples where simply turning up to the interview would get you a testing job, but I don’t really consider that testing, or a proper job.)
Here’s some thoughts on why I believe the message is too simplistic.
- Some companies may see an active social media user as a drain on resource. After all, you should be working instead of tweeting…right?
- Some companies may see an active social media user as a risk. “What are they going to say that could get us in trouble?”
- Some companies may see active social media users as a boon. After all, their business may benefit from the thought leadership, experience or free marketing. They may even be working in the social space and demand people who understand and “get” social media.
- Some testers wield their social presence with great skill and attention, yet aren’t great testers, but great marketeers.
- Some testers wield their social presence with great skill and attention and are amazing testers, and great marketeers.
- Some testers don’t wield their social presence well, and are bad marketeers.
- Some testers sign up to social channels because they think they should and do nothing spam others.
- Some testers sign up to social channels because they think they should and have no interactions, followers or conversations.
- Some testers use social media as a platform to sell services and goods.
- Some testers use social media to communicate ideas and further their skills and thinking.
- Some testers use social media to build communities.
- Some testers use social media to destroy communities.
- Some testers just don’t get it and stay away.
- Some testers are afraid of social media. Their personalities don’t gel with the public display of thoughts and ideas, yet they can be incredibly talented.
- Some companies hire very well without social media.
- Some companies go straight to their social channels for hiring.
I probably know someone who fits all of these, and a whole load more categories too.
I know a great tester who recently landed an amazing job and she isn’t on any social networks. Not even LinkedIn.
I know testers who are on social networks and actively connecting, but they can’t get jobs. They lose out to better candidates.
I know of some testers who have been offered a job directly through Linkedin without even being interviewed.
It’s too simplistic to say that getting a job requires you to be on a social channel and it’s too simplistic to say that being on a social channel will get you a job. It’s clearly not true in either case.
What get’s you a job (apart from “jobs for the boys” mentality) is proving to yourself, your peers and the hiring manager that you can do the job.
Social Media is no different to networking in person at Testing events, presenting at conferences and offering training services to build a wider network.
Getting in front of the hiring manager is easier with more connections, but these don’t have to be online through social channels. Many testers operate very well indeed and that’s purely from the old fashioned face to face networking.
Networking has been the key to getting a job (especially in hard times) for years, and it’s true that Social Media makes that’s easier now for the masses. But any hiring manager worth their salary should be recruiting for the person and their mind, not the number of social channels they are on.
Social channels and social media can help you amplify your thoughts, build your network, generate content and communicate with those who can get you jobs, but your skills and experience will be what ultimately separates you from others.