8 Lessons Learned From Building A Team

Building a successful test team is hard work.

I’ve outlined 8 lessons learned from building a team in an article titled “The Blazingly Simple Guide To Growing A Test Team”.

It has been published in the October edition of Testing Trapeze.

Testing Trapeze October Front Cover
Testing Trapeze October Front Cover

8 Lessons Learned From Building A Team

The article sits along great articles from Erin Donnell, Richard Robinson, Margaret Dineen and Dave Robinson.

Testing Trapeze is a bi-monthly magazine edited by Katrina Clokie and put together with an amazing team.


The 8 lessons I learned from building and growing a test team are:

  1. Appreciate it will take time
  2. Understand the purpose of the testing team
  3. Appreciate that certifications don’t make someone a great tester
  4. Automation specialists don’t kill the need for testers
  5. Find people with a passion for testing
  6. The recruitment process is key to your success
  7. Interviewing is about selling yourself and solving your problem
  8. Quality is not just the responsibility of testing

You can read more in the October edition of Testing Trapeze.

Are you offering a career in testing or just a job?

Many companies are offering a job in testing.

Many companies are offering a career in testing.

Testing - A Job or a Career
Testing – A Job or a Career

A career is a series of experiences. These experiences may come from many  jobs at many companies. Or they may come from a single place of work with a varied set of experiences.

A job is what some companies are offering. And there is nothing wrong with this.

Jobs are good. So too are careers. Some people can experience more in a single company than others may experience in 10 other jobs.

Being honest about the role you have is the key to getting the right person.

If you are offering a job then say so. If you are offering a career then say so too.

Some people just want a job. Some people want a career. It’s important to match the person and the job. By matching the right people to the right jobs you stand a chance of solving your problems. (You are solving a problem..right?)

It seems so simple but it’s a mistake many hiring managers make.

Get it wrong and you spend money recruiting the wrong person.

I once took a role that I believed to be a career. When I joined it was clear it was a job. I didn’t want a job, I wanted a career. I knew I would never get that at this company – I left after just 6 days (3 of which were spent trying to find out who to hand my notice in to).

I’ve also made the mistake of hiring people who wanted a job when I was offering them a career. It’s not good.

One is no better than another. Someone who takes a number of jobs can build a strong career.

Contractors, by their nature, are often looking for the next job. But they can build a strong career from those different jobs.

Your task is to match the right person to the right role.

In the interview be clear about what you are offering and strive to understand what the candidate wants.

Is it a job, or a career?

Be honest with the candidate and you’ll likely get the right match. Get it wrong and you’ll have probably hired the wrong person.

Using Kanban in the recruitment process

We’ve been recruiting heavily for a while now and it’s been an epic journey from those early days 4 years ago when I started here at NewVoiceMedia to where we are today.

The Dev team has grown very quickly indeed and we’re still recruiting for talented Developers to join us.

I’ve been involved in pretty much every hire in to the Dev team over the last 4 years and have thoroughly enjoyed this recruitment process. It’s where the inspiration for this latest series of blog posts has come from. When you’ve interviewed 150+ people at various stages of recruitment you start to find ways to optimise this process, see interesting patterns and work out whether or not you enjoy it. I enjoy it 🙂

When you receive a tonne of good candidates each week it’s important to have a process that is efficient and effective. I like to use Kanban to visualise the process and aid with tracking people as they move through our recruitment process.

Kanban is a visual tool to show you your work in process so that you can start to make decisions from the information. LeanKit have a decent description of what Kanban is on their website so I won’t repeat it here.

A Recruitment Kanban Board
A Recruitment Kanban Board

Above is an image of a typical recruitment Kanban board. The above shows a simple recruitment process from the point of view of a recruiter.

Any Kanban tool will work for visualising your stages and process, from post-its on a whiteboard to a fully fledged Trello board. I like Trello because it’s super easy to use, is free and can pretty much be configured to map out any process. The above is a doodle I did in the Paper App.

For this example I’ll use a standard recruitment process that starts with a backlog of applicants, then a phone interview, then a coding exercise, followed by a face to face and then hopefully an offer and a “Yes”, or it may just be a “No” from any stage within the process. Testers and Scrum Masters miss the Code Review stage unless that exercise is relevant but essentially follow the same flow.

An important part of an efficient system is to ensure you don’t spend time copying and pasting details. If you work closely with your recruiter you should be able to get them to send an email directly to Trello (with or without salary details depending on your level of sharing) so the candidate they are putting forward appears directly in the backlog of your Kanban board.

On the backlog are people who are applying for a role (hopefully you’ll be as quick as possible to provide timely feedback for the person applying 🙂 ). Then it’s a case of moving them through the different phases as you do your interviewing. I think it’s important to include an “Offer In Progress” stage because there are sometimes delays and of course, until you receive the signed paperwork back they technically have not accepted.

Each stage will no doubt trigger some actions for you and your team such as sending the coding exercising, reviewing the code and then getting all of the contract preparation stuff sorted for example. Simply flow people through your system and use the Kanban board to show you where people are in the process, where your bottlenecks are and also give you clues as to how efficient your recruitment and interviewing process is.

I find Kanban a really effective way to see how applications are processed. I firmly believe in responding to every single candidate as quickly as possible and Kanban helps me to do this and avoid bottlenecks in our system.

How do you manage your hiring?

Dedicate time each day to recruiting

Recruiting good people takes lots of time, energy and focus.

In my current busy work schedule it’s almost impossible to find a block of time to sit down and rattle through recruitment tasks. So I’ve tried to focus on doing just one or two recruitment based tasks each day.

I’ve come to realise that this is much more effective than leaving all of the tasks to bunch up and create a daunting wall of project work.

To work through this backlog I’ve found it particularly useful to block time out in my calendar specifically for recruiting. An hour here and an hour there works a treat.

The Dev management team here at NewVoiceMedia share a backlog of recruitment tasks all focused on evolving our recruitment process to be industry leading. We want to hire the best people in the industry to help us achieve our business goals so we need to evolve our recruitment process to be the best it can be too.

Your own recruitment process will likely reflect the amount of effort you put in to it. It takes time (and hence costs money) to create a good recruitment process but it’s time and money well spent.

If you feel you have a daunting wall of recruitment goals and tasks then try breaking them down and kick start daily actions towards your recruiting goals.

Don’t rush in to a hiring decision


We regret to inform you image
We regret to inform you – Image from Caro Wallis “Sweet Sorrow” March 24, 2010 via Flickr, Creative Commons Attribution.

It’s very easy when recruiting to rush in to a decision about hiring somebody, especially so when you haven’t been inundated with a significant number of good candidates. Be cautious though when making a decision and be sure that you’ve truly explored the options open to you.  It’s sometimes better to delay a hire (and sub-sequent knock on to work load) than it is to hire someone who’s not right for the team. Hiring someone who is the wrong fit can have a real detrimental impact to the whole team. One way to mitigate rushing in to a decision is to have a number of different people interview the candidate and then be responsible for making a decision about hiring them.  If you have at least 4 people interview a candidate then you can get a balanced view of the candidate. Ideally the other interviewers will be from other functions across the delivery side of the business (agile, dev, product, service etc). If any one person says “no” then it’s up the others to try and convince them to change their mind (if the candidate is worth fighting for) or the other must simply accept the decision and move on. It can be hard to say “No” to really good candidates, buts it’s a process that helps keep the bar high. It is of course more expensive and time consuming but if you’re trying to hire the best talent then it’s a system I would absolutely recommend.

So why do we rush in?

Of course there are commercial deadlines to meet and hiring constraints to work within but in my experience most people rush in to making decisions on candidates because the candidate is the “best” of the rest. This sounds harsh but it happens very frequently, especially in the testing and scrum master hiring arena (no doubt others too). After a string of pretty bad interviews it can be tempting to say “yes” to someone who stands out. And this may be a good strategy as you may get the person you want, but is the person standing out because they are exactly what you’re after, or just because they are somewhat better than the others? In the future I’ll share ways to get the right candidates first time so you spend less time interviewing bad candidates, but for now take it easy when hiring. Take some time, don’t rush, don’t feel the pressure too much and always try to be objective in your decision making. Hiring someone that doesn’t fit can be a very costly and demoralizing event. Instead, consider a group consensus as a way to get a more balanced view of a candidate.

How to work with recruiters

It’s inevitable that at some point during your recruitment drive you will need to use a recruitment agency.

Argh – don’t worry – they aren’t all bad, in fact a great many are invaluable for finding you the right candidates.

Here’s some of my advice about working with recruitment agencies (note – it is just advice from my experience – experiment and work out what works for you – I’d be keen to hear more stories of success (and failure)).

Meet them face to face

Meeting a recruiter face-to-face gives you the chance to form a strong relationship. There’s nothing quite like shaking their hand and sitting down to chat with each other in the same room. Of course it is possible to chat to them over the phone or using modern comms like Google Hangouts, etc, but in-person is preferable in my view.

Asking them to come to your office is the first step. This can often show you their commitment as some agencies simply won’t come to see you. Scratch them off the list. Sure there are logistical reasons sometimes but the cost of travel is not that high nowadays and it shows willing.

You also get the opportunity to observe their body language. Are they welcoming and friendly or do they act uncomfortable and twitchy, especially if they deviate from their non-verbal “baseline” when asked tough questions.

When you meet someone face to face you will form an impression. That impression may or may not be a true reflection of that person, but as they say, your perception is your reality. If you don’t trust them or gel with them then don’t take the relationship any further.

Trust your instincts. The market is swamped with recruiters, both good and bad. Only work with those you trust.

Remember – all recruiters are in sales (aren’t we all?) and they will all have that elusive candidate you’re seeking – be critical of this and work with those who you trust.

Agree terms you’re both happy with

At some point you will have to agree terms. This is often dealt with by financial teams or HR. If you’re negotiating yourself then work out a good deal for both of you.

Try and find a rate (and overall package) you are both happy with. These details may be mandated by your HR team so you may not need to worry too much about this, but it’s important to ensure both parties are happy about the terms.

Both parties being happy about the deal is important as it’s a relationship you are forming. Of course, you may never know if they are truly happy but don’t push for a rate that they are not comfortable with as you may find they simply don’t invest the time in finding you the right candidates.

The market is competitive for both hiring manager and agencies and the balance appears to be in the hiring managers favor (for now) but that doesn’t mean you should push too hard on the recruiter – they are running a business also and ideally you’re building a long term relationship here so it needs to start on a good note.

It’s also worth taking the time to understand what goes in to finding candidates – it’s best not to assume you know what is involved (and hence how much you believe this work is worth) unless you truly do know what is involved.

However, don’t be pushed to a rate you’re not happy with either.

I’ve had recruiters in the past hold out for sky high rates with no real justification as to why I should pay so much. The reality is that a “sky high” cut of nothing is still nothing.

Learn from them

Good recruiters know their domain deeply. There’s often an assumption that recruiters don’t know anything and that they are “just” sales people – this is derogatory thinking. Sure, some of them simply throw unsuitable people are job openings and are on the whole unprofessional – but good recruiters know their work very well indeed and are genuinely trying to do a great job for you. We can learn a lot from them.

Ask them how you can stand out in a noisy market and what needs changing regarding your current recruitment process, job adverts or interview approach. And if they can’t answer any questions about their domain or industry – well – you need to make your mind up as to whether you trust them after that or not.

Ask them whether they have competing clients in the same locale/area/domain and how this will impact your search for candidates.

Recruitment agents are often recruiting for a number of clients. Some of these other clients may be in the same sector and locale looking for the same candidates. You need to work out how this will affect your recruiting, what you need to do to change these circumstances (i.e. how to stand out in the market – more on that in future posts) and how to ensure you’re the first port of call for suitable candidates.

Review their effectiveness

Every so often it pays to review the recruiters effectiveness. Don’t go on the number of submitted candidates. Instead you need to work out the success ratio of submissions to job offers.

Some recruiters may only send a small number of applicants but all of them may convert to an offer.

If you’re working with a number of agencies don’t be afraid to drop those who aren’t performing well and replace with a new agency. It’s important to keep trying new agencies if you’re not getting the results you want.

Only work with a few agencies

This is an important point. The fewer agencies you work with the easier the management of recruitment becomes. But more importantly the better your relationships may become too.

Another aspect is that good recruiters go out and find good candidates. These good candidates are often hard to find and few in numbers. The more recruiters you work with the more likely it is that these candidates get “prodded” a number of times for the same role by all of your recruitment agents. This is annoying. The first impression someone has of your company is often from the recruitment consultant. Being badgered by several recruiters for the same role is not a good impression.

Working with a small number of agencies allows both sides to really understand what is required from each other and you will hopefully talk more about what is working and what isn’t.

For example, when I hire for testers I only ever worked with just a single recruiter – and he delivers time after time. He now knows the roles I recruit for and the people I look for. We’ve spent years building this relationship and it’s working.

Outline the roles and the people you’re looking for – in depth

The person fit is always more important than the skills alone. It’s great to find candidates with the skills but if that person is a poor team player or causes chaos in the office then it’s probably not worth employing them.

My experience tells me that someone who is a strong team player outweighs someone with perfect technical skills but a poor attitude. Of course, it would be great to find someone with top skills and an awesome team fit in one package but these people are hard to find. Skills, tools and experience can be taught and learned – it’s much harder to change someone’s personality and outlook about their work.

You’ll have to talk about financials at some point so make it clear what your maximum salary is for the role and what the overall package entails – ensure both sides truly understand this. Is the package realistic? Recruiters can’t find the impossible so if you’re salary is WAY too low for the role then don’t expect to see too many candidates. Your recruiter will know the market conditions and you know testing – together it should be a good match.

Outline the process

Ensure both sides clearly know what the process is for submitting candidates, giving feedback and making rejections/offers. It’s important to align expectations so you don’t muddle the process or step on each others toes.

A professional process flow from CV submission to on-boarding your new team member gives a great impression – although expect there to be unplanned problems in all processes and for the process to change over time.

Give them honest feedback so that they can improve their searching

A recruiter cannot change their provision of candidates with any certainty if they don’t have honest feedback about the candidates they have been putting forward. Be clear in explaining why someone wasn’t suitable and make suggestions on what the recruiter can look for to qualify future candidates better.

Only by doing so will you ensure you get the candidate you want. Of course, you may not know what sort of candidate you want – this will make it harder for the recruiter but it’s not something an honest brain-storming session won’t help resolve.

Don’t work with recruiters who rely on certifications

My advice would be to never work with a recruiter who is finding you candidates using certification searches and filters. They most likely don’t know about testing and are simply following a norm that may need breaking.

Of course, you may be happy using this approach (am I also therefore suggesting you don’t know about testing? 🙂 ) but it’s going to generate a lot of unsuitable candidates for you to review.

Recruiting using just certification filters will find you many candidates but they are unlikely to find the right candidates. You require more than certifications don’t you?

Don’t work with recruiters who solely rely on job boards

Posting just to jobs boards is a simple strategy (hint: you could do the same thing) and one that will result in many candidates coming forward – but mostly an abundance of people who aren’t suitable. Jobs boards are mostly ineffective in my experience.

A recruiter who relies solely on jobs boards will be having mostly the same influx of applications as you would get if you advertised there. They will be fending unsuitable candidates off with a stick or putting them forward for you to fend off. Good candidates could get lost in the mass of applications also.

Always ask your recruiter how they find their candidates and be sure you are happy with their approach. I prefer recruiters to head-hunt rather than post job adverts. Some people don’t like this approach. I would advocate avoiding a recruiter who solely relies on jobs boards, LinkedIn job adverts or certification keyword searches; there are many other creative ways of sourcing good candidates.

Work with them to define the candidates you want and a search approach you are happy with.

When I mention this I often get criticized for not trusting recruiters do their job, but in the early days I, as a hiring manager, need to understand where my candidates are coming from and what their first impressions of our business are likely to be. Working with recruiters is about forming a relationship and that can only start to form if both sides talk to each other and align expectations.

Once you find a good recruiter stick with them and keep giving them your business.

In my experience building a long term relationship with your recruiters leads to great success.

This is part of a series exploring how to hire good testers – the reverse “how to get a freaking awesome job” is covered in my book Remaining Relevant – a book for testers who want to take control of their careers. It’s full of advice on how to find good jobs, perform well in an interview and take control of your own self learning.

You can follow more posts in the series using this category link – http://thesocialtester.co.uk/category/hiring-testers/ or by subscribing to my RSS feed

Recruiting for the right talent is time consuming

Recruiting for the right talent is time consuming. Trust me. I spend a huge portion of my day involved in recruitment activities but it’s well worth it for the end result.

Good testers (and programmers/scrum masters etc) are really hard to find and increasingly hard to tempt to your workplace. They are in demand and hence often take extra leg work to get them for an interview – let alone get them on board.

However, over time recruitment becomes easier and quicker, especially if you take recruitment really seriously and work on building a large network and a multi-threaded approach to recruitment (more coming soon). In some instances recruitment can be very quick indeed – but in the early days it’s time consuming.

Or of course you could throw lots of money at a series of recruiters and chance that, but in my experience there is small group of recruiters who are especially good at finding top end testers.

In Johanna Rothman’s excellent book “Hiring Geeks That Fit” she makes the following statement:

“You don’t have to spend gobs of money to find great candidates, but if you don’t, you probably will need to spend time. Remember, potential candidates may not all look in one place to learn about the great job you have open, so you need to use a variety of sourcing techniques to reach them.”

Recruiting is hard work and it is time consuming so as you embark on hiring a team, or a tester, don’t expect it to happen overnight. It can take months before that elusive candidate turns up. Be prepared and don’t start your recruitment campaign days before you need a tester. Start early and be prepared. Or of course, throw gobs of money at the problem.

This is part of a series exploring how to hire good testers – the reverse “how to get a freaking awesome job” is covered in my book Remaining Relevant – a book for testers who want to take control of their careers. It’s full of advice on how to find good jobs, perform well in an interview and take control of your own self learning.

You can follow more posts in the series using this category link – http://thesocialtester.co.uk/category/hiring-testers/ or by subscribing to my RSS feed

Why testers really should learn to code

Testers need to learn to code *. Period.

And here’s why.

The market demands it and the supply is arriving.

As a hiring manager I’ve personally seen a massive increase in the number of testers who can now code. They may not be able to write production grade feature code, or automated tests, but they can write scripts to help them test, or they can write a small app that will inject data or they can extend an Open Source tool to make it work for their needs.

They can basically dig deeper than what you see on the screen, do more with automation and do a more varied set of testing by using tools and code. I see this as a positive thing – I know some people may not.

There is also a growing number of developers who are moving in to a technical testing capacity and learning how to do good exploratory testing and test planning. The market is buoyant for testers who code (and who know how to sell themselves) and it’s good for hiring managers. It’s quite rare now for me to find a tester who isn’t coding, or at least really focused on learning it.

In a nutshell the market is now being supplied with these once elusive testers who can code.

You could of course replace “learn to code” with “learn to do security testing” or “learn to do performance testing” but the reality is that most of these niches also require coding or scripting experience. It’s rare to find a tool that pops out of a box and does what you want it to do – despite what many of the tool vendors sales people will tell you.

To ignore the shift in the market is to leave yourself at a disadvantage when trying to get a job.

Whether you believe that coding is essential to being a good tester or not will soon become an irrelevant moral viewpoint. Testers now need to remain relevant to those with jobs, and those people with jobs for offer are starting to get multi-skilled testers (or T-Shaped as I call them) applying.

Instead of investing energy in fighting the inevitable train of change it might be worth spending that time learning to script, or carving out a niche in another aspect of testing (security, usability, off-shoring, test management etc), or resigning yourself to being outpaced in job applications.

We should be careful though to discern from simply following trends such as certifications and following evolutions in how testing is done. One is very dangerous and leads to lazy recruiting and competitions like certification inflation – the other is a natural progression of our testing craft. Learning to code is deeper than a certification; it’s a skill that can open up many new doors and give you access to tests that were once impossible without some code. Of course you could rely on the developers in the team to help out and that models works well, but the market is shifting and to remain relevant in this market means your skills will need to shift also **.

It’s no longer enough to be a tester who doesn’t code, because when you apply for a job you may be up against a tester similar to you who can code ***.


* Note – there are of course many roles within testing that mean coding or scripting is not essential such as management, strategy, coaching, leadership/directorship roles and problems solvers etc but for the mainstream testing roles the times are changing.

** Note – shifting skills could also mean learning specific tools, learning how to solve problems or, as the post suggests, learning how to code.

*** Note – of course – there are also lots of other ways to get hired rather than applying for a job – thought leadership and networking are two – I cover lots more in my book.

Testing is bigger than just testing

Testers have a main expertise in software testing.

Some testers could even have a more focused area of expertise such as usability, performance, security or exploratory testing.

When recruiting testers, you’re more than likely looking for one of these testers who has a core expertise in testing. Sounds logical.

But this is just an expertise and developing software requires more than just an expertise in testing; it requires doing what’s needed to ship software as a team.

Shipping software results in the company achieving its goals; most typically something to do with money. Ship stuff – get paid – keep the business going.

Shipping software often requires testers to step outside of traditional roles associated with testing, or to evolve what testing “is” so dramatically, that at first it may not appear to be testing.

The same is true of scrum masters, developers and anyone else building software.

So when hiring testers it’s important to hire testers who know their role within the company is more than testing; their role is to help the company ship good quality software and sometimes this means doing something that might not be testing.

In my experience all good testers are happy to evolve their skills to meet market changes and sometimes wear many hats within the business. Testing is increasingly becoming an activity done by everyone and is no longer just a phase done by people with the job title of “tester”.

Interviewing for this outlook and flexible thinking is harder than it may first seem, but it is possible.

If you want to ship good software frequently then it’s important to find people who can support this vision by being solid members of a team.

Team-work is all about the end goal and not just about the individual job title, main responsibilities or specialism.

What problem is hiring a tester solving?

When hiring testers it’s important to know what problem you are trying to solve.

By defining the problem that you have you’ll be able to ensure you solve the problem in the right way, whether that be by recruiting the right person or actually coming to the realization that recruitment won’t help.

Throwing more testers at a test problem is no guarantee that the test problem will go away.

For example, to handle a growing regression suite the answer may not be to hire more testers. It may be to hire a test automation specialist to automate the tests, or to encourage your developers to automate more tests.

Another example is poor quality products going out to production. You may improve the quality by hiring more testers, but you may also improve the quality by slowing down, spending longer designing what you’re building or working out some way to get rapid feedback on what is being built.

Recruiting is often time consuming and expensive so it’s always worth assessing whether recruiting more testers is going to solve your problem. To do this you need to know what problem you are trying to solve.

When you define your problem you’ll also have a clear understanding about the kind of person you are looking for. This will help you articulate your needs to recruiters. It also allows you to communicate more clearly to the candidate.

Being able to explain to a candidate what they will be doing and why it’s important for the business will increase your chances of a candidate accepting your role (assuming that’s what they want to do). Don’t be surprised if a candidate doesn’t accept your job offer if you cannot even clearly explain what their job role would be and in a sense, what problem they are going to solve for you.

Not all testers are created equal so it’s crucial to know whether their aptitudes and skills will solve your problems. Some are better at exploring, some are better at designing, some are better at building bridges between teams and some are very good at automating anything that moves. Each one of the above examples will help to solve a problem but not all of them will solve all problems.

By understanding your problem deeply you’ll be able to work hard to solve that problem. Simply hiring testers because you “feel” like you need resource leads to bloated test teams and ineffective delivery. And of course, the chances are you wont have solved your original problem.

Certifications are creating lazy hiring managers

A certification is not a marker of excellence.

I can say this categorically from the viewpoint of someone hiring testers.

It’s a viewpoint often ignored, dismissed and unheard by those extolling the virtues of the certification as an effective tool for hiring.

I’ve interviewed 100s of testers and I’ve yet to see any direct link between excellent candidates and their possession of a testing certification. Period.

certification of awesomeness
Certification of Awesomeness

To print your own certifications of awesomeness – visit Certification Magic.

Don’t rely on them for your recruitment as they won’t provide you with candidates who have the consistent level of experience, knowledge or aptitude that many people promoting this angle of certifications would have you believe.

If recruiting excellent testers was as simple as pre-filtering candidates based on them possessing a certification, then every software company in the world would have awesome test teams – but this simply isn’t the case.

You cannot presume someone with a certification is a talented tester.

You also cannot presume someone with no certification is a rubbish tester.

Anyone who tells you otherwise is selling you a certification, lying, misinformed or easily mislead in to spouting a message they have not thought critically about.

This doesn’t mean that certifications are bad. They could have their place. In fact I’ve heard some very compelling reasons recently for using the certification schemes. They seem to solve some people’s problems.

But they create problems in recruiting.

Certifications have created lazy recruiters (hiring managers, HR and recruitment agencies) by providing a seemingly easy way to filter applicants. This in turn has given lazy candidates a simple route to getting through the filters.

Candidate no longer need to excel at anything, nor up-skill (other than sitting more certifications) or learn how to sell themselves effectively (Testing Club link to a thread discussing selling yourself) in order to get an interview, and presumably, also land a job.

Overall it’s creating a vacuum of fairly average people all playing certification inflation to keep up with everyone else.

I believe that certifications are not improving our craft in the way many believe they could, or should, and they are negatively affecting the recruitment of testers.

A certification is a business transaction. Lets not pretend it’s anything more. You pay for a course, you get a certificate (and a short amount of coaching/training/lecturing/reading etc).

Certifications are not education and they are no substitute for on-the-job learning, but they may be a useful source of learning and a useful introduction to testing for some.

A certification tells you nothing about the candidate themselves. It tells you nothing about their soft skills, aptitude, motivations, self learning ethos or work ethic. This is important because being a good tester is more about the person than it is about the skills. Most skills can be taught and learned – aptitude, thought patterns and work ethic are much harder to change.

Recruitment is not supposed to be easy. It is hard work finding good testers. Your job as a hiring manager is to find good people. This means your job is going to be hard. Embrace the challenge.

Instead of relying on certifications why not try other innovative ways of getting the right people? It’s not as tricky or expensive as it may first seem. (I’ll be sharing many of these ways over the coming months).

But of course if all you want is another bum on a seat then certifications may be your more efficient way of recruiting.

If however, you want a test team that can help your business adapt to the changes and evolutions it will inevitably go through then you will have to drop your reliance on certifications as a way of recruiting testers; certifications are making both you and the majority of candidates in the testing industry lazy.

Effort on the CV is a good indicator of interest

Picture of a CV Image from The Italian VoiceCurriculum Vitae” March 2, 2008 via Flickr, Creative Commons Attribution.

A general heuristic you could apply when hiring good testers is to assume that the effort on a CV is directly related to how interested the candidate is in working for you.

The more effort applied to tailoring the CV – the more they are someone you want to hire. If you’re looking to hire a great test tester you’ll ideally be looking for people who want to work with you, and not just want any old job.

This is a good heuristic to use, but as always, it is fallible.

Generally speaking though a CV that has been tailored thoughtfully to match your job advert is often the sign of a tester who cares about working for you.

A boiler plate CV with no tailoring is a sure sign of someone who just wants a job – any job.

Be careful though. Not everyone knows how to write good CVs or tailor an application. You may miss some great candidates by applying the heuristic with no other key indicators.

In my opinion though if someone wants to work for you then they will take care and effort over the application.

Care and effort on the application manifests itself in a clean and simple CV with carefully chosen details to support their application. It will contain a sentence or two about each of the aspects in your job advert, even if it’s a simple statement saying “I am currently learning X” or “I am keen to understand more about Y”. It shows they care. And testers that care are the testers you’ll likely want to hire.

In my experience the better the CV – the better the candidate

Your Service Desk is your customer

(In the following post I use the term Service Desk to also mean “support desk” or whatever else you call the people who deal with calls from customers)

Call Centre
Image from Lamont_Cranston “aafad 167/365 call centre-kun” Nov 2, 2008 via Flickr, Creative Commons Attribution. – Image not modified.


Your Service Desk team is your customer.

You may have never thought of them in this way before but they are your customer.

They may not be your only customer but they are your most important customer.

The service desk may not be your company’s most important customer (or even seen as a customer at all to the wider business), but they are most likely your most important customer.

If service desk have a problem, you have a problem. If service desk have no problems the chances are (assuming your service desk is accessible for your end users) you are building and releasing something very good indeed. This is not a problem.

If your service desk are fielding 1000 calls a day about bugs in your product/service then you’re getting something wrong with your development process. And this is your problem.

If your service desk are fielding 100 questions a day about how a feature works then you’ve likely got something wrong with that features suitability, the documentation or the communication around it. And this is your problem.

If your service desk are fielding 10 requests a day and having to do something for your customer that they could realistically do themselves (i.e. resets, unlocking, data configuration) then you’ve likely got something wrong with your product or documentation or communication. And this is your problem.

Your testing (and the Dev cycle it exists within) is there to provide something of value for your end users/customers, whether that be a product or service. And when that product or service doesn’t meet expectations your end users/customers will likely contact service desk. Your customer is now your service desk.

You should do everything within your power to help them support your end users. Your test strategies, approaches and techniques should be designed to support your service desk. Ideally your process will be designed to minimize as many service desk calls as possible, but the reality is you’ll not catch everything. So design your testing to support customer issue resolution.

If service desk have a problem then development has a problem. It’s your job to stop people calling in (by creating something that solves their problems and doesn’t create new ones), but when they do call in it’s also your job to support those on the front line dealing with your end users problems. After all you most likely helped to create their problem in the first place.

Collective Noun for Testers?

The other day I tweeted a picture of our test team testing together as a group (for the first time) in our new test lab.


I asked “What do you call a collection of testers in a test lab?”

Some interesting responses below. Are we heading towards a collective noun for testers? No doubt someone’s done a collective noun post before 🙂


Stephen Newton @SteveAN01 – a play group. Or an informative.

@paul_gerrard – A “Newsroom of testers”

AQtime @aqtimepro – An argument waiting to happen.

George Dinwiddie @gdinwiddie – an exploration?

Darren Hails @rw_testing – correctly located!

mubbashir @mubbashir – Breaking Bad 😉

James Salt @saltpy – happy 🙂

Mark Keats @mkeats – A crash of testers.

neill mccarthy @MccarthyNeill – a curious of…?

Amy Phillips @ItJustBroke – Dangerous

James Lyndsay @workroomprds – an expedition


Thanks to all who responded.

Test Case Completion – A Story

I did a EuroSTAR webinar last week on shipping products and talked about how shipping products using test case metrics is bad news. It reminded me of this story which I share now.

It’s very common for testers and teams to rely on a number of test case metrics and measures to work out when they are done, or to plan the work, or to measure the progress.

This can be very misleading.

A common metric I used to rely on, and see many people rely on often, is the classic “Test Case Completion” metric.

This metric is often used for planning and for measuring completion but more scarily for working out when to release a product or service.

It goes like this

Let’s say we have 10 testers. We also have 1000 test cases. With a bit of magic and maybe past history we can predict that each tester should be completing 10 test cases per day each.

So, that gives us an elapsed time of 10 days to complete all of these test cases. Right?

So now we can plan.

“It’s going to take 10 days to complete our testing”

This happens on almost every single testing project. Test cases and test completion rates are often the guiding factor for schedules and release planning.

We can also use this metric to measure progress.

On day 1 we should have completed 100 test cases. Day 5 we should have done 500 test cases.

If we don’t see these numbers trending in this way (or close to it) then we can adjust. We could *make* people work more hours, maybe achieving 15 test cases per day.

We could add more testers to the mix. Or we could even just not run some test cases.

The Problem

There’s a very obvious problem with this approach. In fact, there are lots of problems yet it doesn’t stop this being the defacto way of planning testing.

One problem is that not all test cases are created equal. Some will take hours to run, some maybe even days and some a few minutes.

Another problem is that there is an assumption that the only testing that needs to be done is contained within the test case.

Another problem is that there is an assumption that testers are like robots who will perform the same each and every day. We all have bad days.

There’s also an assumption that the tester won’t find any problems and hence delay the running of a test case in order to investigate a bug.

A Story

A company once used to run a giant regression phase where all test cases would be run again on the “final” build.

They would print out all 3000+ test cases and stack them on a giant table in the office.

The expectation was that each tester would complete 10 test cases per day – this would allow them to hit the magic release marker of 100% tests run.

Here’s what happened.

At about 5:30am on the day of the regression a group of testers would arrive at the office and rifle through the test cases.

They would pick the really easy ones; the ones that took just a few minutes to run.

They would pick about 50% more than they had to complete.

They did this for two reasons.

Number 1 – they figures that they would be asked to work longer hours to complete more tests – so they already had a stash of easy ones to do whilst eating pizza.

Number 2 – even if they didn’t get asked to stay late they could excel by completing more tests than other people running up to the last few days of the phase.

10 test cases

A second group of testers would come in at 8 am and be left with the really hard test cases. Some of these test cases would require a days worth of setup and config just to run.

Each day the testers would mark how many tests they had done that day on a giant matrix stuck to a wall behind the manager. The first group would mark in the number 10. The second group would be lucky to register 2 or 3.

Some of the first group would surf the web in their spare time, some would help the other testers, some would do exploration, some would go home early.

All of the first group would game the system for a variety of reasons. Yet all of them would be doing what was asked of them according to a simple metric like test case completion.

Not so strangely, the second group of testers would simply not run all of the steps of the test cases (or even mark entire test cases as done without running the checks) in order to try and run 10 per day. When faced with the reality of a 1 day environment build to check one thing….what would you do?

The project shipped late. And it returned to be worked on further.

The scary thing is that this behaviour happens all the time.

When simple measures like Test Case Completion are used to measure progress or to plan projects you’re already skewing the process and opening it up for gaming, abuse and a false start.

What’s the alternative?

I’ve no doubt there are many alternatives to this problem and no system or measure is exempt from being skewed, gamed or misused.

My suggestion would be to move your testing to be nearer the code by pushing for more behaviour driven testing and unit testing which drives out the design, the code and some of the behavioral tests. And then to deliver in to your test environments as soon as possible. If this process works it means you no longer need test cases (or as many of them) as the checking is automated, therefore you don’t need test case completion metrics. Your automated checking becomes a set of results and it frees you up to explore the product and find the things the test cases (or checks) would never have caught….in other words… freeing you up to do testing.

It’s obviously not a simple change (I know I’ve been there) but it is possible and small steps towards these sorts of approaches are entirely possible in almost any context. The real question comes down to how much you’re willing to experiment with testing, reporting and project planning.

No matter what your approach or your context it pays to be aware of the pitfalls of relying on test case completion metrics and to spot the wrong behaviour it drives. At least if you spot it, you may be able to make some changes and encourage the right behaviour by changing your process, or measuring something different.

Michael Bolton blogged about shipping projects and test cases this week also.