Follow The Work - Image

Follow the work – bad news for Test Managers?

I get lots of enquiries from founders of start-ups who reach a certain growth point where they really need to start taking control of the quality of the work being produced. Their companies seem to reach a size and market growth where the focus on quality becomes a priority.

This is usually about the time when I get a call or an email and get posed the typical next two questions:

“How do I go about starting a new test function?”


“Where do I find a good Test Manager?” Read More

Rapid Reporting

After meeting Shmuel Gershon, probably the most energetic and enthusiastic tester on the Planet, at EuroSTAR 2010, I was introduced to a man of many talents. Not only does he like to read, write and talk about testing but he’s also adding back to the community in other ways with probably the most useful tool I’ve ever used whilst testing.

I introduce to you…….Rapid Reporter.

In essence, it is a reporting tool that allows you to jot down your thoughts, bugs, tests, notes, configs and setups as you test. Things I would have normally used Notepad++ for or a simple writing pad. But the tool is more than that really because it not only allows you to write down your testing notes, but it frees you up from breaking your lines of enquiry/testing by being so simple and easy to use.

The interface is incredible lean and minimalist and everything is pretty much controlled from the keyboard. It allows screenshots and rich text and pushes it all out in CSV format afterwards meaning it’s fairly generic.

It’s got some neat little features like the transparancy setting which I find myself using a lot whilst running multiple desktops, each one cluttered with various heads-up feeds or information radiators or browsers. Another neat feature is the countdown of session time and warning when you exceed the session time set. Nice.

Rapid Reporter has become my favourite little tool of choice now and no testing session starts without it running in the background. It’s proven invaluable when troubleshooting, retrospectively analysing sessions and tracking what sequences/patterns show bugs. I find myself typing more notes than I would have traditional entered and I also list out a lot more data than before which means more data when logging defects or replicating bugs.

So why not give it a download and see how you get on with it. And don’t forget to check out Shmuel Gershon’s blog too.

Alternative Paths for Self-Education in Software Testing > Markus Gärtner #agiletd


I can't believe I managed to beat Markus Gartner with a blog post from the wonderful Agile Testing Days conference here in Berlin. Oh wait a minute, that's because he's talking….and here's my blog post of his talk Alternative Paths for Self-Education in Software Testing.

Who Is In Charge of your learning?

Markus introduced his topic and asked the question

Who is in charge of your career?

Your boss, your employer, your family, your school teachers….nope… Yes, just you.

Markus then made a very pertinent point "If you find yourself unemployed in a years time, things you do now will all be contributing to you being employed quickly again."

Markus suggested you could start by:

Creating blog or personal journal
Sign up to a community like the Software Testing Club, Weekend Testing, Coding Dojos
Markus also suggested that testers should start to learn how to program in languages like Ruby or Python or Groovy.

Markus explained that the brain has two sides: The Left and right sided brain.
Left – Hypothesis (books, Rapid Software Testing, Black Box software testing, Bucaanner Scholer)
Right – Synthesis (Black box software testing, Testing Dojos, Challenges, Weekend Testers, Miagi-Do School)

Markus then suggested some Books to read
Cem Kaner – Testing Computer Software
Lee Copeland – A Practitioners Guide To Software Test Design
Cem Kaner, James Bacj, Brett Pettichord – Lessons Learned in Software Testing

Markus then suggested the Rapid Software Testing course by Michael Bolton. "4th day free" Michael added in 🙂

Markus also explained that there is a wealth of excellent material on the AST website. Most of the content for The Black Box Testing is provided content by Cem Kaner and James Bach. Markus suggested this is an excellent selection of courses and there are about 60 videos to learn from as well as slides and other information. There were a wealth of topics introduced as being available in the AST course material including Bug Advocacy and various Testing Techniques.

Markus then introduced The Buccanner Scholar and suggested that there are lots of learning heuristics contained within the book. It's the story of James Bach and his approach to learning. Markus introduced some heuristics like 'SACKED SCOWS', 'Long Leashed Heuristics', 'Obsess and Forget' and 'Procrastinate and Push'

Testing Challenges – challenges to and by the Testing Community are very valuable ways of learning more about your craft.

Testing Dojos – Collaboration, Safe Environment, Deliberate Practice. Normally a mission with a challenge which allows the testers to practice their testing and learning. There can sometimes be observers or facilitators. Sometimes it is done using pair testing.

Weekend Testing – A few hours at the weekend where there is a charter or mission and the group work towards how to test it. It's a great opportunity to learn about the assumptions we make and the traps we can fall in to.

Miagi-Do School of Software Testing – A school founded my Matt Heusser. It's a non-commercial, zero profit school of learning where people can improve skills, learn from others and share knowledge. There is a belt system where people move through the belt ranks, like a martial arts.

Markus concluded that no-one is in charge of your testing other than you.

There are different styles, Feedback, Hypothesis and Synthesis.

Stuart Reid > KeyNote > Agile Testing Days – More video snippets #agiletd

Some more videos of Stuart Reid’s Keynote at Agile Testing Days. The most anticipated talk of the day…controversial talk, but not enough time left for questions……

Not sure about the sound quality (as not had chance to listen back yet), let me know if it’s really bad.

Not the whole talk, but snippets of stuff. Some interesting statements.


Watch on Posterous

Watch on Posterous

Watch on Posterous


Agile Testing Days Berlin #agiletd


It's that time of year again. The time for Agile Testing Days in Berlin.

I arrived bleary eyed this morning in to Berlin Tegel airport and am now checked in to the hotel and ready for my presentation tomorrow.

I wandered across to the exhibition hall at lunch to say hi to some of the delegates and speakers. Was great to see so many familiar faces and I was well chuffed that a couple of people recognised my name from the Software Testing Club.

Jose is on top form again and is one of the most welcoming hosts you could ever wish to have. Looking forward to the speakers dinner tonight and a few drinks after in preparation for the feast of presentations tomorrow. Some excellent keynotes ahead.

The thing that struck me this year was the vibe in the place when I arrived. Lots of joking, fun and laughter. I'm sincerely hoping this continues through the week.

More to come from me no doubt over the next few days. I'll try not to bore you all too much though.

Oh yeah, the image attached in the view from my hotel room. Not too shabby….

Agile. Testing. Days

Agile Testing Days 2009 was one of the first major conferences I ever attended and I loved every single minute of it. It was a real eye opener for me as it brought me in to face to face contact with many of the leading agile testing practitioners like Lisa Crispin and Elizabeth Hendrickson plus many more.

The talks were not only insightful and passionate but the conversations they spawned after were a real pleasure to be a part of. And that’s one of the great things about the agile community; it’s full of people wanting to further the craft of agile and testing. People wanting to sit down and chat about agile, about testing, about development, about anything. I think the social side of any conference are the most valuable moments. The chance to get snippets of information, the chance to exchange ideas and stories and the chance to meet like minded people. I’m truly looking forward to Agile Testing Days 2010 but am getting scaredy now about my presentation.

It’s been through so many iterations that it’s not at all like the first draft. Agile maybe? I’ll be posting it out on here after the event.

And if you get the chance to get along to Agile Testing Days – jump at it, grab it and run with it.

That tester’s incompetent….hide them

Last year I attended an excellent agile testing conference but came away frustrated at one of the keynote presentations.

One segment of the talk mentioned the often painful transition to agile. The presenter made reference to a British TV show character who is quite nice, but quite dim. In other words, an amiable kind of person but incompetent. Commonly known as an idiot.

The presenter then suggested that in an agile environment there is no room for an incompetent tester. Agreed, but surely the same is true for any environment? IS there a room for an incompetent tester in waterfall?

But what really got my goat was when the presenter suggested that we simply hide this person away from management.

Photo courtesy of

The first reaction by most people seemed to be that this was a complete farce. This person is a part of the team and hence contributing to the bottom line. Paying a salary for someone who remains hidden? Really? Why would you hide someone away from management? The reasons given were that it was humane when transitioning to agile; to hide them away rather than expose them, then get rid of them. For real?

This spurned some furious discussions afterwards where I was intrigued to hear that many of the attendees thought the incompetent tester should be sacked. After all they are contributing to the bottom line. Get rid of them. And that’s often a sentiment held with many in the agile community. Incompetent testers (any team members too!) can ruin an entire agile project. No room for the weak. Get rid of them. Maybe this is why many testers are worried and negative about moving to agile?

But even this missed the point entirely for me. I found myself asking the question “Is this tester really incompetent or are we just not trying hard enough to integrate them?”

Are we asking the wrong questions, taking the wrong actions, avoiding the hard work, wanting instant results like we are often led to believe is the norm? Sure, on the surface you may think this tester IS incompetent but is this a result of their skills or their environment and lack of motivation?

Maybe this incompetent tester isn’t being given the tasks and challenges that make them enthused. Have we spent time finding out what makes them tick? Have we sat down and tried to get to know them? To understand them? To challenge them? To motivate them? Have we asked them what they think? Have we even trained them or coached them in what agile is?

Have we invested in them with time and energy? Encouraged them to attend inspiring testing conferences? Training Sessions? Local user groups? Pointed them at online communities like The Software Testing Club?

Have we done enough to really come to the conclusion that they are an idiot? Have we explored all the avenues?

If we have, then maybe that’s fair enough. Let them go, but don’t hide them away. However, I would bet a whole £1 that we haven’t invested in this person as much as we could have. And if we don’t do that, then we may never know that this person is a whizz at building automation frameworks, is an absolute star in front of the customer, can articulate complex ideas to non-tech audience or comes alive when performing exploratory testing.

You’ve only got to take a glance around the testing communities to see that some testers are real customer advocates, some love automation, some thrive on efficiency improvements, others love lean and agile, others teach, some berate, some challenge, some build social networks and communities, some are socialisers, some live and breathe quality. Some are funny, others are sad, some are arrogant, some are happy. Some write about testing, some present at conferences and some are super thorough and live for nothing but quality, quality, quality.

It’s this mixed bag of skills, talents and attitudes that make the testing community so interesting. Surely there’s an element of this in your test teams too? Isn’t it just a case of finding out who’s interested in what and giving them work to inspire, challenge and enthuse?

So instead of saying “That tester’s incompetent. Hide them” we should instead be saying “That tester’s got hidden skills, experience and ability. Let’s find out what it is…..and if they are still incompetent at the end of that process then….well, we’ll sort something out” or something like that.

But this is a difficult concept to grasp for many people who see testers as a quantifiable, measurable, certifiable and replaceable person. Hiding them away ignores the fact that testers are complex and diverse. It’s a simple way of ignoring the truth. It’s an easy way of avoiding hard work. It’s an easy way of applying testers to a project as a resource rather than a person. Resources take little maintenance; people have ups and downs, goals and ambitions. Resources act the same each and every week. Testers fluctuate. They need motivation and inspiration.

Leaving the incompetent tester to fall even further behind, hiding them from management and giving them ridiculous job after ridiculous job is not humane. So to suggest that doing so is a humane action by middle management is misguided and painfully difficult for many to comprehend. The humane thing to do would be to invest some time in them, step up and inspire them, get them all fired up and find out what makes them tick. Take responsibility for motivating and inspiring them and their work. Or could it be that I live in a dream world where we all share some responsibility for making our working environments great places to be?

So if someone said to you: “Bob over there is incompetent. Hide him away” …………………………..What would you do?

Agile scared me

I’ve been noticing posts, comments and tweets about agile recently where people are being incredibly derogatory and negative. Almost to the point where their comments are bordering on offensive. The trouble is most of these people openly admit they have never actually worked in an agile environment. So what gives?

You know what I think? I think agile scares some people. I think they worry that they won’t have a job, that their role will change so much they can’t cope and that maybe, they will be exposed as a tester who is not capable of testing. Harsh, but even a well known testing presenter (who shall remain nameless) identified that this often happened when teams make the transition to agile. The move to agile identified those who are “not so good”.

I’ve been talking a lot about agile recently because for me the turning point in my career came when I started on an agile project.

At first I hated it.
Then I quite liked it.
Then my passion for testing was re-ignited and I started to really thrive in an agile environment.

Agile put me back in the zone. It cut out the bureaucracy, the admin side of test artifacts and left me (and the team) in control of creating great software. And this scares many teams.

What? Us in control? Really? Are you sure? But how will we cope? What do we do when? Eh? You want us to create our own structure? Are you mad?

And hence many teams flounder, fumble and stumble back to traditional ways of working.

I remember my first agile project and the first “kick off” meeting. It was an eye opener. It terrified me. I couldn’t believe people worked this way.

Before the meeting I read the agile manifesto, I read the scrum alliance website, I read some books and I read some blogs. I didn’t understand it at first, so I rewound and read it again until my brain hurt. And I still didn’t get it. It still didn’t make much sense. Theoretically it sounded fab. In reality though, how could it work?

In that first meeting I was informed by the Tech Lead that the UI would be the design and that we don’t need to worry too much about the requirements (backlog). Seriously? No design? No planning? I believe I gave the same look my careers adviser gave me when I told him I wanted to be a ventriloquist (thanks Seinfeld for the joke). I almost choked with rage.

From that point on though it started to make sense. And more importantly, it started to really work. I started to feel passionate about delivering software again. My job felt meaningful. We rolled out software….fast (and to top notch quality).

To sum up why though, here’s four reasons why I believe adopting an agile mindset will help you re-ignite your passion for creating fab software (I also included the slide). Agile gives you:

1.Freedom for the team to choose its own framework based on their local context. (team, customer, skills, experience, organisation ethos)

2.Freedom for the team to choose its own way of providing rapid and accurate feedback. (feedback to all team members as soon as possible is critical to success)

3.Freedom for the team to work to its strengths and weaknesses enabling them to inject the fun back in to software development

4.Freedom for the team to take control. No Excuses. Scary.

Agile Testing > Story : Distilling more information in the story

Continuing my exploration of the life of a story in agile testing. Part 1 is here.

Each story on the backlog has been defined by the business and the customer. These are capabilities that the customer wants. So the next step is to distill some more information in them in the form of acceptance criteria. These are essentially the markers/gates at which this story will be deemed as complete. It's a list of things this feature/capability must do.

Here's an example story using the Software Testing Club:
As a registers user I would like to create a new forum post so that I can start a discussion

And here's some example acceptance criteria

  • User has the option to create a forum post
  • New Forum post window has X fields
  • User cannot add a forum post without a title
  • All error messages appear at X location
  • User cannot add a forum post with no content
  • Mandatory fields are X
  • Optional fields are X
  • Title field holds X characters
  • Description field holds X characters
  • Tags field only accepts tags separated by commas
  • Clicking submit will post to the forum board
  • Any errors when submitting will be reported to the user
  • User will be able to see forum in list after submission
  • Tags entered will be search-able
  • Other users can view the posts
  • etc

You see the point.

If a tester gets involved in the story writing session then the acceptance criteria tends to be more detailed. It's natural for us to want to question the story early before we get the software and the more we can do that, the more accurate that story will be. There are plenty more to add to that story example above. Think of distilling more information in the story as if you are fleshing out a test case(s). The story is the test case. The acceptance criteria are the tests themselves.

In the early days for me stories would arrive with only one or two vaguely described acceptance criteria. Programmers would code against these and then make the usual assumptions they often have to make. I would get software that didn't behave how I thought it should and the customer would need to be consulted all the time, assuming they were available. If not, they would get a demo that didn't appeal and a whole new sprint would be dedicated to getting it right. Time wasted.

With detailed acceptance criteria that the tester, programmer and customer have all agreed on it is possible to avoid this wasted time. Programmers code against it, testers test against it and the customer accepts the software against it. Anything that changes is agreed between all parties and added to the story. In theory, it's as simple as that.

So this is why it's important to get involved early and put some test input in to story writing sessions because the questions and ideas you will raise (just like you would in a test case or through exploration) are essential for the success of that story. How many times do you here a programmer or customer say 'I would never have thought of that' or 'You can't do that, can you?' – The tests and ideas that generate these statements are the same golden content you need to get in to the story…..and early.

If you are not involved in the story writing session then try to shoehorn your way in. After all, most managers will be receptive to ways to stop wasting time. Any ideas how to avoid bugs in the software is a good thing. If you still can't get involved then start collecting defects that you are fairly sure the team could have avoided if testers had been consulted earlier. The tester is the one who thinks about how to find breaks, possesses skills that aid bug hunting, has experience and is often the main champion of customers and end users. So to not include testers in story distilling sessions could be a major mistake.

Agile Testing > Story : On the backlog

Over the next few posts I'll be exploring the concept of a story in an agile environment and what it means to us as a tester. Over the past few days I've been hearing about how testers don't get involved with story writing sessions, how testers duplicate the acceptance criteria in their tests and how testers don't fully understand how a story can replace a spec.

I'll hopefully cover all of these and some more by breaking a story down in to smaller lifecycle steps.

My experience is based around scrum projects. The one thing to consider is that each agile implementation is different, with different teams and customers and different requirements of the team. So no 'one' solution will be suitable for all. But hopefully by sharing my experiences on here it will help you round your view and decide on the best way of working on your agile project.

A story is basically a description of how someone interacts with the software to get a desired response.

A story basically takes the form of:

As a [Actor/Person] I would like to [Feature or capability] so that I can [Value of action]

At the start of a project there is a tedious, but incredibly important job of adding all of the customers ideas, requirements and thoughts to the Backlog. The Backlog is the project holding space for stories. Look at it as your requirements document.

Getting the stories on the backlog is a process of sitting with the customer and adding their requirements to the system in the form of the story. At this point it is unlikely that the stories will have acceptance criteria (i.e. the details of what is involved in the story).

Some of the stories may actually at this point be Epics. An Epic is essentially a story that contains lots of other stories. An example would be "Log In". This would normally consist of several individual log in stories – depends how complex though. When tackling an Epic it is essentially to break it down to manageable stories.

A manageable story is essentially a story that can be completed (to the definition of done) in your sprint. A sprint is normally between 1 and 4 weeks – although there is no law on that one. If the story is not achievable then it is an Epic and needs to be broken down more.

The definition of done can be complicated but essentially this is a set of rules/regulations/guidelines that must be adhered to before the sprint/story or task is considered done. For example, the sprint is not done until all code is checked in, all tasks are done, all stories are done, the stories have all been tested, the demo stack is ready for the customer, the deployment scripts are done, the automation suite is started, no defects outstanding etc etc. A series of gates in which the software and process must go through to be ready.

Right, back to the story. Once all stories are on the backlog the customer should then rank the stories based on priority order. i.e Top to bottom rank order of what's most important to them, at that moment in time. This rank order will change as the customer sees the software, gets new information or responds to market/financial pressures – and this is the beauty of agile. The next piece of work is always the highest priority for the customer.

Once ranked there are two lines of thought as to what to do next.

A: Have the team estimate each story in advance – time consuming, inaccurate as acceptance criteria will not be defined, tricky to estimate with no information on the emerging system
B: Add acceptance criteria to the first few stories and then have the team estimate.

I prefer option B as estimating the whole backlog often proves fruitless and pointless in my experience unless you are using it for forward planning. It's at this point that we, as testers, sit with the customer and programmer and work through the stories adding relevant acceptance criteria (coming in the next post).

Once acceptance criteria is added we then have the whole team estimate. Some teams use time estimation points, others complexity, others a combination of the two. For me, the only one that really matters is complexity, but the others would argue against this.

Estimating complexity is a process of sitting down with planning poker cards (numbers on each card). The scrum master (person running the sprint) would then read out the story and each team member would estimate on complexity, putting their card face down. Once everyone has estimated the whole team then turn the cards and we find a happy ground, negotiating between each other.

Estimation based on complexity is tricky to understand. It's not about how long it will take but about how complicated you think it is. An easier way of working it out would be to take a story that is neither "really hard" nor "really easy", writing in on a piece of card and sticking on a long wall. Then take each other story and write them down too. Now stick them either side of the existing story. Right hand side for more complicated. Left hand side for least complicated. From this you can start to understand that each story will have a complexity level that we assign a number to. I work on 8 being the average story and work either way from there.

Once we agree on an estimate that goes against a story and is then used to work out the team velocity. The velocity is essentially how many complexity points we can achieve per sprint. This is why sprints tend to be kept the same length to maintain a consistent velocity. In the first few sprints though, you will have no idea of the teams velocity as there is no historic data. Over time though the team will slip in to a rythm or groove which allows a much more accurate velocity to be calculated.

We have not estimated all stories at this point, but before each sprint this process needs to take place. This is so that at the sprint planning meeting the team can assign stories to the sprint which have already been estimated and have acceptance criteria. The customer also needs to check the ranking and the backlog as there could be new stories and defects to now consider. This is a continual process and is often referred to as grooming the backlog.

And so that's it really. In a nutshell (and a heavy scrum one at that) we have the basics of stories and backlogs and how they are used in a sprint. The one key point I have missed though is distilling the acceptance criteria in to each story. Something that not only makes estimation easier but also makes programming and testing smoother, cheaper and less dramatic. More on that in the next post.


Pair Programming and Pair Testing

Our programmers here at iMeta now wax lyrical about pair programming and it’s easy to see why. The quality of code coming through to the test team now is exceptional. There’s very few fix-fail episodes and the programmers seem over the moon with how great pair programming is fairing. Sure, there were teething issues and some programmers didn’t feel the groove when pairing but these were soon overcome and they moved forward.


It got me thinking though about whether or not testers should be pairing when writing test cases. And my conclusion is that they should. It brought back memories of when I used to have to send test cases off for peer review at a previous company. I too had to review other peoples tests. It did often become a chore but more importantly was that is missed the point. And here’s why:

  • The review was more a sanity check on formatting, spelling, ensuring every step had an expected result, test case length was ok, etc
  • It became a chore so often it ended up being a skim read.
  • The person reviewing often didn’t have the same product knowledge. This meant the test cases weren’t reviewed regarding how well they tested the application.


And so I sat down with a fellow tester on an after work development project and did some pair test case writing to start with. It was incredible. The thought patterns and processes we entered were remarkable. As a pair we wrote simple, clean and to the point tests. Those pointless steps or ideas were left out. During the tester not doing the main writing would then spawn a mind map charting all of the ideas so that we didn’t miss any.


The tests were succinct and short in a high level guidance way (i.e. no detailed steps). We kept every single test DRY (don’t repeat yourself) extrapolating all setup, preconditions and data out to separate excel documents. It truly was a great experience as each of us brought a different outlook to the table. But more than that we bounced ideas off of each other. In terms of time spent it might appear that we were doubling but the quality of the output was incredible.


So how about actual pair testing?

The next step was to actually see if we could do some pair testing. And we could. This too brought some amazing side effects. We raised more important defects. We generated new and interesting exploratory ideas to run all managed through a mind map. We had to do so little admin to the test cases that we were both truly surprised with how good they were.


It felt like we’d covered more of the system in less of the time. But we also covered a lot more of the system than we had test cases for. This was because as we got to a step, one or both of us would highlight that the documentation hadn’t mentioned this, or the test case didn’t consider this factor yada yada.

The whole process has left me thinking more and more of us should consider pair testing. Maybe as a short trial process for one day a week. Maybe as a permanent idea. Believe me, the tests, the testing and the camaraderie are all enviable positive effects of pairing up. Let’s not just leave the pairing to the programmers. Let’s help take testing to a new level too.