When your stats are telling the Test Manager a glorious lie…Jump Ship

Last night I was told an incredible tragic story. With kind permission of my friend I will summarise it here. I will also apologise now, for my version will not be as funny as the original. It was tragic, but told with great enthusiasm and with

My friend, a senior tester in a large organisation, was asked to explain manual testing (and justify the need for it) to the Test Manager. Yep. You read that right. The Test Manager was asking a Senior Tester to explain testing.

In essence the Test Manager was probing for reasons to keep (or not keep) the test team. It looks like budget cuts and other financial pressures were demanding a drop in head count. After a rather good description of why testing is important and why skilled individuals are needed the Test Manager got hung up on the idea of Test Cases. It turns out the Test Manager couldn’t detach testing from the Test Case or the tests container to be more precise.

It seems obvious to me that this Test Manager believed testing was nothing more than a Test Case. The person running the Test Case needed no other skills than maybe a little domain knowledge and the ability to operate a mouse. It’s a common view held by many in and out of the testing world. It’s a view that irritates and mocks me but none-the-less it’s a view that exists, rightly or wrongly. (That rhymed by the way. By accident)

After my friend explained testing to the Test Manager he was pleased. And rightly so. He’d done a good job. His explanation would have won awards from the Plain English Society.

He explained the human critical thinking side to testing. The experience a tester can bring. The levels if investigation a tester will go to, to root out a bug. The awareness levels a tester has. The unique thinking a tester can bring to the table. He’d expertly described Exploratory Testing (by paraphrasing a few testing gurus)

He’d not forgotten to mention the ability to make judgement calls in the face of changing priorities and commercial pressures. He’d woven in a nice little thread on being able to look past the facts and see the truth (I’m doing a blog post on this soon…). He’s waxed lyrical about testing early through automated tests leaving highly skilled testers to explore and feedback. He’d mentioned how testers approached software development from a unique angle.

His outline of why testing is important and why human testers are needed was award winning. He’d Incorporated testing in to the delivery of good software and without an ounce of “Gold Plating”, “Blinging” or “Bigging Up”. I gave him a standing ovation.

Apparently after this mini training session the The Test Manager sat back in his swivel chair (James Bond Villain style) and laughed “muhahahahahahahah I admire your arrogance” <– Note – this bit is actually true.

The Test Manager, it seems, didn’t share the same view. He was still hooked on Test Cases being the testing. “Get some seniors to write the tests and juniors to execute” was what he said. “We can cut costs even further by automating everything”.

This whole incident happened about 3 months ago. Since then they’ve got rid of some good senior testers and brought in juniors to do the manual work. It seems they are continuing the extraction of human thinking by going down the QTP automation route too. The Test Manager appears to still believe that the actual test case is the test. That the actual automation test is the test. And now these “tests” are being run by juniors or machines his testing world is sweet.

My friend doesn’t share the Test Managers enthusiasm. He’s already seen a decline in quality as the Seniors are being stretched and the juniors don’t have the same people skills and experience to work closely with the rest of the development team. Things are slipping despite the code coverage and the number of tests executed stats going through the roof. The stats are telling the Test Manager a glorious lie. But he seems happy with this. He hasn’t a clue to be fair.

My friend is not at all happy with this hence him handing in his notice a few weeks back. In fact, it could well be the beginning of the end of this company’s quality output as two other seniors have jumped ship leaving just three seniors. These other three hate it too. Sinking ship anyone?

Anyway. I guess the moral of this story is “If you have a Test Manager who doesn’t know anything about testing, has an evil laugh and your stats are telling a different story to reality then jump ship. Jump ship fast. Or push everyone else over board”

Image from the cool galleries of : http://www.flickr.com/photos/wili/

Just making an observation part 1

I can’t help but wonder why so many people feel so passionately about their organisations appraisal process yet are accepting of long feedback cycles in project work? I see too many similarities

Why do we hate one yet are accepting of the other when in reality they exhibit the same problems? Do we enjoy working for so long with no-one offering any feedback? Do we enjoy derailing projects? Do we enjoy seeing misery and anguish on the face of management the week before release? Do we enjoy the pressure of being the bottleneck with only days to go? I know the answer is “YES!” for some.

Long feedback cycles…….vague goals aimed at being relevant over a long and changing period of time…..last minute feedback…..no time to make changes…

I know I prefer the regular feedback cycle of the new wave of appraisal schemes (not new wave to some…) and the regular feedback cycle that agile offers me and the team. I guess I just like to know where I, my career and the project stand.

Just making an observation that’s all……:)

Don’t be a follower, be a tester.

I’ve had this blog post in draft for about 5 months now but yesterday morning a twitter post inspired me to finally get it published. It was a link to an article from Yvette Francino and included in it was a video interview (brief one) with James Bach. http://itknowledgeexchange.techtarget.com/software-quality/james-bach-the-buccaneer-tester/

I would suggest anyone who has even the slightest interest in testing reads this article and watches the video. The video is inspiring; it’s direct, it’s James’ usual straight talking no nonsense stuff and it’s also incredibly motivating.

A few months ago I put out a blog post about plagiarism and copyright that generated some incredibly heated comments as well as direct mails and tweets along the same topic. I studied the comments and concluded that they all alluded to the three following sentiments:

We should not challenge the best practices of testing We should not challenge the experts in testing We should not talk about testing publicly unless we are an expert or we know the experts.

The comments left on the post intrigued me. It re-enforced my belief that many people in the testing community simply follow the leader. They love best practices. They love the norm. They do what others say is right. They strive for conformance. They don’t ask “is there a better way of doing this”. They don’t challenge things.

I had a fairly heated exchange at a testing conference with someone who refused to believe that testers should ever make suggestions for new ways of doing thing, improvements to workflow, improved designs, enhanced usability or the changing/modification of features in the software they test. They didn’t think it was their job. For real? Not your job? Then what is your job? It turns out that running scripted test cases based on a spec written months ago, that had locked in a whole load of ignorance with a design created to solve a problem that actually didn’t exist, or did exist but now doesn’t. That’s what a testers job is apparently. Not in my book.

I believe that when a team/tester doesn’t question the norm it leads to stale testing, uncreative work environments, bored staff, uninspiring tests, dull testing jobs. Sadly that’s the stereotype image of testing. In fact, it’s not just a stereotype, it’s a reality for many.

So I bring you this advice:

We absolutely should challenge the best practices of testing (if they actually exist) We absolutely should challenge the experts in testing (it’s good for the industry) We absolutely should talk about testing regardless of whether we are an industry expert (who says we are/are not experts…is there a list? How do I apply?)

So why not write that blog post, submit your talk to that conference, challenge the Best Practice at work, challenge the heavy scripted testing, introduce some exploratory testing sessions, drop a metric in your report, provide something different to the people that matter, learn some coding, challenge the experts if you don’t believe they are right……….?

To sum up what this post is saying:

Don’t be a follower. Be a tester.

Accessibility Testing and Ownership

I received a question the other day from one of the guys I mentor asking about how to go about testing the accessibility of a website.


It’s fairly common for many testers to have never had to consider accessibility testing before. It’s something I’ve been doing for some time now but I can’t quite remember how I first got interested in it, by luck I think. I think it’s fairly essential for most web testers now even if the site under test is not aiming to achieve any type of compliance. Especially as the tools available can also help in other areas of testing too.


I also believe that you can only really test for accessibility by understanding why you are doing it. Some people run a few checks, reckon it’s all good and don’t really understand why they are testing for accessibility. Useful article here with some good insights : http://www.cnn.com/2009/TECH/12/15/cnet.web.accessibility/index.html I think nothing can replace meeting people with accessibility requirements and getting first hand stories of why accessible sites are so important. There are lots of Internet based activities many of us take for granted. It’s a really powerful experience learning and understanding from people who have problems accessing sites and other online material that I take for granted.


I’m not just talking about none sighted disabilities, accessibility encompasses a lot more than that, including cognitive, speech, hearing and the incredibly common colour blindness. All of these ultimately marginalise people, exclude people from the web or make their experience online all that bit more complicated and difficult.


But why should this be the case when we have professional testers (and other disciplines) on the team? The main reason really is because most website/app teams don’t spend time considering, designing, coding and testing against internationally recognised accessibility standards (WCAG: Web Content Accessibility Guidelines); maybe because they don’t know they should be or maybe because they don’t really know where to go for information.


The irony is that creating a site that is accessible opens your reach up to a much wider audience and doesn’t really cost much more to produce. (there could be a steep learning curve and extra costs for specialist testing though). It often results in a much better design and user experience too. But ultimately the main reason for improving the accessibility of your sites and apps is to empower more people to interact with the web. Right?


I believe it’s time we, as testers, take a little bit of ownership with accessibility. If no-one else on the project team will then why not be the champion of accessibility? After all it takes less than about 5 seconds (after installing tools) to say whether a site fails compliance and believe me, many will. A little bit longer if the online checkers give a yes it’s all clear.


A quick check and you can raise your concerns with the team. How passionate you feel about pushing for more accessibility after that is down to you, your environment and your products purpose, audience and context.


Before I rush in to explaining some basic tools it’s important to point out that the WCAG standards have shifted over the last year from WCAG 1.0 to WCAG 2.0. Many of the tools for testing accessibility are still using 1.0 so there could be some extra leg work. This is not necessarily a problem as they are still valid (in most cases) but if you have requirements double check what compliance level and version they are against. As with all things these things evolve over time.

Compliance to the standards isn’t as simple as leaving it to an online checker or automated tool. It takes seconds to point out a failure but much longer to prove compliance. Checking is fine…testing is another thing.


Here’s an example.

If a site has a field for the text entry of the users “name” then it needs to have a label for the field, primarily so screen readers can inform the user what they need to enter. So if the field has no label an online checker or tool will instantly fail this. No label = no pass.

But if the field does have a label then the checker will say “yes, that is OK” even if the “name” field has a label of “Do you like blue cheese?”

This is a very basic example, some are not quite as clear cut as this and many are nowhere near as simple. There are lots of subtleties and loads of documents to read to get your head around it but the main message is; don’t just rely on an automated/tool based check, human inspection is required.

It is also not enough to assume that a quick scoot around the site with a screen reader is enough to prove it’s OK. There’s simply no replacement for real end users testing the site/app. For example, I once tested a site for screen readers and a number of other devices and gave it an all clear. But we also sent the site off to an external charity for real end user testing who gave it the once over and reported loads of things back. Things I would NEVER have found. “Accessible sites still need to be usable” was one of the quotes fed back. They could get end to end, but not as smoothly as they should be able to.

As a starting point my advice would be to download and install FireFox web browser and then install the accessibility toolbar (https://addons.mozilla.org/en-US/firefox/addon/5809).

Then open a site and on the accessibility toolbar click Reports > Accessibility Report and see what it says. The toolbar is awesome and has loads of options but some good highlights for me are the Scripting > Disable Scripting which disables all scripts on the page. There’s also a really quick way of turning off CSS (Style > Author CSS > Off) without having to search for the option in the browser settings and the ability to show text equivalents of objects and images (Text Equivalents > List of Images/List of Objects)

The FireFox add-on is not the only one out there, but it’s one of the most complete and a really easy way to quickly assess the state of your site’s accessibility.

Here’s some useful links and also a link to my ‘delicious’ accessibility feed which will grow over time.

WCAG guidelines

Actual Guidelines- http://www.w3.org/TR/WCAG20/


Overview of WCAG – http://www.w3.org/WAI/intro/wcag.php


Understanding WCAG 2.0 – http://www.w3.org/TR/UNDERSTANDING-WCAG20/


Techniques for WCAG 2.0 – http://www.w3.org/TR/WCAG20-TECHS/


FireFox 2.0 add-ons

TAW3 en un clichttps://addons.mozilla.org/en-US/firefox/addon/1158

This is a really good add on which gives you a link in the Tools menu: Tools > TAW It

When clicked it grabs the page you are on and gives you a new page with each validation error, query or warning highlighted with relevant failure information included. It also gives you a good list of areas that require human checking. Their website also offers a validation process as well as a WCAG 2.0 Beta checking site. (http://www.tawdis.net/ingles.html?lang=en)

FireFox Accessibility Extensionhttps://addons.mozilla.org/en-US/firefox/addon/5809

The FireFox add on.


Online Checkers

W3C validatorhttp://validator.w3.org/



T.A.W checker (as mentioned in FireFox add on section)http://www.tawdis.net/ingles.html?lang=en


There are loads more links and useful tools over on my delicious feed under the accessibility tag, link below. The beauty of using the following delicious feed is that the list will grow as I add more social bookmarks rather than a static web page that I have to keep updating.




I hope this gives you a really basic introduction to accessibility testing. I’ve not even scratched the surface here, but it should be enough to get going.

Some useful add-ons for testing

Firefox’s add-ons are proving invaluable to me as a web tester but there are also some awesome tools for improving the efficiency of everyday tasks too. I’ve compiled my favourite Firefox Add-ons here (in no particular order).

Note: IE, Chrome and Opera also have some useful add ons. (http://www.ieaddons.com/gb/) (http://www.mychromeaddons.com/) (http://www.opera.com/docs/plugins/)

Selenium IDE https://addons.mozilla.org/en-US/firefox/addon/2079

Web automation. Fast becoming an industry standard tool. This is the IDE version. Visit http://seleniumhq.org/ for more information on the RC version and grid.

IE Tab https://addons.mozilla.org/en-US/firefox/addon/1419

Flips your Firefox tab in to Internet Explorer mode. Super useful.

Firesizer https://addons.mozilla.org/en-US/firefox/addon/5792

Re-sizes your window to preconfigured sizes

Webshots https://addons.mozilla.org/en-US/firefox/addon/6168

Super little screen capture tool.

Fire cookie https://addons.mozilla.org/en-US/firefox/addon/6683

The best way of tracking and testing of cookies in your browser.

Delicious https://addons.mozilla.org/en-US/firefox/addon/3615

Social bookmarking. Never lose a site of interest again and share them with the community too. Perfect for bookmarking testing sites, information and snippets.

Clear Cache https://addons.mozilla.org/en-US/firefox/addon/1801

Super shortcut for clearing out the cache within the browser.

Copy Plain Text https://addons.mozilla.org/en-US/firefox/addon/134

Tool for copying text in plain text format only. Drops all of the text formatting giving you just the text.

Fiddler Hook http://www.fiddler2.com/fiddler2/addons/fiddlerhook/

Firefox addon for integrating with fiddler. How can you web test without some sort of HTTP debugging?

Download Status https://addons.mozilla.org/en-US/firefox/addon/26

Embeds your downloads in a toolbar at the bottom of the browser. Keeps it nice and tidy.

Xmarks https://addons.mozilla.org/en-US/firefox/addon/2410

Superb tool that allows you to upload your Firefox bookmarks to a central server and then download them to any browser. Perfect for multiple computers; keeps it all in sync.

W3C Page Validator https://addons.mozilla.org/en-US/firefox/addon/2250

Check your page against W3C standards and compliance.

Pencil https://addons.mozilla.org/en-US/firefox/addon/8487

Brilliant for mocking up screens.

SQL Injection https://addons.mozilla.org/en-US/firefox/addon/6727

Does exactly as the name suggests

Quick Restart https://addons.mozilla.org/en-US/firefox/addon/3559

Restarts Firefox maintaining all tabs and open docs

Firebug https://addons.mozilla.org/en-US/firefox/addon/1843

How can you test websites without this one? Inspect your page elements.

Regular Expressions Tester https://addons.mozilla.org/en-US/firefox/addon/2077

Useful tool for testing Regular Expressions.

Quick Locale Switcher https://addons.mozilla.org/en-US/firefox/addon/1333

Genius tool for quickly switching your browser between locales. Really useful for multi locale sites.

Http Fox https://addons.mozilla.org/en-US/firefox/addon/6647

Brilliant tool for tracking the http traffic in your browser.

TAW3 https://addons.mozilla.org/en-US/firefox/addon/1158?src=api

Accessibility checking tool. Awesome.

Firefox Accessibility Tools https://addons.mozilla.org/en-US/firefox/addon/5809?src=api

Accessibility checking tool. Combined with TAW3 above you should have some indication of how accessible your sites are.

For more tools and testing hints and tips, check out the wiki at The Software Testing Club. http://wiki.softwaretestingclub.com/Software-Testing-Tools

Alan Richardson also sent me a link to a post he had done on browser add-ons over at Evil Tester. http://www.eviltester.com/index.php/2009/04/29/how-on-earth-did-we-test-the-web-without-these-tools/

Image from Flickr. http://www.flickr.com/photos/jakematesdesign/

Stick it to me

I’m in the process of putting together a fun little eBook consisting of a collection of sticky notes from testers around the globe.


The simple premise is that each sticky note contains a personal response to the following question:


“Why do you enjoy software testing?”

Although the eBook will offer no quantitative measurement of testing prowess, knowledge or skill, it may enlighten us as to what gets testers out of bed in the morning….

And even if it doesn’t do that, at least it will add a little bit of visual testing fun to our testing world.

If you want to take part do the following:

1. Write down your answer to “Why do you enjoy software testing?” on a paper sticky note

(Post-it note or whatever you personally call them – http://en.wikipedia.org/wiki/Post-it_note). Your answer can be as long or short as you like. If you don’t enjoy testing then let me know that too. One submission per person.

2. Take a photo of it.

3. Email the photo to rob@softwaretestingclub.com

4. Wait some time whilst I collect the stickies from people (no deadline yet – I’ll see how it goes).

5. Wait some more time until I find time to put them all in a eBook

6. Watch this blog space for the announcement of where you can download the eBook from

7. Download the eBook. Basque in glory at your post-it note and check out what keeps others motivated.

8. Ping the eBook around to your friends, colleagues and peers.



(I reserve the right to not include your sticky if it is poor quality or offensive, etc

To get things started I’ve included an example.

Please let your friends, colleagues and peers know about this request for stickies. The more the merrier.

Team Work Poster

This is the team to get it done

A fair few years ago I encountered a story about an environment where “bean counting” was priority.

  • testers should produce X number of tests per day.
  • testers should be raising X number of defects per month
  • Monthly prize awarded for most defects raised
  • No exploratory testing – no time for it – no need for it – test cases in advance was the norm
  • Test cases must have X number of steps irrelevant of complexity or feature set or skill set of tester
  • Step by step test cases only – no vagueness or open ended questions. Must have expected outcomes for each step. Nothing should be left to the tester to think about
  • No data extraction to central repository meaning test cases were a maintenance nightmare.
  • Detailed test plans up front and then ignored by whole team
  • Detailed resource scheduling months in advance – no contingency and all staff working 100% capacity
  • Staff treated as resources rather than people

I’m sure this sounds familiar to most testers. It’s certainly a common discussion point on most testing forums.


Team Work Poster
Team Work Poster


At this work environment there was a lot of negativity. The atmosphere reeked with contempt, finger pointing and blame. There was a lot of tick boxing just to meet quotas. Random checks of “passed” test cases revealed several defects that were not found. Was the test really run?

People were raising multiple defect reports for the same issue but in all the different flavours they showed (like operating systems, locales) just to raise their totals. Don’t forget, there was a prize available for the most defects raised, regardless of how sensible they were. 62 individual reports for one issue was the record. A one minute, one line code fix solved all 62 issues. Oh and yes, there was also a prize for most defects resolved by the programmers.

Then something strange happened. Through no serious planning or collaboration some test leads (and their teams) started flying below the radar. Doing things the best way (to them and their team), not the corporate way.

They continued providing the metrics needed for the managers but let their teams “just get on with it”. So they prepared shells of test cases as guidance and to report metrics on, but encouraged exploration and learning. After all, one of the most difficult challenges facing the team was the lack of communication between departments and the use of bad specs. So they explored and wrote tests during the testing. Constantly learning about the system. Constantly fine tuning the process.

They started to automate the tedious processes and the deployments to test. All off the record of course. Within about 3 weeks the “off the record” teams were motivated, dedicated and passionate about testing. They were also working fast, but disciplined. They were creating their own structure.

They were, as the agile world would call them, self organising, even though this was waterfall through and through. They had automated huge amounts of tedious regression freeing up time for more interesting exploration and scripting.

All the time they ticked boxes to appease management. All the time they flew below the radar. All the time the Test Leads risked the wrath.

Instead of pandering to the enforced corporate structure they instead created their own environments and worked to their own strengths and weaknesses.

More importantly though, they started to have fun doing their jobs. They started to find meaning in what they were doing. It made sense and it provided interest. It was now a meaningful job. When something is meaningful we want to do it. We want to get out of bed in the morning. It becomes a passion for us. It is highly motivating.

Eventually the “off the record” teams were exposed and the leads were reprimanded and the teams were forced back in to the enforced structures. The leads were moved around or their roles were changed. In essence, the structure took over again. The “trouble makers” were banished from the project and watched more closely than ever before.

And then something obvious happened. The reprimanded team members started to leave, moving off to other jobs. In the space of about 2 months 25 people had left.

Despite the real success and improvements these people made the management couldn’t see the value in good software, motivated teams and rapid delivery. No reports or formal processes were being followed – so to management it was all wasted effort. Null and void in the grand scheme of delivery. Software isn’t tested unless a test case is produced. Reports and charts were more important to the management than good software. Defects don’t count if no test case rooted them out.

Also though, the management saw the testers as resources. The numbers were more important than the people. The numbers were more important than the actual end quality. The numbers were more important than the ethos, environment and vibe. The numbers were deemed to tell the full story. The numbers were mostly faked, twisted or irrelevant. The numbers were valued more than the tester creativity, enthusiasm and flow.

I’m not advocating low flying testing.

I’m also not advocating no reports or metrics. We need some of this in order to record what has happened and to give us an indication of where we are in the process. Management still need to know where we are in the grand scheme of things. But balance is needed. Balance for the good of the product, the customer and the organisation – not for the good of management who can’t see past false numbers.

In the end the test leads essentially lost their roles and ultimate this low flying ended their careers at this organisation but it also resulted in this organisation losing extremely valuable and skilled testers. It showed clearly to me that when structures and enforced constraints start to affect morale then people will go to extremes to do a good job.

More importantly, if these people fly below the radar and still can’t get the job done there will be 2 outcomes.

1 – they will leave and the organisation will lose invaluable testers. 2 – their enthusiasm and creativity will be sapped, they will stay but offer little value. Essentially the organisation still lose invaluable testers.

A good manager should be isolating teams from the corporate politics, enforced structures and metric only assessment of quality and testing. We should all be persuading those around us that the effectiveness of the tester is at risk with excessive structure in place.

A balance should be sought between providing useful indicators of testing (metrics, burndowns, defect numbers) and letting the teams just get on and do the job.

A creative, happy, empowered and proactive tester can be worth a thousand demoralised testers structured out of self thinking and creativity.

Instead of looking at how to measure, structure and enforce global testing “best practices” we should instead be empowering the team to solve the problems. We should grab the talent we trust and give them the platform to grow. And we should be standing on our soap boxes and advocating that:


“This is the team to get it done” Picture from Jeffk on flickr. http://www.flickr.com/photos/jeffk/92047417/

Agile scared me

I’ve been noticing posts, comments and tweets about agile recently where people are being incredibly derogatory and negative. Almost to the point where their comments are bordering on offensive. The trouble is most of these people openly admit they have never actually worked in an agile environment. So what gives?


You know what I think? I think agile scares some people. I think they worry that they won’t have a job, that their role will change so much they can’t cope and that maybe, they will be exposed as a tester who is not capable of testing. Harsh, but even a well known testing presenter (who shall remain nameless) identified that this often happened when teams make the transition to agile. The move to agile identified those who are “not so good”.


I’ve been talking a lot about agile recently because for me the turning point in my career came when I started on an agile project.


At first I hated it.
Then I quite liked it.
Then my passion for testing was re-ignited and I started to really thrive in an agile environment.


Agile put me back in the zone. It cut out the bureaucracy, the admin side of test artifacts and left me (and the team) in control of creating great software. And this scares many teams.


What? Us in control? Really? Are you sure? But how will we cope? What do we do when? Eh? You want us to create our own structure? Are you mad?


And hence many teams flounder, fumble and stumble back to traditional ways of working.


I remember my first agile project and the first “kick off” meeting. It was an eye opener. It terrified me. I couldn’t believe people worked this way.


Before the meeting I read the agile manifesto, I read the scrum alliance website, I read some books and I read some blogs. I didn’t understand it at first, so I rewound and read it again until my brain hurt. And I still didn’t get it. It still didn’t make much sense. Theoretically it sounded fab. In reality though, how could it work?


In that first meeting I was informed by the Tech Lead that the UI would be the design and that we don’t need to worry too much about the requirements (backlog). Seriously? No design? No planning? I believe I gave the same look my careers adviser gave me when I told him I wanted to be a ventriloquist (thanks Seinfeld for the joke). I almost choked with rage.


From that point on though it started to make sense. And more importantly, it started to really work. I started to feel passionate about delivering software again. My job felt meaningful. We rolled out software….fast (and to top notch quality).


To sum up why though, here’s four reasons why I believe adopting an agile mindset will help you re-ignite your passion for creating fab software (I also included the slide). Agile gives you:


1.Freedom for the team to choose its own framework based on their local context. (team, customer, skills, experience, organisation ethos)


2.Freedom for the team to choose its own way of providing rapid and accurate feedback. (feedback to all team members as soon as possible is critical to success)


3.Freedom for the team to work to its strengths and weaknesses enabling them to inject the fun back in to software development


4.Freedom for the team to take control. No Excuses. Scary.

In pursuit of agile coolness

I’ve been doing a series of talks recently about agile being a mindset and not a methodology. At the core of the presentation (and the end talking points) are three questions:

1.At what point are you “agile”? 2.Are too many teams craving structure? 3.Is agile a methodology?

I’ve also had some feedback that the graphs and cheesecake pie sum up the current state of agile testing.

So here they are, included in this post. Let’s include some perspective on them though.

The Graph

Agile is cool. No doubt. The graph shows how the more you use the word “agile”, the cooler you become. It starts to take a downward turn the more you use the word “agile” though as you start to become a sort of freakshow if the only thing you say is “agile”.

“You want to go out for lunch?” “agile” “hmm, ok, did you want me to grab you a sandwich?” “agile” “right. You fancy a beer later?” “agile” “Got anything planned for the weekend?” “agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile, agile” “right. Freak.”

The thing is though, you’d still be cool, but only on YouTube and daytime TV.

The Pie Charts

The pie charts show agile adoption and failure within the IT industry in 2009. As we can see EVERYONE tried agile in 2009. And EVERYONE failed. At least that’s what it feels like.

The Agile Cheesecake Pie

Agile Pursuit is by far the most common behaviour I see. It’s based around the board game “Trivial Pursuit”.
Playing pieces used in Trivial Pursuit are round and divided into six sections. A small, plastic wedge can be placed into each of these sections to signify when a question from a certain category has been correctly answered
. – http://en.wikipedia.org/wiki/Trivial_pursuit

Instead of questions, development teams use techniques or processes that once mastered, result in a tiny piece of cheese, or pie, or cake – hence the name “cheesecake pie” being available.

Some teams are so determined to become agile that they set themselves a list of about 6 – 8 techniques they think they need to be doing. They then enter the lengthy, and often never ending game of agile pursuit, even if the process they had to start with was working fine.

They face problems, questions and queries on their way to filling their agile cheesecake pie. Only when the cheesecake pie is complete do they declare themselves agile. Ironically though, in the pursuit they loose sight of the reasons why they wanted to be agile and end up with so many techniques that they just can’t get the software out of the door. Either that or the game gets abandoned because everyone got bored…or left…or retired.

As a team relaxes about declaring themselves “agile” they actually end up gravitating towards the complete cheesecake pie as they strive to offer rapid feedback and tweak their process. (sounds painful).



After each of my talks there has been a resounding and common conclusion to the three questions I pose:

1.At what point are you “agile”?

You are agile when “you stop wondering whether you are agile and just get the job done”

2.Are too many teams craving structure?

Structure is a killer for most teams, especially in the testing world where we are used to formal documents, formal test cases and formal metrics. Making the shift is terrifying and for some it’s a step too far. But for some organsations and teams that *can* be a good thing.

3.Is agile a methodology?

Agile is not a methodology. It’s a mindset. It takes a new outlook on software development. It’s an attitude. It’s about inspecting and adapting. It’s an approach.

Let me know what you think (and no doubt there will be some interesting feedback) as comments to the blog post.

Outside the loop

There exists a loop. An environmental loop.

And in a simplistic way, you are either in it, or out of it. The loop I refer to is the loop that surrounds your team, or your project or your organisation or your current test case. The loop in which you work. Big or large. Wide or narrow. A loop. Maybe you belong to several loops. You will most certainly be in at least one.

If you are in a loop you often don’t see the problems with the loop. You are blind to the real reasons why your particular loop might not be working efficiently. You have narrow focus.

To see the problems with the loop, you often need to be outside the loop. Once outside the loop you can have fresh eyes and with a clear mind you can see the problems. You get a wider picture of the loop. You will see the problems, bottlenecks and efficiency gains. Some of the problems may seem obvious, as many problems are once understood, yet others may take a good deal of searching to be uncovered.

If you are outside of the loop you often see them quicker and easier than those people inside the loop. Look at how often a fresh set of eyes finds bugs in apps you’ve tested well; or when a customer reports a blatantly obvious bug the whole team missed.

Some problems persist in the same loop for many years. Often until fresh eyes point out the problems or the loop itself collapses under a mega weight of inefficiency and problems.

But I digress.

The problem with being outside the loop though is that you often cannot change the loop. And this is a common problem faced by many consultants (or new starters) who lack the pursuading skills needed to change the loop.

Consultants often point out the problems, notice the bottlenecks and clearly see a more efficient path of working, but because they are outside the loop they find it tough to change the loop. Resistance to change from outsiders is natural.

And it’s not just consultants who can point out problems. I’ve heard testers comment or berate other test teams for doing X when Y would be better. Do these testers work in a perfect loop themselves…really?

“You need to be outside the loop to see the loop. But when you are outside the loop it becomes much harder to influence the loop”

The above quote was taken from Patrick Neate’s awesome book “Where You’re At” (http://www.patrickneate.com/page.asp?p=3008).

So being inside the loop is hard work because you can’t see the problems as clearly. But being outside the loop is hard work too because you can’t always influence the loop.

So here’s some advice.

Don’t be afraid to step outside the loop…often. Taking a step back from your testing world and looking at the project as a whole is very valuable.

Taking a critical look at your project from another viewpoint is essential. Taking a massive long look at the whole organisation from a different perspective can be mind blowing.

  • Why not swap people from one team to the other after the project finishes (unless of course you find the all elusive high performance team of heros)?
  • Why not sit in on other team meetings?
  • Why not have floating testers who simply switch from team to team every week. This works wonders but takes a certain type of tester (and supporting team) to cope with it.
  • Why not share your observations and collaborate to improve? Why not help each other to see the problems?

I’ve seen this working first hand with major success. You’d be surprised at how valuable this can be. A real eye opener for some teams. Heart breaking for others.

Why not get a consultant in to help you with your testing work, but ensure you give them the mechanism to make changes? After all, it’s a royal waste of time paying a consultant and then not letting them change anything. Believe me though, this happens a lot. Consultant are valuable, you’ve just got to work with them closely.

Outside viewers, consultants or observers are not a bad thing. Not always anyway. They are loop observers. They will see things you cannot. They will notice problems you might never see. They are incredibly valuable. Work with these people to improve your process.

Be conscious of the fact that you CAN influence the loop. And provided with the right information there is no stopping you.

Observers can feed you the observations, you can make the changes.

And there’s no better collaboration effort than that.

Photo courtesy of Alex Kess (flickr – http://www.flickr.com/photos/akc77/)

Don’t judge people too quickly

In the testing world there often appears to be a lot of hatred, resentment, hostility and anger between various camps of people. Obviously this is not just limited to the testing world but it’s the world I mingle in, so it’s on my radar.

All too often people are jumping to conclusions about the way someone else tests or the theory they have come up with or the way they write their test cases or the fact they have released software without all tests passing yada yada. You’ve seen the blogs or forum posts where someone says you’re not doing it right unless you do this or why would anyone even consider doing that or this?

They can be quite offensive at times and more often than not, show a deep misunderstanding and lack of human respect. It’s a complicated world that we live in and too often we jump to conclusions, fed by our own experiences, the media, our testing heroes, our norms and cultural values, our education, our training, our mental states and models and our general outlook on life plus lots of other things too. We base our views about other people and their testing on our own worlds and ideas. Often that is all we can do. We see things as black and white. We don’t see past the immediate actions, words or theory to the person, their context or their world. It’s not that we can’t. It’s often just that we don’t/won’t.

And this hit home with me this week with a very personal story.

Over here in the UK we are in the middle of a very unusual (apparently) weather system that is dumping masses of snow, dropping the temp to well below freezing and generally causing a boat load of disruption. We aren’t prepared for the snow. We don’t have the correct training or the temperament to deal with this. We weren’t designed for this weather. We had no spec. No requirements. No user stories. We don’t know what to do.

So we panic buy food. We stop gritting roads (controversial). We issue severe weather warnings. We have the snow on the news 24 hours a day yet not really explain how to cope with it. We finally grind to a halt. And then we all complain about the snow. Even those who don’t have the ability to work from home and are down the park on their sledges.

And so last night with a heavy heart, a cup of hot tea and a faint feeling of wild panic I sat and watched the news on the TV. On it there was a policeman, don’t recall his name but some high ranking policeman, telling people to stay at home unless they absolutely have to go out as car accidents were causing chaos. There was some other bloke from the RAC telling us to stay at home and that more accidents and stuck cars is the last thing we need at the moment. Then there was some MP telling us all not to go out and not to panic buy. Stay in and keep warm they said. And with accidents at a record high and thousands of people having to sleep in cars over night due to being stuck it’s a sensible message they were all saying. Stay at home. Don’t risk your life and the lives of others by trying to get out.

Then came the typical happy story to end the bulletin focusing on a series of hardened English folk who braved the weather to open their local school. The teachers heroically fought their way to work. The kids did too. (oh how they must have cried when they saw the thousands of other kids on the news sledging around the hills of olde England). Everyone in the community pulled together, put their lives at risk and made the long and painful journey to school. “We shall teach these kids, no matter the weather” was the mantra they all chanted (note: this mantra bit might not be true).

So two dualities exist. Stay at home, put safety first. Make the journey, brave the danger, prove how tough you are, get your face on the TV.

And then this morning just as I optimistically dragged my wheelie bin out for collection I was nearly squashed by some idiot trying to get to work in an over sized Audi. With 30cm of snow on the ground, frozen over by a night of minus 5, the roads are literally like a skating rink. And here was this loony trying to get to work. After he had bounced of the pavement edge a few times, taken out next doors bin, crushed two miniature conifers, dragged a hedge for 10 metre and then spun around twice he finally settled in a large mound of sand, which lay nicely hidden under snow in someones front garden. My immediate reaction was to ensure this person was OK. He was. I then proceeded to berate him for stupidly putting his own life at risk, as well as mine and potentially too the Stubborn Cat. Poor cat.. (note: the cat was OK)

However I had jumped to conclusions too soon. I had assumed he was an idiot for ignoring all of the sage advice on the news. I had assumed he lived in a similar world to me, where he could work from home. Where he valued his life above journeying to the office. I had assumed he didn’t need to make this journey. As it happened he did need to make this journey. He was a local doctor (thankfully not mine) and he was on an emergency call; to a local resident. I felt foolish. I felt ignorant to other peoples needs. I learned a valuable lesson. I felt really bad.

So whilst I was digging out his car with the help of some neighbours, whilst he went by foot to the call, I was giving this situation some thought. I was milling it over. I associated it to testing and the behaviour I see too often in our community. I vowed I’d try to be more empathic. That I would try to understand that there are reasons people do things I disagree with. Reasons why people can’t do things certain ways. They have contexts and worlds I don’t occupy. And frankly, sometimes I’m glad I don’t occupy them.

I was wondering why all of these people make the journey out in the snow and realised that many had too, even if it meant putting their lives at risk. Some for good reasons like the doctor, or the nurse or the policeman or the parent visiting their sick child in hospital. Some for money. Some because they feel guilty for not trying. Others because their company would dock them a holiday day if they didn’t go in (made even more tragic by the fact that in this example the employer was the local council who didn’t clear the roads – note this wasn’t my local council…)

And so there are always reasons why people make, what seem on the surface, to be stupid decisions; but in the end most are based on some deeper context, understanding or reason, often which you will have no insight into. So to blindly say someone is stupid for sitting the ISEB exam is misplaced. To say someone is a fool for using record and playback automation is a mistake. And to say that someone is an idiot just for not using your BEST PRACTICE is frankly mad.

So my new years resolution is to be more understanding of other people and their situations and to help them, work with them, learn from them and in turn maybe impart some of my understandings to them. I’m going to respect other peoples opinions, even if they seem wrong.

We are a community of testers. And the more we push our own agendas at the expense of others, neglect the contexts of others in this complicated development world and don’t appreciate that software testing is a human activity, the more we are becoming fractured, incoherent, inconsistent and untrusted. And we can only sustain this level of negativity for so long before the whole thing collapses. And that, would be a very bad thing.

Apparently there’s nothing new..

At a conference I attended a few weeks back someone asked a simple question. In actual fact it wasn’t a question, it was a statement. And an angry one at that.


“What you’re talking about isn’t new. There’s nothing new at this event. Where’s the new content?”


He then proceeded to walk out in disgust. I must add that it wasn’t me talking…or making the statement.


At first I agreed with him. There wasn’t anything new. There wasn’t anything groundbreaking. But then I sat down and really studied what was said at the event. The talks were indeed a simple rehash of “common sense” things or ideas we’ve all heard somewhere before. It was indeed old stuff. But then 99% of the stuff about testing and methodology is not new either. It’s been done before, maybe not in the testing arena, maybe not in the computing arena, but most likely done or said in some context before.


But looking at it like that is missing the point. It’s too simplistic a view.


Most of the content I see actually is “new”. It has to be unless someones just copied it word for word (some do by the way). The idea might not be new but it’s bundled together in a new form. Therefore it’s new. It’s recompiled in to different contexts. It’s repackaged, merged, renamed, tweaked, built upon, defined more clearly, analysed further and given cool and funky labels. So it is new. It’s just a new version of something that’s gone before.


Like Die Hard 2, 3,12000 etc. They’re all the same thing. Different location, different baddie, different time and place – same concept. New film.


“New” theories may be made more palatable, more relevant, more concise, easier to understand or more intuitive to apply in real life. It may well be old but if repackaged well it can still be new. It can appeal to the new crowd. It can appeal to the people paying attention at that moment in time. In that context.


Every once in a while someone does genuinely come up with something new, but to attend a conference and expect groundbreaking, never heard before content is optimistic to say the least.


But I’ve come to appreciate that very few conferences have “new” stuff. It doesn’t mean they are not valuable. I still learn things from conferences. I still go off afterwards and look things up, do some research or make some notes. New understandings or re-branding of old ideas is important at furthering our craft of testing. For spreading the word. Reaffirming ways we do things. Building on ideas.


We can learn from the past. We have to. And no doubt we will go around in circles but that’s life I’m afraid. Like the time when flared trousers came back in fashion. Or 80’s retro games came back. It’s a new audience (may be different, may be older, may be the same), with different outlooks on life all at a different time and place. Doesn’t mean it’s not valuable. Doesn’t mean it’s not new to some person or society or the testing community or me. Doesn’t mean it’s not acceptable. Doesn’t mean we should walk out of conferences. Doesn’t mean we still cannot learn something.


I’m just waiting for shell suits to come back in fashion, I’ve got a cracking green and black one in the wardrobe. 🙂

Talent imitates, Genius steals

Talent imitates, Genius steals


A very pertinent phrase for what’s been happening in the testing world recently. There’s been some high profile stealing or imitating of ideas and concepts but also some low key ones too. It seems we are in an increasingly competitive (and open media) world and those with the really great ideas are in grave danger of having them stolen. After all, there are too many testers and not enough good ideas.


But it really grates on me when the “theft” of ideas is so blatant, when it’s a straight copy, when little is done to even mask the fact it’s been nicked. Taking an idea and building on it, merging it, reforming it – fair enough I guess. Taking an idea and just running with it as your own – poor show.


Like the training provider that blatantly nicked Lisa Crispin’s and Janet Gregory’s Agile Testing book chapters for it’s course content, or the testing forum that nicked SQA heading ideas. But closer to my world is the “potential” borrowing of the Software Testing Clubs magazine content ideas. I say “potential” because it could just be coincidence.


Now, creating a magazine is not a fresh idea, there are loads of mags out there and there’s no harm in creating more of them. It’s a free world. It’s not a patented idea. But to steal the ideas, concepts and content of a fresh and funky new magazine and pass it off as your own idea would be just plain wrong. Right?


Now I do wish the peeps who appeared to have copied The Software Testing Clubs ideas all the best with their magazine.


I genuinely do. Because it’s a lot of hard work and it requires a lot of good content, good design, creative ideas and bags of enthusiasm. It also takes a lot of integrity to make sure the contents good and in keeping with the style. It could just be a coincident that their magazine forum post reads awfully similar to the STC magazine forum post. It could also be a coincidence that it also reads like one of my articles, it’s got my style and language tone. Coincidence indeed.


And maybe it’s a bizarre coincidence that the content ideas are also identical to the STC ones. And maybe the publishing date being the same is truly a coincidence too. Small world and all that. It is funny how the testing world works. Great ideas appear to come along at the same time. Well, actually in some cases about 15 days after others 🙂 …. but let’s not quibble. Let’s not make any accusations. Good ideas are good ideas. And good luck to all who ride with them.


It will be an interesting January for the testing community. Two magazines with the same intended content. Oh how we spoil you guys. Oh how you have choice. And what a lot of great choice you have.


I’m really looking forward to the release of the STC magazine (January 2010). I’m well chuffed with the content; and the enthusiasm and hard work that’s gone in to it will show through in the end. The design is awesome, the contributors are perfect and the funny stuff is funny. Very funny. And a big thanks to all those who have contributed or been involved in some way shape or form. Even those who submitted blatant copies of other peoples work (it opened my eyes to how stupid and underhand some people can be…even to the person who tried with two different (copied) articles….I salute your persistence.) And if the other magazine is packed full of similar cool content then I’ll be over the moon. Yet more great places to go for testing information and a bit of down time.


There is plenty of room for both magazines, I’m just awfully glad we didn’t advertise on the forum some of the really really really cool features in the mag. phew.
2010 will be an interesting year and only time will tell who is either Talented or a Genius.




P.S – thanks to Faris Yakob for the title idea. http://farisyakob.typepad.com/

100g of ability, pinch of willing, bags of passion, 50g of interest, a slice of learning. Pan Fry. Testing……Done.

The new series of the Gordon Ramsay’s F Word is back on TV here in the UK. This series Ramsey et al are in search of Britain’s best restaurant by cultural cuisine type (i.e. Chinese, Thai, French etc).


I wouldn’t normally be referring to Gordon Ramsay or British TV except that something struck me about the restaurants, in fact more precisely, the chefs that make it in to the final cook-off on the F Word. The chefs all seems to share something in common……Passion.


Their passion is for cooking. But not just for doing the action of cooking but also helping other people to learn more about cooking as well as building their own knowledge during that process. Each of the chefs run very successful restaurants, with awesome food but more importantly from my view is that they all also run weekend or out of hours cooking courses.


In the testing world over the past few weeks there have been lots of discussions about training. Matt Heusser posted some stuff on his blog the other week about training (or lack of) http://blogs.stpcollaborative.com/matt/2009/11/18/you-say-you-want-a-revolution/.  Selena Delesie posted about coaching testing skills http://selenadelesie.com/2009/11/25/coaching-testing-skills/ and twitter has been awash with talk of certifications and training.


So what’s my point? Well, my point is that for those of us who are passionate about what we do it is inevitable that many of us will want to share and contribute to community projects and training. It’s not every ones calling and some people don’t thrive in social environments. But for those that do, training and coaching local testers is a really great way to share your passion for testing. It’s also a fantastic way of building on your own knowledge.


So maybe the time has come for us to get involved in promoting testing, cultivating testing, getting local community testing groups flourishing but most important of all; offering a place for people to learn more about testing.


Too many people sit back and complain about lack of training for testers. Instead, why not create yourself a community group, join a local testing group, build your network and try to get some local learning going.


But most of all. Enjoy it.


(picture courtest of Fernando on flickr: http://www.flickr.com/photos/fernando/)


P.S – For those who don’t know about Gordon Ramsay, try watching an episode of F Word. Done.

Taking a break from the old routine

I’m a regular on many of the testing forums and mailing lists and I’ve noticed recently a very worrying trend. That is that many testers simply cannot appreciate different ways of working. Their way is a Best Practice. But it’s more than that, it’s not just about Best Practices but the vitriol in which they are delivered.

I see posts on forums asking for help on how to solve a problem. The responses are usually stories of success of working X way to which someone replies with Y way. Then someone chips in with Z way and pretty soon the whole thread is hijacked in an argument over which way is better. Mine, mine, mine.

Some of these threads are extremely useful where people learn a new way of approaching a similar problem. Other threads though have a slightly more sinister feel to them.

Now, I’m not one to hold back and I often respond with critical comments to posts but I like to think I never tell people how they should do something…my way. I can offer help, suggestions, stories, advice, mentoring and guidance but I can’t *tell* someone what to do. I have no best practices that will work for other people, just some honest suggestions from experience that could work.

However, when answering forum posts many testers don’t appear to consider the fact that there are thousands of testers out there, all working under different methodologies or interpretations of methodologies, time frames, technology, budgets, skills sets and software development life cycles. Some are working in highly structured environments, some with helpful programming teams, some with middle managers bean counting and some with processes that are old, destructive or wasteful. But it’s their environment and it’s their problems they are asking for help with. So let’s help – not dictate and ridicule.

Over the years I’ve created many many many Best Practices for myself. In fact, every time something goes right, it’s a best practice. Yippee. I have also developed some worst practices too. Boo. And along the way forums, user groups and mailing lists have been enormously helpful in helping me forge my way through my testing life. However, the time has come to stop visiting certain testing groups, forums and sites. To stop subscribing to some LinkedIn and Yahoo Groups; the job feeds are taking over and too many articles have Best Practice in the title.

The sad thing is, the resources are the ones I’ve long been supporters of. Resources that have helped me enormously in the past. Resources that have become the main staple of the testing community. And it’s a shame they are becoming hunting grounds for the Vitriol spitting testers pushing Best Practices.

If the negativity and anger included in some of the replies on these sites is the routine (which it appears it is), then I’d recommend we take a break from the old routine. Unfortunately for many of these forums, this seems to be the action many testers have taken too. No longer are people going to sit through a barrage of Best Practice suggestions, people calling other peoples suggestions ‘stupid’ and people leaving sarcastic comments like ‘maybe you didn’t read the question correctly’. We are not finding it helpful.

There are far too many new testing forums, groups, projects and communities who are there to help. Who still have a community feel. Who still serve the community they are part of. Communities who help testers and grow the community. Whose members don’t try to out-do one another, push their Best Practice as the only solution and degrade and belittle anyone who doesn’t agree – although no forum is ever immune to this, but some forums have a much lower rate of angry testers and vitriol Best Practice pushers.

Now let me clear something up. I’m not talking about people who offer criticism, constructive feedback, honest statements and points of views – sometimes in a heated way – this at times can be healthy – very healthy. I’m talking about people who pick apart other peoples ideas for fun, who say that other ways can’t work, who are so hung up on terminology that it’s painful, who actively take it upon themselves to counter argue forever despite real world success stories, who simply can’t understand that people work in different ways, who have black and white views on automation and who plainly abuse and call people stupid (believe me – it happens quite a lot – and a lot worse sometimes).

And I do sincerely hope that these testers who can’t appreciate contexts come to understand that the testing community is complicated, multi-threaded, diverse and evolving. It’s made up of people with no experience, some experience and a wealth of experience. And in a world where collaboration, community and learning are becoming more valuable it seems alien to be abused for posting on a forum a real and genuine question, problem or response. Sure there are “better” ways of working but there are very few absolute “best” ways of working.

But I’m not going to end of a low note. Nope. For every one of the negative, bullying and belittling testers, there are a thousand welcoming, positive and helpful testers. Testers who are happy to help. Testers who give honest opinions and feedback. Testers who understand that their way may not be the “best” way. Testers who are happy to improve, learn and share knowledge.

And that is a really great thing. In fact, it’s a truly great thing. It makes me glow with pride.