There comes a time when we all use numbers to represent some fact or information. These numbers *can* be highly effective at communicating your information.
Image courtesy of : http://www.flickr.com/photos/fragmented/
These numbers could become the defacto representation of some outcome, fact or finding. These numbers could become the metric you make a decision by. These numbers could start to guide your day to day work.
At work, we use numbers like these all of the time. We use the project velocity. We use the number of open bugs. We use the number of tests that fail testing. We use a whole army of number to represent information like the top ten applets used in our system to the number of end user using a particular browser. We use them as indicators, not the law. They guide, not force.
It’s important to remember though that these numbers are just that. A number. It’s not the number that’s important, it’s how you feel about that number that matters <– I believe it was Michael Bolton I first saw using that phrase.
Sometimes I look at a number and wonder why I use it. Because I always have?
I’m no longer accepting that so I’ve dropped or changed a large number of metrics recently because they are no longer adding value.
Yet many people use numbers, with little meaning, to guide their testing, their releases, their company strategy.
Like my metrics and numbers, they may once have had meaning. They could once have been a way of locking down a fact, or a snippet of information or even a whole business process. Then our numbers grew. We collected more of them. The data got bigger. The variance even larger. They lost any symbolism they may have once held. They lost their context and no longer describe the idea or concept they once did. Time and business moved on and the numbers didn’t keep up.
But they have become the norm. They are the defacto number. They have not been re-factored, or re-worked or even questioned. So they still guide testing. They still inform key decisions. And the scary part, they may be wrong.
As Testers I believe it is our role to treat any metric or number with a little skepticism. We should question it’s use or at least ask more about its meaning.