Push The Button

During a conversation with a group of testers at an event I soon found myself outnumbered in my views around “enhancements” to the product. I was the only one who saw a Tester’s role as more than just verification.

I was a little amazed at how this group of Tester’s (or shall we call them Checker’s?) narrowed the focus of their roles down to simply following a script, clicking buttons and ticking boxes. They were also incredibly passionate about not suggesting improvements to the process or product. It’s not so clear to me though where the line is between enhancement and bug. It’s not always easy to categorise.

Through sheer persistence and the use of pursuasive language and examples of real bug stories I convinced them that there was more to Software Testing than checking. I also bought them all a drink, but I don’t suspect that had anything to do with it. 🙂 I’m not saying they weren’t doing a good job, far from it, but to not look at the wider context around your checks is to ignore the purpose of the system.

I promised to blog about the things we talked about, and here it is, about a year later than it should have been.

This post stemmed from a point one of the guys made “A button is a button. It’s the same in every system. We test it does what the spec says it should do.”

At a basic level this is absolutely correct, but to look at a button with this narrow focus is to miss the point of Testing (as I see it). The button is just one action, or element, or interaction in a much bigger process or system. It may fire off something or trigger an event but the interesting questions for me are:

  • Why would I click the button?
  • What are my expectation?
  • Do I have any choices around clicking the button?
  • Will I like what happens?
  • How will you encourage me, force me or convince me to click the button?

And a whole lot more. To ignore the context of the button is to not question the system it lives in. To check the button does what the spec says, is to put too much trust in the spec. So I blurted out a few ideas about testing, which actually cross over in to design. I’ve included as many as I could remember here in this post. Some may be helpful for you, some may not.

 
Is the button accessible to all users?

  • Should it be?
  • Do certain users get more or fewer buttons? Why?
  • For example, admin users or permissions based systems.

 

Why would someone click this button?

  • Are there other choices?
  • What will drive someone to click this button?
  • What happens in the grand scheme of things if they don’t click it?

 

How will your users “feel” before and after clicking the button?

  • Do you know?
  • Does it matter?
  • Can you redesign to cater for this “feeling”?
  • For example, submitting your tax returns will give your users a different feeling to submitting a newsletter subscription form

 

Is this behaviour monitored or detected or regulated?

  • Does this change the button form?
  • Does this change the behaviour?
  • Who detects it and why?
  • Are there privacy issues or data protection concerns?
  • What is done with this data?
  • For example, audit trails in systems.

 

If your users access the site via a mobile device, does the button still have the same look and feel?

  • Does it still do the same thing?
  • In the same way?
  • Should it?

Do previous choices or data entered alter the button choice and action?

  • Should it alter it?
  • Could it alter it to get a better outcome for all?
  • When does it alter it, and when doesn’t it?

 

Does the button have the potential to do damage and therefore require more consideration or a confirmation?

  • Status updates?
  • Sending personal details?
  • Firing missiles?
  • Turning off life saving machines or devices?

Do you need further steps in the process or is clicking this button enough?

 

Is it possible to design the overall page and the button to deter or encourage certain behaviour?

  • For example, to eliminate the “Donkey” vote the candidate voting lists are often randomised

Does anything surrounding the button affect the choice or decision?

  • Should it?
  • Could it?
  • Can we affect choice by anchoring adverts or information or other users choices?
  • For example, peer review scoring on sites like Amazon

Could the button or process be disabled but visible to encourage upgrades, add ons or extra steps?

  • Should it?
  • Does it fit with the business model?
  • Would your users appreciate this?

Is the action the button performs pleasurable for the user? Or negative?

  • Could this process be sweetened if negative?
  • Or rewarded if good?

Is the button consistent with other buttons in the process or system?

  • If not, why?
  • If yes, should it be?
  • Would it be better with a different design..to stand out maybe?

Does it matter whether you are the first person to click this button?

  • Do you get extra features?
  • Do you disable a process for other users?
  • Is the outcome clearly communicated?
  • For example, one time downloads or data collections.

Can you click the button more than once?

  • Why?
  • How?
  • Does it function consistently?
  • How does this affect the system or data or process?
  • For example, submitting a form more than once

 

Does the button support multiple users?

  • Can more than one user click their own button at the same time?
  • At different times?
  • At all?

Can you double click the button?

  • What does it do?
  • Should it be possible?
  • Could you deter this behaviour with warnings or informational text?
  • For example, some users click hyperlinks twice.

Is the design of the button modern, or old?

  • Can it be skinned?
  • Does it look old for a reason (functional, to encourage upgrades etc)

Do you need supporting help text?

  • Is it obvious it’s help text?
  • Can it be on or off?
  • Or on all the time?
  • Is it obvious the text is relating to this button or process?
  • Is the help system being designed against a recommended compliance like the elmer guidelines for form design?

Can you create an emotional attachment when the user clicks on the button?

  • Can you get buy in, repeat business or a brand fan?
  • Do you want to?
  • Do you need to?
  • For example, the Like button on Facebook

What action does the button perform and is that consistent with what your users want to do?

  • Submit, next, back, details, open, close, print, etc
  • For example, the final submission button in a wizard should not be called continue, unless it has supporting text to inform the user what that button will do.

Is there a time limit surrounding the button?

  • A limit before clicking or does it start a timer?
  • How would this affect the design and testing?

Is the button grouped with other like minded buttons, or dejected and lonely?

  • Is the like minded group logical?
  • Does it convey the right message?
  • Is the lonely button there for a reason..to draw attention?
  • For example, all text formatting buttons should be logically grouped, rather than spread across a busy toolbar.

How does the button perform with no supported visuals?

  • Text only
  • Screen readers
  • Accessibility

Are you using colour to symbolise or signify meaning?

  • Red for danger
  • Green for go
  • Are these signs common amongst your users?
  • Ambiguous?
  • Should you be using colour? Accessibility? Colour blind?
  • Do they display well on different monitors?
  • If you had no colour could you still use the system?

Does the button actually do what the user expects?

  • Regardless of the spec, is the button action logical?
  • Are the expectations wrong?
  • Is the action wrong?
  • Is your oracle wrong?
  • Is it by design, fake affordance maybe?

Is the button part of a sequence?

  • How is the sequence represented?
  • Is the sequence logical?
  • Can the sequence be tricked, skipped or broken?
  • Is the sequence communicated well?

Does the action of the button mirror actions or features in other systems?

  • Will users see a link between systems, for example a status update in Facebook?
  • Could you redesign to take advantage of that association?
  • Would you want the association?
  • Is it worth redesigning a common process, or will the baby duck syndrome kick in

Is the process surrounding the button common?

  • For example installers.
  • Does the process require much thought or is an almost natural process of unthinking clicking.
  • For example, the browser add on tools that are hidden in the middle of a software installer. They work on the basis that you will not read it and hence install the browser add on.
  • What could you do to encourage more thought around the process? Make it harder?

Does the button itself represent the action?

  • Delete icon, rubbish bin icon, cross, plus sign.
  • Are they logical?
  • Are they repeated on the same screen adding ambiguity?
  • Are the symbols universal to your audience?

If there is a breadcrumb trail, how will clicking this button affect that?

Does the button require data fields to be completed?

  • Should the button be visible until all fields are complete?
  • Is that clearly communicated?
  • Greyed out? Disabled? Button text change? Disappear/Reappear
  • When does it validate? Where? How?
  • Is it consistent?

Does the button recall previous data?

  • What happens when clicked more than once?
  • How is the data represented or affected

Is the button generating real time feedback?

  • How real time?
  • Does this affect performance?
  • Is it required?

Does the button work under multiple instances?

  • Multiple browsers and/or tabs?

What lies beneath the button?

  • Can you glean any further information about the button by using a tool like Firebug?
  • Does it have alt text?
  • Is it logically labelled and suitable for automation?
  • Are there any comments in the code or other clues?

Is the button the final action?

  • Does the user need to know anything before they click the button?
  • Do they get a confirmation?
  • Are they reassured enough to click it?

If the button is part of a process, would it better to push the process through a wizard?

  • Will the user experience be better? Less prone to mistakes?
  • Will the process be done in one go, or can they save and return?

Does the button need an undo?

  • Does the action need a reverse or opposite action?

Is the button multilingual?

  • What happens to the button when it tries to show different languages or locales?
  • Does it still make sense?
  • Does it hold longer text?

Can the user press this button without using the mouse?

  • Mobile devices. No right clicks. Etc
  • Shortcut keys? Tab order?
  • Should they be able to?

Can it be made easier or harder to press the button depending on your objectives?

  • Many large organisations appear to be making it hard to find the button which leads the user to the telephone numbers or listings.  Self service is cheaper but is it always better?
  • Big red buttons with lots of “click me” appeal
  • Button in the middle of the page, or hidden in the top corner.

If you removed the button and that part of the process how would it affect your system?

  • Would it make it better?
  • Unshippable?
  • Un-usable?
  • Better?

A lot of these ideas come about when reading about design ethnography and UX, particularly around lenses, but for each one I’ve always been able to relate this back to a bug or “enhancement” I’ve seen in systems I test.

Questions about the bigger process will lead to more test ideas about the button (and the bigger process also). The button is a part of a bigger system, which is operating in a context. I think it is at this level that individual actions start to make more sense as a whole, and you can begin to question who, what, why, where and when.

If you find yourself working in an environment where you’re not allowed to raise questions about the checks you are running and you really want to (and attempts have fallen on deaf ears), then it may be time to move on and see what else is out there.

 

14 Replies to “Push The Button”

  1. awesome post – no wonder though that people think testing gets in the way, look at all the questions we raise just for a button :)I’d love to sit in a design meeting and raise all of those – wonder if I’d get halfway through before being asked to leave. Or be firedDouble clicking a button is on of my usual quick tests and more often than not it finds a problem

  2. Thanks Phil.I reckon about 10 minutes before getting thrown out :)There’s a lot to consider and I can’t help noticing how my testing is moving more towards understanding human interaction. Some of these are pure design comments, but there are a number of tests in there.As I wrote them out I actually recalled a genuine bug I’d raised over the last 10+ years for each point. Funny how the mind works. If I had tried to think of each bug I wouldn’t have had so many..The double click test is a classic. I refer to it as “The Parent Heuristic” as almost everyone I have ever worked with has mentioned that their mother or father or elder relative double clicks EVERYTHING!Thanks for commenting. Rob..

  3. I’ve spent the last couple of years trying to show how important and useful it is to get the test team involved at the design stages, because we train ourselves to think though the possibilities that will result from a design decision. This is effectivley the same process that you go through when thinking of what tests need to be run, just done earlier in the project life. On almost every project, the conclusion is that yes the testers should have been involved earlier, but when the next project is being designed, the testers are not involved. It is the creativity that comes from good testing that I enjoy from this job , and that creativity has often helped to find problems that are not bugs, but do affect the value of the product to the customer (sometimes, I’ve come up with things that have not been done, only for the customer to request it anyway).

  4. Hi Andrew,Thanks for the comments and it sounds like you’re doing an excellent job of bringing your thinking to the design stage.The interesting thing from your comment was the assumption that testing is done after the design. I guess I need to clarify that in the post. The Testing I typically advocate is right at the design stage; during the story planning and analysis and even more so during the development.Saying that, if people can’t get involved that early then it’s better to ask those questions later than never at all.How do you think you could get your Testers involved earlier? What are the main barriers to this?And I’m so glad you mentioned creativity and testing. The two go hand in hand and it’s great to hear of other people who have the same thinking.thanks again for commentingRob..

  5. “The interesting thing from your comment was the assumption that testing is done after the design”I think I need to be clear in stating that the assumption is held by others in my organisation, and not the by me. I take the view that testing is gathering information on the product/project and that this can be done at any time. There are always questions to be asked and information that can be garnered. The trick is persuading others that the information we can give at these times is important, relevant and useful.

  6. This is marvellous post Rob – a 101 in questioning a product! Its amazing how many questions you can generate for such a simple action as pushing a button.Its a great example for aspiring testers – what are your pointers for generating such a list please? Think got about half of them… I’m guessing the Phoenix Checklist played a part? (Thanks for that btw!)

  7. Hi Andrew,I think that’s what I meant in my badly worded response! You did make that clear in your first comment that you’ve seen success when testing early.The stereotypes of Testers and what we do is hard to beat. I’ve always found that even when you can see the light there will be setbacks. It’s almost as though we have to prove we are more valuable than others. I have no scientific evidence for this though :)Rob..

  8. Hi Duncan,Most of the ideas came from reading around (and implementing) UX, design and Ethnographic research techniques. Reading sources on Testing are great to read, but texts about social sciences are more insightful in my experience. But then I studied social sciences for many years, so I would be biased.A lot of these questions, tests and ideas came from experience too. More than I care to remember. In turn this came from moving jobs often (sometimes through boredom, but sometimes through relocation) which exposed me to different environments, products and approaches to software delivery; this aided me greatly. But if you get to test on a reasonable complex system in a creative environment with good mentors…well, that’s awesome too. Problem is, those environments are few and far between.The Phoenix Checklist is kind of ingrained in my mind. I often don’t realise I’m questioning so much. I’m a natural skeptic in real life too and I think this helps. But more than anything I spend a lot of time “observing”. Observing design ideas, the world around us, human interactions with the environment (tech, physical environment and other people) and also the effects poor design can have on people, their emotional state and their next actions. (including my own)I’ve only recently started trying to articulate this in my blog. Glad you’ve enjoyed it so far.Thanks for taking the time to commentRob..

  9. Thanks for the post, really. There’s a big relef in knowing you’re not totally wrong in what you think. From what I have experienced, it’s everybody in the development process who could participate in improving the testing situation. What Phil said tells me, that management themselves need to see in testers more than clicking apes (or scapegoats if something goes wrong 😉 ), but not only in a theoretical way as Andrew mentioned: you have to be consequent about it, not only talk and consider. On the other hand I might also make the testers itself (partially) responsible for our situation: do we really dare to contradict, to say what we don’t agree with even if we feel (so far) nobody cares? And do we back up others that do? On the other hand also Development needs to learn that having issues with their design / concepts we don’t want to argue about their genuity, but we’re more than just dumb clickers.Anyway this post definitely has found its way into my favourites. Thank you! (And you definitely earn a medal for investing your time into listing all those details!.)

  10. Thanks for the post, really. There’s a big relef in knowing you’re not totally wrong in what you think. From what I have experienced, it’s everybody in the development process who could participate in improving the testing situation. What Phil said tells me, that management themselves need to see in testers more than clicking apes (or scapegoats if something goes wrong 😉 ), but not only in a theoretical way as Andrew mentioned: you have to be consequent about it, not only talk and consider. On the other hand I might also make the testers itself (partially) responsible for our situation: do we really dare to contradict, to say what we don’t agree with even if we feel (so far) nobody cares? And do we back up others that do? On the other hand also Development needs to learn that having issues with their design / concepts we don’t want to argue about their genuity, but we’re more than just dumb clickers.Anyway this post definitely has found its way into my favourites. Thank you! (And you definitely earn a medal for investing your time into listing all those details!.)

  11. Hi Sebastian,Thanks for taking the time to comment.It should absolutely be a holistic approach to “quality” by all involved. I think the times are changing and it’s no longer acceptable to leave the testing until the end. We absolutely do more than just click…well, some of us. There is a worrying majority who simply do verification, but with automation, we can soon do something about that :)Thanks for the nice comments.Rob..

  12. Great post. I just thought about my last job where I was a button checker. I hated it (and it totally wasn’t supposed to be like that). I’ve always been working at places where my input on projects was valued (as well as the rest of the team’s input) and it made the product better in SO MANY WAYS.I do work with a former game tester now who disintegrated my beliefs that game testing would be awesome. They’re pure checkers with lists of things given to them by the PMs and engineers to check. No wavering, no exploring, just checking.I guess we’re lucky to be at a place that promotes not only testers, but developers as well to be inquisitive about the products we’re creating.

  13. Hi Justin,Thanks for taking the time to comment. It’s great when you find somewhere that values Testers and good Testing. You’re right, it makes the product so much better, and most importantly, it allows us to really enjoy our jobs too. :)I heard the same thing about Game Testing. It’s the same part of the game, over and over, driven by check lists. Although there surely has to be some people doing interesting and exploratory game testing?CheersRob..

Comments are closed.