Specification by Example – book review

I’ve just finished reading Specification By Example. Sorry Gojko for taking so long!

Specification by Example is an interesting read. I was going to paraphrase what it’s about but it’s best to come straight from the source:

“Specification by Example is a set of process patterns that helps teams build the right software product. With Specification by Example, teams write just enough documenta- tion to facilitate change effectively in short iterations or in flow-based development.”

I really like the idea of Living Documentation. It’s an interesting idea which I believe could solve a number of communication problems in many businesses.

As Gojko says:

“Living documentation is a source of information about system functionality that’s as reliable as programming language code but much easier to access and understand.”

The book is incredibly easy to read and is split in to many different sections, each one building on the idea of Living Documentation, and each one accompanied by a number of quotes from people implenting these ideas.

The book has a compelling introduction which sets the scene well before moving through the various stages of thinking about Specifying through examples:

  • Key Benefits
  • Key Process Patterns
  • Living Documentation
  • Initiating the Changes
  • Deriving Scope From Goals
  • Specifying Collaboratively
  • Illustrating using examples
  • Refining the specification
  • Automating validation without changing specifications
  • Validating Frequency
  • Evolving a documentation system
  • Case Studies
  • Appendix

Each section builds on the idea of specifying using examples, but there were a couple of standout chapters for me, particularly Specifying Collaboratively and Validating Frequently. I enjoyed these sections because they hit a chord with my own thinking and some of the challenges I’ve seen in organising automation (and story chats).

It’s also very interesting reading about the companies Gojko interviewed in the Case Studies section, especially when these companies have incredible well respected people offering these insights.

I took away a huge amount from this book. It was especially well timed as we embark on a more “specification by example” approach to cover some of our automation here. It’s good to have some examples of approaches for legacy products to refer to when we try it.

Sometimes books only concentrate on the greenfield approach, but here Gojko has added lots of advice for people in differing contexts.

Although the book is really aimed at people looking to automate using examples I still think this book would be useful for others who have a general interest in automation. Some of the hints and tips I took from the book were more agile process related too, so I think there could be something for everyone in it.

http://specificationbyexample.com/

Estimations, Approximations and a Million Dollars

I’m loving this YouTube channel explaining cool and interesting scientific stuff : Minute Physics

I especially liked this video on how to weigh a million dollars with your mind. It struck a chord with me because it reminded me of how we sometimes come up with estimation for unknown future features or capabilities.

[youtube http://www.youtube.com/watch?v=-zexOIGlrFo?wmode=transparent]

 

Good form design – ELMER

I’m a huge fan of keeping things simple, usable and accessible when it comes to developing stuff (if possible).

A cool source of ideas for building usable and accessible “forms” is the ELMER guidelines. These guidelines are aimed at those building public sector forms, but I think the guidelines are good for anyone having to build forms.

Here’s a little from the website:

“Simplification of public forms is important to improve communication between the users and the public sector. The proceeding transition to electronic reporting may be an important simplification measure for the respondents, but only if the Internet-based solutions are felt to be more user friendly than the old paper versions. By applying good pedagogical principles, electronic forms may also ensure a better understanding of the task, better control of the data before submission, and by that even better response quality and more efficient processing by the relevant authority.”

The guidelines aren’t that tough to read and they make a lot of sense. The forms that we ended up with when building against these guidelines just flowed well and felt much better than the original prototypes we’d built.

Here’s some of the “stand-out” ideas I took from the guidelines, which helped to build much better forms with greater usability and accessibility. I must also add that these excerpts below were taken from the guidelines about a year ago (I know…. my draft list of posts is too large). I’ve checked a few of them against the current guidelines and they are still the same, but some might have changed in the last year. Another document to re-read 🙂

“The page order in forms must be locked where:

   1) the order is significant with regard to response interpretation and quality, or
   2) the order will depend on responses given on previous pages.”

“User-requested help must appear when the user clicks on a standardised help icon in
connection with a question. The user can make the help text disappear from the
screen by clicking again on the same symbol, and must be replaced by a different text
if the user clicks on the icon connected to a different question”

“In forms where a significant number of questions are irrelevant for specific form filler
groups , or where different form filler groups shall complete significantly different

sets of questions, different tracks must be developed. Each individual user shall be

directed to the relevant track. Several tracks may consist of identical pages/question

sequences”

“Both labels and input fields for response dependent questions must be greyed on the
form pages and only be opened for completion if previous answers indicate that they
are relevant”

“Each individual page should be delimited with a view to avoid an unreasonably long
download time. The download time is affected by such factors as graphics use, the
amount and type of controls and the number of fields.”

“If there are several interim sums which are not simultaneously visible to the user, a
separate summations group should be created at the bottom of the page. If the interim

sums are located on different pages, they should be transferred to a summary page”

“Fonts, font sizes, colours and other graphic elements, must be used consistently and
uniformly in all forms issued by the same inquirer. The forms must differentiate

clearly between various types of elements (headings, category headings, labels, error

messages and warnings, etc.). As a general rule, sans serif fonts should be used and

colours should provide good contrast.”

Links to information that is irrelevant to the form completion should be avoided. “
“The readability of help texts should be increased through use of typographic means.
Except for very brief phrases, the texts should be broken into a series of easily read-

able chunks with highlighted headings and keywords.”

“In cases of incorrect completion of individual fields, an error message must appear
automatically as soon as possible after the error has occurred. The text must appear in
the information area and the relevant field must be clearly marked. Messages presented
in a separate window (dialogue boxes or pop-ups) must not be used”

 

As you can see, there are some interesting ideas that might work for you.

Patterns of Design

I stumbled across this excellent site of mobile phone UI patterns. It’s a great collection. I think it’s interesting because we can start to see trends and this library will give us clues as to how the designs for mobile phones is changing.

I’m not entirely sure what this has to do with Testing, but I just thought it was worth sharing.

Mobileui

Volunteering – a good way to learn Testing

I often get asked by people new to Software Testing what the best approach to learning more about Testing is.

Is it certification, books, blogs or other courses? These are the usual categories that get listed.

I very rarely hear people ask whether “practice” is a good approach to learning. I think this stems from many people thinking that “practice” and “work” are the same thing. That one cannot exist without the other.

How can I get experience in Testing unless I have a job Testing?

I think this view is out-dated. I practice my testing outside of work. I practice my testing inside of work. If you want to get good at something..practice.

Over the last year I’ve been working with a great organisation who are building wonderful technology, mainly aimed at people without access to the basic infrastructure that many people take for granted. It’s got a great user base and is making a positive change to many people’s lives.

Over the last year I have been trying to organise some Testing for this organisation. The group of Testers I’d gathered are great; a mixture of domain experience, general testing experience and a few new to the scene.

Last week it all came together and hopefully this year we’ll starting adding a load of value for our client. And all of this is free, outside of work and purely voluntary.

My stock response for the last few years to the question “How can I learn more about Testing?” has been “Volunteer your time”.

3859852439_ca615b8876_z

There are so many organisations out there building software that *could* benefit from your enthusiasm and skills.

You’ll potentially learn loads about testing, working with people, time management, commitments, decision making, technology, reporting/articulating your testing and how to work within constraints.

It’s all great for your CV.

So how to do it?

  • Find an organisation that you feel some synergy towards.
  • Get in touch with them and volunteer your time and energy.
  • Be sure to set expectations about levels of testing, time and experience you are offering.
  • If you need a more experienced person to help you get it started then head to The Software Testing Club and ask for help, or drop me a line.
  • Persevere – communication can often be tough, expectations may need aligning and relationships will need to be built.
  • Commit and deliver on your promise.
  • Ask for help in the community if you get stuck or you realise there is too much work for you.
  • Document and audit your learning. (any good interviewer will ask you about your experience)

There are so many organisations, open source projects, charities and not for profits that would benefit from some volunteer help….so what are you waiting for?

Image courtesy of wblj – http://www.flickr.com/photos/imageclectic/

Visualising Changes in Your Product

There are so many ways of working out what is changing in your product. As a Tester I look for this information from any source. It helps me filter my test ideas.

One of the tools we’ve been exploring recently here at NewVoiceMedia is a cool tool called Gource. One of our programmers Wyndham has been experimenting with it and been tempting me with the insights this tool could offer our Testers.

There simply isn’t time to test everything so any technique or information source that will help me filter and target my Testing is very welcome.

Gource is a tool that “visualises” SVN check-ins. We can visualise where the check-ins are happening, with what frequency and with what intensity. We can get access to check-in information anyway, but what this tool gives you is a timeline of change and that all important visualisation.

Here’s a cool vid of Gource in action for Flickr:

 

http://www.vimeo.com/11876335 w=400&h=300

Flickr/Gource from Daniel Bogan on Vimeo.

So we’ve been playing with this and immediately I can see areas of code check-ins not immediately obvious from story cards or other workflow tools.

We’re going to be exploring this tool more over the next few months and seeing how well we are finding the insights it can give.

I’ve also been having crazy ideas about how I can use the SVN checkin process and Gource for visualising where we are targetting our exploratory testing. We could see how much testing is going on around which component…..but I digress. I’ll explore that and let you know.

Here’s Wyndham blog on how to get Gource set up and running.

 

Observing to help our Testing

Whilst on the train a few months back I spent some time observing how people were using technology.

Some were using the tech as I assume it was intended, some were “street hacking” the products, whilst others had adopted unique ways of utilising technology (and other devices) to fit the context they found themselves in.

I often wonder how effective testing can be (and overall design and development) when we have little insight or understanding of how our applications/systems are used in everyday life.

We can make sure it’s “functionally complete” or “performant” or “meeting requirements” but that doesn’t mean we’ve helped to create a great product right for the audience it’s aimed at.

I think Testers can help greatly in this respect, either by simply asking questions at all stages of the design and build process but also by getting out and seeing end users in situ (if you can).

Many of us assume, predict or speculate how our end users are using the software, it’s often all we can do, but I wonder how much of this speculation is accurate. I often hear it said that Testers “pretend” to be the end user, which is fine if you know who your end user is, and in what context they will use your product but this approach should never be used as an absolute point of view.

No-one will ever be able to use your applications in exactly the same way as your real end users, but we can certainly work hard to get close to it.

Here are some of the things I observed:

  • Several people were using multiple mobile phone handsets. Work and Play? Feature versus Feature? Or simply taking advantage of the best tariff? 
    • What about data on each phone? Did it need to be synched?
  • One person was swapping sim-cards in and out of one handset to make use of the best tariffs.
  • Two people were using a technique called “flashing” or “beeping” to communicate with someone else via mobile phone. (flashing is where you ring a phone and then hang up before they answer. There are all sorts of social norms growing around this practice – good article here: http://mobileactive.org/mobile-phones-beeping-call-me)
  • One person was trying desperately to get a good photo out of the window on his mobile phone whilst the train was moving. I’m not sure he was happy with any of them. Misplaced expectations?
  • One lady was using a USB expansion device blue-tacked to her laptop lid. I “assume” she was doing this to expand the capacity and/or maybe to reduce the overall width of the machine (the two USBs she was extending were bigger than the expansion USB itself). Street hack? Do we need to test any usages like this (i.e. overloading original intention)
  • The person next to me complained that his phone was running out of power too quickly. He’d been playing Angry Birds for the entire journey whilst using the phone to play music.
    • Expectations of battery life needing to be longer to support modern multi-usage?
    • How did both apps perform?
    • Which one used the most power?
    • Should we be thinking about how much power our apps consume on mobile devices?
    • Can we realistically even measure it?
    • Should we rely on the underlying platform to manage such usages?
  • Many of us live in an age of “smart phones”. How does this “one for all” device fit with other electronic devices like eBook readers, Tablets, Laptops etc
    • Can we read something on a Kindle and start reading where we left off on an iPad at home or Laptop at work? 
    • Can we seamlessly share information between devices?
    • How is our information distributed, stored, secured, managed, protected? Do we care?
  • Another lady caused minor fits of giggles as she took out her “early designed” portable DVD player. It was pretty huge by modern standards. She proceeded to play episodes of Friends on it as she slept for the journey. Old tech and usages is a major challenge for many Testers.
    • At what point do we stop supporting old tech or old versions of our software?
    • Do we need to test all versions?
    • Are there techniques for testing many platforms/versions at once?
    • At what point does this old tech stop being “acceptable” in society….which society?
    • How do we know what versions our customers use?
  • One man was reading his Kindle on the train. He had created a hook for his Kindle. The hook was made from a coat hanger. This hook was suspended around the ceiling hand rail so he could essentially suspend the Kindle at eye height. At the bottom of this bizarre Kindle concoction was a ribbon so he could stop the Kindle swinging around with one of his hands. Street Hack?
    • Could we have ever envisaged it being used this way?
    • Would it have changed the way we tested or designed or implemented?
  • Whilst on the train it is inevitable that the phone signal will come and go. With bridges, buildings and shady coverage it would be ambitious to assume total connection for extended periods. Yet I observed a number of people “hating” (i.e. swearing and getting annoyed with) their devices when they lost connectivity.
    • Two people physically threw their phones on the tray table when they lost a voice signal. Others uttered offensive and angry rants at their devices.
    • Yet the problem existed outside of the device or software in use; it existed in the infrastructure. Are expectations outgrowing reality?
    • The infrastructure no longer supporting the norm and expectations of device usage. This is an interesting challenge for those creating devices relying on technology infrastructure like broadband and mobile networking.
    • What’s the minimum connectivity? What’s the risk of it going down? What happens if it does go down? Does it recover? Do we lose data? Does it matter?
  • One person was drinking heavily whilst attempting to use a Smart Phone keyboard. I won’t repeat what he said, but he struggled to type his SMS. One interesting point he made was that he used to be able to type whilst drunk on his old phone. Loss of capability? Extreme contexts? Trustworthy observation?
  • Two people were “chatting” to each other via their mobile phones (maybe Skype or other chat system). They were sat opposite each other.
    • Are cultural changes in communication reflected in your products?
  • I was sat in a quiet carriage. As usual there were a few complaints and confrontations about noise. One of which was about the noise being made by the keyboard of someone’s laptop.
    • At what point do we pass social thresholds of acceptability? Do we need to do more Inclusive Design?
    • Should we consider more extreme contexts when testing or write these off as edge cases?
    • Could we ever imagine all of the contexts our products may be used in?
  • I kept turning on my wireless hotspot on my phone to sync my field notes (Evernote) as the train stopped at each station. This raises questions of synchronisation, offline working issues, data storage on the cloud and a whole host of privacy issues. There are some interesting examples of how people are using tools like Evernote in the wild.
    • Do you have stories of how people are using your applications?
    • Do you actively seek stories from end users?
    • Do you use this data to build user profiles and system expectations?

Does this really have anything to do with Testing?

There are a few people starting to talk about social sciences in Testing and I believe the application of many areas of social science are going to grow and diversify as Testers seek to find out more about people and technology and how our products can find the right audience(s).

Social sciences can give us a deep insight to ourselves, culture, mass media, communication, research, language and a whole lot more.

Observing people is just one element of research. Research will give you insights, clues and information about the things you research.

Observations and research will help you to make decisions on what to focus on, and what to overlook.

Testers are also natural skeptics; we should help to experiment, prototype and challenge simple assumptions and social categorisation (i.e. all young people use Twitter, wear hoodies and listen to hip-hop) *

Observing people in every-day life is often the biggest eye opener for any Tester wanting to learn more about people and tech.

Why not have a go? Focus on the world around you. I bet you’ll see stuff you never saw before. I bet you’ll see people using tech in ways you’d never seen before. I bet you’ll learn something new.

The challenge is how you can bring what you learn to your Testing.

As usual, if you have a go, let me know what you think.

* This is a real generalisation I heard someone say at a technology, culture and local council meeting!

The myth of the standard tester

I always tell candidates looking for a job to have a standard CV, but to never send this CV to anyone. Never.

The ‘Standard CV” is a starting point for a contextually specific CV that gets the hiring manager giddy with excitement about speaking to you.

Yet the other way around, it’s very common for companies to have a “standard job spec” for most roles.

We have another Testing role available, someone send me the Tester’s job spec please!

This standard spec gets rolled out whenever there is a new position available or an existing employee leaves. It’s got some generic stuff on it and it’s pretty vague about everything. It asks for a broad range of skills (cos we want a wide pool of people) and doesn’t specify much in the way of actual expectations or outcomes.

It contains lots of buzzwords and leaves the door open for the widest selection of candidates the company can attract.

I think this is sad.

I think this is sad for two reasons:

1. If your business is that generic (i.e. that anyone can fill a testing role) then I suspect you’re solving the wrong problem by hiring generic people.

2. I don’t believe any two Testers are ever the same, so why should the roles they fulfil be?

As your business moves and grows (or shrinks), your products mature or you start to move to new technologies, techniques or processes it is inevitable that the roles you need will change also.

One of the major challenges a manager may face is ensuring that your teams skills and goals are in alignment with the business needs, and that your teams skills can/will change in line with your business.

I think that each team needs diversity of mindsets, skills and experience. I think each business needs different Testing skills at different times. A great team will acknowledge this and flex and grow with the business. Standard test teams won’t.

I think that all Test Managers who hire from a “standard” job spec do so because they know no different (or HR are forcing their hand). I did this for a few months and spent that entire time griping about how few good candidates there are. I was casting the net too wide and wasn’t 100% sure who I was looking for). I then narrowed and recruited an excellent team (If I say so myself 🙂 )

  • Do we just need a “standard” tester (whatever that may be)?
  • Or someone with Exploratory Testing skills?
  • Automation skills?
  • Database Testing skills?
  • Security Testing skills?
  • Web Testing skills?
  • Messaging skills?
  • Email and multimedia skills?
  • Specialist domain knowledge?
  • Managerial skills?
  • Leadership skills?
  • People skills?

I could go on for ever.

Why do we try to cast the net wide and grab the most generic person we can find when deep down inside we want someone with some specialisms, or a certain outlook on life, or certain skills and abilities?

  • Is it because it’s hard work initially to write new job specs and think long and hard about the actual objectives we want this person to achieve? Yep. Probably.
  • Is it because this process will take a longer time than the generic advert? Yep. It will do, but it will be cheaper in the long run.
  • Is it because recruiters are asking for generic job specs? Maybe. But believe me, there are some great recruitment consultants out there.
  • Is it because it’s always been done this way? Yep. Absolutely. But why should that stop you changing it?
  • Is it because I may have to get creative about the hiring process and work hard to find someone right, rather than someone who *might* be right? Yep. But that’s your job right?

So here’s what I believe anyone hiring for a tester (or building a new team) should/could/might do:

  • Define the personality traits of the person you want to hire.
  • Define the actual skills that person will need.
  • Define the activities they will do.
  • Define the outcomes you expect that person to achieve.
  • Write a new job spec for each position.
  • Screen, screen and screen – face-to-face interviews are expensive for everyone involved.
  • Interview based on the things you’ve defined above. Test them. Show them around. Invite them to meet other people. Ask varied questions. Tough questions. Be Open and Honest; make it clear what the objectives of the role are.
  • Be subjective in your decision making.
  • Be Open Minded and willing the flex on some things.
  • Be hardline about the things that really and truly matter.
  • Do not discriminate.
  • Get a second opinion.
  • Read Cem Kaner’s excellent interviewing article (www.kaner.com/pdfs/QWjobs.pdf).

Offer to the right person and you will have no regrets. Offer to a generic person, for a generic role and I suspect you’ll have your work cut out in the months and years that follow.

Hiring the right person is going to take much longer but in the long run, but it’s a price worth paying.