One of my favourite blogs (http://www.experientia.com/blog/) carried an essay/article on humans and machines by Marina Gorbis. It’s a breezy article but makes some awesome points and each point Marina made rang true for what we are seeing in the Testing world.
Marina makes a point that machines are replacing the mechanistic jobs traditionally done by humans, which is allowing humans to concentrate on critical thinking and situational response and other “thinking” activities.
This is true in the Testing world where many people utilise machines to perform the jobs machines are good at, leaving the humans to do exploratory testing, critical thinking and creative endeavours.
I believe Michael Bolton summed this up perfectly when he made the critical distinction between Testing and Checking. Checking could be considered the mechanistic tasks, potentially best performed by a machine. Testing is the human task best performed by a….human.
Marina lists out a number of ways in which Machines are becoming included in everyday work. I’d like to briefly list her ideas out here and draw the parallel to what I see happening in the Testing world.
So what are machines good at?
Machines are awesome at repetitive, mechanistic tasks.
Those tasks that are mechanistic and repeatable. The tasks you do daily by rote are perfect for a machine.
If you can programmatically check something, aim at getting that code written.
So if you spend your day ticking boxes, checking hyperlinks work or simply clicking through the same User Interface over and over again, then you need to think about using a machine to do this.
Machines are awesome at doing the things humans cannot do.
You want to deliver 100,000 requests to a server at one go – Use a machine.
You want to check how well a device works in sub zero temperatures whilst being bombarded with Acid Rain – use a machine. (believe me, someone I know tests products in these conditions)
If you need to “test” something but it’s impossible for a human, then a machine could aid you in achieving your goal.
Machines are awesome at analysing complex data and applying rationality
A quote from Marina’s article :
“For centuries, economists have built models based on the assumption that humans behave as rational economic actors. But thanks to advances in neuroscience and behavioral economics, we’ve come to realize that humans aren’t good at thinking through probabilities and risks and making rational economic choices based on those probabilities. While we don’t want to use pure rationality when making moral or ethical decisions, more rationality would be helpful in situations such as when making financial decisions.”
Should we start using machines to aid us in decision making?
Could we start using machines to inform our decisions about what and where to focus testing? You don’t already?
How about using Pairwise tools to filter down your combinations?
How about using machines to number crunch previous system usage or bug counts to find high risk areas to test?
How about using machines to analyse performance metrics gathered from a Load Test?
Machines are awesome at doing tasks which are too large or too small
Those pesky tasks that are really small or really large could be completed by a machine.
Filling in compliance reports with test data.
Adding all of the change sets to the release notes.
Producing reports on Test Coverage and Results.
So what are humans good at?
Humans are awesome at Thinking
A quote from Marina’s article:
“Thinking and computation are different processes, and machines are not good at thinking”
So we have these machines that can do loads of cool things, but as of today, they cannot really think. Testing is a very human process as we aim to discover more about the system under test.
In order to understand more, we need to peel back layers to discover new insights. These insights require a human to think about them. To understand them. To respond to them. To base new Tests on them.
Humans are awesome at Social and emotional intelligence
Being able to connect to your domain and subject matter, in my opinion, leads to better software testing. I’m a firm believer that a Tester can do great testing in any domain and on any product, but I believe they will do awesome testing when they have an interest and an emotional connection to the product or systems they are testing.
When you are emotionally connected, at any level, I believe your senses are hightended and you become more channelled and focussed. As of today, there are some emotionally aware machines, but they are still in early development.
“Feeling is as complicated as thinking, if not more so, and just as the machines we’re building aren’t thinking machines, the emotional and social robots we’re building aren’t feeling machines—at least not yet.”
Without emotions how can you understand why your end user is finding the UI so frustrating?
How can you connect and build relationships with your colleagues and stakeholders?
How can you begin to understand the contexts in which your customers work?
Do your automated tests design themselves in this way?
Do they have an awareness of feelings?
Humans are awesome at creativity, intuition, and improvisation
A quote from Marina’s article:
“……. the comparative advantage of humans is in doing things spontaneously, responding to unique circumstances of the moment, and making decisions accordingly.”
Do your automated checks respond to a new error and design a new test on the fly?
Do your automated checks create a new hypothesis on spotting a path through the system not previously taken?
Do your automated checks make use of other tools (that you hadn’t told them to use) to test a new path of interest?
Replacing the Tester
A manual tester can be replaced by a machine. They absolutely can. But that wouldn’t be a good strategy if that Tester thinks, connects emotionally and in general, uses their skills, experiences and crucially, their brain.
So you were worried about your job being replaced by a machine? You should be if you do rote checking with no thinking, but not if you do Testing.
A great quote from the essay tells a better story than anything I could cobble together with tenous links and parallels.
There hasn’t yet been a technology that has resulted in our working less. This is because machines don’t just replace what we do, they change the nature of what we do: by extending our capabilities, they set new expectations for what’s possible and create new performance standards and needs.
Although I’m not so sure that sentence is absolutely true, look at how the unemployement rate went up when robots entered the car industry, it does offer a really interesting way to frame the use of machines; as amplifiers of our skills, abilities and approaches.
But does the risk of replacing menial, rote jobs mean we shouldn’t advance the use of machines and robots in more domains, like Testing? I’ll leave that for your own judgment and feelings but it’s clear machines are empowering many to achieve new heights of Testing ability.
I’ll leave you with a great quote from the article, which I would encourage you read.
“The combination of humans partnering with machines and using superior strategies opens up new worlds for exploration.”
Curse you, Lambert. First you post about books that I have to read and now I find an awesome blog site that I have to add to my reader.The risk of replacing menial jobs ? Isn’t there a clue in the word ‘menial’ ?
Hi Phil,I like to keep you flooded with information and cool sources of goodness.There is indeed a clue in the word “menial” but my reason for toning down this post (it was originally a lot more “cutting”) is because I believe the majority of the testing world to be affected by machine replacement. Instead of a culling I think we need something bigger than this….a movement by management to see different value…or maybe a movement by Testers to champion their own value.Thanks for commenting Phil – enjoy the blogRob..
Excellent post, I don’t know if I should choke with raging jealousy at how good it is or merely give up blogging as there is just no point 🙂
Hi Eliza,Thanks for the nice words and glad you enjoyed it. Keep blogging!!!! Please.ThanksRob..
Hi Lambert!Just one remark. In order to implement checking you need a skillful programmer with brain and ability to think. In any level of testing, from unit test up to system test.
Important points, covering and extending my mantra: “humans are good at: understanding what is important; judgment; dealing with the unknown; separating right from wrong.”However, I have to object to “Machines are awesome at analysing complex data and applying rationality”Machines are awesome at computing complex data, and applying statistical models.After that, you want a human that understand details, that can judge what the numbers really mean, and that can make good decisions (using both rationality and feelings.)I have no experience of your first example – where to focus testing – and don’t really understand how that could work well.For filtering down combinations, I always prefer importance, error-proneness, best representatives, over Pairwise.When using computations to find risky areas, I never get better information than the testing team can say at once (although the combination is nice.)For analysing performance metrics I use visual analytics, i.e. software that display the results, that are analyzed and acted upon by a human.The quote in the same section dubiously imply that financial decisions never have moral/ethical considerations…
Rob, First off, thanks for linking to this. I now have a new subscription waiting to be read in Reader. However, I don’t agree with your point that automation in the car industry lead to the unemployment trends that you linked to. Taking a deeper dive into the article you linked as well as some supporting documentation linked from there ( http://en.wikipedia.org/wiki/Relationship_of_automation_to_unemployment ) actually states the opposite. That the number of available jobs increased due to the freedom of time from not having to spend on repeatable labor and the increased spending power of people leading to the creation of new jobs for products that weren’t possible before.Just like the essay talked about the fact that before the vacuum cleaner was invented it was OK to have dusty floors. How many jobs did that industry create on its own? It didn’t even replace any workers.However, the biggest section of that essay to me was the final one. Marina gave the example of the Playchess.com contest. It wasn’t the superior humans + computers that won it, rather the superior process. I think this was the biggest takeaway for me from a testing standpoint.
Hi Rob, great post, as always.I think you hit the nail on the head with your response to Phil – “or maybe a movement by Testers to champion their own value.”Some of the Testers I come into contact with appear to be quite happy with their manual checking – perhaps a more cutting article would them realise their jobs (as they know them) are potentially at risk.But that said, if they’re happy with manual testing, they probably wont be reading around their chosen subject & thus wont be reading this post! (a nice sweeping generalisation there – might spark a reaction…)so, could it be that Managers looking for different value start pointing Testers towards the greater testing community (if they themselves are aware of it)?
Hi Karlo,Thanks for the comment. Agree you need skilled programmers but I think there are also a number of tools someone can use that don’t need any programming experience.ThanksRob
Hi Rikard,Thanks for your comments. I didn’t elaborate too much in the post as I have some further posts coming up, but you are right, there are other ways to deduce risk and to filter your testing. I was responding directly to comments in the referenced blog.In finding problem areas I often use defect tracking tools to highlight high defect areas and to find patterns. But yes, a human still needs to assimilate and accommodate that information and then make decisions.An experienced test team with knowledge of the product and the domain is always the best source of knowledge for decisions, but this isn’t always a luxury many testers have. For example, a new test team with a vast product full of legacy quirks example…a tool *can* be helpful in these situations.Insightful comments as usual and thanks for taking the time to comment. Some good points for me to chew over.ThanksRob..
Hi Justin,Thanks for commenting. You are spot on with that article reference and that source of information. It depends how you look at it. It destroyed the jobs for those with only the skills to put cars together hence a massive decline in the industrial style jobs. It did indeed create a number of different jobs which I guess in a sense I skirted around in the blog post.I basically didn’t want to say “If you are doing industrial, mechanical testing then you will have no job, but if you have the skills to “test” then you will be ok”It creates different jobs. Tricky to say what they will be. Even trickier when many companies still approach testing as a rote, mechanical job. I do like that take-away also. The process and the mindset is more important. Great stuff.Many thanks for commenting.Rob..
Hi Duncan,Thanks for the comments.It’s a tricky one because you are spot on. The majority of people who do rote, mechanical testing probably aren’t reading this blog. There are also a huge number of testers who seem quite happy with checking. So when I get all “frustrated” about the testing world I need to tone my thinking down because there are many contexts where checking “appears” to be working. I don’t believe a Test Managers role is to “Manage”. I think it is to “Lead”. And I believe that people who see themselves more as Leaders or Enablers or Encouragers will help their team achieve great things. Just some thoughts. Thanks for commenting and I think you are spot on. Managers themselves (and every tester) needs to be aware of what’s going on in the wider community. But how do we encourage them to do so?Always a tough questions.ThanksRob..
“I don’t believe a Test Managers role is to “Manage”. I think it is to “Lead”. And I believe that people who see themselves more as Leaders or Enablers or Encouragers will help their team achieve great things. “That is bang on the money – As adults with families & financial commitments, I like to think we’re professional & responsible enough to manage ourselves, get the job done & earn our crust.I nkow that when i’m feel that I’m being “managed”, it really saps my drive & motivation
Managers themselves (and every tester) needs to be aware of what’s going on in the wider community.We need to keep spreading the word & get Testers hooked on testing – enable them to see its far more than following scripts.It only takes 1 Tester on a team to bring awareness of the wider testing community – you’d hope a Manager would latch on to this & nuture that development…
Hi Duncan,I think you’re spot on. Get one Tester fired up and motivated about better ways to do something and the rest will follow. We, as a community, need to keep motivating people and offering insightful ideas for people to try. It’s all about finding the angle of Testing that ignites someones passion. Thanks for commenting.Rob..