The Future of Software Testing
I’ve been thinking recently about the future of software testing.
I’ve been wondering why some people are still testing the same way they did 10 years ago and why others are trying new ways, pushing boundaries or at least experimenting slightly. I’ve been wondering why some people are complaining about the way testing is happening, yet are doing nothing about it. I’ve been wondering why we aren’t, as a community, pooling resources more than we currently are. Why we aren’t open sourcing our learning or collaborating more on solving tricky testing problems.
The Business world is moving fast and we need to move with them. We need to remain relevant. We need to add value, not hinder delivery.
This post isn’t about them and us. Or Agile versus Traditional. Or scripted versus Exploratory. It’s about testing. It’s about what challenges face testing now and in the next few years.
Before I list my thoughts though I will add the simple caveat that these are my *feelings* about what will happen. You may not feel the same. You may agree / disagree / not care either way. This list is not scientific. It’s not complete. Some or all of it may not come true or be relevant or even right. I can’t predict the future, but I can lay down some thoughts to start a conversation.
An Observation
One thing I have noticed over the last few years is that we seem incredibly reluctant in many corners of the Testing community to talk about the future of testing. We *appear* to be stuck in the past. We don’t want to discuss the future (or the present for some), or maybe we feel we don’t need to talk about it.
Maybe those that care are busy getting on with it, forging the future and moving on; always pushing the boundaries, not looking back. Maybe some people don’t even see any challenges to their cosy existence and can quite comfortably meander on with retirement in mind. Maybe some simply don’t have the engagement in testing to care.
We seem reluctant to experiment, to day-dream, to try things out and to open our minds to what the future may hold. We close ourselves in our testing box and berate anyone who tries to change that. I wonder whether people are clinging to the past because they are worried that any changes we make might not work; that we might not be better off than where we are now? We’ll never know unless we try though.
But I firmly believe that the world is changing fast around us. The way we interact, communicate, do business, use and create software is changing rapidly. Most apps are now web only, many are cloud hosted, some are purely mobile apps and almost all of them are being built and released in an increasingly fast moving market. The streets and our cities are rapidly becoming technology platforms in themselves. There is a bewildering array of devices to support, all with different quirks and nuances. For many people, we have no single demographically identifiable “end user” so our test combinations and usage patterns are in the gazillions.
Yet we seem reluctant to ask where we fit in to this future. We seem reluctant to experiment with ideas, tools and techniques en masse. Very few people are having sensible conversations about how to push/pull testing forward to meet the demands of businesses. Fewer people still are even aware such change could be coming.
For such a big question as “how do we fit in to the future?” I think we need new ideas, we need to make new mistakes, do big experiments, create new tools, collaborate and discuss, but ultimately start to make changes. Big changes. Small changes. Some changes…And share our findings.
So what will face us in the future………………..?
User Experience, Accessibility and Adoption of Tech in Society
Pretty much everything is moving online or to the digital world (certainly in the markets I work in). The rates of online adoption vary depending on who you speak to, but there is no doubt our worlds are becoming focussed around communication via mobile networks and the Internet.
More and more products are building in social channels and wanting to utilise the benefits of multi channel communication (the channels and platforms themselves will come and go, but the concept is here to stay and evolve). It might not be now, or 2 years or even 5 years, but it will happen.
With more online adoption though we run the risk of marginalizing people who are unable to interact with the web in an efficient and effective way. Physical and cognitive disabilities, user interaction problems and mindset changes *could* make the adoption of products and applications a hurdle for many in society.
Testers can play a huge role in identifying these pitfalls, championing good design and accessible sites/applications as well as figuring out how to test a bewilldering array of devices and platforms, for a bewildering cross section of users in a bewildering series of contexts and uses.
Search and Discovery
Search and discovery for personal growth is the most neglected element of testing. I’m not talking about the Search & Discovery using Google to find your answer, then copying it, pasting it and passing it off as your own.
I’m talking about Search and Discovery as a way to learn, self improve and grow our skills and experience. This is a key skill neglected on almost every training course available to testers (there are thankfully a few exceptions). To an extent many rely heavily on certifications and our day jobs for information, learning and knowledge. Not so bad if your day job challenges you…and you like certifications.
Many testers haven’t heard of the thought leaders in the community, some testers are only at conferences because they’ve been told to be (this really surprised me) and many see testing as a lower grade career choice or a stepping stone to programming/development.
Many testers are becoming so ingrained in big vendor tools that a conversation around testing can’t happen unless it’s about the tool.
Many testers have no awareness (or acknowledgement) of Exploratory Testing, Accessibility, Acceptance Test Drive Development, Security Testing or UX. Many don’t read trade publications or blogs, join local user groups or read books on testing. Some may never need to know more than they know now…but how do they know that?
Testing isn’t a career for many, it’s a job. For those of us serious about testing it represents a massive challenge.
How do we convince people to do search & discovery? To self teach? To teach others? To mentor? To be a mentor? To build their awareness fields?
How do we encourage people to make testing a career and not just a job? Or how do we separate the test heads and lifers from the people who just want to get paid? How do we further our craft without reliance on standardisation techniques and exams? How do we offer companies and markets the right people, for the right job with the right mindset?
I believe we can do more. I believe we can provide a cheap (or free) worldwide learning network to encourage growth and development for the mainstream. I believe we can offer those who want to learn a safe and welcoming environment to express their opinions and share knowledge with more like minded people.
There are courses and learning resources available but they aren’t mainstream. Maybe it’s best we keep it that way. It might ruin what we have. Surely not?
There are some great examples of learning sources here
Communication
Those that have followed me over the last few years know how strongly I feel about communication skills for testers. It’s my number 1 requirement for any tester joining my team. I firmly believe a tester needs to be able to articulate their views on testing. They need to be able to persuade, to be assertive, to be a good listener and to be able to communicate their findings in clear and conscise ways.
Yet communication skills are one of those soft skills that many believe aren’t essential. Most people have over inflated views of their own abilities which results in zero time spent on improving their communication. Most people aren’t self aware about their communication in the first place. Being a good communicator is one of the most powerful skills you can learn.
I feel we need more people writing, presenting and talking about testing. The future of testing will need “test champions“; people who can talk about testing to a variety of audiences, using a variety of mediums extolling the virtues of great testing.
Remember – being able to find a bug is great, but not so great if you can’t describe it, explain how you find bugs and pursuade someone to take your views seriously. A good communicator will go far. Very far.
Stereotypes
The future of testing needs us to break the stereotype of testers – one of the most prolific is the Checklister
We don’t need checklists to work.
We can fit in to agile teams.
We are flexible and dynamic with our test strategies.
We don’t run tedious, boring, repetitive tests.
We can do……
We are more than that. Aren’t we?
Communities
Our sense of community and our need for places to hang out with like minded people are essential to our future growth. It’s why we do so much work at The Software Testing Club to make people feel welcome. It’s essential for a culture of learning and sharing to grow.
The barriers of entry to any testing community are now very low. You don’t even have to take part if you don’t want to (simply listen to the conversations), but I’d encourage more people to join any testing community that “feels” right. Sharing knowledge and experience is a great way to learn, it’s also a great way to build ties with like minded people. It’s also a great way to have a global and collaborative conversation about the future of testing or your everyday testing problems, or just a chat. It’s a great way of belonging. But be careful, it’s also the stomping ground of many “Best Practice, my way or no way” testers.
I think the challenge for the future is encouraging more people to join these communities and to share their views and opinions in a constructive manner.
Agile
We all know small iterations of work, with regular and rapid feedback is a good thing. We can call this agile if you like (or common sense software development)or a flavour of iterative, but Testers continually struggle to see how they fit in to this process. I see it as logical move, but the future will hold many challenges for many testers who need to drop the “long haul planning” mindset for the “short term releasing” mindset.
Oh yeah, and coupled with this is the inevitable question about automation of tests. Agile is sure to continue to scare some people, fox others and liberate others. Some people love it. Some people hate it (even though they may not have tried it). The question is “how can companies remain competitive to changing markets unless they adopt some sort of shortened iterative development cycle? And if they must adopt shorter releases schedules to market, how can testers (and the team as a whole) help them?
Above are some of my thoughts on testing and the challenges we may face. What are your thoughts on the future of testing?
More crowdsourcing? More certification? Even bigger divides in testing domains (heavy structured, highly scripted, agile, exploratory, schools of testing)? What about our Managers and leaders of the future? Do they need new skills?
Feel free to let me know.
As a wise philosopher once said “Mongo not know… Mongo just pawn in game of life.”Rob,I would love to give my my deep thoughts and musings for this, but right now I’m just like a lot of other people trying to stay working and making a paycheck. I used to have this passion, but have had it partially beaten out of my (after awhile your head starts to bleed from pounding on the brick wall) and have focus on other things in my life (my wife and daughter).I guess I was born at the wrong time, fifteen years ago I was just like you in this regard. I’ve done the good fight (more times than I want to admit), but in the end dealing with a bunch of people who don’t care (until it hits them in the pocketbook) has taken the fire out of me.So I agree with all your points (I even did a presentation at STPCon 2010 about “Selling Testing to Them”), and I support you and wish you the best of luck in this endeavour.Regards,Jim HazenVeteran of the Software Testing Trenches
Rob,Good post. One additional prediction. In the future, many more testers will have a deeper awareness and more appreciation of: (1) structured variation in test cases, (2) the applicability of Design of Experiments methods (very common already in other industries) to selecting test cases, and (3) the critical importance of learning new test design approaches throughout a tester’s career. I guess that makes me more of an optimist than Jim. :)- Justin
Great post Rob. In my opinion most of what you’ve mentioned here *is required* now and in my experience I could already see it happening with a very small section of people (testers). But as you said *most* are still taking it as a money-making job rather than a career. And some who have taken it as a career are still old school. Hoping to see the change happen soon.
Hi Rob,I like the following quote from Max Planck used in Kuhn’s book, The Structure of Scientific Revolutions (see http://en.wikipedia.org/wiki/Paradigm_shift#Kuhnian_paradigm_shifts)Sometimes the convincing force is just time itself and the human toll it takes, Kuhn said, using a quote from Max Planck: “a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.”I think that the same holds for testing and agile development. I think that the Context Driven School of Testing is the /new science/ of testing. It is almost impossible to convince managers but also testers who grew up with the traditionalist way of testing (ISTQB and TMap) of the benefits of CDT. Time is ticking…Initiatives like The Software Testing Club, The Swedish Workshop on Exploratory Testing (SWET) and recently The Dutch Exploratory Workshop on Testing (DEWT) are examples of a changing testing landscape.- Ruud Cox
Hi Jim,Man, it’s hard to hear your words. I know how you are feeling though, keeping the pay check going and looking out for the family. It’s core to most people in the industry.It’s a shame you’ve been beaten down though cos change needs the experienced testers like yourself on board. The people to help guide and influence and direct.Thanks for the morale support. It’s nice to get feedback that I’m banging my head against the right wall :)ThanksRob
Hi Justin,Thanks for commenting. Absolutely. There’s a huge amount of of learning to be done from other disciplines and especially so when it comes to combination and pairing tests and testing.CheersRob
Hi Farid,Many thanks for commenting. I too see it as the current day challenge but unfortunately I believe we are in the minority. That is a scary thought. But I see light and I see change and I’m optimistic we can help businesses get value from us rather than road blocks and out of date thinking.ThanksRob
Hi Ruud,Good to hear from you as always. Great quote. I read that a while back and couldn’t help drawing the same conclusion. Eventually time will change the norm and we *should* be able to rejoice in the new norms. Then the next set of challengers will come along and complain about us :)Rob..
Rob,Yeah, sucks having to be in this mode. I just hope that you are wearing a padded helmet when you’re banging your head on the wall.A brief self history for folks here. I started in the Software Industry in 1987 as a programmer. After about a year of doing maintenance work with a lot of debugging and semi-formal testing (it wasn’t totally shotgun style) I shifted over to the newly formed Test group in my company. I was the Right Hand man of the Test Manager. We formalized and built up the testing group. That was 1988, and a light went on in my head that I actually liked Testing and that I could make a career out of it. It was my niche.At that time in the PC Software world (DOS) the idea of Testing as a seperate function/group (let alone a career) was even more strange than today (24 years later). I was lucky in that my manager was encouraging and understood this type of work (he had done it before, not a newbie). So Test group startup number 1.Next I went to work for a company doing the same thing, starting up the test group and acting as the Right Hand man. The company/environment was very supportive (they hired me and the manager to start the thing) and we grew from 2-3 of us to 60+ in one year. It was crazy. But again I had a great manager who had lots of experience and he was a great mentor. Test group startup number 2.After that place I bounced around to a couple of companies and was in Test groups that were already established. Both of which were not in good shape, and management saw us as a burden and not as helpful. That was 6 years of getting beaten over the head for doing my job.Next I went to a company to do the startup thing again, and again as the 2nd man in the door. The manager there I knew from a previous company and he was just as good as manager at company #2 above. We got lots of support from the management (like full support) and other teams (especially Development, surprise), and we did some great work. Again the manager was a great mentor, and now long time friend. But we got bought out and things changed, so I left. Test group startup #3This time it was me as the Test Manager, and I gave it my best shot but I kept running into a VP who thought test was a waste of time & money. After about 18 months I said “C’ya!” Test group startup #4Then I went into consulting work for a couple of consulting companies, and watched as others in the test group were getting beat up. Sometimes I tried to help those poor souls, and sometimes it worked. Moral of this long and boring story is that you need to be in a company that “want’s testing to succeed”, and have some level of respect from your peers in other groups. That is what we need, and need to promote. Mutual Respect.So yes, we need to promote our work & its benefits and to communicate effectively with ‘other’ groups. But we need that environment of mutual respect and desire to do a good job that is necessary for our work to flourish. Without it we are in a world of hurt.Unfortunately I’ve only seen that at 3 of the companies (of more than I want to admit) I’ve been at or consulted to over the years.Finally, one of the most effective things I’ve found (taught) to use to get peoples attention (especially Sr. Mgmt.) is to put it all in money terms. Meaning how does testing affect both the Hard Dollar (revenue, P&L) and Soft Dollar (reputation, rework, customer satisfaction, etc.) factors. As I’ve said before using an analogy: (US Football) “Testing is the Offensive line protecting the backfield of the Quaterback (Developer) and others (Mgmt.). You need us there to help protect you from getting sacked.”JimP.S. I’m actually an optimist, I look forward to things going wrong. 😉
Hi Rob,What’s your views on testers playing part in, or fully designing UI’s?Cheers,Darren
Hi Darren,Thanks for commenting.I’m all for anyone using the skills they have to fulfill a need or purpose in the business.It shocks me sometimes how businesses pigeon hole people in to roles without looking at the wider potential. “You’re a tester, so test”, yet many testers I meet run their own businesses outside of work, can write really well, have a solid grasp of social media, can present well, can mingle well at conferences and can use their design skills and experiences in other fields. Yet many don’t see the potential. They don’t capitalize on the skills many testers have. They certainly don’t capitalize on the creativity, ideas and critical thinking side of what we do.Testers are testers wherever you go. Always the same set of skills that they show.So, in a nutshell. I think if a tester has the skills to design the UI completely – then so be it. Design, coding, marketing, PR, Sales etc etc. We each have skills that can be used in other elements, let’s not squander that talent.And your views?Rob..
Hi Rob,Thanks for the reply. I was curious of your views on testers designing or aiding the design of UI’s because it’s a bit of a touchy subject for some.I’m with you on the view that if you’re good enough then the company should make good use of you. I got put in charge of UI design in my new job, even though I’m a tester. Is that a wise move? Possibly not, or perhaps it’s a smart move? I’m certainly good enough, my only concerns which I’ve voiced are that it will leave me biased to flaws with my designs. As such we’ve decided to take a collaborative approach to UI designs, which makes sense anyway. I can still oversea design and make sure the right designs are approved, yet the responsibility for producing initial wireframes doesn’t solely lie on my shoulders, thus preventing as much bias from me.When you think of UX, accessibility, and usability, I’d argue that we are key components to getting the correct designs in place. Today it might be an uncommon thing testers designing UI’s; some might even think it’s not our role; in the future though I think it will become much more common practice.Cheers,Darren.
Hey Darren,That is great news about being involved in the UI design.I think no “one” person should ever be in charge of the design and UX. It should be a collaborative process with maybe one person leading the charge, planning and decision making based on input from others.The other side of this is that sometimes you can end up with design by committee which often results in hours of deliberation and discussion about sometimes irrelevant points. I’ve seen many of these too.I remember speaking to a group of testers once and suggesting that if we aren’t involved in almost element of the life-cycle then we aren’t doing our job correctly. I was lucky to get out alive. It seems we have been programmed to only be involved at the end only, to never question anything other than the spec / feature relationship. Scary stuff.Times are changing. We need to be more involved in everything. We need to add value through our unique skills, no matter what those skills are. It sounds like you are having a great time with your new role. Good luck with it all.Maybe we should collaborate on an eBook about UX, design and test involvement. Drop me an email if you are interested :)Rob..
Hey again Rob,Thanks for the quick reply, some very good points. I’m of the mindset that if our involvement is only at the end of a project then that project has already failed and is almost certain to overrun its deadline! Getting into UX, I think you’ll enjoy Patton’s “Getting Software RITE” (http://www.agileproductdesign.com/writing/ieee/patton_getting_software_rite.pdf) if you haven’t already read it of course. Not everything is perfect or indeed applicable to all scenarios from this article, what I did really like though was salesforce.com approach to doing remote evaluations of their features with users, and the recording of these sessions for later analysis. That I think would work well in most projects, although it’s my view that evaluations should begin long before that, even back at the initial wire frame designs. After all you can apply a proactive approach to testing designs via techniques such as paper prototyping and wireframe evaluations for very little cost, and great payback when you consider that initial designs will almost certainly be flawed.I’d love to do an eBook with you, however I don’t even have the time to write articles on my own blog at the moment though, so sadly I’ll have to pass on that one. Thank you though for the offer, it’s very kind of you.Cheers,Darren.
Hi Darren,Yeah – that article by Jeff Patton is excellent. I’ve tried to build some ideas in to the process here.It sounds like you are getting lots of great challenges in your role and you’re jumping at them with gusto. That’s a sign of a great job :)There’s no rush for the eBook. Even if it’s a few years from now. I have bags of patience.Rob..