I may well be opening a can of worms with this blog but I am genuinely interested to see what people think about crowdsourcing and in particular services like uTest.
I’ve read some glowing reports about uTest and heard nothing negative – which is unusual in our community. We often see past the gloss to reveal the real story. But maybe uTest and crowdsourcing really is the future.
So I’m genuinely interested to hear from people about why crowdsourcing is becoming increasingly popular amongst the testing community.
I have my own opinions and views on crowdsourcing but they are formed from the little experience I have with uTest. Let’s just say I found the payment, project allocation and script assignment somewhat confusing and unfair – but that was right back in the early days, things have changed since then.
I also have a concern that lots of testers are working for nothing. No bugs, no pay. And the stats seem to show that too (on the assumption that everyone registered is testing – which I know is not right). I know it is their decision though.
In the UK there are 778 registered testers. They have had 270 test cycles between those that take part. And they have found 1159 defects. Not bad. But it works out about 1.5 bugs each (assuming all take part – which we know not to be true).
It fairs slightly worse in India. 5096 testers, 430 cycles and just 4242 defects. On basic and simple maths that’s less than 1 defect each.
Spain has 142 testers, 32 cycles and just 78 defects. Half a defect each. Not brilliant.
America fairs a bit better but I know these numbers don’t tell the whole truth. It could be one tester raising hundreds of bugs. But why are so many registered and not taking part?
(numbers taken from http://www.utest.com/meet-testers – assuming this is up to date)
So please do leave comments and let me know what the benefits are of crowdsourcing services.
I am genuinely interested to see how the crowdsourcing model works for those doing the testing.
- Are you making money? (note: please don’t disclose how much you are making – a simple yes/no.)
- Are you learning more about testing?
- Is the 20,000 + community really all experienced testers?
- How long do you test on average for before you find something?
- How often do your bugs get bounced back to you?
- Are you simply running basic scripts or being utilised for the creative, analytical and questioning minds that you have?
- Is the paid per bug scheme based on luck (i.e. which test scripts you get?) or is it based on skill (i.e. being picked for your ability?)
- Should more companies be getting involved in crowdsourcing?
- Does it solve the testing problem? Or simply solve the outsourcing problem?
There’s some suggestions to get you started.
I look forward to finding out more.