If you want to go fast, go alone.
If you want to go far, go together.
In a space with mind-boggling delivery-speeds and changes that can be as unexpected as earthquakes, testing assumes a new challenge every day.
So many platforms, so many OS creeds, all these devices pouring out of labs and factories- could our challenges have gotten any more slippery and tricky? Releasing a perfect app or product happens to be an exercise tied in a ponytail of these trends and potholes. It is a challenge the scale and complexity of which is simply hard to handle for any organisation or team on its own. But everyone can use some help in slicing away these problems.
The 2014–2015 World Quality Report pegged how by 2017 some 29% of IT budgets would get going to quality-assurance and testing. Yet, just around the corner of 2015 alone, the spends on testing jumped to 35% of IT budgets.
The year 2016 threw a new buffet of top-tier testing tools around function, performance, cloud, Continuous Integration, Parallel Testing, readiness for device-versatility and compatibility with OS options as well as major ALM environments (languages).
This new state of testing world and the way app users are turning fussy every year, have only opened floodgates of new possibilities and work for an interesting part of this world — that was not in there, but always out there. Yes, the power of the Crowd.
More so in the case of mobile app testing where the complexities of problems are way different than those of desktop or web apps.
Hey, but Crowd-sourcing?
Yes, it is happening and possible. Thanks to what technology can allow us to do today, organisations can now spread the burden and pressure of testing apps on-time, thoroughly and through more lenses/perspectives than one — to a crowd, i.e. interested and equipped testers outside the walls of the organisation. This need and trend are gaining a lot of traction for some obvious and latent reasons. Let’s attack them briefly here.
The Challenges of Mobile-App Testing
We know the usual issues that Mobile-app testing is usually wrought with, don’t we?
A big picnic of screen sizes, aspect ratios, pixel-densities, screen elements, access factors and other differences that boil down to more work and more confusion for testing.
An intimidating variety of mobile standards, power-side factors, battery-aspects, connection types, carrier details, service changes, data volumes, bandwidth ranges and availabilities — as distorted as this may appear, no specific need can be sidelined into the real-world environment. It has to be tackled during testing stages with confidence and comprehensiveness.
Add to these lists the differences erupting from OS versions, fragmentation in Android landscape, APIs, user mobility, language differences, usability nuances, localisation needs, Internationalisation requisites etc. which make it unwieldy for a single team.
The depth and breadth of an app’s nature — native, hybrid or web application as well as the growing traction and ever-climbing expectations of agile development. Do not forget to add the aspect of contextual testing tools here.
So we can get a drift now of what is making this new room for crowd-testers. There’s more to the plot when we consider why more and more organisations and products are sinking their teeth into this approach. Lean in:
Cheap vs Expensive
If testing misses any error or usability-flaw, it will immediately pop on the bottom-line, customer-experience and user-retention figures. That makes missing to opt for crowd-testing akin to a big hole in one’s pocket. After all, the more people help in testing an app, the faster the app rolls out; but more importantly, the chance of having unchecked-flaws is also considerably reduced. The tendency of oversight and slipping on defects decreases, when there are more eyes, and more than one kind of eye, checking and watching what can go wrong. This polishes the stellar impression of your brand for future apps and offerings too.
Lack of in-house test environment
No matter how much an organisation prefers to keep things in a walled garden, there are practical knots of network availability, scalability, resource-utilisation, load-readiness, QA burdens etc.
With Crowd-testing, a team not only stretches the network parts but also creates an environment where needs and nuances of a variety of users can be looped in at the testing stage.
Actual checks for load bear-ability, dips, performance, spikes, and execution can happen better, faster and cheaper when the massive scale offered by crowds gets tapped easily.
Lack of sufficient time to test
The sheer pace at which apps keep evolving, adapting and adding to new devices, user whims, network factors, functions, features and technology-innovations can be quite a nightmare for testers.
New builds can drive through these extreme-level scenarios (that DevOps, Parallel Testing and Continuous Integration have only made more pressing) better with Crowd-testing even if they are rolling out updates quite frequently.
Crowd-sourced Testing for Scalable Mobile QA. Pros & Cons.
It pays to know, no matter how promising and glossy an alternative sounds. So here’s a quick and objective list of both the sides of the coin:
Pros:
- Speed and versatility that a global pool of professional testers offers.
- Significant savings on time and money.
- Quality of feedback and defect-observation gets sharpened considerably.
- Testing-cycle length gets a refreshed and optimal effect with crowd-sourced options, thus, reflecting on time-to-market cycles
- Access to a wider variety and capabilities of testers.
- Better test-coverage and ramp-up times (hours instead of weeks).
- Possibility of test-on-demand.
- Insights into real environments, real devices, real networks and real users.
- Closer alignment with Target demographics.
- Scalability and automation impacts.
- Cloud-advantages.
- Additional scope for any performance testing tool and scope for simulation of local network conditions, or traffic load.
- Access to a wide and latest range of skills.
- Faster feedback to developers without denting customer experience.
- Neutral feedback and objectivity that raises the bar of apps.
- Ease and depth of customisation as per target segment.
Cons:
- Control issues.
- Information confidentiality and integrity.
- Adequate motivation and availability of crowd-testers.
- Getting work done as per time-needs, standards and complexity and risk factors.
- Communication challenges.
- QA rigour.
Developing a Strategic Approach to Faster QA
Both the sides, benefits and risks spelt out above, have to be looked into smartly and attentively, so that crowdsourcing turns out just the edge that it appears on paper. For that, one needs an expert with experience and understanding that this space needs. Addressing the areas of information sanctity, resource-readiness, control, organisation, coverage, time and standards — these are just some red-lined qualities that wipe away chaos from crowds and turn them into arms that extend and turbo-charge the impact of your team.
Do recall how Statista had reckoned that by March 2017, Android users would be choosing from a pack of 2.8 million apps, while Apple users from 2.2 million. Can we notice a small detail that was tucked somewhere? What is worrying here is the mad rush by developers to meet such demand-appetites and let go of proper testing. That’s not just possible but natural.
Yes, faster time-to-market should never mean loose-ends. Apps are useless if they are full of user gaps and vulnerabilities. It is all about getting the big picture right. It is all about making QA count — with or without crowds.
Because the only place that is not crowded in this market is — an app-maker that users love and stick by because they know that they can always look forward to work that is –slick and kick-ass! Is that you?
