Every silver lining has a cloud. Yes, as cynical and downbeat as it sounds, this happens to be that pinch of salt that every ambitious app needs to swallow well. Not even the likes of Niantic spared this harsh truth. While we thought they must be hardly getting anytime to wriggle away from celebrating the explosive popularity and feverish adoption of Pokémon GO, there was also a lot of hand-wringing going on over insufficient preparedness to handle the gaps of back-end testing. It turned out to be a fantastic app, but as it burst many charts and scale-limits alike, its makers confronted its real performance limits too.
Load, performance, soak, spike, shrink — these are words that are keeping people awake — whether it is a gaming major working to keep pace with a breakneck-speed love its fans shower on a new app or update; or an immigration/Government website trying to make sense of mercurial demand-blips and flips.
There is simply no room, then, for an app to suffer network request failures, or a server error or an operating system-mismatch or feature-update snag or any other goof-up. Especially no wiggle-room whatsoever when the organisation and teams are too complacent to test them in the real context.Be it this device or that, this form-factor or that, this screen-size or that, this native language or that, be it iOS or Android or Windows or Blackberry — it is the software’s team onus to shake the app well for consistency, performance and scale.
Yes — enter the world of Mobile Device Testing.
Top challenges faced by Mobile App Dev and Test Teams
Just like no self-respecting automotive major would let a new car slide out without brutally testing it on a test-road that simulates all aspects of traction, turns, temperature, winds, heat issues, manoeuvrability, water-pressure etc. The same way developers and test-teams cannot rest (and should not open their beer cans) until an app is thoroughly tested on real devices.
The terrain is full of challenges — such as heterogeneity and consequent-incompatibility of devices and OS;network variations;battery problems; connectivity issues;differences in environments, features, user preferences, usage nuances, screen characteristics; and back-end muscle. This further accentuates the need and objective of deploying real devices,irrespective of the strength and frequency of emulators and simulators put to use in mobile app development. The intervention of real devices brings an app closer to actual, and bare, introspection; and more adjacent to a deep-dissection of anything-under-the-sun-causes that can ruin an app.
Challenges related to hardware issues
Solutions that promise a comprehensive toolkit and provide reusable test-case repositories while empowering the app’s test on every ripple of hardware possible.This is where performance, as well as stress and security testing, should be taken into account, lest the app forgets to check its sturdiness in the face of the most unpredictable factors. It is often hard as well as expensive to equip it for unknown vulnerabilities, third-party integration pitfalls, chip-part-reasons, memory support, interoperability-bottlenecks, fault-tolerance, battery consumption, power usage, network-strength volatility and other hard-side aspects. It is only through a well-designed test on real devices that one can ascertain both software and hardware readiness of any app.
Challenges related to software issues
With a spree of endless platforms, OEM customisations, OS categories, designs, UI genres; the task of adequate app testing gets twisted and tangled in new software-side challenges. Impact on users and anticipation for user needs, responsiveness-strengths, interfaces and workflows is something that cannot be comfortably sidelined while releasing an app. The new app has to be checked on all angles and varieties of user-frustration. Un-installations have to be successfully and timely averted. Developer updates and regular checks are vital here. So is the need for running tests for various browsers, UI versions, features, APIs and OS-types.
Decoding Mobile Continuous Testing Process
It is no surprise that a study by Dimensional Research spotted 53 percent of users experiencing ‘severe’ app issues (these mean apps that crashed or didn’t perform as expected). When 61 percent expect apps to start within four seconds, can we rely on isolated and fragmented testing approaches anymore?
If we peek into a Q2 2017 ‘State of Mobile Device Performance and Health’ report, we would easily see how the Android device failure rate hit 25 percent worldwide. The numbers for iOS device-failure-rate were about 12 percent. That’s not the whole story — the crashing app rate was some 54 percent on iOS devices; while for the Android breed it was somewhere about ten percent.
This ‘failure’ — as study-organisation Blancco defines it — entails a slew of first-party and third-party issues, Wi-Fi connectivity problems, headset malfunctions, phone reboots, and anything that can set the app conking off.
This ‘failure’, however miniscule or temporary it may be, can immediately convert into frustration, alienation, abandon and competitor-switch for users who are already impatient and fickle in this device-inundated world.
This endows new rigour and planning-regimen for mobile testing, and bringing in ‘Continuous’ underpinnings here is what app-makers need in ensuring that their testing stands the test of real-time.
Continuous testing is a strong answer to help navigate the testing challenges that new devices, new Operating Systems and new capabilities are throwing up — and these are often so odd and case-to-case that conventional testing may not hold much water here. Companies need to look beyond expensive, clunky, unpredictable, lengthy, ineffective and inefficient release processes that were oriented to support one update every couple of months.
It is the age of continuous delivery and, hence, of continuous testing. As the Dimensional Research study reminded well, even ‘mobile first’ thrust is not going to suffice unless organizations adopt a ‘performance first’ mindset that entails, among other things, monitoring application performance continuously.
What’s harder than wooing a new user? Retaining a user that has just experienced a bug or some load-gripe on your app. No doubt then that thorough and meticulous testing assumes new proportions and significance in a world where users are not just fastidious but also unforgiving.
This is not just a hygiene issue; it is also a financial one. When reports from firms like Gartner are cautioning how just 0.01 percent of mobile applications will be considered financially successful by 2018 — it is time to pause and take stock of one’s testing mettle candidly enough.
Apps need the ruthless testing that leading car-makers invest in every new and promising beauty or beast that they bring out. They still know that no matter how well they intend to and try, the real road is going to be always unpredictable.
It’s not enough to install air-bags. It is precious to know where to place them and why.
Because no one likes to be a deer caught in the headlights. Or facing one.