“Let’s try that again.”
Ordinarily, these words do not get anyone’s ears pricked up; but when it is the Apple Senior VP Craig Federighi uttering something as he fiddles with Face ID at an on-stage demo for iPhone X global launch, you cannot help but notice.
It was a great feat for Apple again to usher in a new system of Face-recognition instead of fingerprints and other sensor/access input. But what developers and designers accomplished spectacularly with AI and Infrared thingamajig inside their labs, fizzled out for a second outside those labs. That is not desirable. But let’s face it- it can happen to anyone, any company, any demo.
This was when Apple had also enlisted the best-known hands from Hollywood special effects’ universe to test and secure Face ID with the most bizarre recreations of the human face. So what does it tell you about the tail-end of testing? Is there something you have been missing out or neglecting?
What is User Acceptance Testing?
What we know as beta testing is essentially the phase of UAT or User Acceptance Testing (UAT). This part of end-user testing nails a crucial phase of software-development — checking the software for the actual world and the audience that is supposed to use the application. Why test an already-checked software code again, you may ask. Let’s see that from a user’s eyes.
Objectives
It is only when the experiences of some users are captured and shared with developers, that software assumes the real-shape that will delight its users in the best-case scenario. For the worst-case scenarios, UAT will, at least, help the developers discover what did they missed, the one that troubles a user outside.
This part is paramount before the commercial release of an application as this will reveal important pitfalls and hidden issues that only surface when a user puts it to use.
With UAT, one can be sure that a product has been thoroughly tested for both functionality and business dimensions. UAT helps to validate key business functionalities and unravels many aspects that can translate into bugs, contractual gaps, QA red flags, delivery promises, and final performance areas. With both alpha and beta-testing, the organisation can be sure of having (at least to a reasonable extent) taken notice of all nuances and factors that will make or break the application in the user’s hands. This can be done in-house by leveraging the expertise of users similar to real users; or by having early-stage users. Yet, it is not as easy as it sounds.
Challenges Faced
Improper test planning
No matter how much one invests or what amazing users one has put together for this process, the exercise can easily backfire in the absence of a systematic plan.
It should be as good as (and well-aligned to) system-testing. The inclusion and focus related to requirement-analysis, design-phase, delivery tick-boxes, strategic implications, functionality-and-feature roadmap and form-factor issues should be weaved well. Sharp and lucid definition of test objectives, along with easy-to-word templates for the team for writing test-cases, scripts and strong environments, play an important role too. So does the setting of priorities for business objectives.
What will also make UAT deliver its outcomes is the time management aspect as this happens to be the fag end of the testing cycle; so critical setbacks can be quite heavy. Overlaps between system-testing and UAT can also be quite serious. When time is less and deadline pressures are huge, getting the best out of UAT can be quite a struggle.
Incorrect UAT users
The purpose of this testing is defeated at the very outset when user-selection is wrong. The thrust should be resolutely on the inclusion of end users. Careful and meticulous selection of beta users and internal team members requires special attention. Their level of familiarity, training and understanding of business needs, and time allowed for getting well-versed with real UAT intents can be of great significance.
Managing new biz-requirements
UAT phase can throw a whole new set of confusions, questions and requirements. Last minute change requests can add to the number of panic attacks without any warnings. Project managers should keep an eye and hand ready for accommodating nick-of-the-time demands and changes without leading to any release failure.
Communication gap between teams
A lot of the problems mentioned so far can be wiped on the spot by fine-tuning communication between all aspects of a team: from development, testing to user-testers and between onshore and offshore ones, where applicable. This will help immensely in mitigating incident-response latencies and improving proper planning and collaboration. Use of latest tools and workload management can curb a lot of duplication and bring out a new teamwork impact.
Solutions Framework
Planning the UAT in advance
It is important to make sure that there is sufficient time, resource-availability and attention for giving a good structure to UAT. This is where identification of user roles, differences in expectations and actions, outcomes-related variations can be given an early spot. Similarly, use of User Stories as Test Cases and clarity in communication of client requirements can be quite an active constituent in making UAT surpass its intended goals. Adequate room for language that works for users when they use and report is a prominent point. Due push should be given to the creation of UAT templates, hands-on training interventions, the timing of UAT, test sequence and test scenarios while not losing a firm grip on the actual expected results as well as business-impact.
Involve actual users
This may sound like common-sense advice, but it is quite a clincher in UATs. Negligence or pushing UATs down the neck of functional test-teams can sound risky but can happen without raising any warnings. Lack of time or resources should, nevertheless, be not an excuse for not galvanising proper users to conduct these tests. Compromising the kind of users is as good as making these tests plunge down the drain. At the same time, positive communication and interface between user-testing and development teams are required so that negative feedback and blame-game can be supplanted with constructive feedback and a focus on the big-picture outcome.
Baseline the real-time conditions
UATs pass or fail in an environment, and if the environment itself does not reflect or simulate the real-world scenario, then the tests become pointless to undertake. Use of appropriate data, tapping it smartly and including tools that help in bridging the gap between UAT environment and the actual environment is highly recommended. So is the importance of control on release cycle to have good navigation on different software versions and their tracking on test and UAT environments.
Understanding UAT well
UAT is an excellent and decisive way of aligning expectations, requirements and application-performance in the minutest way possible. Development of products should make users comfortable, satisfied and not left for fighting fires.
Review of proper standards
Test-acceptance is an indication of a confirmation of specifications and expectations. But in situations where meeting specifications still do not translate into acceptance, it becomes a struggle to handle the after-test effects well. The math can become more confusing when there is a real business-value despite all shortcomings. Clarifying the slippery-spot of acceptance is essential and helpful for everyone in the process.
It would propel UAT to success faster when one knows how to define and work on UAT feedback. The range should cover positive areas as well as spotting of bugs or quality gaps. There should be a good room given to change-requests and new functionalities as well. After all, the main aim of UAT is to get closer to what the user wants or dislikes — no matter how late or where he/she realised that insight. Interactions with early-stage application allow them to find out more about their own behaviours and inclinations.
Act on everything as soon and as well as you can. Remember these wise words from Gerard Huerta.
“When you are stuck, walk away from the computer and draw. It will teach you how to see:”
When done well, with use of latest automation, tools and breakthroughs, UAT can sometimes be that very drawing.
