How to Run User Acceptance Test (UAT): An Actual Example From a Singaporean Tech Company

Overlooking User Acceptance Tests (UAT) in favour of software releases is a myopic and potentially expensive business decision. This is what we’ve learned as a technology start-up in Singapore.

We are Novocall, and here’s our take on the importance of UAT, based on our own experience.

Why UATs Are Important

As a technology company, we understand the tendency to rush software releases and get our product into the hands of our users as quickly as possible. Testing, especially user acceptance tests (UAT), are often deprioritized, given the minimal resources and time urgency a start-up possesses.

Looking ahead, we realize that this is simply thinking short-term. Software releases without proper testing from the user could result in unsatisfactory user experience, bad product reviews, and poor NPS (Net Promoter Score) scores.

In today’s digital world where user experience even trumps price, UAT is becoming more important, regardless of how large or small your company might be.

Functional Tests Are Not UATs

After building a product, a development team needs to ship it without bugs. That’s when they do functional testing. Functional testing is done by the developer through writing code, to test whether the functions they developed run without hiccups. If the software passes the functional test, that means they work. That means it’s good to go, right?

Wrong.

Functional tests are important, but you’ll never know how your system will behave in the real world unless it’s being tested by real users. This is because humans function in ways that the development team might not foresee, and this might break the system. With UAT, it takes both system performance and human behaviour into account.

Planning a UAT

Before we implement anything, it’s important to plan the UAT, especially since our end-user profiles are business owners who aren’t necessarily tech-savvy.

We start with defining and prioritising our business requirements (both functional and non-functional). Prioritising is important as not all tests can be implemented given certain projects time constraints.

The next step is to craft our UAT acceptance criteria (UAC), which is a focused description of each test case.

This gives a decision table outline of an acceptance criteria.

 

Description
Given(i) Input
(ii) Preconditions
When(i) Triggers
(ii) Actions
Then(i) Output
(ii) Postconditions

The UAC is one of the most important parts of the UAT because this is where compromise happens. Here are some questions you need to ask yourself when setting up the UAC:

  • How many defects and bugs in the system are we willing to tolerate for release to the public?
  • What will happen if the system is delayed?
  • How much delay could we accept?
  • What are the damages if the system is delayed?

If you want to go through less technical acceptance criteria, the following is one example of validating a user story. We do a similar acceptance criteria with our call-back feature in Novocall. For this one, let’s assume we’re using it for a more generic contact form.

 

Example #1
User StoryAs a user, I want to submit a contact form successfully.
Acceptance CriteriaUser submit a form.
“Given my role of a website user,
When i click on the ‘Contact Us’ page,
I get redirected to the page.
I fill in all the required fields
I click the ‘Submit’ button
My feedback successfully gets submitted to the company email.
CriticalYes
Test resultAwait result
CommentsWill not release until requirement has been met 

The UAT Process: A Run Through

After planning, it’s time to build the user testing process as well as the status report. Multiple test cases will be written up with scripts (shown below). With these test cases, we then prepare a status report to be filled out after the user testing is done.

Here’s an example script for a web application’s login test case:

Example #2
PurposeUser able to log in with user ID and password
PreconditionsUser has signed up. User has not logged in
InputCorrect user ID
Correct password
Process (Actions & Triggers)
  1. Enter user ID
  2. Enter password
  3. Click ‘Login’ button
OutputUser logs in and sees the dashboard

Finding Users and Executing Your Test

After setting up our process, it’s time to test! There are numerous ways of user testing, with each test depending on the type of software you are testing and the data you are looking for. This could range from surveys and feedback to more expensive options such as focus groups.

However, if you are running lean or want to save costs in general, you could opt for a more affordable yet reliable testing method: remote user testing. This allows for real users to test your system with video and feedback to boot, without the price tag of a focus group. There are multiple platforms that do this for you: Usertesting, Userfeel, Usabilityhub.

After the test is done, we’ll compile the data collected, make a status report, and evaluate the test results. This will determine whether we:

  1. Leave bug and go ahead with the release
  2. Push back the release date and fix bug
  3. Add change request for future release

Given the complex nature of software and human behaviour, UAT can be a tailored process that is customised to your product. These examples I’ve shown worked for my company’s web application, but it may not work for you. Thus, it’s important that you take the time to examine your test processes and get your UAT done well. With a properly executed UAT, it could bring about confidence to your product releases and deliver exponential returns on your UAT investment.

This was originally published in the Construct Digital Insights.

Written by

JJ Huang

JJ is the Co-Founder of Novocall. When he’s not busy building the Novocall brand, he spends his time watching crime shows and documentaries.

Written by

JJ Huang

JJ is the Co-Founder of Novocall. When he’s not busy building the Novocall brand, he spends his time watching crime shows and documentaries.

Related articles

How to improve your Digital Marketing ROI by Tracking Your Calls

How to improve your Digital Marketing ROI by Tracking Your Calls

Tracking your phone calls helps you know where your customer calls are coming from and that is critical to making smart business decisions. We can do that with call tracking software. Not all businesses can work with only social or chat. There are many businesses...

5 Reasons Your Business Needs Call Scheduling

5 Reasons Your Business Needs Call Scheduling

There will come a time when you realize that a call scheduling process is a business necessity. There are so many people who would like to do business with you, but sometimes your sales reps are unable to get to the phone and you miss a lead. In other cases, you are...