In July of 2022, our studio manager Udi Salant gave an enrichment presentation to employees of the Tipalti company at their offices in Galil Yam. The Salant ID studio has been working with Tipalti for over three years. Tipalti is a leading unicorn in the field of fintech not only in Israel but also around the world. The lecture dealt with user testing / usability testing and emphasized how to conduct user testing correctly, as well as what important insights can and should be derived from the process. The methodology Udi presented reflects the procedures carried out at the SalantID studio, and this article summarizes the main points of the lecture.
When we work on a system for a client, regardless of how well we know the content of their field, one of the most important principles that guide us is the knowledge that although we have extensive experience and knowledge, there is also the modesty that we do not know better than users themselves. Sometimes we produce a system that we believe in, that we gave our best to create an optimal system, but then when we go and test the product in front of potential users and receive completely different feedback than expected. Essentially, the guideline for success for any product will always be from users themselves, and if auser fails to use the system correctly, the system is not accessible enough for them. As a result, we will make changes to the system and check it again.
So What are User Tests and What is Done in Practice?
User tests consist of two groups of people – researchers and participants. The researcher gives specific instructions to participants of the user test, and those participants then go and complete pre-written tasks. While the participant operates the system and begins completing those previously-mentioned tasks, the researcher monitors participants’ behavior as well as listens to them and summarizes their feedback.
User testing has many advantages:
-
Monitoring problems: user tests make it possible to find problems with a system that we as UX and UI designers did not notice and as designers did not notice.
-
Uncovering opportunities: Sometimes opportunities emerge that we hadn’t thought of before.
- Learn about the potential users: it is always important to notice how users behave and what they prefer during user tests.
User tests can be divided into four different categories:
- Moderate synchronous user testing is when a researcher holds a meeting with the participant and guides them to act according to a pre-written list of tasks. This can be done in a physical meeting or remotely. The experimenter can change and direct the participant as needed.
- Asynchronous and non-moderate user testing is similar to synchronous testing only that the instructions are not delivered by an experimenter but by the computer into which the various tasks have been entered. From there, the user can perform them in their free time. The performance can be saved and recorded and viewed at a later stage. This is sometimes problematic because the user does not always properly understand the task or conveys irrelevant information that later costs valuable time for the analysis of the test findings.
- Guerrilla user testing involves having colleagues at work or in the field examine your UX and UI designs and give feedback on them from a usability and visibility standpoint.
- Testing with existing customers: is when you examine a platform or system in the real world with real customers who actually use your product.
Types of users and the different levels for examination
The users of a system can simultaneously be a very diverse population or a very limited one. Users can differ in basic parameters such as gender, age, nationality, religion, socio-economic status, and more. They can also differ in their personal preferences such as interests, level of technological mastery, and a variety of interpersonal differences that exist in each of us. Each person is different and our goal is to create a system that will be easy and accessible in the best way for each and every user. Of course there is no way to characterize and design a system that will perfectly fit the needs of one hundred percent of your users, but we will create a common enough denominator to meet the needs of most users, as well as create a support system for those who have difficulty (just like training wheels on children’s bicycles).
User tests can be done at several levels or layers depending on the needs and according to the mental and financial investment of a company. Systems can be tested on colleagues at work or on company employees outside of the product team. Potential users can be gathered, given tasks, and have their performance tested. Moreover, companies with higher levels of products can conduct user tests on beta versions and their systems, as well as real-world testing or performance testing of customers already using their systems.
How Many Users Do You Need to Test a System?
Nielsen Norman, the godfather of all things user experience, sums it up in the most clear way – “The answer is 5, except in situations where it is not the case”. Google, for example, will bring in between 5-8 users, depending on the situation. In general, there is a general agreement in the field that this can and should be a small, single digit number of users. The idea behind these numbers is that 80% of the problems that are in a system will be fixed in user testing with these five participants, and the remaining 20% of problems will only be fixed after many users have used the system, so in relation to the investment versus the outcome, the most cost-effective way to improve the system is by conducting user testing at a smaller scale initially, with at least five users.
The Main Questions We Ask Ourselves in User Testings
-
Value Proposition: Are you contributing something that your potential users actually want or need and is not adequately addressed elsewhere in the market?
-
Relevance: Does the system convey what the potential customers expect? Does it match their needs and wants?
-
Clarity: How easily is your value proposition communicated? It is reflected dynamically in your product with images, microcopy, and calls to action?
-
Urgency: How do you meet and utilize the internal motivations of your potential users versus creating external incentives.
-
Distractions: Is your platform pulling users away from the main purpose they came in for?
-
Anxiety: Are there processes that create uncertainty or fear among your users when they go to perform actions that you want them to perform?
What Not to Do in User Testing
So after we talked about what user tests are and how to perform them, let’s dive into important guidelines and steps for conducting user tests so that you can make the best out of them:
-
Don’t examine users in groups – instead, examine users one at a time. We as humans have a tendency to imitate other people’s behaviors and agree with arguments they put forward. Therefore it’s important to conduct user testing one on one to limit any biases. By doing this, we can form a full picture of the problems and malfunctions we have in a system since each user will have genuine, unbiased feedback while interacting with the system.
- Don’t (inadvertently) add stress to the subject by sitting in front of them . Instead, we recommend you sit next to subjects to help them feel as comfortable as possible during a user test. You should create a pleasant and welcoming environment for them that feels comfortable. Keeping user tests one-on-one can help with that (one participant and one facilitator), as can sitting next to the participant rather than across from and especially not behind them.
How Do You Prepare for User Testing?
User testing is not a trivial process. For user tests, you need to organize and prepare a script in advance in order to avoid variables that may interfere with receiving feedback. It is extremely important to carry out a similar if not identical procedure between all users, for example by asking the same questions and providing the same setting for all users being tested. These are the steps you need to follow in order to build user tests the right way:
- Preparation and assessments:
- Prepare a test plan document
-
Set up a user profile
-
Build user scripts for the tests
- Administering the tests:
- Create an interactive prototype for the tests
- Conduct a conversation with the subject
- Collect findings during a session
- Analysis and findings:
- Consolidate findings into a unified document
- Analyze findings
- Draw conclusions and create an action plan for improving your product
Who Should Do the User Testing?
Example 1: We examined why the “about” page diverts users from the main flow of filling out a referral form
Example 2: Opinion rating on a business is stuck in the process
