User testing is a one-to-one research method based on the performance of the task or tasks determined for a product or prototype. In this process, feedback on flow, design, and product features is collected and evaluated.
Doing user tests with too much detail is a waste of all resources. To get the best results, run the maximum number of tests with up to five users. This location can only change if the control product has several different types of users.The main user test methods can be listed as follows:
Hallway testing: It is a very quick and inexpensive method of control, which is usually done with ordinary people.
Remote testing: It is a method of testing users by leaving them in their own organic environment.
Expert review: These are the tests created by coming together with expert users in usability tests.
A / B test: These are tests based on decision making by examining the choices made by the user within two options.
What is A / B Test?
Each control provides a learning curve for the product’s development, and the more tests are conducted, the closer to excellence in user experience. A / B testing is also one of the most ergonomic ways to this excellence. So, what should we pay attention to before, during, and after the test? What kind of path should be followed?
The tests that are formed to determine which of two different versions (A and B) of a product or a feature of a product will perform better are called A / B tests. For the version to represent the current state of the product and the B version to represent the newly developed version, the versions need to be on the air at the same time. The versions must be separated by only one feature at a time to be able to observe exactly which different feature success comes with. When there is more than one change between versions, it will not be possible to tell for what difference the test was successful.
To create two different versions of a product or feature, it is necessary to write a hypothesis. A / B testing, with the method of verifying or rejecting the written hypothesis in the context of certain product usage metrics; enables a product development decision to be made or abandoned. When the successful version is removed from the control tool and published so that all visitors can see it, the testing process ends and the product development decision is implemented.
What are the steps of the A / B testing process?
After the hypothesis pool document listing the hypotheses is filled, A / B tests are performed by following the following steps respectively (McCloskey, 2015):
Developing the version of the product or a feature of the product that meets the hypothesis
- Dividing users into two groups randomly
- Show different version to each group at the same time
- monitor the behavior of users
- Evaluating the performance of versions
- Select the winning version and publish it to show to all users
- Repeating the test if needed
How are A / B tests evaluated?
Many control tools give two conversion rates as a result of A / B tests; one for the control group and one for users who see the variation. Conversion rate can measure clicks or other actions taken by users.
Possible to be seen in the report “control group (2.1%), Variation group (+ / – 2.3%).” The meaning of the result is that those who use it with a margin of error of 2.1% prefer the new variation. It is not very correct to think that the conversion rate is between .7 and.3; To be sure of this, it is necessary to repeat the same test more than once because the control was set up with a statistical error rate of 5% from the very beginning.
At this point, Kaiser Fung, the founder of the Applied Analysis Department of Columbia University, states that there is no assurance of the conversion rate and the common sense of the people performing the test in this part. A conversion rate of only is a good rate even considering all margins of error; In this case, it might be a good decision to publish the design in the variation. While taking this decision, of course, it is necessary to evaluate the labor cost created by applying the new design (Gallo, 2017). Therefore, making these decisions before applying the tests by creating hypothesis pool documents will provide attitude from time and prevent effort for a control that will not be applied at all.After the hypothesis pool document listing the hypotheses is filled, A / B tests are performed by following the following steps respectively (McCloskey, 2015):
Developing the version of the product or a feature of the product that meets the hypothesis
- Dividing users into two groups randomly
- Show different version to each group at the same time
- monitor the behavior of users
- Evaluating the performance of versions
- Select the winning version and publish it to show to all users
- Repeating the test if needed
A few A / B testing tips:
Perhaps the most important rule when performing A / B testing is to test the features to be checked simultaneously, simultaneously.
The decision on the results of A / B tests should not be made too soon. According to the concept called statistical significance, it is possible to have an idea of whether the test results are valid or not. This rate is generally € and above, but it is the general acceptance standard. Ultimately, if only a few conversions or visits appear, the results will not be a valid result, as the results will have very high, unrealistic rates.
If a change in a major issue is to be tested, it is a better approach to target new users rather than existing users. Thus, considering the possibility of failure of the test, it is prevented from showing a change that will not be applied to existing users and making them unhappy.
It is very important that the A / B testing is consistent across the entire website.
A / B tests have three results; no result (due to statistical significance), a negative result, or a positive result. It is very unlikely that you will always see a positive result in every test. Therefore, it is important to learn from the results of the tests and continue to apply new tests. The more tests carried out, even if the results bring small changes, the most optimum product will be reached when viewed cumulatively (Chopra, 2010).
Although the duration of the tests varies according to the traffic of the website, the minimum time is one week. The reason for this is that user behavior can vary on different days of the week. Therefore, the tests should remain on the air for at least seven days.
Similarly, it is wrong to interpret the results without taking into account special days. For example, an e-commerce site that sells souvenirs should not consider the conversions it received before the New Year as the success of the test that was broadcast at that time.
How is a hypothesis developed?
With the help of hypothesis development, product development teams can act more focused. Because when a thought is expressed as a hypothesis, its feet become more grounded; When it is clarified what the hypothesis should be provided by changing and which metrics should be measured, the idea put forward for that hypothesis ceases to be a “dream” or an “arbitrary request”, it becomes rationalized.
For a hypothesis to be controllable, the metric to be followed must also be determined. Continuing on the above example of the registration form, the form conversion rate to be found by dividing the number of members who completed the form and the number of individual visits to the format page should be the main metric that needs to be followed. However, to follow this metric, the baseline value in the current situation should be checked and it should be noted how much an increase is expected after the change.
Consideration of the effort and resources required before, during, and after the test is carried out; By predicting the likelihood of the test-taking place and doing this for all the hypotheses listed, it is possible to compare and prioritize the hypotheses. The main purpose of doing this study is to focus on hypotheses that support the basic success criteria of the product, which can be applied simply and that can achieve high-performance increases.
User Test Preparation
User tests; It is the most important of usability techniques, one of the central resources of the usability world. From the outside, it may seem incredibly simple and easy to do. So is it so or is it difficult, and what about the preparation process?
You observe the people who use your product and then apply what you’ve learned to make the user experience better. However, as with all seemingly simple ideas, user testing is easy to do, but it’s hard to do a good job. Beforehand, there is a very comprehensive user test preparation step.
There are multiple factors to take into account in these tests. As they say, “If you fail to prepare, prepare for failure”, this is the case in user tests as well. Let’s look at a few things you should take a look at before your user test sessions, which can help you overcome quite difficult processes such as writing a test script and selecting participants.
Selection of participants
Preparation for user testing includes the step of identifying the users of the product or service and finding the people who will best represent them. When choosing your participants, make sure that there are people who will reflect the characteristics of the people who will use your products. For example, if your product will be mostly used by managers but you are doing your test with their secretaries, most of your time is going to waste.
Why should the User Test be done?
Sometimes, the purchaser of a product may find the money spent on user testing quite unnecessary, and “correcting known mistakes and coding the missing parts in the first place” may have priority. It is very wrong to perceive user testing as making the design or software incomplete or wrong. To explain this better, we can show the automotive sector as an example. As we all know, Audi engineers are pretty good at their job, but when launching a new model car, they first check the car over and over again; They never put the car in the showroom without performing a crash test and driving tens of thousands of kilometers, fine-tuning the ergonomics and performance of the car based on feedback. These engineers are not incompetent, on the contrary, they are very competent at their job because they renew their work by evaluating user feedback. Web and mobile software can be as complex as automobile design.
A good user experience cannot be done without real users getting involved. User tests also come into game at this stage. User testing sessions consist of a participant, a computer or mobile device that the participant will use, and sometimes a control manager who will task the participant to complete. Although this test manager observes and notes the problems that may arise in the website or mobile application, it is very necessary to record the user as video and audio. Thanks to video and audio recording, the individual doing the research can understand a lot from the expressions and gestures on the user’s face along with the images on the screen.
Benefits of user testing:
- Improving the design process
- Uncovering yet undiscovered issues
- Fixing problems before the design process has yet been finished
- Testing assumptions
- Ensuring to be objective and resolving differences of opinion
- To reduce the risk of the project
- Increasing conversion rates
- -Yo increase customer satisfaction
Market research and quality assurance testing are the most fundamental elements of a project, but they should never be confused with user testing.
Usability testing components
User tests consist of 5 components;
Learnability: Can users easily complete simple operations in an interface they have tried for the first time?
Efficiency: Can users complete the assigned tasks quickly?
Memorability: Are users still able to use the software efficiently and effectively after using the software for the first time without using it?
Errors: How many mistakes does the user make, how important are the mistakes they made, or can the user correct them easily?
Satisfaction: Are the users satisfied with the software?
Usability testing methods for software
Usability is measured using a variety of test methods, let’s examine a few of which we can use for the software.
Online Eye Tracking: It is the process to gain information about how long the user looks at which part of the screen. In this way, the items that need attention are placed in the most viewed points, and in this way, attention is concentrated there.
Mouse Movement and Click Map: The aim is to improve the design by looking at the average of the user’s mouse movements.
Laboratory Tests: It is a type of test in which the user is examined by the person performing the test during the test and notes about user behavior.
Feedback Tests: Users state their comments on the software and an improvement is provided about the software by reviewing these comments.
Modified Online User Test: It is a remote test method controlled by the researcher, where the users are directed by the researcher during the test.
Unmodified Online User Test: The difference between the unmodified online user test from the modernized online user test is that the scenarios created before are applied by the user in order.
Hypothesis Development and Testing
The concept of testing, “design thinking”, which has gained popularity recently, constitutes a very important part of design-oriented thinking. The cycle that creates this methodology consists of the steps of researching and defining the user’s needs, producing solutions to meet this need, developing and testing prototypes if possible, and applying the feedback and implementing the solutions. Using this cycle, we can say that the most important stage that opens the door to improvement is the control stage.
The test-oriented approach is not based on your instincts because it is highly likely to mislead you, it allows you to make decisions based on data and reduces the risk you will take on.
Nowadays, especially since the cost of digital testing methods and tools has decreased considerably, it has become easier to apply this approach. Comprehending the logic of testing and making this approach a part of your thinking processes, whether in your business or private life, will benefit you more than you anticipate. Of course, don’t accept my hypothesis as it is, test it first!
Low-cost user tests
Many of the analyzes generated in the user experience (UX) space hold ways in which you can measure the usability of a website. This type of analysis and comparisons you will make with your competitors’ site support the user experience designer to understand the current state of the site. Thus, it can identify problem areas and develop suggestions for improvement.
It’s also impossible to fix or improve something when you don’t know what’s wrong. Imagine that you have taken your poorly performing car to a service and the workshop said “we fixed the problem” without any diagnosis or check. The fact that they have made many changes related to your car does not mean that they have detected the tank of the problem. This is like trying to redesign a site without recognizing real users and identifying the problems they are having on the site.
Qualitative or quantitative tests are not preferred, with the excuse of time and budget. There are many ways to get these precious details while working with a limited time and a small budget. The guerrilla test is one of them.
Let’s take a look at some of the helpful guerrilla test patterns that can be used to gather important information while measuring user habits and site usability without straining the budget or the project plan.
Destroy the broker
Finding users to participate in tests can sometimes be costly, especially if you want to reach a specific user group that is suitable for certain profiles, you will have to pay both the casting agency in between, as well as the participants. Remove agents and find attendees on your own. You can achieve great success by using colleagues, families, relatives, and your environment. Your job should not be difficult, especially when you consider that people enjoy giving ideas and they may feel more important because they give ideas.
One of the most successful and simple methods of collecting data is Google form. In terms of use and creating the feedback form you desire in a short time, these types of forms are both useful and provide a good data visualization for quick analysis of the results.
Take advantage of the software
You can perform your user tests using much software on the market. You can even do your user tests based on profiling, which I mentioned at the beginning of the article (which can be expensive).
It will be enough to extract the appropriate profiles and enter the criteria attached to your profiles from the dashboard area provided by the software. In this process, you can both control your tests and at the same time have user tests delivered to you as video recording.
Get quick feedback
If you want quick feedback and an actionable report based on the feedback generated, I can recommend Zurb’s product called Verify. There are alternatives in multiple edits such as obtaining measurable user feedback, 5-second inspection, click test.
Take small steps
There are also web-based instruments that contain many setups such as question-answer, preference-based control, and navigation test. For example, UsabilityHub is one of them. After you create your test on the site and enter your questions and project information, you will have the opportunity to evaluate your product in an online panel with many individuals in this sector, based on the audit setup you choose.
As a result…
As you can see, user test or interview setups are not a fortune. The important thing is that you have information about control, interview types, and related fictions, know the steps you need to take until that stage and define your needs correctly.
Interested in learning more? Try and check out other amazing contents on Computer and Information Technology click HERE
