In-Depth Guide to Website Usability Testing Methods

9 min read
Mar 25, 2015

Making sure your app or website works the way you intended isn't always easy. If your site is not functioning properly, that could be the reason you aren't seeing the results you are looking for. The solution is to find the right testing methods to verify your site's functionality and usability.

Finding testers can be difficult. Lucky for you, there is the internet. User Testing is option number 1, and you can find it easily by googling 'user testing.' However, a platform recommended by Oli Gardner, Co-Founder of Unbounce (and the man who has seen more landing pages than anyone in the world) is Usability Hub. This option is more than likely the right choice for you if you aren't able to round up some of your peers to give your website a try.

That said, it's usually a good idea to give your peers a crack at your usability testing, because that kind of testing should only cost you a six pack of their favorite beer.

Bonus: Download our Modern Web Design Checklist as a PDF. Easily save it to your computer or print for reference anytime.

Below, take a look at website usability testing methods that will make sure your website is functioning in a way that resonates with users to create a better user experience.

Do a Pilot Test to Avoid Embarrassment

This is a test run, plain and simple. The reason for the pilot test is to work out some of the kinks in your testing session before sending it off for real testing. The pilot test should not be viewed as optional or a waste of time. This step is important because fixing some of the simple issues that come from human error, poor coding or page speed beforehand will save you and your testers time in the long run.

Make sure that, in your pilot test, you approach it as if you were a true tester — going through the exact testing process with a plan for exactly what you will be asking to be tested. Treat the pilot as a true test, even if you won't be recording any data at this time.

Creating a Usability Test Plan To Show Your Process

Before testing begins, you must create a checklist — an outline of what the test will entail in a way that will be easy to understand. It doesn't need to contain all the technical information that you may know, but it should provide an idea of what is being tested and why.

Tomer Sharon, Author and UX Researcher at Google Search, was kind enough to provide us all with a simple example and explanation of what a usability test plan may look like:

1. Title — What you’re studying and the type of test.

2. Author and Stakeholders — Everyone involved in conducting the test.

3. Date — Don’t forget to update this every time.

4. Background — A brief history of the study, under five lines.

5. Goals — Try to sum it up with one sentence, but if you have multiple goals, use a short bulleted list.

6. Research Questions — Make it clear these are the questions you hope to answer with the study, not the questions you’ll be asking the participants.

7. Methodology — Since we’re outside of an academic environment, a simple what, where, and for how long will suffice.

8. Participants — The specific characteristics of the people you’re looking for and why.

9. Schedule — Include the three important dates: when recruitment starts, when the study takes place and when the results will be ready.

10. Script Placeholder — Until the full-study script is available, a simple “TBD” is fine.

With this checklist, you will be able to ensure your testing remains on point and on target.

Website Usability Testing

User testing is a mandatory part of success on the web. There are over a billion websites in existence and users become accustomed to certain website attributes that determine usability. If your website is deemed unusable by a user, there is nothing stopping them from moving on to your competitor's website.

Setting Expectations for Website Usability Tests

When setting up website usability testing, keep in mind that just about anyone, including a dog, can use the internet. Therefore, you must set expectations for when testing begins.

1. Behave naturally - Natural behavior on the web can result from open-ended tasks, without one correct answer. By asking testers to find a piece of information on your website and let them do the rest, you will be able to identify what makes the most sense to users — based on natural reactions.

2. Complete the Task However You Want - Users in the real world won't have you whispering in their ear if they go off track. Instead of trying to put them back on track, let them wander and try to determine why they lost focus.

3. Test Other Websites - Testing a competitor's or peer's website will help provide context around your results. Having test participants complete the same task assigned to them for your website on another website will grant insight into how users actually behave on the web. It has almost nothing to do with your website in particular, but creating a website that works in the way users are familiar with is important.

4. Don't Say Which Is Which - People, in general, will tend to shy away from the truth if they think it will reflect poorly upon them. In other words, some people will be unlikely to share with you the difficulties they may have experience with your website. Allow the test participants to critique another website before your own, if possible, to put them in the right frame of mind. Don't reveal that you are testing your website earlier than is needed.

In general, you want to keep the testing environment open and free for the users to experiment, while capitalizing on the opportunity to meet your testing goals.

Six Criteria to Focus on for Website Usability Tests

When conducting a website usability test there are six criteria that should be focused on according to Jacob Gube, founder of Six Revisions — no matter what type of website you have.

1) Task Success — Measuring the success rate of open-ended, and direct, tasks is an important part of usability testing. You'll want to determine the intuitiveness, ease-of-use and repeatability of the task. Making sure that users didn't complete the task the first time based on blind luck is just as important as determining if they could complete the task at all.

2) Navigability — Card sorting and site tree testing are perfect for answering these questions. If you are a note card kind of person, you can use Trello. A question to ask of your website is: how fast and in how many clicks does it take users to get where they want to go? The answer should be no more than 3 to 4.

3) UX Design — It is not enough to just see if users can complete a task. Conducting interviews or asking for feedback with users will help you to determine the user experience. Aim to solve frustrations and delight users in the future.

4) Readability — Content is king, so make sure it is easy to read. Pay attention to font family and font size if users have to zoom their screen in. In addition, test user comprehension. Use a tool like Readability Score to determine the reading level of your content. Your score should be between 6.0 and 8.0 for the Flesch-Kincaid Grade Level formula.

5) Accessibility — Testing your website across multiple browsers to ensure compatibility is the major goal here. While it may seem like nothing will be different, there is definitely a difference between Safari, Firefox and Google Chrome. Believe it or not, there is even a difference between Chrome on a Mac and Chrome on a PC. In addition, check your color choices that create contrast for readability. You can find tools to assist you from W3C. The tool most recommended for cross-browser checking, however, is Browser Stack.

6) Speed — Site speed can be a real downer on a website's user experience and that's not all. A slow website will also impact your SEO. Follow the best practices set forward by Google and use tools like Google Page Speed or Pingdom to monitor your website speed. Sometimes the answer may just require you to move to a faster server.

Mobile Usability Testing

While you are still testing the same subject — your website — the criteria and experience will be completely different between the desktop version and the mobile version. It is important to understand the different norms for device type in order to nail usability testing for a better user experience.

Adapting Usability Testing to Mobile Devices

Obviously, designing a website for mobile use is different than design a website for desktop use. Otherwise, we wouldn't be talking about it. But what needs to be taken into consideration for mobile, in particular?

Have the Right Testers — This is huge! Don't have iPhone users test anything on android, or vice versa. If you gave me an android device, it would be a struggle for me the figure out how to turn vibrate off and on when the sound is off. The tester needs to be familiar with the device they are testing on. Otherwise, it will be nearly impossible to determine if any problems reported are with your site or with the use of the device, in general.

Allow for Customization — People are more prone to have custom settings on their phone than on any other device. Therefore, it is a good idea to let them use their own device for testing, meaning you have to find a way to get your app or website on their phone. A website isn't difficult — it can be done by exchanging a URL, but an app can pose problems. We use TestFlight for our mobile usability testing of our iOS apps and have had great success.

Here are a few quick tips for mobile usability testing that you can apply immediately.

1. Bring chargers — You can't test on a dead phone and phones tend to die much quicker than you might expect.

2. Observe Body Language — The body language and facial expressions of mobile users is very apparent, especially in showing signs of frustration and disappointment.

3. iOS and Andriod aren't always the same — Just because your app or website works as expected on iOS or android, doesn't guarantee that it will work on the other type of device. Therefore, you may need to have two different test groups. As noted above, you should really utilize users of each specific operating system.

Get your Moblie Usability Testing Method Done in 1-hour

Worried that a mobile usability test will take all day? Nonsense. Marina Lin, an Interaction Designer for Mobile Apps at Cars.com, wrote and excellent article in which she explains her mobile usability testing methods. Below, we will take a look at the summary of her methodology.

1. Build Awareness (10-15 minutes) — This all about creating a frame of mind — that is, inform the users consciousness of how they actually use the app or website. Start broad and then narrow the focus. Be sure to ask questions about when, where, what, why and how — you know the basics. One word answers seem to work just fine.

2. Create Documentation (20 minutes) — This exercise is particularly helpful if it encompasses not just when the participant used the app or website, but the actual goal they were trying to accomplish when using this product as a tool. Create a document that allows them to record their actions with the website or app and their goal for each action they take. In addition, have them note their emotional response. The emotional response should be either positive or negative. It will help to reveal where usability issues arise or if they landed on a page they didn't find useful.

3. Have them Help Solve Problems (20 minutes) — Have your testers help you solve the issues they experienced while testing. Ask them to design a feature that they would like to see. Start by giving some suggestions or baseline to get the ideas flowing and to ensure the solutions are possible on your end. For example, if you had two ideas for a feature in the app or on the website and they reacted negatively to the feature you chose, preface this section with a question that may lead them down the other path you were considering.

4. Get Feedback (5-10 minutes) — Take a final round of feedback and ask for elaboration on any ideas or comments that weren't clear to you.

Conclusion

Mobile and website usability testing is often seen as the same thing, but that is not the case. There is a clear separation in key components of testing between devices. Desktop usability and mobile usability will often have different goals, criteria and tasks to be completed. Be sure to document your different methods and keep the methods and results associated with their respective devices. With this process, you should be able to create a testing process for your website or app that results in a design for the best user experience possible.

Want more web design expertise for your business? Check us out!

Photo Credit: Dog on Computer

Sign up for blog updates!