Frequently-Asked Questions (FAQs)
usability testing expert review contextual inquiry
- How many users does it take?
- How do we get users for studies?
- How do we plan a test?
- Who makes up a team?
- What happens in the Executive Viewing Room?
- What are the test deliverables?
- Is lab testing the only option?
- What types of data do we collect?
- How can the benefits be measured?
How many users does it take?
This is one of the most frequently asked questions! The simple answer is, it depends. But, more specifically, we recommend small studies with focused users who represent the same type, so that results from a small sample can be realistically compared. Our typical studies use 6 users: a walkthrough (tolerant) user to “test the test,” a pilot to confirm the test (typically, the pilot data can be used), and four users representing the specific user group we´re observing.
If you have budget for 15 to 20 users, we recommend recruiting several smaller groups of users and creating scenarios that work for each user sub-group. For example, with 18 users, you could get results from three different user groups. Or, you could conduct three separate studies during the development cycle for your product.
Return to Top
How do we get users for studies?
Our liaison for the Usability Center, Dawn Ramsey, is the Dean of Extended University. In her capacity as Dean, Dawn oversees continuing education, special grants, external projects, and private industry relationships. As a result, she has access to a vast network of resources from business, industry, and academe. Using the user profile we create in our planning meeting, Dawn´s resources provide her with the ability to identify and get inside companies with users that specifically match your real users. For home or casual users, she has similar resources.
Unlike some companies that either do not provide these services or use temporary employment agencies for recruiting, our user recruiting process is highly effective in contacting and contracting your specific, targeted users in a timely and cost-effective process that brings results.
How do we plan a test?
Because we use a team approach to user testing, planning is the essential starting place. In a half-day meeting, typically, our team — composed of usability experts from the Usability Center and the key members of your organization — meets to establish all the essential elements for the usability test. These include setting goals, determining tasks (which become scenarios), creating post-task and post-test questionnaires, and creating the all-important user profile for recruiting the users from the specific subset(s) of the user population.
Whenever possible, this planning meeting is conducted in the Executive Viewing Room of the Usability Center at Southern Polytechnic. However, when logistics make this difficult, we set up web-based conferencing capabilities for this meeting.
Who makes up a team?
The Usability Center provides a team lead and one or two usability associates. We perform all the usability aspects of an evaluation, including facilitation of meetings, creating a positive user experience, logging during testing, and reporting after the test.
Your organization provides two or three key members to the team. Typically, this includes the appropriate person from product development, marketing, documentation, quality assurance, technical support, or other areas with an interest in the findings of the usability study.
What happens in the Executive Viewing Room?
We encourage you to invite as many people as possible to see the action of a usability test from the Executive Viewing Room. In this room, members of your organization, not just executives, can directly observe the user during a test, see and hear what´s being recorded, and talk among yourselves about what you´re learning from users.
Because we know from experience that “seeing is believing,” and live sight is better (more realistic) than remote viewing or video highlights later, we make it possible for you to bring up to 30 people into our expansive Executive Viewing Room.
What are the test deliverables?
Test deliverables are typically, but not exclusively, the following:
- All materials for each user (completed screener, pre-test questionnaire, post-task questionnaires, post-test questionnaire, log of user experience)
- DVD for each user
- Spreadsheet of team findings, immediately following the last user test
- Report (executive summary) describing the goals, users, scenarios, findings (a week later)
» View a sample usability study report.
Other deliverables we can provide on request include:
- Highlights tape of top findings
- Formal report with analysis of findings, including data from questionnaires
- Oral presentation/PowerPoint with clips to company personnel to share findings more broadly or specifically
Is lab testing the only option?
Lab testing is only one of many options we provide to help you understand your users and their experience with your product or interface. A user-centered design process includes many elements, such as:
- Site visits (contextual inquiry) to learn about users, tasks, and environments
- Persona creation to develop specific profiles of your users
- Expert review (heuristic evaluation) to inspect a product or interface against a set of rules or guidelines
- Cognitive walkthrough to understand learning issues for your product under development
- Focus groups to get answers to marketing questions
- Surveys, web-based, paper-based, or phone based
- Training to educate your staff on the benefits of a user-centered design process and to give them the tools to conduct user testing
What types of data do we collect?
Usability testing produces two types of data: qualitative (the reactions of users working with your product), and quantitative (time on task, number of mouse clicks, number of errors, calls to help, responses to questionnaires, and more).
This information can be applied to the continuing development of the product before release to market.
How can benefits be measured?
ROI can be measured in a variety of ways, including reduction of support calls, increased conversion rates, and greater sales, to name just a few.
To see the way in which one company demonstrated ROI after conducting usability testing at Southern Polytechnic, view this PowerPoint™ presentation (2.8 mB).
To read the results of a survey of usability professionals concerning ROI, see the white paper UPA Usability in the Enterprise Project: An Analysis of ROI Measurement and Metrics across Usability Professionals