User research & testing:
small appliance with
digital interface
CLIENT: INDUSTRY-LEADING SMALL APPLIANCE COMPANY
ROLE: UX RESEARCHER
SKILLS: UX RESEARCH, STUDY DESIGN, USABILITY TESTING, IN-PERSON MODERATING, DATA ANALYSIS/ SYNTHESIS, REPORT DEVELOPMENT
TOOLS: 1 BEYOND VIDEO, USTREAM
DESCRIPTION
Conduct user research during the development phase of a new physical product from an industry-leading small appliance company. Our team at the User Experience Center (UXC) at Bentley University was hired to conduct research and make UX recommendations on a number of aspects of a new product with a digital interface. Areas of interest included the machine’s cleaning/maintenance process, instructional materials, and product differentiation on packaging and labeling. I was part of a small team working on this product, which included a UX manager and one other researcher, for a duration of 5 months. The product was released to market in late 2018.
CHALLENGE
Develop and run user research studies to test key aspects of the product. This particular study tested user understanding of the cleaning/ maintenance process for the machine based on the interface and user guides/ instructional materials. The goal was to understand if users were able to understand and complete the cleaning procedure correctly. Our client was also interested in understanding whether or not users understood the urgency of messaging and implications of not completing the process as prompted by the machine. The elements we tested were unfinished prototypes that were continually changing - sometimes just days before testing began. Our challenge was to adapt quickly to prototype changes, and present unfinished prototypes in ways that still allowed us to gauge the experience and made sense to users unfamiliar with this new type of product.
METHOD
Develop a testing protocol using the combination of digital and physical prototypes available to us from our client to gauge user understanding. Synthesize and analyze findings into a report and develop key design recommendations for improvement.
PROCESS OVERVIEW:
STUDY DESIGN —> IN-PERSON USABILITY TESTING —> DATA ANALYSIS / SYNTHESIS —> DESIGN RECOMMENDATIONS & FINAL REPORT
study design
Study design involved creating ways to test users’ understanding of the cleaning process and instructional materials. Participants were introduced to the product concept first, then asked to imagine they unboxed the product and found the instructional materials. It was important to give enough context to understand the product without biasing their understanding of the process we were testing.
STUDY GOALS
We were interested in understanding:
are instructional materials clear and easy to understand?
will users be able to correctly complete the cleaning process?
what, if any, are the pain points of completing the cleaning process?
do the instructional materials provide enough information to complete the cleaning process?
method
Participants were asked to think aloud as they reviewed prototypes and completed tasks. Additionally, specific questions were developed to measure understanding of the cleaning process and instructional information and to identify pain points and successes in terms of usability.
KEY TASKS
INSTRUCTIONAL MATERIALS
Have participants review the instructional materials and provide feedback
Questions on look and feel and content understanding
CLEANING PROCESS
Observe participants completing the cleaning process using a think-aloud method
Questions about UI design, time involved, and overall thoughts on the process, in order to identify pain points and successes in terms of usability
OVERALL PRODUCT FEEDBACK
Gather ratings from participants regarding the simplicity, intuitiveness and overall feel of the product and experience
OVERALL PRODUCT FEEDBACK
Gather ratings from participants regarding the simplicity, intuitiveness and overall feel of the product and experience
KEY TASKS
They were asked to review the instructional materials and provide feedback.
Participants were given a set of tasks related to the cleaning process of the machine, which they completed using a mixture of physical and digital prototypes of the machine.
Interactive high-fidelity digital prototype with inter (InVision) (interface)
allowed participants to interact with a prototype of the machine’s digital interface
Medium fidelity physical prototype with limited functionality
allowed participants to test the physical aspects of the machine
The digital prototype was interactive and
•Participants were shown a prototype of the machine in physical and digital UI formats. Because both prototype formats could only individually complete a portion of the overall cleaning process participants were directed to complete steps incrementally and switch between the UI and the physical machine.
We conducted 90-minute sessions with 11 participants, who were a mix of male and females who met our recruit criteria.
Conducting an expert review first allowed us to:
Familiarize ourselves with the user journey
Identify surface level problems quickly so that valuable user testing time could be used to take a deeper dive on specific usability issues
Use best practices and established heuristics to develop hypotheses to test
Helped shape the order of tasks and prompts in testing
Sample finding from my individual Expert Review:
Bolded items in the data section appear clickable, but are not.
To find the data indicated here, the user must know to look inside the ‘details’ tab.
Recommendation: Make the bolded data elements active links that connect to the details page, and open to the corresponding item.
discovering this allowed us to test the hypotheses that users would struggle to access the data shown here:
——> during usability testing, all 8 participants unsuccessfully tried to click the bolded items
CONTEXTUAL INQUIRY
An observational visit was made to a hospital Emergency Department (ED) that currently uses this technology alongside traditional radio communication. I observed the context of use for the application, differences in workflow between the two technologies, and varying user needs in the ED and ambulance environments.
This involved observation and informal interviewing of several types of end users including:
Charge nurse
EMS dispatcher
EMS Director
EMT
contextual inquiry highlights
USABILITY TESTING
Using insights from our heuristic review and contextual inquiry, a moderator's guide was developed, with tasks specifically intended to:
Investigate the workflow for receiving notifications and filtering emergency cases
Understand the experience for first-time users
Eight 1:1 Usability sessions were conducted (in-person and remote) using test accounts within the app.
recruit: 1 ems manager; 3 neurology residents, 2 physicians, 1 emergency physician, 1 registered nurse
Key tasks included:
Viewing incoming ER Cases
Receiving Notifications from other members of the ER/EMT staff
Filtering for ER Cases relevant to user type
Filtering Notifications
Setting a logout/end time
USABILITY RESULTS
QUALITATIVE
QUANTITATIVE
selected quotes helped build the story of the user experience for the target demographic
“These categories aren’t quite [specific] enough...but if they were too narrow I’m worried I’d miss a patient.”
“I got an alert! So...I press View?
...I would have thought it would bring me right to that case.”
Average Ratings of Task Difficulty
At the end of each task, users were asked to rate the process according to level of difficulty, where 1= Easy and 5 = Difficult.
FINAL FINDINGS & RECOMMENDATIONS
finding
Overall, the app is intuitive to navigate and use, even for first time users
Certain features only apply to specific user types and confuse others. Slowing down or confusing users in the context of emergency care can be costly and impact life-saving outcomes
Following the data journey can be challenging for users; viewing media files and incoming cases can sometimes be difficult
Filters for cases and notifications do not have a clear distinction, leading users to wonder what they will be notified of
Recommendation
Maintain simplicity of the design, especially considering use context
Support several “profile types” that are more tailored to user needs. Allow users to sign into profile types as part of their initial signup; allow them to change their profile type if necessary
Resolve situations where interface does not meet user expectations & add search and sort functionality to cases to increase data traceability
Expand customization options, including clear choices for which notifications get pushed to the user’s phone and clear sign-out notifications