Hunches | Case Study
Designed an intervention to collect users' feedback on Alexa's proactive experiences
Role
Product designer
Timeline
16 weeks
This project was part of an Amazon-Sponsored Externship mentored by Amazon's UX Researcher, Monica Chan.
Contributions
Co-design workshop
Developed majority of the activities and supporting materials for the workshop
Co-hosted the workshop
UI explorations
Assisted in UI explorations
Generated Hi-Fi mockups for user testing
Research
3 interviews, user-tests, and Reddit users' insights
Competitor analysis
Context
Proactive actions?
Actions taken by a smart device without user's input. In Alexa's term - Hunches
Scope
1 in 4 smart-home interactions is initiated by Alexa. Number bound to increase
Design space
Imagine you're pulling a late-night work session
Suddenly, Alexa turns the light off at 10pm
Confused, you have no idea what happened and why
Problem
No Hunch awareness
Feedback system inefficient
Security and privacy concerns
Why need user feedback?
To reduce incorrect actions and improve user trust
In a nutshell
Awareness
More User Feedback
Fewer Incorrect Actions
Improved User Trust
Current mediums
To collect user feedback
Alexa phone app - A simple yes/no
Alexa Echo show - Digital and voice UI
Research
Expand to see
Ultimate Goal
To get contextual feedback from users on the proactive actions
Problem space 1/4
Discoverability
We realized nobody knew what Hunches were.
How can we collect contextual feedback on a feature that users aren't even aware exists?
In the current model, Hunches activities are buried within the settings, requiring users to navigate four levels deep in the hierarchy. This makes Hunches difficult to locate and the reason for lack of awareness within users.
Solution
A new "Hunches" section now appears on the home screen to alert users of actions initiated by Alexa
A new tag introduced in the device tile will inform users of proactive actions performed on it
Interations
1
Nav bar access
Initial - Hunches section on the bottom nav bar
Problem - Not feasible for this significant a change. It's a new feature, still under R&D
2
Tab on the home screen
Iteration - Tab on home screen (leveraging scalable model), avoiding significant change
Problem - could get lost among the others as they shuffle.
3
On device tiles
Iteration - additional tag on device tiles to inform proactive actions
More research
Expand to see what we did
Important insights
Prefer notifications to be semi-proactive, except for critical situations
Do not prefer to provide verbal feedback or engage in verbal conversations for long
Give feedback only when something doesn't work as expected
More likely to give feedback when the impact of feedback was transparent
Users do not prefer verbal feedback
I was thrilled at the start of the project, eager to dive into Voice User Interface and craft conversational designs. However, the insights we gathered suggested otherwise. We had no choice but to pivot.
So how do we notify the users?
Problem space 2/4
Hunch awareness
Turned off by default
Hunch notifications are off by default - missed notifications on Hunches
Toggling applies to all - notification overload
Solution
Step 1
While installing device - set alerts without additional steps
Allows notifications only for critical devices (eg. main and garage door, baby room's devices)
Step 2.1
By leveraging the Echo Show's always-on display, we can grab the user's attention to notify them of proactive actions triggered by Alexa without being intrusive
Current Widget
(to invoke Alexa)
(to invoke Alexa + Hunch notifications)
Re-Designed Widgets
Step 2.2
By leveraging the Echo Show's always-on display, we can grab the user's attention to notify them of proactive actions triggered by Alexa without being intrusive
Interations
Expand to see
Problem space 3/4
Contextual feedback
A simple 'yes/no' feedback is not sufficient to improve future actions
Solution
Step 1
Added a two-step contextual feedback system. Hunches are based on assumptions that Alexa makes based on user's daily patterns. This solution makes the the assumptions transparent and asks which assumption was wrong
Step 2
If they choose to provide further details, Alexa shows the context on which it's Hunch was based asks which assumption was incorrect. This make's Alexa context-aware and can improve it's future proactive actions
Interations
Expand to see
Problem space 4/4
User retention
Users
"Giving feedback a couple of times doesn’t sound so bad, but I honestly can’t see myself doing it for weeks or months"
How can we encourage users to provide feedback for long?
Solution
Research suggested users were more likely to give feedback when the impact of feedback was transparent. To encourage this, we made the process transparent by showcasing Alexa taking actions based on the feedback previously given.
Recap
Let’s take a quick look at the app’s flow to recap 16 weeks of effort put together by me and my team.
Hunches on home screen
Impact
Accessibility to Hunches
Improved from 10% to 80%
Interaction with Hunches
70% increased
Improved user flow
Expressed low frustration navigating Hunches
User Trust
Showed confidence in Alexa's proactive actions
After more than 10 usability tests
Personal significance
Amazon externship
Unique opportunity to collab with an experienced researcher
Learnings - problem solving, user-centric solutions, efficiency, agile methods
Smart devices
Conversational design - new and exciting domain
IoT devices - A deeper understanding of how they function
Brainstorming exciting and innovative concepts for virtual assistants
Reflections
Working in big teams
Initially, poor communication in our large, unfamiliar team led to missed meetings and incomplete tasks. Assigning ownership based on individual interests helped, each person took charge of their area and got others involved. Within weeks, collaboration improved naturally, and I plan to use this method in future large teams.
Maintaining interest when things change
The project began with a focus on voice design, which excited me, but research showed it wasn’t ideal, forcing a pivot. With limited flexibility in Amazon’s app, it was demotivating at first. But I found purpose in improving even small details, which reignited my interest. It taught me that priorities shift fast in the industry, and adapting to make the most of the scope is key.
Team
