Rye Skill on Echo Show

Rye

Rye, a Skill for the Amazon Echo Show, prevents food from being forgotten or thrown away by recommending recipes based on the most efficient usage of your inventory.

Project Roles: I led the voice design of the prototype and designed the multi-turn VUI interactions. For the visual design, I created the concept video, Hi-Fi storyboard and branding guidelines.

Timeline: 10 weeks
Team: Corey Brown and Kelly Hwu
Tools: Illustrator, Photoshop, XD, Premiere Pro, After Effects and SaySpring

design challenge

Smart Cities

How might we develop a smart system that helps consumers efficiently manage their food consumption?

final concept

Rye

Rye connects to your Amazon account and creates a virtual food pantry from Amazon and Whole Foods purchases. Rye uses your food preferences along with the estimated expiration dates of your foods to optimally manage your consumption and inventory.

Rye addresses wasteful food habits by not changing the way people purchase food, but rather how they use the foods they purchased.

Expiring Items gif

Opportunity space

Why Rye?

"30-40% of food in the United States goes uneaten, ending up in landfills where it contributes to greenhouse gases."                                                                                                                - Food Waste Prevention and Recovery Assessment Report

We found that people have trouble managing their personal and household food consumption which can lead to food waste. Despite these wasteful habits, there is not a widely-used, or effective solution to tackle food waste at the consumer level.

Through our literature review, we found that in most Americans cities, waste at the consumer level is sorted into three categories: landfill, recycle, and compost. We questioned what was being discarded and saw compost as an opportunity space to explore, especially in the area of expiring food. We made three assumptions about unwanted food and probed further into people’s food waste behavior.

assumptions

01

People prefer not to waste their food and desire a useful alternative that requires minimal effort.

02

People are willing to receive another person's food that would otherwise be thrown away.

03

People are willing to exchange or trade their unwanted food with community members.

in-city intervention

in-city intervention

Testing Assumptions

“I actually would like to, but I have the means to afford what I need.” - C2

We staged an in-city intervention to test our assumptions about exchanging expiring food. We placed a Neighborhood Food Pantry in a residential area and in a community park. The Neighborhood Food Pantry had an assortment of opened, unopened, cooked, and uncooked foods.

We quickly learned through this experiment and user interviews that our food sharing network did not fit with most people’s mental models. Instead of a food sharing network, many people saw the Neighborhood Food Pantry as another form of food donation for the needy.

30 concepts suite

IDEATION

Broad + Narrow

Using our insights from the in-city intervention, we created a response suite that explored 30 diverse concepts to address food waste; from augmented reality cooking classes to smart countertops to personal assistants. We wanted our concept suite to go as broad as possible and include designs for today and designs for the future.

We then narrowed our 30 concepts down to five which we transformed into storyboards to highlight different food waste scenarios. We did not want to reinvent something that addressed food waste because we found current interventions limited to already learned behaviors. Instead, we wanted to explore a solution that tackled behaviors leading to food waste. Our storyboard concepts explored a range of ideas; from virtual currency for farmers markets to a meal prep voice assistant.

hi-fi storyboard

We combined our storyboard scenarios and insights from our initial research phase to create paper prototypes that explored three possible design directions.

design directions

01

Food Sharing Network
Although, the response of a food sharing network was lukewarm during our in-city intervention, we did not want to dismiss the idea completely. We created FoodMatch, an app, that encourages neighbors to exchange unwanted foods.

02

Virtual Cooking Assistant
Let’s Cook, an app, recommends recipes that use the greatest number of foods from your inventory. If you’re missing an ingredient for a recipe, you can search your neighbor’s “virtual food pantry” and ask them for the item.

03

Meal Preparation Voice Assistant
Meal Prep takes into account your calendar and food inventory to schedule meal prep sessions. We felt that a voice assistant would be useful because it offers a hands-free cooking experience.

usability testing

pivoting

User Feedback

"I wish there was something else I can do, but what do I do with bad food?" - P1

We tested our three paper prototypes with six users  and asked each participant to think aloud as they completed their assigned tasks.

At the beginning of each test, we asked the user to describe their current behavior around food waste. All six users mentioned that they would often waste food and felt “bad” or “guilty” about it, but they didn’t know what else to do with expiring foods.

From our user interviews, two insights resonated with us because it focused on changeable behavior. We decided to pivot from a food sharing network to a food management system.

insights

01

Users felt apologetic when they wasted foods and desired a solution that required minimal effort.

02

Users preferred to make efficient use of their food before it expires rather than giving it away.

Voice User Interface (VUI)

Asking Alexa for Help

During our initial round of user testing, more than half of our users were drawn to Meal Prep because of the hands-free cooking experience. Users also felt that the interactiveness of the device would help them manage their food waste better because it held them accountable for their food inventory. Through user testing and feedback, we decided to pursue a VUI for our final concept.

However, during testing we noticed several times that the user would ask “Alexa” to repeat a direction or phrase. To address this cognitive overload, especially in a kitchen environment where multitasking and interruptions can be common, we decided that a visual aid is needed.

While testing Echo devices at the Amazon Bookstore, we interacted with an Echo Show. We were instantly drawn to the Echo Show because the screen complemented the main voice functionalities, but gave the user agency to refer back to the screen for reference.

As smart systems rapidly grow and integrate into our home environment, we felt that the Amazon Echo Show was the right platform to initiate widespread change in food consumption and food waste.

Interaction flows

Mind the Gaps

We created interaction flows for three usage scenarios that addressed potential gaps such as onboarding for a novice user to substituting an ingredient after the recipe has started.

Overall

This is the overall flow of Rye's main features.

Onboarding

This flow outlines Rye’s onboarding scenario and highlights key features such as the user's inventory and recommended recipes.

Low-confidence

This flow outlines an instance when Alexa has “low confidence” in an ingredient because it was either purchased a long time ago, has not been used in awhile, or has expired. 

Recipe + Feedback

This flow shows when the user does not have an ingredient after starting a recipe and Rye makes a substitution recommendation.

View prototype

user testing

Creation of Rye

We designed our HiFi prototypes for the Echo Show and conducted a second round of user testing. During this round, our users stressed the importance of seeing the ingredients for each recipe step. They also wanted the flexibility to see previous and future steps. We took their feedback into consideration and created the final design for Rye.

View a recipe gif
Next step gif

UI SPEC

"Alexa, Open Rye"

Our UI Spec highlights Rye's design process, information architecture, and interaction flows.

Next Steps

Beyond Amazon

For future iterations of Rye, we would like to expand the experience beyond Amazon and include grocery purchases from other stores. While creating Rye, we toyed with the idea of tracking non-Amazon purchases through store rewards cards or credit cards. However, we need to further explore these tracking techniques as well as other methods to find an effective solution.

reflections

Learn Often

Throughout the design process of Rye, I learned that it is important to quickly adapt to change and proactively learn new skills.

One of the challenges that we faced is our lack of familiarity with the Echo Show. Initially, we read a lot about the Echo Show through the developer guide and secondary research. However, we did not fully understand the Echo Show's voice interactions or style guidelines until we interacted with one. When designing for an existing interface it is crucial to interact with the device early on in the design process because it can lead to better design decisions.

Despite this challenge, we quickly learned about VUIs and prototyped a working model with limited resources. We also learned new software such as Adobe XD which gave us more flexibility in creating Hi-Fi prototypes of the Echo Show and SaySpring which allowed us to create multi-turn VUI interactions. Ultimately, our flexibility during the design process and eagerness to learn helped us create a successful project.