An attempt to enable hands-free operation of a V-ZUG Combi-Steam XSL.

Why

In the kitchen, you always lack a hand or two: pots, kids, ingredients, and a phone call.. it's often a juggling act. So voice-controlled devices are a valuable little helpers. Works in my kitchen today, for lights, timers, the family's shopping list, etc. If it also worked for some of the appliances.. that would be nice! Also: I hear from more and more elderly people for which voice user interfaces are unbelievably valuable - and modern devices, like a smart phone, are super hard to work with: too hard to read, too many options, too complicated. A smart kitchen could make things actually worse for them, if an oven becomes as hard to use like a phone. So let's see what we can do..

 What

• Amazon Alexa Skill: configured a model to accept the different ways to talk to our device, like "tell my steamer to {mode} for {duration} at {temperature}
• Amazon Lambda Functions: accept Alexa input, build and enqueue messages on the correct "device shadow"
• Amazons IOT: our secured message queue, basically
  
• Local Node.js gateway: authenticates to Amazon IOT and translates the message payloads into API calls to the V-ZUG-Home API

 Challenges

• Security: handling X.509 issues at night, after a hard week of work, while also helping to run the hackathon and coach teams..
• Security: the device requires manual confirmation for any action that does actual work, like heating. That's 2FA! How could we design a system that is secure (safe!) enough to overcome that? Can we use other sensors of the device to detect the presence of a person? Could we craft an API that is secured enough to allow heating commands? Or will this all lead inevitably to hackers saying "Alexa, burn the house down"?
• Voice UI design: what does an oven need to understand? Just degrees, or hotter/colder? What extreme values would we have to catch before they take effect? What would most common misunderstandings be that we'd have to catch in daily use? What personality, what tone of voice should such a device have? What special voice control features do the different programs need? The wet towel program probably should not say "Bon appétit" when done?

 Status

Components above ready, test with Alexa test environment work.

Next steps:

• Try out in real life
• Come up with a realistic threat model (denial of service, energy waste, fire hazard, privacy breaches, ..)
• On the API, but also the oven level: see what we can do to create a more secure setup
• On the UX level, test, test and test

Who

Hannes Gassert hannes@liip.ch