2016

Project Soli: Interacting with audio in mid-air

Putting Google’s Advanced Technology and Projects group’s (ATAP) high-performance radar-based gesture sensor in action for connected home audio.
A team at Google’s Advanced Technology and Projects group (ATAP), led by Dr. Ivan Poupyrev, are testing its Soli sensing technology in different contexts to enable people to control devices without touching a single button, knob or screen. We recently carried out a close, embedded collaboration with Google ATAP on Project Soli, putting the high-performance radar-based gesture sensor in action for connected home audio. 
Approach

Soli is a miniature radar that detects touchless gesture interactions. The purpose-built interaction sensor can track sub-millimeter motion at high speeds and accuracy, allowing creation of what Google describes as “ubiquitous gesture interaction language that will allow people to control devices with a simple, universal set of gestures.”

Working with the technology in the JBL by Harman speaker, we set out to provide the ATAP team with a deep understanding of users’ experience interacting with audio in mid-air. Although it may initially be thrilling to magically control volume, skip songs and change radio stations with a wave of your hand, what changes when people can no longer feel or touch a device? And what will it take to get used to using gestures rather than touch? 

One of the main factors to consider with this new form of interaction is how users receive feedback. With physical controls and an interface, the feedback is instant and more easily defined – you can feel a device turn off when you press a button, or see numbers countdown on a microwave screen. If there’s nothing but air, how do you guide the user? 

UI (User Interface) is our specialty, so to answer these questions our multi-disciplinary team, offering a unique skillset of engineering, service design, UX design and industrial design, set out to explore the changing audio landscape and trends. We then visited people in their homes to discuss how they currently use connected audio, their likes and dislikes, and future possibilities. And, armed with the prototypes our team created to test a variety of interaction forms with potential users, we shot videos to show how the gesture controls worked, how users reacted to it, and to tell the story of their overall experience.

Result

The insights derived from these tests allowed us, in a very close cooperation with ATAP and JBL by Harman, to recommend and develop various industrial design adaptions. And as I/O 2016, Google’s yearly developer conference, was fast approaching our team continued to support ATAP and JBL by Harman in producing a functional prototype that was presented at the event.