Here’s How Apple’s HomePod Can Hear You Across The Room

Ever wonder how the Apple HomePod is able to hear you say, “Hey Siri” from across a noisy room? It took a lot of very smart people to make that happen.

A new post on Apple’s machine learning blog details all the thought and technology — both hardware and software — behind the marshmallow-shaped speaker’s ability to hang on your every word. The challenges, it turns out, were pretty immense.

“The typical audio environment for HomePod has many challenges — echo, reverberation, and noise,” Apple explains. “Unlike Siri on iPhone, which operates close to the user’s mouth, Siri on HomePod must work well in a far-field setting. Users want to invoke Siri from many locations, like the couch or the kitchen, without regard to where HomePod sits.”

To deal with these problems, the team used a mixture of what it calls “supervised deep learning” and “unsupervised online learning” to determine how to interpret the signals from its multiple onboard microphones. Essentially, the speaker is able to use this machine learning algorithm to determine which is the optimal audio stream for listening at any given moment, allowing it to always have a keen ear out for where it might be hearing, “Hey Siri.”

There is a heap of cool software that allows the speaker to optimize speech detection and tune out things like music, environmental sounds, and other non-speech audio. But even when it is focused on speech, there are other challenges.

“Far-field speech recognition becomes more challenging when another active talker, like a person or a TV, is present in the same room with the target talker,” reads the post. “In this scenario, voice trigger detection, speech decoding, and endpointing can be substantially degraded if the voice command isn’t separated from the interfering speech components.”

The team developed a unique formula for dealing with this issue — the actual formula can be found on the company’s website — and even developed special ways of dealing with echo cancellation and suppression.

The fact that so much thought went in to creating a product like the HomePod and making it actually work isn’t all that surprising, but being able to hear exactly what the challenges were and how they were solved straight from Apple — one of the most secretive companies on Earth — is refreshing. We recommend you check out the full blog post for more information.

While you’re at it, be sure to check out our list of the best smart speakers if you’re in the market.