The explosive growth of their AI voice assistants has Google, Apple, and Amazon racing to put your entire smartphone in an earpiece.
In October 2016, an impressive group of tech industry royalty took the time to get a demonstration of a product from a startup called Doppler Labs. Microsoft cofounder Bill Gates and CEO Satya Nadella each got one, as did Apple Internet chief Eddy Cue and Jimmy Iovine, head of Apple’s Beats headphone group. So did C-suite executives from Amazon, Facebook, Google, and Tencent.
Donning pre-production versions of Doppler’s Here One wireless earbuds, they experienced the device’s ability to cancel out unwanted background noise, amplify the voice of a particular person in the room, and even converse with people speaking in another language. About a half-second after a Doppler staffer asked a question in Spanish, the wearer heard a computerized translation back into English.
At least two of the companies made informal acquisition bids, but none offered a high-enough price to convince Doppler to give up its dreams of launching a momentous new product. Sales failed to take off, and a year later the company was shuttered. But that’s not the end of the story. Within weeks of its closing, more than half of Doppler’s top technologists were working for the tech giants.
Now, Fast Company has learned that Amazon, Apple, and Google each have high-priority projects to pick up where Doppler left off. All three are working on products that combine the utility of the hearing aid with the entertainment value of a pair of high-end headphones, and potentially much more, say sources. Since all three have announced plans to get into healthcare, they could easily add fitness and health monitoring sensors for everything from counting steps to measuring oxygen saturation. And while it may take years to happen, none want to be left behind should it become possible to create a general purpose, in-ear computer that allows consumers to leave their phone in the desk drawer.
“Ultimately, the idea is to steal time from the smartphone,” says Gints Klimanis, Doppler’s former head of audio engineering. “The smartphone will probably never go away completely, but the combination of voice commands and hearing could become the primary interface for anything spontaneous.”
Spokespeople for Amazon, Apple, Google, and Microsoft all declined to comment for this article.
WHY HEARABLES, WHY NOW?
For half-a-decade, Doppler and other startups have been trying—and failing—to come up with a “hearable” with the combination of sound quality, battery life and cool factor to become a mass market hit. So. why the sudden interest from the big guys? Because personal voice assistants such as Amazon’s Alexa, Apple’s Siri, Google Assistant and Microsoft’s Cortana, have suddenly emerged as the biggest interface revolution since the iPhone popularized the touchscreen.
RELATED: Are smart assistants good for users? An honest look at the pros and cons
Our desire to use technology without the hassle of touching it has made smart speakers the fastest growing new hardware market in years, says Strategy Analytics’ analyst Cliff Raskind. By 2023, 63 percent of U.S. homes will have a device like the Amazon Echo or Google Home, up from .03 percent in 2014 and 16 percent in 2017.
By then, Americans will speak rather than type more than half of their Google search queries, predicts Comscore. The market for ads delivered in response to voice queries will be $12 billion, according to Juniper Research. And these predictions don’t even contemplate a future when consumers have computers in their ears for more of their waking hours, providing tech giants with even more data on their movements and desires—not to mention a channel into their brains that makes shopping as frictionless as saying “Alexa, buy (fill in blank).”
THE FUTURE IS EAR
“There’s much more that tech companies can do with ears than amplify music and make phone calls,” says Satjiv Chahil, a former Apple marketing executive who has advised hearing aid maker Starkey Hearing Technologies in recent years. “It’s about allowing your virtual assistant to whisper in customers’ ears throughout the day, while also enhancing their health and well-being.”
RELATED: How I Heard The World With Nuheara’s Bionic Earbuds
Your ears have some enormously valuable properties. They are located just inches from your mouth, so they can understand your utterances far better than smart speakers across the room. Unlike your eyes, your ears are at work even when you are asleep, and they are our ultimate multi-taskers. Thousands die every year trying to text while they drive, but most people have no problem driving safely while talking or dictating messages–even if music is playing and children are chatting in the background. Ears are not in the front of your face, so it may be easier for the Jony Ive’s of the future to come up with fashionable or even invisible designs for the ear than for the eye. Remember Google Glasses?
Those are just the obvious advantages. With the right sensors and processing on board, a hearable can tell if your head is pointed toward a store shelf in front of your face or at a billboard down the road. Add in a heart-rate monitor to measure stress and an electroencephalogram sensor to analyze spatial brain activity, and it could know what you are thinking about to some degree—say, how much of your attention is being paid to the sound of footsteps coming up behind you, says Poppy Crum, chief scientist at Dolby Laboratories.
Yes, AI-enhanced hearables in the future will be able to understand more than the words we speak. A Cambridge, UK-based startup called Audio Analytic is already licensing the ability for a device to recognize the sound of a window breaking or a baby crying. At this rate, it won’t be long before Amazon can send ads for Robitussin when it hears you cough.
THE HEARABLE CHALLENGE
The ear also presents nasty challenges for any company hoping to sell a mass-market computing device. Such a device must be tiny, nearly weightless and fit perfectly in each person’s anatomically unique ear canal to be comfortable for long stretches of time. At the same time, it must have enough battery power to last at least as long as a smartphone, not to mention a strong antenna and on-board processor. There’s also the problem of earwax, and the unsolved mystery of how to use an ears-only device without too much head-shaking, hand-waving, ear-tapping, or self-talking. According to one recent study, only six percent of Americans said they were comfortable talking to their voice assistant in public.
Then there’s the stubborn stigma against hearing aids. While hundreds of millions of people think nothing of wearing head or earphones, only 16 percent of the 48 million Americans who could benefit from hearing aids have purchased a pair, says the Hearing Loss Association of America. Those that do buy tend to put it off for an average of seven years.
Tight regulation of the industry hasn’t helped. Because hearing aids have been defined as medical devices, manufacturers must get products approved by the Food & Drug Administration, and consumers need to get a doctor’s prescription and pay to see an audiologist, usually with no help from insurance. Content to go for profit margins over sales growth, a stodgy oligopoly of five companies has been able to dominate the $6 billion-a-year hearing aid industry , selling products that cost an average of $2,700 per pair, according to Consumer Reports. A top-of-the-line pair will set you back $10,000 or more.
Now, that regulatory anchor is about to come loose. Last August, Congress passed the “OTC Hearing Aid Law of 2017”. When it goes into effect in August 2020, if not sooner, companies will be able to sell hearing aids over the counter to people with mild to medium impairment online or at any drugstore, just like glasses makers sell $10 readers to people who don’t want to bother with an optometrist.
This opens a large and growing market. The World Health Organization says that 1.1 billion children and young adults around the world are at risk of hearing loss, having grown up with earphones blasting away at point-blank range.
The law could have dramatic impact. Suddenly, anyone who finds themselves saying “what?” more often than they would like will be able to walk into a Walgreens and buy a consumer-y looking device for a few hundred dollars. Very likely, they will pick it up in the electronics aisle next to colorful iPhone covers and FitBits, not in the aisle for Depends and other products for the elderly. The device may not be marketed as a hearing aid at all, but as Bluetooth earphones with “hearing enhancement” or “personalization.”
“I’ve been waiting for this moment for 20 years,” says KR Liu, Doppler’s former vice president of accessibility, who has worn hearing aids to battle severe hearing loss since she was three. “You have these amazing companies that can do amazing things and have the branding power to de-stigmatize hearing aids.”
DOPPLER’S INFLUENCE
Doppler didn’t invent the hearable, but it had outsized influence during its brief existence. Music industry exec Noah Kraft and former Microsoft executive Fritz Lanman created the company in 2013 to come up with a product the Coachella crowd could use to customize the sound of live music, such as the ability to add a “fuzz” effect or put an upper limit on the volume. By early 2016, it had assembled an impressive team of audio experts and was working on the Here One, which added more “hearing augmentation” features as well as the ability to make phone calls and stream music.
As word of the product’s capabilities began to spread, Doppler began getting inquiries from tech giants interested in catching the hearable wave. Although the Here One was supposed to come to market within months, Kraft, the company’s CEO, declared that October to be “Demo Month.” A small team set up shop in a swank conference room overlooking San Francisco Bay in the offices of Universal Music Group, one of Doppler’s first investors.
Visitors included venture capitalists such as Mary Meeker and Yuri Milner, and teams from Amazon and its Silicon Valley-based R&D unit, Lab 126, as well Google, Apple, and Facebook. While a few companies put some informal bids on the table, none offered anything close to the valuation Kraft and Doppler’s board felt the company was worth.
RELATED: What Doppler’s Here One Earbuds Do–And How They Do It
Any unicorn dreams faded away after the Here One went on sale in early 2017. The tech press praised the device for its innovative design, but abysmal battery life and difficulty explaining why this wasn’t just another wireless earphone led to poor sales. There was one bright spot, however. Nearly a quarter of the buyers had purchased it as cheaper, better sounding alternative to hearing aids—without any marketing effort by Doppler to reach this audience. With Apple’s AirPods taking the consumer earphone market by storm, Doppler decided to pivot. While the engineering team focused on hearing functionality, Liu took a lead role in lobbying for the OTC bill in Washington D.C.
By the time the bill became law last August, Doppler was in dire straits. Kraft re-approached potential acquirers, who immediately agreed to meet. A team from Microsoft, including Nadella, explored whether hearables could be used to boost worker productivity. The companies collaborated on several interesting ideas. Since the Here One had an inward facing microphone that amplified the sound of the wearers own voice, why not create software commands Word or Excel users could whisper so quietly that workmates wouldn’t even notice. In the end, Microsoft decided to pass.
Several teams from Google took another look and passed, including one from the X “moonshot factory” and one from the company’s hardware division, which was looking for help finishing up its soon-to-be-panned Pixel Buds (aka “Pixel Duds”).
In September 2017, Apple sent a large team for another round of talks. It clearly didn’t need Doppler. Apple had been learning about hearing technology since 2011, when it began forging partnerships with hearing aid makers, so customers could pipe sound picked up by the mic in an iPhone directly into their hearing aids (a student, for example, could put their iPhone near a teacher at the front of the room to hear the lecture more clearly). The company had poured big money into creating technology such as the W1 communications chip, which has helped make the AirPods a stand-out in terms of sound quality, battery life and ease-of-use. AirPods captured 24 percent of all wireless earphone sales in the first half of this year, far ahead of runner-up Beats with just 3 percent, according to NPD Group. Still, Apple remained interested in an aqui-hire of key Doppler technologists, particularly those working on hearing algorithms, but wasn’t willing to pay enough to interest Doppler.
Talks with Amazon lasted the longest and were the most serious. With a powerful hearable, it’s customers would be able to shop via Alexa even when not near a smart speaker—and without having to depend on an iPhone or Android device. The Lab 126 team had been looking for years for a way to “get Alexa out of the house.” But aware of Doppler’s fast-declining finances—and possibly because it had learned so much about Doppler’s technology and business during negotiations–Amazon’s deal-makers only offered a low-ball bid
Rather than accept any of the bids, Kraft chose to shut down the company a few weeks later. He later sold Doppler’s intellectual property to Dolby, which specializes in audio software to enhance the sound of movies and other media. Dolby has not confirmed any new products based on Doppler’s patents, but “we are spending time identifying how our technology, ecosystems, and knowledge are relevant to the hearable marketplace,” says Crum, the company’s chief scientist.
“It’s good to hear that Doppler’s vision lives on even though the company doesn’t,” says Kraft via email. “We’re proud of what we built and proud that the Doppler team is helping others bring the in-ear computer to fruition.” Kraft declined to comment on any discussions with Amazon, Apple or any other suitors.
NEXT UP
Doppler is gone, but the vital signs of the hearables market are getting stronger. Salaries for audio technologists are soaring, with big tech companies often paying $200,000 salaries to top talent from startups and the traditional hearing aid companies. Mobile chip giant Qualcomm introduced its first family of chips specifically for hearables in March, and other chip companies are expected to follow suit by the end of the year.
Amazon, Google, and Apple are keeping their cards to the vest. Three former Doppler employees say Amazon already had a team of 70 people working on hearables when the companies were in talks last year. While Google’s hardware team continues works on Pixel Buds and other products, Google’s X unit is looking at developing fully independent in-ear computers, while the Google Voice unit focuses on ways to make that personal assistant more accessible via ear-based devices, says a person that’s had dealings with all three.
Apple is also marching ahead in its deliberate way. Rather than build a revolutionary new product to usher in the hearable era, it will continue to add new capabilities in familiar form factors, sources say. According to Bloomberg, the company will announce high-end headphones for music lovers by the end of the year, and will introduce a water-resistant upgrade of the AirPods, that includes the ability to activate the device by saying “Hey, Siri.”
Other pioneers of the hearables market are already preparing for the big guys’ arrival. Bragi, a German company founded shortly before Doppler, recently decided to stop selling its hearable devices in favor of licensing its software.
“When you’ve got Apple and others coming directly after you, you need to change where you invest,” says CEO Nikolaj Hviid. “On the other hand, it’s nice to suddenly be getting all this company.”
