Smart speakers have proven adept at monitoring certain healthcare issues at home including detecting cardiac arrest or monitoring babies’ breathing. Now, the speakers can be used to track the minute motion of individual heartbeats in a person sitting in front of the speaker.

The system monitors both regular and irregular heartbeats without physical contact. It sends inaudible sounds from the speaker out into a room and based on the way the sounds are reflected back to the speaker, it can identify and monitor individual heartbeats. Because the heartbeat is such a tiny motion on the chest surface, the system uses machine learning to help the speaker locate signals from both regular and irregular heartbeats. When tested on healthy participants and hospitalized cardiac patients, the smart speaker detected heartbeats that closely matched the beats detected by standard heartbeat monitors.

While many people are familiar with the concept of a heart rate, doctors are more interested in the assessment of heart rhythm. Heart rate is the average of heartbeats over time, whereas a heart rhythm describes the pattern of heartbeats. If a person has a heart rate of 60 beats per minute, they could have a regular heart rhythm — one beat every second — or an irregular heart rhythm — beats are randomly scattered across that minute but they still average out to 60 beats per minute.

Heart rhythm disorders are actually more common than some other well-known heart conditions. Cardiac arrhythmias can cause major morbidities such as strokes but can be highly unpredictable in occurrence and thus difficult to diagnose. The key to assessing heart rhythm lies in identifying the individual heartbeats. For this system, the search for heartbeats begins when a person sits within 1 to 2 feet in front of the smart speaker.

Then the system plays an inaudible continuous sound that bounces off the person and then returns to the speaker. Based on how the returned sound has changed, the system can isolate movements on the person — including the rise and fall of their chest as they breathe.

The motion from someone’s breathing is orders of magnitude larger on the chest wall than the motion from heartbeats, which poses a challenge. Also, the breathing signal is not regular, so it is difficult to simply filter it out. Since smart speakers have multiple microphones, a beam-forming algorithm was designed to help the speakers find heartbeats.

The team designed a self-supervised machine learning algorithm, which learns on the fly instead of from a training set. This algorithm combines signals from all of the smart speaker’s multiple microphones to identify the elusive heartbeat signal.

The heartbeat signals that the smart speaker detects don’t look like the that are commonly associated with traditional heartbeat monitors. The researchers used a second algorithm to segment the signal into individual heartbeats so that the system could extract what is known as the inter-beat interval, or the amount of time between two heartbeats.

The researchers tested a prototype smart speaker running this system on two groups: 26 healthy participants and 24 hospitalized patients with a diversity of cardiac conditions including atrial fibrillation and heart failure. The team compared the smart speaker’s inter-beat interval with one from a standard heartbeat monitor. Of the nearly 12,300 heartbeats measured for the healthy participants, the smart speaker’s median inter-beat interval was within 28 milliseconds of the standard monitor. The smart speaker performed almost as well with cardiac patients: of the more than 5,600 heartbeats measured, the median inter-beat interval was within 30 milliseconds of the standard.

Currently, the system is set up for spot checks. If a person is concerned about their heart rhythm, they can sit in front of a smart speaker to get a reading. But the team hopes that future versions could continuously monitor heartbeats while people are asleep — something that could help doctors diagnose conditions such as sleep apnea.

For more information, contact Sarah McQuate at This email address is being protected from spambots. You need JavaScript enabled to view it.; 206-543-2580.