A system called Marko uses signal reflections to provide scientists and caregivers with valuable insights into people's behavior and health. Marko transmits a low-power radio-frequency (RF) signal into an environment. The signal will return to the system with certain changes if it has bounced off a moving human. Novel algorithms then analyze those changed reflections and associate them with specific individuals. The system then traces each individual's movement around a digital floorplan. Matching these movement patterns with other data can provide insights about how people interact with each other and the environment.
The system was tested in real-world use in six locations: two assisted living facilities, three apartments inhabited by couples, and one townhouse with four residents. The case studies demonstrated the system's ability to distinguish individuals based solely on wireless signals — and revealed some useful behavioral patterns.
When deployed in a home, Marko shoots out an RF signal. When the signal rebounds, it creates a type of heat map cut into vertical and horizontal “frames,” which indicates where people are in a three-dimensional space. People appear as bright blobs on the map. Vertical frames capture the person's height and build, while horizontal frames determine their general location. As individuals walk, the system analyzes the RF frames — about 30 per second — to generate short trajectories called tracklets.
A convolutional neural network — a machine-learning model commonly used for image processing — uses those tracklets to separate reflections by certain individuals. For each individual it senses, the system creates two “filtering masks” — small circles around the individual. These masks basically filter out all signals outside the circle, which locks in the individual's trajectory and height as they move. Combining all this information — height, build, and movement — the network associates specific RF reflections with specific individuals.
To tag identities to those anonymous blobs, the system must first be “trained.” For a few days, individuals wear low-powered accelerometer sensors that can be used to label the reflected radio signals with their respective identities. When deployed in training, Marko first generates users’ tracklets, as it does in practice. Then, an algorithm correlates certain acceleration features with motion features. When users walk, for instance, the acceleration oscillates with steps but becomes a flat line when they stop. The algorithm finds the best match between the acceleration data and tracklet and labels that tracklet with the user's identity. In doing so, Marko learns which reflected signals correlate to specific identities. The sensors never have to be charged and after training, the individuals don't need to wear them again.