Researchers have developed a sensor package that monitors multiple phenomena in a room using machine-learning techniques. The prototype contained 19 different sensor channels, including sensors that indirectly detect sound, vibration, motion, color, light intensity, speed, and direction. The sensor board is plugged in to a wall outlet, eliminating the need for batteries.
After developing the prototype, the team gave the sensor “training” data by providing it with hundreds of real-world examples of what 38 different devices and appliances sound like when in use. Once the sensor “learns” this data, it can then be presented with a new object where it can pick out what it is, based upon the 38 patterns it’s already recognized.
Transcript
00:00:00 [Music] the promise of Smart Homes offices classrooms and indeed the entire Internet of Things relies on robust sensing of the environment there are two ways to achieve this today one option is to upgrade one's home with smart appliances however these are expensive and rarely talk to one another a more flexible option is for owners to tag
00:00:26 existing objects with aftermarket sensors this adds some level of smartness but requires many tags and can be socially and aesthetically obtrusive we explored an alternate general purpose sensing approach wherein a single highly capable sensor board can indirectly monitor an entire room we started our research by building an inventory of sensors used in commercial and academic
00:00:49 systems we decided to include all of these sensor Dimensions but no camera as this sensor is particularly sensitive to users our sensor board is plug-and-play uses wall power and connects to our Cloud backend over Wi-Fi a single sensor board in a room can capture a wide variety of events on the right you can see our board's raw sensor streams this pait running is easily seen in both the
00:01:14 acoustic Channel and by our high-speed accelerometers here we can see data from three different motor powered appliances a garbage disposal a blender and a coffee grinder note how each results in distinct sensor signals now the user turns on a stove burner note how the thermal image
00:01:42 reveals not only that the burner is on but what burner and how hot it is of course this low-level sensor data is rarely of interest to users instead we use machine learning to automatically recognize patterns of sensor activation and expose these highlevel Environmental events as synthetic sensors although virtual they can be treated just like traditional physical
00:02:07 sensor feeds triggering userdefined functions or used by developers to build responsive applications importantly raw sensor data is never sent to the cloud instead it's featurization [Music] [Music] various bathroom accessories the toilet flushing and the state of
00:02:39 lighting we can also detect multiple events in the house including when a fireplace is on the water tank is heating the dryer is running and when the hbac is on in a workshop or industrial setting we can detect multiple events such as when the dust filter is running or the exist fan is
00:03:02 on different tools such as sawce a shot bag a drill press a grinder and various handheld Tools in a workplace or an office we can detect a suite of events such as if a phone is ringing we can detect writing on a whiteboard and also erasing a water fountain running urinals
00:03:34 flushing and paper towels dispensing the synthetic sensors seen thus far we call first order synthetic sensors however these basic synthetic sensors can be fed into second order synthetic sensors able to capture higher level semantics such as count duration and state for example A first order towel dispense sensor can power a second order sensor that tracks the number of towels used
00:04:00 [Music] with such a sensor a facilities manager could automatically receive alerts or schedule restock requests in this example A first order closet running sensor is used to power a second order water consumption sensor metrics like this can inform monitoring Behavior change and other applications finally more complex
00:04:29 devices can have multiple States Beyond just on or off like this microwave here you can see first order synthetic sensors responsible for recognizing individual states by building on top of these sensors it is possible to create a secondorder sensor that tracks the state of the device armed with this high level understanding richer assist of applications can be built
00:04:53 [Music] finally there's no reason to stop at second order synthetic sensors these can feed into higher order sensors able to capture more complex environmental facets like human activity and the mechanical health of objects please see our paper for more details thanks for watching our video [Music]
