A synthetic sensor that turns your room into a smart environment, Internet of Things Made Simple by CMU researchers

Ubiquitous sensors seem almost synonymous with the internet of things (IoT), but some Carnegie Mellon University researchers say ubiquitous sensing — with a single, general purpose sensor for each room — may be better.

The plug-in sensor package they have developed monitors multiple phenomena — sounds, vibration, light, heat, electromagnetic noise, temperature, etc. — in a room. With some help from machine-learning techniques, this suite of sensors can determine whether a faucet's left or right spigot is running, if the microwave door is open or how many paper towels have been dispensed.

"The idea is you can plug this in and immediately turn a room into a smart environment," said Gierad Laput, a Ph.D. student in CMU's Human-Computer Interaction Institute (HCII). "You don't have to go out and buy expensive smart appliances, which probably can't talk to each other anyway, or attach sensors to everything you want to monitor, which can be hard to maintain as well as ugly. You just plug it into an outlet."
Laput, along with Chris Harrison, an HCII assistant professor, and Yang Zhang, a Ph.D. student, built their platforms using sensors used in other commonly available smart home devices — with the exception of a camera, which raises privacy concerns.

Machine-learning algorithms can combine these raw feeds into powerful synthetic sensors that can identify a wide range of events and objects — for instance, distinguishing between a blender, a coffee grinder and mixer based on sounds and vibrations. Even soft, more subtle sounds, such as writing or erasing on a whiteboard, can be detected. More than just recording whether a device is in use or not, synthetic sensors can track the state of a device — whether a microwave door is open or closed, if cooking is interrupted, and whether the microwave has completed its cooking cycle.

"It can tell you not only if a towel dispenser is working, but can keep track of how many towels have been dispensed and even order a replacement roll when necessary," Laput said. A faucet left running when a room is unoccupied for a long time might prompt a warning message to the user's smartphone.
Even more advanced sensing can infer human activity, such as when someone is sleeping, showering, watching streaming video or has left home for work. Most of this processing occurs on the sensor platform itself, so detailed and sensitive data need not be transmitted or recorded, he added.

The sensor platform can be manually trained to recognize various phenomena, such as the cycling of water heaters or heating and air conditioning units. It also would be possible to pre-train the sensors to detect many popular devices and brands of home or office products, making it possible for the sensor platform to begin functioning as soon as it is plugged in, Laput said.

Plugging the units into a regular electric socket eliminates the need for batteries or special wiring. As a practical matter, each room likely will need its own sensor platform, though it would be possible to have each sensor platform communicate with other nearby sensors to create a home-wide sensing environment with just a few sensors, not hundreds.

Google, through the GIoTTo Expedition Project, supported this research, as did the David and Lucile Packard Foundation. For more information, please visit the project website.
Via CMU