Where Objects Talk to Each Other

Where Objects Talk to Each Other

Inside the Mobile, Pervasive and Sensor Systems Laboratory at the University of Arkansas, among the wires and the circuits and the chips and batteries and quarter-sized solar panels, Nilanjan Banerjee and Pat Parkerson toil in that nebulous environment where man and machine seamlessly converge. It is a world in which computers have intelligence, electronic environments are sensitive and responsive to the presence of people, and information processing has been integrated pervasively into everyday objects and activities. In short, it is a world in which, as their colleague Craig Thompson puts it, “everything is alive, and objects talk to each other.”

It sounds like science fiction or a world of the future, but it is neither. All of this is already happening, especially with mobile phones, global positioning systems, laptops and tablets, all types of sensors, and retail systems that gather mountains of data about products and consumers. Banerjee and Parkerson, assistant and associate professors of computer science and computer engineering, respectively, devote their time to improving and enhancing these systems, making them faster, more efficient and especially capable of performing new tasks. They focus on three areas: renewable energy-driven systems, health-care systems and mobile phone-based systems, popularly referred to as apps. Their designs and products include small, sound-powered sensors that read and store medical data, solar-powered sensors used for disaster relief and large monitoring systems for “green” homes. These will help paramedics find an unobstructed route to tornado victims, empower parents with sick infants and enable homeowners to monitor and more efficiently control energy.

An App for Green Homes

In the green home project, an iPhone-based recommendation system uses information collected by the measurement system to give users advice regarding energy consumption.

In the green home project, an iPhone-based recommendation system uses information collected by the measurement system to give users advice regarding energy consumption, including warnings in advance of critical battery situations, recommendations for the best times to execute high-power tasks and opportunities to adjust the power states of specific devices to reduce energy consumption.

Although expensive to build, green homes have many personal and social benefits. Because many green homes rely on clean energy sources such as sun and wind, they decrease reliance on fossil-fuel energy production and do not harm the environment.

Green homes can be “off-grid,” that is, not connected to the U.S. power grid, or “grid-tied,” relying on the grid for only some of their power. If they are set up and function properly, both off-grid and grid-tied homes can create significant tax rebates for the homeowner and in some cases can lead to rebates through consumers selling energy back to power companies.

Currently, off-grid homes account for only 1 percent of all dwellings in the United States. However, as the federal government focuses attention on clean energy sources, the number of grid-tied homes with solar power has increased significantly in recent years. In 2009, the number of grid–tied homes grew by 40 percent.

In 2011, lab director Banerjee, a self-professed software guy, received grants totaling $450,000 from the National Science Foundation to further investigate energy generation and consumption in off-grid and grid-tied homes and to develop a business model to commercialize the work. Collaborating with Sami Rollins, a usability and mobile-systems specialist and professor at the University of San Francisco, Banerjee had already designed software and an iPhone mobile application to perform such a task. To build the system, the two researchers partnered with Parkerson, a hardware guy from the university’s High Density Electronics Center, who has more than two decades of experience with electronic circuits and sensors.

With help from students, Parkerson is designing hardware circuits and indoor-light-driven sensor boards compatible with Banerjee’s software. Together the researchers developed an automated energy-management system that monitors generation and consumption of energy in homes that rely on sun or wind for power.

This prototype measurement system in an off-grid home in Arkansas helps the researchers understand how energy is generated and consumed in homes powered by renewable sources.

This prototype measurement system in an off-grid home in Arkansas helps the researchers understand how energy is generated and consumed in homes powered by renewable sources. Image submitted

“We are building a system that strikes a balance between totally automated control, which might irritate homeowners by turning off the television while they’re watching a program, and reactive or manual techniques that really are not sufficient to prevent critical battery situations or energy outages,” Banerjee said. “Our system simply will alert the homeowner of critical situations and then suggest which appliances to turn off. From anywhere, as long as they have their smart phone, homeowners can then use the software to direct the system.”

In early 2011, the researchers installed a suite of monitoring tools in an off-grid, solar-powered home in Fayetteville. They collected generation and consumption data for 55 days, 14 days during the summer and 41 days in November and December. In addition to monitoring the photovoltaic system’s power-generation devices, the system also tracked individual appliances, including a refrigerator, washer and dryer, hot-water heater and even a television and lamps.

Data collected by the monitoring system demonstrated that energy harvested from the solar panels and energy consumed by the house were both highly variable. This was true within a single day, across several days and across seasons. Although generation and consumption varied greatly, they did so in a predictable manner.

Their findings also provided evidence that traditional energy-management techniques are insufficient in off-grid homes. For example, the widespread assumption is that the ideal time to run appliances that require a lot of energy is between 7 p.m. and 7 a.m. However, the researchers discovered that because the battery was receiving and storing more energy during the day, it was better to run appliances from 10 a.m. to 8 p.m.

Finally, the data showed that manual and reactive methods to manage energy consumption do not prevent critical battery situations. The homeowner in the study was conservative with energy consumption and carefully monitored battery voltage, but he still had to rely on a backup generator approximately 25 percent of the time, because he could not predict times in which harvesting solar energy was low.

“All these findings point to the need for a feasible, automated and proactive energy-management scheme,” Banerjee said.

The automated system will include important control functions, which perform three important tasks. First, the system predicts when energy storage is likely to be critically low and will notify homeowners in advance so that they may take proactive measures to reduce consumption. Second, by predicting when energy harvested is at its peak, the system will advise homeowners of ideal times to execute tasks that require a lot of power, such as running a dishwasher or washing machine.

Finally, relying on information collected on each appliance, the system will suggest energy conservation techniques. For example, it can recommend users adjust or reduce the temperature of the refrigerator by a few degrees without negatively affecting its performance. All of this information reaches the homeowner via the iPhone application, which can then be used to send commands to specific appliances.

More recently, to strengthen monitoring and control functions, the researchers have expanded the project to sharpen their understanding of the performance of specific appliances, which they hope will explain why the appliances consume energy the way they do. When all the data is in, Banerjee and Parkerson predict nuance in performance will be related to ambient light and temperature, both inside and outside the house.

The Sound of Flu

a sensor- and mobile-phone-based system captures specific nuances of sound within a child's cough that signify the coming of flu or strep throat.

A sensor- and mobile-phone-based system captures specific nuances of sound within a child’s cough that signify the coming of flu or strep throat.
Stock Image

Who knew you could hear the flu? Banerjee and Parkerson have modified HiJack, a hardware and software system developed at the University of Michigan, to fabricate a sensor- and mobile-phone-based system that captures specific nuances of sound within a child’s cough, for example, that signify the coming of flu or strep throat.

HiJack uses sensors to harvest power and bandwidth from a mobile phone’s headset interface. The modified system includes a harvester that scavenges sound-wave energy emitted from recordings on a smart phone. Because the “sweet spot” for harvesting energy from sound varies from phone to phone, Banerjee and Parkerson built a feedback-based system that converges to a frequency in which energy is harvested optimally.

Future work will focus on augmenting the harvester with biological assays and enzyme sensors to further detect viruses, Banerjee said. They are also building software that uses machine-learning techniques to screen further. Combined, these tools will provide early, point-of-care diagnostics for non-medical personnel, such as parents at home with infant children, or doctors and nurses in third-world countries or other areas lacking medical services.

“For adults, early detection of the flu isn’t critical,” Banerjee says. “But it doesn’t hurt, and it can be a matter of life and death for infants. This tool could give parents real peace of mind when their baby develops a cough.”

Providing Emergency Information During Natural Disasters

the solar-powered emergency communications system would provide continuous, uninterrupted service during tornadoes and other natural disasters.

A solar-powered emergency communications system would provide continuous, uninterrupted service during tornadoes and other natural disasters.
Photo by Matt McGowan

A bizarre phenomenon of the massive tornado that struck Joplin, Mo., in 2011 was that people in California or Hawaii or anywhere else in the world that receives CNN or the Weather Channel knew more about the storm than people living only a few miles from its path. More importantly, because power and communication systems, including the Internet, shut down due to damage caused by the tornado, victims did not know where or how to seek aid, and emergency responders struggled to find clear routes to reach victims. Unfortunately, this was not a unique experience.

To address this problem, Banerjee, Parkerson and Jack Cothren, associate professor of geosciences and director of the university’s Center for Advanced Spatial Technologies, are working on a system that could provide seamless geographic information to mobile phones or laptops within areas that have experienced disaster-related loss of wireless communication. This technology also could be used in other scenarios, such as hiking in extreme wilderness areas or military operations in deserts or other remote locations.

In September 2010, the researchers received a $485,000 National Science Foundation grant to develop what they call a solar-powered emergency mesh. The mesh can be thought of as a network of nodes that blanket a geographic area. Similar to a server, each solar-powered node contains data – geographic information – that can be downloaded to a user or communicated node-to-node, if necessary.

Because it relies on renewable energy and is not dependent on the power grid or Internet service providers, the system provides continuous, uninterrupted service. If a node fails due to variability inherent in renewable energy scavenging or extreme environmental conditions during the aftermath of a natural disaster, the mesh will automatically redistribute data on the failed nodes to maintain service.

A system demonstration displays a disaster area in red and an unobstructed or optimal route by a green line to an aid station or hospital. This information reaches the user through a web browser on a smart phone, laptop or tablet.

There are several issues that must be resolved before a practical and fully functioning system could be deployed. For sustainable operation on small solar panels, the nodes must operate on extremely low power yet still have enough power to serve map-based information to users. Parkerson and his students are developing and testing hardware in an effort to strike this balance. Another challenge is the geographical placement of nodes to ensure that optimal connectivity can be maintained regardless of topography. Additionally, Cothren’s group must investigate and test sophisticated GIS software that can function well on a device that has low power and limited resources. To address these obstacles and test a combination of hardware and software, the researchers plan to deploy a 40-node mesh in Fayetteville toward the end of 2012.

Check out more cool stuff at the Mobile, Pervasive and Sensor Systems Laboratory.

About The Author

Matt McGowan writes about research in the College of Engineering, Sam M. Walton College of Business, School of Law and other areas. He is the editor of Short Talks From the Hill, a podcast of the University of Arkansas. Reach him at 479-575-4246 or dmcgowa@uark.edu.

University Relations Science and Research Team

University Relations Science and Research Team

Matt McGowan
science and research writer
479-575-4246, dmcgowa@uark.edu

Robert Whitby
science and research writer
479-387-0720, whitby@uark.edu

Looking for an expert?

The University of Arkansas Campus Experts website is a searchable database of experts who can talk to the media on current events.

Trending Topics:
State and local economy
Environmental economics
Immigration politics

More on University of Arkansas Research

Visit the office of Research & Innovation for a complete list research awards and more information on research policies, support and analytics.

Connect with Us