image:
Guha Balakrishnan, assistant professor of electrical and computer engineering and the project’s principal investigator.
Credit: Photo by Rice University
Rice University has received a National Science Foundation award to develop artificial intelligence-powered cameras that could dramatically reduce the data, energy and cost needed to operate large-scale camera networks.
The project, called Generative Cameras for Low-Cost and Multimodal Distributed Sensing, aims to rethink how cameras capture and process information. Instead of capturing and transmitting every pixel in a scene, the system would collect only essential measurements and use generative AI to reconstruct detailed images. The approach could make battery-powered camera networks practical for remote monitoring and other applications where power, bandwidth and computing resources are limited.
“We’re asking whether cameras can leverage AI to generate remarkably rich information from surprisingly little input,” said Guha Balakrishnan, assistant professor of electrical and computer engineering and the project’s principal investigator.
Making cameras smarter
Modern camera networks support applications from wildlife conservation and traffic management to disaster response and robotics. But capturing, processing and wirelessly transmitting high-resolution video requires substantial power, making cameras difficult to deploy in remote locations or operate for long periods on battery power.
The researchers will develop generative cameras, or GenCams, which combine simple, low-power sensors with generative AI to reconstruct detailed images from sparse measurements or a small amount of carefully selected data instead of every pixel in an image.
By relying on learned patterns instead of transmitting complete images, GenCams are designed to reduce bandwidth, computing requirements and energy consumption without sacrificing useful information. That could make large, battery-powered camera networks more practical for applications such as wildlife monitoring, infrastructure management and disaster response while using far less power and bandwidth than conventional systems.
Building the next generation of AI imaging
This research will develop the algorithms behind GenCams, combining sparse sensing with generative AI.
The project has three phases. Researchers will first develop GenCams using conventional red, green and blue image sensors. Next they will incorporate depth sensors, which measure the distance to objects, and event-based sensors, which record changes in a scene instead of capturing complete images. Last, they will integrate audio and language models to create systems that combine images, sound and text.
Moreover, the research team will build prototype systems to test the technology outside the laboratory and help bridge the gap between theory and real-world deployment while guiding future research.
Expanding the possibilities for distributed sensing
Funded by a three-year, $900,000 grant, the project could shift more of a camera’s intelligence from hardware to software. Instead of simply recording images, future systems could interpret scenes while collecting far less data.
This work advances several areas of computer science, including computational imaging, multimodal learning and inverse imaging, which reconstructs images from incomplete measurements. Together, these advances could produce sensing systems that deliver more information while using fewer physical resources, Balakrishnan said.
“Generative AI gives us an opportunity to redesign the camera itself,” Balakrishnan said. “By moving more intelligence into software, we can build imaging systems that are both more efficient and more capable.”
Balakrishnan is leading the project with Rice co-principal investigators Vicente Ordonez, Vivek Boominathan and Chen Wei.
Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.