A leap in science photography – Schlieren imaging enhanced by event cameras

A recent TU Berlin’s press release, interviewing Science of Intelligence (SCIoI) PI Guillermo Gallego, reveals an exciting breakthrough in scientific photography. Researchers from his group merged schlieren imaging with advanced event cameras. This practical technique improves traditional schlieren photography and has broad applications in a string of sciences and industries, from aeronautics to robotics.

Capturing the unseen through a serendipitous discovery

Schlieren photography has long been a standard method in research and industrial applications, offering insights into transparent media’s movements, such as flows in wind tunnels or gas leaks in industrial systems. By introducing event cameras to this well-established field, researchers Guillermo Gallego, Shintaro Shiba and Friedhelm Hamann redefined the way we capture and interpret images of transparent substances.

Unlike conventional cameras, event cameras do not capture entire images at once. Instead, they record changes in brightness – the so-called “events” – for each light-sensitive pixel independently. This unique feature allows ongoing research at SCIoI to further explore and analyze event camera images, potentially enhancing the capabilities of robots, self-driving cars, and smartphone cameras.

“It was more or less by chance that we discovered that event cameras are an excellent way to improve the schlieren photography process,” Guillermo recalls. This revelation unfolded during the research endeavors of Shintaro Shiba, a doctoral candidate from Keio University who worked with Gallego at SCIoI, and Friedhelm Hamann, doctoral candidate from SCIoI.

How the combined technique works and what it promises

Schlieren photography visualizes the movement of otherwise invisible transparent media, creating “streaky” images (“Schlieren” in German) by capturing density variations in front of a patterned background. Event photography, inspired by human vision, works without an exposure time. Each light-sensitive pixel independently contributes to the overall image when the incident light intensity changes. By combining these two techniques, the research team captured the apparent movement of background patterns, allowing for the estimation of directional movement.

The researchers developed a mathematical theory for digitally evaluating Schlieren images with event cameras. In comparison to traditional Schlieren photography cameras, the event-based approach demonstrated a tenfold improvement in temporal resolution and a fourfold increase in dynamic range. The further integration of AI streamlined and facilitated the data processing, requiring only a fraction of the usually needed amount of data. Consequently, the improved data processing allows for the development of smaller cameras.

Applications beyond photography and the benefits of basic and interdisciplinary research

The implications of this novelty extend to aeronautics, astronautics, fluid mechanics, and industrial applications. The improved slow-motion recordings and higher resolution hold potential value for laboratories involved in diversified research, while the opportunity to operate in unfavorable light conditions makes event cameras ideal for operational and safety tasks in industry. The fruitful collaboration between TU Berlin and Keio University has not only advanced smart applications in robotics and vehicles but has also unexpectedly contributed to flow research and industrial implementations. “Research into event cameras was originally driven by neuroscientists working to establish a model for human vision,” notes Guillermo. This unexpected success underscores the importance of basic and interdisciplinary research, showcasing its potential that extend beyond initial expectations.

In conclusion, the fusion of Schlieren photography with event cameras marks a paradigm shift in imaging technology. As researchers continue to explore the multifaceted applications of this innovative approach, the sectors of photography, robotics, and industrial processes now practically stand before a transformative leap forward.