Researchers develop optical microphone that can see sound, distinguish instruments - Shout4Music

Researchers develop optical microphone that can see sound, distinguish instruments – Shout4Music

A research team at Carnegie Mellon University has recently unveiled an incredible optical microphone that can capture and isolate sound through a dual-shutter optical vibration-sensing system.

This imaging unit is being developed by a team of researchers, professors and students at the School of Computer Sciences Robotics Institute (RI).

“We have invented a new way of seeing sound. It is a new type of camera system, a new imaging device, that can see something that is invisible to the naked eye,” said Mark Sheinin, a member of the research team and a postdoctoral research assistant at Illumination and Imaging Laboratory. (ILIM).

This unique system uses two cameras and a laser to capture surface vibrations, reconstruct sound waves and distinguish multiple instruments at a distance.

The idea behind this rests on the very concept of how sound is produced. Sound is simply a series of traveling pressure waves created by vibrating objects or sound sources. By capturing these vibrations from the surface of a sound source, the optical microphone can create an image of the specific sound.

But even advanced microphones can not distinguish different soundscapes from each other, let alone isolate them or completely eliminate them. This new technology uses a unique way to capture individual sounds through a laser that creates an exact spot pattern in symphony with sound vibrations.


How the optical microphone works

With the help of two pairs of cameras and a laser, this dual shutter camera can capture and reconstruct sound waves from not just one, but multiple sound sources played simultaneously – even though they produced two completely different sounds.

The laser captures and creates an exact spot pattern that is distorted when the surface vibrates. The two cameras, which operate on two different shutters (global and rolling), document this pattern, which can then be sent to a software algorithm that analyzes all images to reconstruct the audio signal.

The cameras use a frame rate of 63 fps, which may seem a bit slow. However, the use of both a global shutter and a rolling shutter allows the device to capture two different aspects of the sound vibrations, one that can read an entire image sensor and another that can read thousands of horizontal lines in a single frame. Together, this allows the device to read up to 63,000 Hz sound.

To really test the device’s capabilities, the optical microphone was used to capture vibrations from several guitars playing a duet, two speakers playing another song, on tuning forks and wildly enough, a bag of Doritos chips placed in front of a speaker.


The importance of sound image technology

Optical microphones are certainly nothing new. In fact, the research project was in part a tribute to the work of researchers at the Manhattan Institute of Technology (MIT), who created the first visual microphone ever in 2014.

However, this new imaging device improves significantly compared to its predecessors in both technology and algorithm.

“This system pushes the boundaries of what can be done with computer vision,” said RI Assistant Professor Matthew O’Toole about their project. “This is a new mechanism for capturing high speed and small vibrations, and presents a new area of ​​research.”

The invention will allow sound engineers to fine tune ensemble mixes and manufacturers to closely monitor the health of their machines on the factory floor.

“If your car starts to make a strange noise, you know it’s time to look at it,” adds Mark Sheinin. “Now imagine a factory floor full of machines. Our system allows you to monitor the health of everyone by sensing their vibrations with a single stationary camera.”


Clock: Microphone with dual shutters

#Researchers #develop #optical #microphone #sound #distinguish #instruments #Shout4Music

Leave a Comment

Your email address will not be published.