Scientists from the Massachusetts Institute of Technology (MIT) and Watson Ai Lab Mit-IBM are investigating the ways of using spatial acoustic information to help machines to better represent their environment. Scientists have developed a machine learning model that can capture how any sound in the room will travel in space, enabling the model to simulate what the listener hears in various locations.
Thanks to the exact simulation of acoustics of this place, the system can learn the basic 3D geometry of the room from sound recordings. Scientists use acoustic information that their system collects to create a precise visual representation of the room, as in the way people use sound during the assessment of the properties of their physical environment.
In addition to all potential ways of using in virtual and extended reality, this method can help AI agents better understand the world. Thus, according to Yilun Du, co -author of the article describing the model and a graduate in the Department of Electrical Engineering and Computer Science (EEC): “Modeling of acoustic sound properties in his environment, an underwater exploratory robot can sense things that are further than just vision.”
“Most researchers focused only on vision modeling. But as people we have a multimodal perception. Not only the vision is important, the sound is also important. I think that this work opens an exciting direction of research on better use of sound for modeling the world,” says Du.
Learn more about how the use of sound can model the world https://news.mit.edu/2022/sound-model-ai-1101
