relation: http://miis.maths.ox.ac.uk/miis/749/ title: Measuring Vibrations from Video Feeds creator: Brambley, Ed creator: Fletcher, Helen creator: Hill, Roger creator: Johnston, Ifan creator: Liu, Jessie creator: MacKay, Robert creator: Mathews, J. creator: Ockendon, John creator: Piette, Bernard subject: Aerospace and defence description: By using a high-speed camera, researchers at MIT in 2014 where able to recover human speech from videos of minute vibrations of objects in a room. For example, in one experiment a 2,200fps camera was positioned outside a room behind sound-proof glass, videoing an empty crisp packet on the floor inside the room, while a researcher shouted “Mary had a little lamb” at the crisp packet. By detecting minute oscillations of the crisp packet of 1 μm (0.001 mm), and using hours of computer processing, a ten second audio clip could be produced that was recognisably “Mary had a little lamb” in an American accent. The purpose of this study group was to investigate whether this tech- nique could be used in practice, with emphasis on the recovery of intel- ligible speech from a video feed of a room. During the week, the group investigated several aspects of the problem, including: • how much an object vibrates due to sound; • what can be done to maximize the vibration; • how the MIT technique detects minute vibrations in videos; • what affects the quality of the resulting recording; and • how good a recording is needed for intelligible speech. It was discovered the MIT experiments would not have recovered intel- ligible speech from an ordinary conversation; their success depended on loud sounds and prior knowledge of “Mary had a little lamb”. Camera vibrations were also ignored by MIT; these are expected to be signifi- cant, but the technique could be adapted to be resilient to them. Other possibilities for enhancing their technique, by exploiting resonances or reflections, are discussed in the report. A high-speed low-noise cam- era is essential, and any existing video footage (such as from CCTV) is unlikely to be of sufficient quality. Further experiments with high-end high-speed cameras are needed to assess the feasibility of the technique in practice. date: 2017 type: Study Group Report type: NonPeerReviewed format: application/pdf language: en identifier: http://miis.maths.ox.ac.uk/miis/749/1/mod_report.pdf identifier: Brambley, Ed and Fletcher, Helen and Hill, Roger and Johnston, Ifan and Liu, Jessie and MacKay, Robert and Mathews, J. and Ockendon, John and Piette, Bernard (2017) Measuring Vibrations from Video Feeds. [Study Group Report]