The MIIS Eprints Archive

Measuring Vibrations from Video Feeds

Brambley, Ed and Fletcher, Helen and Hill, Roger and Johnston, Ifan and Liu, Jessie and MacKay, Robert and Mathews, J. and Ockendon, John and Piette, Bernard (2017) Measuring Vibrations from Video Feeds. [Study Group Report]

[img] PDF


By using a high-speed camera, researchers at MIT in 2014 where able to recover human speech from videos of minute vibrations of objects in a room. For example, in one experiment a 2,200fps camera was positioned outside a room behind sound-proof glass, videoing an empty crisp packet on the floor inside the room, while a researcher shouted “Mary had a little lamb” at the crisp packet. By detecting minute oscillations of the crisp packet of 1 μm (0.001 mm), and using hours of computer processing, a ten second audio clip could be produced that was recognisably “Mary had a little lamb” in an American accent.
The purpose of this study group was to investigate whether this tech- nique could be used in practice, with emphasis on the recovery of intel- ligible speech from a video feed of a room. During the week, the group investigated several aspects of the problem, including:
• how much an object vibrates due to sound;
• what can be done to maximize the vibration;
• how the MIT technique detects minute vibrations in videos; • what affects the quality of the resulting recording; and
• how good a recording is needed for intelligible speech.
It was discovered the MIT experiments would not have recovered intel- ligible speech from an ordinary conversation; their success depended on loud sounds and prior knowledge of “Mary had a little lamb”. Camera vibrations were also ignored by MIT; these are expected to be signifi- cant, but the technique could be adapted to be resilient to them. Other possibilities for enhancing their technique, by exploiting resonances or reflections, are discussed in the report. A high-speed low-noise cam- era is essential, and any existing video footage (such as from CCTV) is unlikely to be of sufficient quality. Further experiments with high-end high-speed cameras are needed to assess the feasibility of the technique in practice.

Item Type:Study Group Report
Problem Sectors:Aerospace and defence
Study Groups:European Study Group with Industry > ESGI 130 (Warwick, UK, Sep 4-8, 2017)
UK Study Groups > ESGI 130 (Warwick, UK, Sep 4-8, 2017)
Company Name:Ministry of Defence
ID Code:749
Deposited By: Bogdan Toader
Deposited On:21 Jan 2019 22:43
Last Modified:21 Jan 2019 22:43

Repository Staff Only: item control page