eprintid: 749
rev_number: 7
eprint_status: archive
userid: 17
dir: disk0/00/00/07/49
datestamp: 2019-01-21 22:43:45
lastmod: 2019-01-21 22:43:45
status_changed: 2019-01-21 22:43:45
type: report
metadata_visibility: show
creators_name: Brambley, Ed
creators_name: Fletcher, Helen
creators_name: Hill, Roger
creators_name: Johnston, Ifan
creators_name: Liu, Jessie
creators_name: MacKay, Robert
creators_name: Mathews, J.
creators_name: Ockendon, John
creators_name: Piette, Bernard
corp_creators: David Smith
corp_creators: Andy Newell
title: Measuring Vibrations from Video Feeds
ispublished: pub
subjects: aerodef
studygroups: esgi130
companyname: Ministry of Defence
full_text_status: public
abstract: By using a high-speed camera, researchers at MIT in 2014 where able to recover human speech from videos of minute vibrations of objects in a room. For example, in one experiment a 2,200fps camera was positioned outside a room behind sound-proof glass, videoing an empty crisp packet on the floor inside the room, while a researcher shouted “Mary had a little lamb” at the crisp packet. By detecting minute oscillations of the crisp packet of 1 μm (0.001 mm), and using hours of computer processing, a ten second audio clip could be produced that was recognisably “Mary had a little lamb” in an American accent.
The purpose of this study group was to investigate whether this tech- nique could be used in practice, with emphasis on the recovery of intel- ligible speech from a video feed of a room. During the week, the group investigated several aspects of the problem, including:
• how much an object vibrates due to sound;
• what can be done to maximize the vibration;
• how the MIT technique detects minute vibrations in videos; • what affects the quality of the resulting recording; and
• how good a recording is needed for intelligible speech.
It was discovered the MIT experiments would not have recovered intel- ligible speech from an ordinary conversation; their success depended on loud sounds and prior knowledge of “Mary had a little lamb”. Camera vibrations were also ignored by MIT; these are expected to be signifi- cant, but the technique could be adapted to be resilient to them. Other possibilities for enhancing their technique, by exploiting resonances or reflections, are discussed in the report. A high-speed low-noise cam- era is essential, and any existing video footage (such as from CCTV) is unlikely to be of sufficient quality. Further experiments with high-end high-speed cameras are needed to assess the feasibility of the technique in practice.
date: 2017
date_type: completed
citation:   Brambley, Ed and Fletcher, Helen and Hill, Roger and Johnston, Ifan and Liu, Jessie and MacKay, Robert and Mathews, J. and Ockendon, John and Piette, Bernard  (2017) Measuring Vibrations from Video Feeds.  [Study Group Report]     
document_url: http://miis.maths.ox.ac.uk/miis/749/1/mod_report.pdf