Digibug. Repositorio Institucional de la Universidad de Granada
Publicación
Contribución de congreso (ConferenceOutput)
Learning Visual Voice Activity Detection with an Automatically Annotated Dataset
This work has been funded by the EU H2020 project #871245 SPRING and by the Multidisciplinary Institute in Artificial Intelligence (MIAI) #ANR-19-P3IA-0003., Visual voice activity detection (V-VAD) uses visual
features to predict whether a person is speaking or not. VVAD
is useful whenever audio VAD (A-VAD) is inefficient either
because the acoustic signal is difficult to analyze or because it is
simply missing. We propose two deep architectures for V-VAD,
one based on facial landmarks and one based on optical flow.
Moreover, available datasets, used for learning and for testing VVAD,
lack content variability. We introduce a novel methodology
to automatically create and annotate very large datasets inthe-
wild – WildVVAD – based on combining A-VAD with face
detection and tracking. A thorough empirical evaluation shows
the advantage of training the proposed deep V-VAD models with
this dataset., European Commission 871245 SPRING, Multidisciplinary Institute in Artificial Intelligence (MIAI) ANR-19-P3IA-0003