The MAD Project (2014-2018) is dedicated to general problems where missing parts must be recovered in partially-observed audio data.
Audio inpainting : reconstructing missing parts in sounds.
A general framework for various reconstruction applications using advanced models and techniques.
Audio inpainting is a recent generic framework for reconstructing missing parts in sounds or in their representations : restoration of old recordings, declipping, spectogram retouching and other modifications are thus achieved using signal processing and machine learning tools. Following a proof of concept in 2012, the MAD project aims at proposing fundamental research works on sound modeling and at developing new approaches for audio inpainting. In particular, it provides advanced techniques for settings in which holes in signals may be small or large, and it promotes and addresses time-frequency audio inpainting, where some time-frequency coefficients are missing. The MAD project also expands the recent audio inpainting concept by extending the range of its applications and by disseminating its results.
Exploiting partial observations to learn elementary components and reconstruct sounds.
Audio inpainting techniques are relying on the observed parts to reconstruct the missing ones. The main approaches developed in the project model the data by decomposing them in dictionaries made of elementary patterns (sparse decompositions; nonnegative matrix factorisation, NMF) and reconstruct the sounds by paying attention to their intrinsic properties such as the phases of the oscillatory components. Other sound structures were exploited to improve those models, such as the autosimilarity in music and speech signals (interchannel similarity, slow variations of the contents, non-local repetition of elementary patterns). In addition, the inpainting problems were also addressed jointly with source separation and compression problems. From a methodological and more abstract viewpoint beyond its applications, audio inpainting also offers an appropriate framework for testing and assessing the relevance of models and estimation algorithms : it shows to which extent the model estimated from a partial observation remains valid on the missing parts that are predicted.
Within the MAD project, the recent audio inpainting concept has been developed along several axes : a scientific axis with a large diversity of research works (advances in audio declipping, multichannel and structured models, spectrogram inpainting, phase inpainting, caracterisation of complex-valued time-frequency matrices) ; the animation of collective research dynamics (new national and international collaborations, scientific meetings) ; the development of a set of Python packages for audio inpainting called skmad-suite.