Explainable Audio Classification of Playing Techniques with Layer-wise Relevance Propagation - Laboratoire Traitement et Communication de l'Information Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Explainable Audio Classification of Playing Techniques with Layer-wise Relevance Propagation

Résumé

Deep convolutional networks (convnets) in the time-frequency domain can learn an accurate and fine-grained categorization of sounds. For example, in the context of music signal analysis, this categorization may correspond to a taxonomy of playing techniques: vibrato, tremolo, trill, and so forth. However, convnets lack an explicit connection with the neurophysiological underpinnings of musical timbre perception. In this article, we propose a data-driven approach to explain audio classification in terms of physical attributes in sound production. We borrow from current literature in "explainable AI" (XAI) to study the predictions of a convnet which achieves an almost perfect score on a challenging task: i.e., the classification of five comparable real-world playing techniques from 30 instruments spanning seven octaves. Mapping the signal into the carrier-modulation domain using scattering transform, we decompose the networks' predictions over this domain with layer-wise relevance propagation. We find that regions highly-relevant to the predictions localized around the physical attributes with which the playing techniques are performed.
Fichier principal
Vignette du fichier
wang_ICASSP23_final.pdf (471.91 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
licence : CC BY - Paternité

Dates et versions

hal-04029145 , version 1 (14-03-2023)

Licence

Paternité

Identifiants

Citer

Changhong Wang, Vincent Lostanlen, Mathieu Lagrange. Explainable Audio Classification of Playing Techniques with Layer-wise Relevance Propagation. 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Jun 2023, Rhodes, Greece. pp.1-5, ⟨10.1109/ICASSP49357.2023.10095894⟩. ⟨hal-04029145⟩
221 Consultations
200 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More