I. Deep Generative Modeling
Consistency-type Models
[arXiv]
Unified framework for "consistency" on diffusion models
(ICML2023 SPIGM workshop)
GibbsDDRM
[arXiv] [code]
Achieving blind inversion using DDPM
(ICML2023 Oral)
Applications:
[DeReverb]
[SpeechEnhance]
II. Multimodal NLP & Commonsense AI
III. Music & Cinematic Technologies
STARSS23
[arXiv] [Dataset]
STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events
(NeurIPS2023)
BigVSAN Vocoder
[arXiv] [code] [demo]
BigVSAN: Enhancing GAN-based Neural Vocoders with Slicing Adversarial Network
CLIPSep
[OpenReview] [arXiv] [code] [demo]
CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos
(ICLR2023)
hFT-Transformer
[arXiv] [code]
Automatic Piano Transcription with Hierarchical Frequency-Time Transformer
(ISMIR2023)
Audio Restoration with ViT-AE
[IEEE] [arXiv] [demo]
Extending Audio Masked Autoencoders Toward Audio Restoration
(WASPAA2023)
Diffiner
[ISCA] [arXiv] [code]
Diffiner: A Versatile Diffusion-based Generative Refiner for Speech Enhancement
(INTERSPEECH2023)
Automatic Music Tagging
[arXiv]
An Attention-based Approach To Hierarchical Multi-label Music Instrument Classification
(ICASSP2023)
Vocal Dereverberation
[arXiv] [demo]
Unsupervised Vocal Dereverberation with Diffusion-based Generative Models
(ICASSP2023)
Mixing Style Transfer
[arXiv] [code] [demo]
Music Mixing Style Transfer: A Contrastive Learning Approach to Disentangle Audio Effects
(ICASSP2023)
Music Transcription
[arXiv] [code] [demo]
DiffRoll: Diffusion-based Generative Music Transcription with Unsupervised Pretraining Capability
(ICASSP2023)
Singing Voice Vocoder
[arXiv] [demo]
Hierarchical Diffusion Models for Singing Voice Neural Vocoder
(ICASSP2023)
Distortion Effect Removal
[poster] [arXiv] [demo]
Distortion Audio Effects: Learning How to Recover the Clean Signal
(ISMIR2022)
Automatic Music Mixing
[poster] [arXiv] [code] [demo]
Automatic Music Mixing with Deep Learning and Out-of-Domain Data
(ISMIR2022)
Automatic DJ Transition
[arXiv] [code] [demo]
Automatic DJ Transitions with Differentiable Audio Effects and Generative Adversarial Networks
(ICASSP2022)
Sound Event Localization and Detection
[IEEE] [arXiv]
Multi-ACCDOA: Localizing and Detecting Overlapping Sounds from the Same Class with Auxiliary Duplicating Permutation Invariant Training
(ICASSP2022)
Sound Separation
[video] [site]
Glenn Gould and Kanji Ishimaru 2021: A collaboration with AI Sound Separation after 60 years
DCASE Challenge
[DCASE Challenge2023]
Sound Event Localization and Detection Evaluated in Real Spatial Sound Scenes