Timbre-Trap

This demo page is for the paper Timbre-Trap: A Low-Resource Framework for Instrument-Agnostic Music Transcription

Source code: https://github.com/sony/timbre-trap

Table of contents

1. Audio Samples
2. Visualization

Audio Samples

Timbre-Trap

Timbre-Trap reconstructs the CQT coefficents when the latent space switch is in "on" state (Reconstruction). It removes timbre from the audio clips when the latent space switch is in "off" state (Sonification).

File Name	Mixture	Bassoon	Clarinet	Saxophone	Violin
Original Audio
Reconstruction
Sonification Transcription

Timbre-Trap with Skip-Connections

With skip-connections, Timbre-Trap reconstructs CQT coefficents very close to the original audio, resulting in a very high SDR.

But the sonification retains more timbre information than the Timbre-Trap without skip-connections, resulting in a slower F1-score.

File Name	Mixture	Bassoon	Clarinet	Saxophone	Violin
Original Audio
Reconstruction
Sonification Transcription

Visualization

The reconstruction and transcription produced by Timbre-Trap with and without skip-connections

URMP test set

Timbre-Trap	Timbre-Trap with Skip-Connections
Ground Truth Prediction Reconstruction Transcription	Ground Truth Prediction

Bach10

Timbre-Trap	Timbre-Trap with Skip-Connections
Ground Truth Prediction Reconstruction Transcription	Ground Truth Prediction

Su

Timbre-Trap	Timbre-Trap with Skip-Connections
Ground Truth Prediction Reconstruction Transcription	Ground Truth Prediction

Guitar Set

Timbre-Trap	Timbre-Trap with Skip-Connections
Ground Truth Prediction Reconstruction Transcription	Ground Truth Prediction