MelGAN

Generative Adversarial Networks for Conditional Waveform Synthesis.

Table of contents

• Abstract
• Results
  • Spectrogram Inversion on Unseen speakers
  • TTS examples
  • Unconditional Music Synthesis
  • Music Translation
  • Samples along Training
  • Ablation

Abstract

Previous works have found that generating coherent raw audio waveforms with GANs is challenging. In this paper, we show that it is possible to train GANs reliably to generate high quality coherent waveforms by introducing a set of architectural changes and simple training techniques. Subjective evaluation metric (Mean Opinion Score, or MOS) shows the effectiveness of the proposed approach for high quality mel-spectrogram inversion. To establish the generality of the proposed techniques, we show qualitative results of our model in speech synthesis, music domain translation and unconditional music synthesis. We evaluate the various components of the model through ablation studies and suggest a set of guidelines to design general purpose discriminators and generators for conditional sequence synthesis tasks. Our model is non-autoregressive, fully convolutional, with significantly fewer parameters than competing models and generalizes to unseen speakers for mel-spectrogram inversion. Our pytorch implementation runs at more than 100x faster than realtime on GTX 1080Ti GPU and more than 2x faster than real-time on CPU, without any hardware specific optimization tricks. Blog post with samples and accompanying code coming soon.

Results

Spectrogram Inversion on Unseen Speakers

Original

Reconstructed

End-to-end text-to-speech examples

Unconditional Music Synthesis

Original

Reconstructed

Sampled

Music Translation

Example for source domain: Bach Solo Cello

Samples along Training

50 epochs - 1.35 hours

100 epochs - 2.71 hours

200 epochs - 5.42 hours

400 epochs - 10.84 hours

800 epochs - 21.68 hours

1600 epochs - 43.36 hours

3200 epochs - 86.72 hours

Ablation

original

Baseline

l1_observed_no_feat_match

l1_observed_space

no_dilations

no_group_disc

no_multiscale_disc

no_patch_gan

no_weight_norm

spectral_norm