Jesse Engel

@jesseengel

Guitarist, Researcher Google Brain. Opinions are my own.

Vrijeme pridruživanja: travanj 2009.

Tweetovi

Blokirali ste korisnika/cu @jesseengel

Jeste li sigurni da želite vidjeti te tweetove? Time nećete deblokirati korisnika/cu @jesseengel

  1. Prikvačeni tweet
    15. sij

    Differentiable Digital Signal Processing (DDSP)! Fusing classic interpretable DSP with neural networks. ⌨️ Blog: 🎵 Examples: ⏯ Colab: 💻 Code: 📝 Paper: 1/

    Prikaži ovu nit
    Poništi
  2. proslijedio/la je Tweet
    28. sij

    バーコーダーセッション🏪 バーコードリーダーのスキャン信号をレジではなく、スピーカーに直接接続することで音を鳴らす。 昼はバイトでレジ係、夜はクラブでバーコーディストになれます┃┃┃┃

    Prikaži ovu nit
    Poništi
  3. 28. sij

    "The goal, he told me, wasn’t to sell an ideology or a vision of the future; instead, it was to convince people that “the truth is unknowable” and that the only sensible choice is “to follow a strong leader.””

    Poništi
  4. 15. sij

    Finally, many thanks to Dr. Xavier Serra of and Dr. Julius O. Smith III of , whose pioneering work on Spectral Modeling Synthesis () was the foundation and inspiration for a lot of this research.

    Prikaži ovu nit
    Poništi
  5. 15. sij

    This was a really fun collaboration with , , and . Also thanks to for all the help, and to for the designing a sweet logo.

    Prikaži ovu nit
    Poništi
  6. 15. sij

    14/ For applications, early experiments are promising. We can push the model size down very small (a single 256 unit GRU, 240k parameters, ) and still get pretty good performance, opening avenues towards realtime neural audio synthesis and manipulation.

    Prikaži ovu nit
    Poništi
  7. 15. sij

    13/ From a research perspective there's a lot of potential followup. We've added some new modules (Wavetable synth, ModDelays, etc.) but still many more possible (Polyphony, IIR, etc.). Also, models using DDSP create samples during training, which is ideal for GANs, EBMs, etc.

    Prikaži ovu nit
    Poništi
  8. 15. sij

    12/ Since we're excited about the possibilities of this approach, we've worked really hard to make a clean and modular code base (), several colab tutorials (), and we welcome contributions! DSP is hard to get all the details right 😇

    Prikaži ovu nit
    Poništi
  9. 15. sij

    11/ One of the key priors these models exploit is perceptual invariance to relative phase for steady state signals (which was the basis of GANSynth, a differentiable phase vocoder). Notice how all these examples sound the same even though the waves look quite different.

    Prikaži ovu nit
    Poništi
  10. 15. sij

    10/ One of the coolest things about using these priors is it takes much less data and compute to do ML with audio. All examples here use less than 13 minutes of data and a few hours on a V100. Hanoi could even train a model on his own Salo instrument ().

    Prikaži ovu nit
    Poništi
  11. 15. sij

    9/ Here's an example of turning into a violin. You can try this yourself with the colab demo (). You can also train your own models and use them in the demo. We can't wait to see how you use it!

    Prikaži ovu nit
    Poništi
  12. 15. sij

    8/ Further, we can get features such as pitch and loudness from one signal and use the trained model to resynthesize (timbre transfer). There are small hiccups when the features don't match training, but the model does surprisingly well given that it wasn't trained to do this.

    Prikaži ovu nit
    Poništi
  13. 15. sij

    7/ Since the model directly uses interpretable inputs like frequency in rendering the waveform, it allows generalization outside the domain of the training data. Here we transpose the violin an octave lower than is physically possible and it sounds somewhat like a cello.

    Prikaži ovu nit
    Poništi
  14. 15. sij

    6/ The key idea of DDSP is that simple DSP components can be quite expressive when precisely controlled by a neural network (e.g. high-quality reconstructions). We can also exploit modularity and interpretablity to swap components and get things like dereverberation for free.

    Prikaži ovu nit
    Poništi
  15. 15. sij

    5/ We combine a harmonic additive synth (sum of sinusoids) with a subtractive noise synth (filtering white noise), and a learned room reverb, to generate a waveform. The loss is then L1 on a multi-scale spectrogram. Since everything is differentiable we can train with SGD.

    Prikaži ovu nit
    Poništi
  16. 15. sij

    4/ While the DDSP components can be used in any end-to-end system, we focus on the effect of the components themselves by performing experiments with a simple audio autoencoder (no autoregression or GAN required). The DDSP components are the yellow blocks. F0 is frequency.

    Prikaži ovu nit
    Poništi
  17. 15. sij

    3/ An example DDSP module is an Additive Synthesizer (sum of time-varying sinusoids). A network provides controls (frequencies, amplitudes), the synthesizer renders audio, and the whole op is differentiable . Here's a simple example with harmonic (integer multiple) frequencies.

    Prikaži ovu nit
    Poništi
  18. 15. sij

    2/ tl; dr: We've made a library of differentiable DSP components (oscillators, filters, etc.) and show that it enables combining strong inductive priors with expressive neural networks, resulting in high-quality audio synthesis with less data, less compute, and fewer parameters.

    Prikaži ovu nit
    Poništi
  19. 8. sij

    Things that should go without saying still need to be said. Deescalate now.

    Poništi
  20. proslijedio/la je Tweet
    26. pro 2019.

    Bayesian methods are *especially* compelling for deep neural networks. The key distinguishing property of a Bayesian approach is marginalization instead of optimization, not the prior, or Bayes rule. This difference will be greatest for underspecified models like DNNs. 1/18

    Prikaži ovu nit
    Poništi

Čini se da učitavanje traje već neko vrijeme.

Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.

    Možda bi vam se svidjelo i ovo:

    ·