NSynth
Deep Mind, Magenta | |
---|---|
Initial release | 6 April 2017 |
Repository | github |
Written in | Python |
Type | Software synthesizer |
License | Apache 2.0 |
Website | magenta |
NSynth (a
Overview
The model generates sounds through a
Technology
Dataset
The NSynth
Machine learning model
A spectral autoencoder model and a WaveNet autoencoder model are publicly available on GitHub.[11] The baseline model uses a spectrogram with fft_size 1024 and hop_size 256, MSE loss on the magnitudes, and the Griffin-Lim algorithm for reconstruction. The WaveNet model trains on mu-law encoded waveform chunks of size 6144. It learns embeddings with 16 dimensions that are downsampled by 512 in time.[12]
NSynth Super
NSynth Super | |
---|---|
Pitch bend, ADSR | |
External control | MIDI |
In 2018 Google released a hardware interface for the NSynth algorithm, called NSynth Super, designed to provide an accessible physical interface to the algorithm for musicians to use in their artistic production.[13][14]
Design files, source code and internal components are released under an
Influence
Despite not being publicly available as a commercial product, NSynth Super has been used by notable artists, including
Grimes reported using the instrument in her 2020 studio album Miss Anthropocene.[5]
Claire L. Evans compared the potential influence of the instrument to the Roland TR-808.[21]
The NSynth Super design was honored with a D&AD Yellow Pencil award in 2018.[22]
References
- arXiv:1704.01279 [cs.LG].
- arXiv:1704.01279.
- arXiv:1609.03499 [cs.SD].
- ^ "Google's open-source neural synth is creating totally new sounds". Wired UK.
- ^ a b "73 | Grimes (c) on Music, Creativity, and Digital Personae – Sean Carroll". www.preposterousuniverse.com.
- ^ Mattise, Nathan (2019-08-31). "How YACHT fed their old music to the machine and got a killer new album". Ars Technica. Retrieved 2022-11-08.
- ^ "NSynth: Neural Audio Synthesis". Magenta. 6 April 2017.
- ^ "NSynth Dataset". Machine Learning Datasets. Retrieved 2022-11-08.
- arXiv:1907.08520 [cs.SD].
- ^ "The NSynth Dataset". tensorflow.org. 5 April 2017.
- ^ "NSynth: Neural Audio Synthesis". GitHub.
- arXiv:1704.01279 [cs.LG].
- ^ "NSynth Super is an AI-backed touchscreen synth". The Verge. 13 March 2018.
- ^ "Google built a musical instrument that uses AI and released the plans so you can make your own". CNBC. 13 March 2018.
- ^ "googlecreativelab/open-nsynth-super". April 1, 2021 – via GitHub.
- ^ "Open NSynth Super". hackaday.io. Retrieved 2022-11-08.
- ^ "NSYNTH SUPER Hardware". GitHub.
- ISSN 1059-1028. Retrieved 2023-01-19.
- ^ "Cover Story: Grimes is ready to play the villain". Crack Magazine. Retrieved 2023-01-19.
- ^ "What Machine-Learning Taught the Band YACHT About Themselves". Los Angeleno. 2019-09-18. Retrieved 2023-01-19.
- ^ Music and Machine Learning (Google I/O'19), retrieved 2023-01-19
- ^ "NSynth Super | Google Creative Lab | Google | D&AD Awards 2018 Pencil Winner | Interactive Design for Products | D&AD". www.dandad.org. Retrieved 2023-01-19.
Further reading
- Engel, Jesse; Resnick, Cinjon; Roberts, Adam; Dieleman, Sander; Eck, Douglas; Simonyan, Karen; Norouzi, Mohammad (2017). "Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders". arXiv:1704.01279 [cs.LG].