Excited to share recent work on music transcription w/ and , appearing today at #ISMIR2022! We release Sheet Sage, a system which can transcribe music as lead sheets 🧵
📄 arxiv.org/abs/2212.01884
⭐ github.com/chrisdonahue/s
🔊 chrisdonahue.com/sheetsage
Conversation
Replying to
Lead sheets are scores which depict melody as notes and harmony as chord names, and are often used by musicians to perform new renditions of existing songs. Sheet Sage can convert music audio into this format w/ little setup:
> ./sheetsage.sh <ANY_URL>
⭐
1
1
18
Lead sheets represent the essence of a piece of Western music (especially pop). Transcribing in this format could help bridge the gap between machine understanding and human perception of music, but robust *melody transcription* remains an open challenge in MIR research.
1
5
We improve melody transcription by leveraging representations from Jukebox, a foundation model of music audio. Our previous ISMIR paper* demonstrates that Jukebox is useful for many MIR tasks - with this work, we add melody transcription to the list
*
1
5
Data was another key component of improving melody transcription. Alongside Sheet Sage, we also release a dataset of 50 hours of melody and harmony transcriptions derived from user contributions to . We hope this data will be useful to the broader MIR community!
1
6
For more info, please check out our #ISMIR2022 presence this afternoon in Session 4, or virtually: ismir2022program.ismir.net/poster_300.html
1
5
Finally, my co-author is actively on the job market!! He is an extraordinarily gifted researcher and a wonderful human being
1
8
am I hallucinating or did I just hear MapleStory music in a ML research project
I love it. Have you tried in on faster things? Coltrane’s giant steps solo… etc.
Such a great work 🤘also love the Real Book vibes!
1
Show additional replies, including those that may contain offensive content
Show