Description
MuseNet is a deep neural network created by OpenAI that can generate 4-minute musical compositions with up to 10 different instruments. It combines styles from different genres such as country, Mozart and the Beatles.
It is based on the same general-purpose unsupervised technology as GPT-2, a large-scale transformer model trained to predict the next token in a sequence, whether audio or text.
The model is trained on sequential data, by asking it to predict the upcoming note given a set of notes. It uses chordwise encoding, which considers every combination of notes sounding at one time as an individual ‘chord’, and assigns a token to each chord.
Additionally, the composer and instrumentation tokens are used to give more control over the kinds of samples MuseNet generates. The model is able to generate music that blends different styles and instruments, while also being able to remember long-term structure in a piece.
It is trained using a dataset collected from various sources such as Classical Archives and BitMidi, as well as the MAESTRO dataset.
Pros & Cons
Pros
Generates 4-minute compositions
Combines different music genres
Supports up to 10 instruments
Uses chordwise encoding
Composer and instrumentation control
Blends styles and instruments
Long-term structure memory
Trained on diverse dataset
Advanced and simple modes
Embeddings provide structural context
Visualize embeddings
Transposition and volume augmentation
Timing and mixup augmentation
Inner critic for training
Upload compositions to services
Trained on MAESTRO dataset
Sparse transformer utilization
Context of 4096 tokens
Able to generate musical melodic structures
Impactful music generation
MuseNet experimental concert performances
Large scale transformer model
Generates music blending different styles
Cons
Limited genre blending
Poor with odd instrument pairings
Instrument selection not guaranteed
Style selection not strict
Composer style not strictly enforced
Potential copyright issues
Limited control over notes
Doesn’t support live interaction
Complex token encoding
Difficulty generating long structures