VAE for music generation with tension control

Congrats to Rui Guo, who was an intern at AMAAI Lab, SUTD, published a paper on 'A variational autoencoder for music generation controlled by tonal tension', which will be presented next week at 'The 2020 Joint Conference on AI Music Creativity'.

Many of the music generation systems based on neural networks are fully autonomous and do not offer control over the generation process. In this research, we present a controllable music generation system in terms of tonal tension. We incorporate two tonal tension measures based on the Spiral Array Tension theory into a variational autoencoder model. This allows us to control the direction of the tonal tension throughout the generated piece, as well as the overall level of tonal tension. Given a seed musical fragment, stemming from either the user input or from directly sampling from the latent space, the model can generate variations of this original seed fragment with altered tonal tension. This altered music still resembles the seed music rhythmically, but the pitch of the notes are changed to match the desired tonal tension as conditioned by the user.

Link to code repo on github.

Link to demo generated samples.

View paper preprint.