Exploring how a Generative AI interprets music

This paper investigates how Google's MusicVAE, a Variational Auto-Encoder, interprets music. The researchers analyze the latent space of the model, identifying 'music neurons' that encode musical information and 'noise neurons' that remain largely inactive. They explore how pitch, rhythm, and melody are represented within the music neurons, finding that pitch and rhythm are primarily encoded in the first few neurons, while melody becomes more apparent in independent neurons for longer musical sequences.

Gabriela Barenboim, Luigi Del Debbio, Johannes Hirn, Verónica Sanz