Automatic self-similarity based form labelling of classical-period piano sonata movements from audio recordings

Abstract

Musical form refers to the overall structure or organisation of a musical composition. It is a complex and high-level property of music that requires musical training to identify. A review of previous research in this field indicates that the focus has been on the task of detecting section boundaries and that automatic audio based form label recognition is a field of study that remains largely unexplored. This study explores the complex task of automatically determining musical form from audio. It demonstrates the ability of a novel methodology to label eight different form types that occur in the movements of Classical-period piano sonatas. The methodology makes use of self-similarity matrices, generated from features extracted from raw audio, as input to a convolutional neural network. The superiority of our approach was confirmed by evaluating it against a neural network model based on state-of-the-art features. We also report an evaluation of self-similarity matrices based on automatically transcribed piano rolls for the task of form recognition. Piano rolls are demonstrated to be superior for this application when compared to a range of other feature representations. Additionally, the performance of the model is shown to be robust in handling variations in performer choices. These range from different interpretations of the same score to actual deviations from the score where performers may elect to play or not to play notated repeats thus highlighting its ability to generalise across different performances of the same piece.