The aim of the Information Dynamics of Music (IDyOM) project is to relate dynamic changes in the information-theoretic structure of a piece of music with cognitive and neural processes during the ongoing psychological experience of listening to that piece. In particular, the project aims to understand how expectation and surprise can be quantified in terms of measures derived from information theory and related to redundancy reduction in perception, musical aesthetics and music theoretic analysis. The project has several components starting with the development of the information-dynamic models and then using them to make and test predictions about cognitive processes in music perception, neural responses to music and musicological analyses of particular pieces of music.
Information Dynamics
The first goal of the project is to developing and evaluate a range of probabilistic models of music in the audio and symbolic domains. Initial work in the project developed new methods for audio onset detection based on information dynamics concepts using Markov models (Bello et al., 2004), and duration models in audio segmentation (Abdallah et al., 2006), as well as sparse representations to map from the audio to the part-symbolic domain.
In the symbolic domain, we have extended the multiple viewpoints approach of Conklin and Witten (1995) to produce dynamic probabilistic models of melodic prediction that use variable-order contexts, long- and short-term information and combine information from multiple musical features in predicting note attributes such as pitch, onset time and duration (Pearce & Wiggins, 2004; Pearce et al., 2005).
We have also developed a novel information-dynamic model based on the concept of predictive information rate, which measures how much information is gained by current observations about the future, but which is not already known from past observations (Abdallah and Plumbley, 2009).
Auditory Perception
The information dynamic models that we develop can be used to make predictions about listeners' responses to music, which can then be tested empirically. We have shown, for example, that information-dynamic models of surprisingness predict listeners' pitch expectations well (Pearce & Wiggins, 2006; Pearce, Ruiz, Kapasi, Wiggins and Bhattacharya, 2010). Notes whose pitches are improbable given the preceding context are perceived as unexpected and vice versa. These results generalise across a range of melodic contexts including single intervals, English folks songs, chorale melodies and English hymns. The information-dynamic models predict listeners' expectations better than existing rule-based models of melodic expectation (Narmour, 1990; Schellenberg, 1997). We have also found that notes tend to be more expected for familiar melodies and musically trained listeners (Pearce et al., 2010).
We have also used the information-dynamic models to predict other aspects of musical perception such as phrase segmentation. We hypothesise that grouping boundaries in music correspond to points where the context fails to inform the listener about the identity of the next musical event. This might happen when an unexpected (low probability) event arrives or because the listener is simply uncertain about what will happen next (high entropy). We have found some evidence to support this hypothesis both at the level of phrase boundaries (Pearce, Müllensiefen and Wiggins, 2008, 2010) and of high-level form (Potter, Pearce & Wiggins, 2007). We are currently investigating whether these results generalise to the perception of word boundaries in language.
Neural Responses
We are interested in understanding the dynamics of neural responses to music, in particular the neural mechanisms that generate expectations and produce responses to expected and unexpected musical events. For this we primarily use EEG which gives us the temporal resolution necessary to associate brain responses with specific musical events. Given our interest in dynamics, we also focus on ongoing oscillatory activation (e.g., time-frequency and phase-synchronisation analyses) rather than methods such as ERP analysis. Using these methods, we have shown, for example, that listeners find low probability events in melodies unexpected and that unexpected events generate characteristic patterns of beta band activation and phase-locking at centro-parietal sites (Pearce et al., 2010).
Computational Musicology
We also use musicological analysis to test the predictions of our information-dynamic models. Here we hope to be able to retrieve the structure of a piece of music (as stated by the composer or interpreted by a music-analyst) from the information-dynamic output of our models. In the symbolic domain, we have used information-dynamic models of pitch expectation and uncertainty (Pearce and Wiggins, 2006) and also predictive information (Abdallah & Plumbley, 2009) to predict structural boundaries and other musicologically and perceptually important events in pieces of minimalist music (Potter et al., 2007). Minimalist music is attractive because many minimalist pieces are by their nature constrained such that structural variations appear in only one musical dimension such as pitch or time. This gives us tight experimental control whilst also maintaining ecological validity.