A musical illusion

One of our primary research interests is in how we enjoy the illusion of unified experiences even when presented with conflicting sensory information from different modalities.  Given its ubiquity and antiquity, music is a particularly interesting domain through which to explore such questions.  For example, we have discovered that expert percussionists use gestures to control an audience's perceptual experience of their performances.    To demonstrate this, we recorded a world-renowned marimbist performing notes using both long  (left video)  and short (right video) physical gestures. We then crossed (swapped) the audio and visual components of these gestures, and asked participants to judge the duration of the sound independent of the image. In a separate condition, we also asked them to judge note duration when presented with the audio alone. The results indicate that although physical gesture length does not affect note acoustics, it does affect note perception. In other words, expert musicians use this illusion to control an audience's musical experience beyond that which is acoustically possible!   Duration ratings (y-axis) of marimba notes presented as audio alone (left) and audio-visual stimuli (right). Notes produced by long gestures (red) are not rated any differently than notes produced by short gestures (blue) when presented as audio-alone. However, they are rated differently when presented with the accompanying gesture. For more information on how this research applies to music performance and music education, please visit  www.michaelschutz.net. To see Michael Schutz demonstrating the illusion on television, visit the Press Room.  To download sample videos to use in presentations/ demonstrations, click here.  Please cite Schutz and Lipscomb (2007) as the source, and email us to let us know this was useful for educational purposes!