Here, I offset the auditory and visual components of the gestures in order to explore the effect of temporal synchrony. The video on the left depicts a stimuli in which the sound leads the moment of visual impact, whereas the video on the right depicts when the sound lags. Because sound travels more slowly that light, the brain is far more experienced in dealing with audio lags (right), rather than audio leads.
These results are consistent with the idea that the illusion is driven by perceived causality. The gesture had a meaningful influence only on the notes it could have caused (those synchronous or subsequent to the moment of visible impact).
It did not meaningfully influence notes leading the moment of impact - which is physically impossible given that sound travels more slowly that sight. Therefore, it appears that once again the brain is strongly attuned to causal relationships across modalities, using this powerful cue to determine when to integrate, and when to separate.
|
 Visual influence was negligible when the note led the moment of impact. However, the influence was retained (albeit diminished) when it lagged by the same amount.
|
|
|
