聽覺節奏對視覺注意力的影響

Abstract

對於接收外在世界的訊息，視覺與聽覺是我們最主要且仰賴的感覺形式。隨著媒體業的蓬勃發展，視覺傳達媒體也逐漸趨向運用結合聲光效果的多媒體形式呈現，聲音可視為視覺設計的要素之一。因此，理解視覺與聽覺整合的知覺現象，以及如何讓聲音與影像能有最佳的配合成為設計者需思考的問題。本研究針對聲音節奏對於視覺注意力的影響進行實徵性的探討，我們假設在視聽刺激節奏一致的條件下，聲音節奏可引導觀者的視線，而將注意力投注在與聲音節奏呼應的影像上。根據上述假設，我們利用心理學實驗將視覺刺激以快速序列性的方式呈現 ( rapid serial visual presentation，RSVP)，測量受視者於螢幕上某定點的視覺注意力，並且於螢幕上同時播放兩個節奏不同的RSVP序列形成雙重作業(dual task)。於作業中加入與其中一序列契合的聲音節奏，測量受試者於視聽覺訊息節奏同步與不同步RSVP作業的表現。 透過操弄視聽覺訊息節奏是否契合的變因，進而測量受試者投注於兩RSVP序列的注意力，以分析獲得的結果探討注意力引導與視聽覺整合的問題。實驗結果有兩個主要的發現：1.聽覺節奏能引導受試者注意節奏同步的視覺訊息。2.利用節奏的一致性能將視聽覺刺激整合，屬於基本層次的知覺現象。節奏可運用圖形的明暗、大小和音符的強弱、高低變化而形成。因此實驗結果應用於影音媒體的製作，我們可以思考如何使視覺媒體更富有節奏，並且利用畫面與聲音節奏的設計形成觀眾注意力的動線，或藉由同步的節奏讓語意矛盾、或無語意關聯的視聽覺元素產生連結，待觀者理解後產生諷刺幽默等情緒反應。 另外，於實驗中發現兩個特徵不同的聲音刺激，對視覺訊息注意力的影響程度不同，隱含視聽覺元素之間是可能存在契合的形式，如聲音的音高、音強與畫面的色相、明度能相互類比。這類更高層次的視聽知覺對應，有待未來更進一步的研究。

Vision and audition are the two major modalities we use to receive outside messages. There is evidence indicating that vision and audition do not function independently. A better understanding of the interaction between these two senses is of great value to a designer. The focus of our study is the effect of audio rhythm on visual attention. We hypothesize that if the visual and the auditory stimuli are synchronized, viewer can be cued by the auditory rhythm and would pay more attention to the synchronized visual event. The aim of this study is to test this hypothesis. We used rapid serial visual presentation (RSVP) as a means to probe subjects’ visual spatial attention on a given spot of the screen. Two RSVP streams of different rhythms were presented to the viewer on each trial. One of the RSVP stream is synchronized with an auditory stimulus while another is not. If the viewer’s attention can be guided by the auditory rhythm, one would predict that the performance in the synchronized RSVP be better than that in another stream. The results show that: 1. The auditory rhythm, while being task-irrelevant by itself, does cue subjects’ attention to the synchronized visual event. 2. The power of cueing visual events is critically dependent upon the acoustic properties of the auditory stimulus. 3. Some rhythms are more potent than others in binding visual and auditory events. 4. As most viewers were not aware that one of the RSVP streams was synchronized to the auditory event, we believe the enhancement effect by synchronization occurs at an early, preconscious level.