多媒體與遊戲環境中閱讀靜動態數學與科學圖像分析:心智模式、認知負荷與眼球移動分析

Abstract

本計劃之總目標擬以眼球追蹤技術分析閱讀多媒體靜/動態圖像訊息的注意力、理解與認知負荷。我們要研究的圖像分屬數學及科學領域，由簡到繁將列出等級，從靜態的簡潔幾何圖(graph，如幾何相似性質)、示意圖(diagram，機械系統結構與功能)、圖表(chart，數獨的數字二維表)到動畫(科學主題的過橋遊戲)等。閱讀圖像的認知歷程將著重在認知負荷與眼球追蹤分析，概念理解將採用心智模式區分出等級。交大與清大(資工與學習心理)的三位教授將實質合作，提出兩項個別型計劃。除了本計劃，另一項計畫是清大師培中心鍾文宏教授申請之個別型計劃【多媒體科學學習的認知雙軌呈現方式—學習結果與眼球移動分析 】，我們期盼能購置眼動儀，分享實驗設備及技術，將以眼球追蹤為主要分析技術，研究多媒體學習的認知歷程，研究目第分三年描述如下。 第一年研究目的：架設眼動儀硬體與軟體，自建注視動畫之分時眼球軌跡與定位軟體，設計能檢驗多媒體(靜動態)圖像理解的刺激物(以 e-prime 控制)，分為兩學科：數學及物理。測試各種圖像的複雜度、認知負荷程度、心智模式的可能層級。 第二年研究目的：根據第一年研究結果區分出認知負荷三種層級的數學與科學靜態圖像，(1)交叉檢驗各圖像內部外部認知負荷與眼球移動軌跡指標(凝視、掃視、迴視)之關聯，(2)交叉檢驗注視圖像眼的球移動軌跡指標與心智模式層級之關聯。繼續測試實驗適用的動態圖像，測試動態圖像的複雜度、認知負荷程度、心智模式的可能層級。 第三年研究目的：根據第二年研究結果區分出認知負荷三種層級的數學與科學動態圖像，(1)交叉檢驗動畫內部外部認知負荷與眼球移動軌跡指標(凝視、掃視、迴視)之關聯，(2)交叉檢驗各注視動畫的眼球移動軌跡指標與心智模式層級之關聯。

This project will analyze students reading process and understanding of visual displays addressing math and science concepts. The visual displays include levels of complexity, from static graphs, diagrams, charts to animation in games. The process will focus on cognitive load rating and eye movement indicators while conceptual understanding will be drawn through levels of mental model. We, three professors in education and computer engineering from NCTU and NTHU applying two independent projects, will cooperate substantially. We particularly will share the equipment and techniques of eye tracking to address the critical issue about understanding of multimedia visual displays. For the first year, we hope to purchase a set of eye tracking system including computer controllers, camera, screen displayer, monitor software, and cognitive experiment system, e-prime. We will design several programs for our more advanced purposes to deal with eye tracking in viewing animations. A management program of eye movement data that we developed last year will be incorporated in to the eye tracker. We also will design visual displays that could be differentiated along a series of conceptual complexity or cognitive load. Their relevant mental models will be explored at the same time. We hope to build some baseline information about viewing visual displays in learning math and science. We will finish data collection and write at least one paper for the submission to SSCI journal.