多媒體英語文能力檢定暨適性化網路評量系統之建置

Abstract

本研究的目的是建立一套多媒體英語文能力檢定暨適性化網路評量系統，此系統功能如下所示： □ 符合英語聽、說、讀、寫四種能力的檢定機制，並以多媒體試題呈現之。 →改進傳統英語評量系統中缺乏支援多樣化多媒體試題的能力。 □ 題庫適性值，將隨著不同受試樣本做動態的調整。 →改進傳統英語評量系統中缺乏動態適性值能力的調整機制。 □ 將提供網路介面適性測驗的試題特徵曲線，方便研判試題適性值。 →改進傳統英語評量系統中缺乏試題特徵即時統計與曲線呈現機制。 □ 整合不同的IRT分析器，進行適性測驗。 →改進傳統英語評量系統中缺乏整合不同特性的IRT分析器的能力。

本系統在題庫的建立上可做同等級試題適性值的驗證與兩段式的動態調整；並可多樣化呈現多媒體試題，為一多媒體適性化測驗平台。在聽力測驗與閱讀測驗中，由電腦自動評分，學生可立即知道成績，得到回饋。在口說測驗與寫作測驗中，更提供友善的介面，方便教師線上閱卷評分。測驗後的結果，亦可提供學生、教師及家長作成績查詢。

本系統可針對學生的作答反應資料，作IRT參數估算的分析，分析試題適性值，並分別產生試題的ICC圖、IIC圖及試題難易度、鑑別度及猜測度等相關資料以進行研判。此系統不僅可動態地調整題庫之適性值，同時亦可編製成適性化測驗。當IRT試卷編製完成後，可以透過系統，進行試測，而系統將可立即性劃出作答情形、能力估算及標準誤圖形曲線，方便使用者研判。

Recently, test items based on multimedia presentation for computerized test approach have received considerable attention due to the use of animation and video presentation features for the stimuli in the test question. The dynamic features incorporated in these multimedia test items can simulate the daily life that we encountered in the real world. This is particularly important for the language learning area such as English language ability evaluation. Traditional computerized English test questions lacking of the use of animation presentation features makes it a counterpart of the pen-paper based test approach except that it is conducted in the computerized environment. Also, the current IRT English test delivery systems do not support test analyzers the real-time display of Item Characteristic Curve (ICC), Item Information Curve (IIC), and Test Information Curve (TIC). Furthermore, only single IRT analyzer (such MLE or EAP) is used to estimate the test taker’s ability, which may become inaccurate when abnormal response patterns are detected during the test. This thesis study is to design and implement a multimedia computerized English test delivery system that overcomes those problems mentioned above. Specifically, the proposed system supports the following features: 1) it is equipped with listening, speaking, reading, and writing tests in multimedia presentation; 2) it is equipped with a two-staged IRT item attribute adjustment feature to normalize test question attribute indexes; 3) the analysis system allows analyzers to upgrade, downgrade, or modify the item attributes such as difficulty level of the analyzed test question by using the real-time display curves: Item Characteristic Curve (ICC), Item Information Curve (IIC), and Test Information Curve (TIC); and 4) it integrates several IRT analyzers to give a more accurate estimation of the test taker’s ability when abnormal response patterns are detected. Application examples are studied and illustrated using the proposed approach. Besides, the proposed test delivery system and corpus analysis system has been used for the National English Test of Proficiency For All on the Web (NETPAW) to demonstrate the applicability and feasibility.