電腦化測驗與診斷系統發展及試題屬性與學習表現研究

Abstract

在教育情境中，測驗的目的在於評量學習者之成就，診斷試題品質及協助改善教學。由於資訊科技之快速進展與普及，電腦化測驗已被廣泛應用於非正規及正規測驗中。多媒體教材對於學習者學習抽象概念有極大助益，多媒體形式測驗題目在電腦化適性測驗中是否會影響受測驗者對於題目語意了解度及造成試題屬性差異，是值得探索的問題。為探討此問題，本研究提出IDEA模式及混合評量架構且進行相關實驗。 合適的能力估測引擎是任何電腦化適性測驗系統的關鍵成份。運用不同的能力估測方法，對於受測者的特殊反應樣式，將產生一些問題如發散或慢速收斂等狀況。能力估測引擎影響適性測驗的結果及效能，針對此問題，本研究實作四種電腦適性系統中常用能力估測程式(OWEN, EAP, MLE和WLE)，且評估及比較其收斂狀態及動態行為。 無論在數位學習、遠距教學甚至是傳統教學中，診斷學習者之學習問題一直是極高難度的任務。由Sato所提出S-P表及其注意係數可被用來診斷學生之學習表現中不正常現象與試題的適用性，然而S-P表忽略受測者之反應時間之因素。本研究提出加強式SP模式，此模式將時間因素列入考慮，且推導出敏捷度及試題解題能力指標以描述受測者在測驗中反應敏捷度及問題解決能力。 基於加強式S-P模式，本研究發展了網路化測驗系統，系統中提出許多有用的圖表及學生之處方籤，用此診斷教學及學習問題。為了近一步診斷學生心智中錯誤概念，本研究以電磁鐵概念為例，發展二階段式診斷測驗及引導補救教學實驗。教學實驗結果顯示實驗組之補救學習成效顯著優於控制組。 在現代測驗理論或古典測驗理論中試題參數是了解試題品質的重要指標，在S-P表或加強式S-P模式中的診斷性資訊亦是了解學生狀態表現及試題品質的重要索引值，本研究試圖引導電腦化英語能力測驗之實驗以探索上述測驗理論中試題參數間之關係，及受測者反應參數間之相關性。且以迴歸分析推導出預測學生表現之相關方程式。

The purposes of testing in the educational context are to assess the learners’ achievement, to diagnose the test items’ quality and to evaluate the educational goal. Due to the rapid progress and widespread dissemination of information technology, the computer-based test is applied on informal or formal tests. Multimedia materials can facilitate the students in terms of reading, understanding, and interpreting abstract concepts. How multimedia presentation form affects the item attributes (i.e., difficulty level, discriminatory power) and the students’ ability to understand the test items’ semantic meanings in IRT-based computerized test is a valuable topic to survey. An IDEA model and a hybrid assessment framework were constructed and an experimental study was conducted for this purpose. An adequate ability estimation engine is the vital component of any efficient and accurate computerized adaptive test system. When estimated using different ability estimation methods, different test takers’ response patterns, especially extreme ones, would produce some problems such as divergence or slow convergence. The ability estimation engine affects the test with respect to its outcome as well as efficiency. To address these issues, this study developed and implemented four ability estimation programs (OWEN, EAP, MLE, and WLE) widely used as the IRT ability estimation engines for a computerized adaptive test system. Then different response patterns were fed to the IRT ability estimation engines so as to evaluate and compare the convergent state under various response patterns, and investigate the dynamic behavior. How to diagnose the learners’ learning problem is a challenging task in either e-learning or the traditional classroom. S-P chart and its caution indexes can be used for diagnosing the students’ abnormal performance and the test items’ suitability. But S-P chart neglects the response time factor. This study proposed an enhanced S-P model, taking the time factor into consideration, and deduced a nimbleness index and test-item solving index for depicting the test-takers’ responding agility and problem-solving ability in the test. Based on enhanced S-P model, this study developed a web-based test system. Various useful charts and prescriptions can be generated by the system and used by instructors to diagnose their instructional approaches and by students to diagnose their learning performance. In order to further diagnose the learners’ mental misconceptions, this study conducted a two-tier diagnostic test and remedial learning experiment by adopting the electro-magnetic concept as an example. The remedial learning effect of the treatment group was significantly better than that of the control group. The test items’ parameters in Item Response Theory or Classical Test Theory are important indicators for understanding the quality of test items. Scoring and diagnostic information in S-P chart or enhanced SP model are essential indexes for understanding the students’ learning effect and performance. This research adopted an English proficiency test as an example and attempted to conduct an experiment for studying the correlations among different item attributes such as the difficulty index, discrimination index, guess index in IRT model as well as the student caution index, and the correlations among different learning performance indicators such as test score, problem-solving ability, and nimbleness index (or test-taker’s response time) in the enhanced S-P model. Further, regression analysis was conducted to deduce the related index’s power of prediction for learners’ performance.