以認知診斷模型分析國中數學科基本學力測驗並以蒙地卡羅法編製平行測驗

Abstract

大部分基於項目反應理論所發展出來的標準化測驗，通常只提供單一個分數來表 示學生的學業成績表現。然而，除了一個分數之外，測驗應該提供有效的診斷資訊以 幫助了解是否學生已經掌握了測驗所欲評估的主題。一個具有認知診斷功能的測驗可 以針對每一個參加考試的學生在各方面能力精通與否的剖析圖，藉由這個剖析圖，能 夠從中得到學生在哪一方面的技能需要加強的資訊。在三個參數的項目反應模式的架 構中，當數個題本的測驗訊息相同時，這幾個題本可被視為微弱的平行測驗 (Samejima, 1977) 。大部分現有的方法來建構多個平行測驗的題本的方式，均屬於有限的組合最 佳化的方法。Chen (2005)根據三個參數項目反應模式的架構，以模擬參考題本的難度 與鑑別度參數的聯合分配而提出採用蒙地卡羅模擬法(Monte Carlo simulation)來編 製多個平行題本。本研究的目的有二:(一) 將台灣國民中學基本學力測驗的數學科的 考試資料，根據認知診斷模型對題目所欲測量的數學能力來做分析。(二)在根據蒙地 卡羅法的基礎下，應用認知診斷指標來進一步發展出自動產生多份富含診斷考生能力 資訊的平行測驗，以期各平行測驗題本對考生能力能提供出相似的診斷資訊。本研究 將使用一個模擬題庫及台灣國民中學基本學力測驗的數學科的考試資料，根據認知診 斷模型的DINA 模型來做分析。本研究將提出四個不同的抽樣與分類方法，並以業界風 行的整數線性規劃方式為基礎來做比較。

Most large-scale assessments developed based on item response theory (IRT) model only provide a single summary score that indicates the overall performance level of a student. However, assessments should provide useful diagnostic information in addition to single overall scores to know whether the students have mastered in assessed topics. A cognitive diagnostic test can offer a skill mastery profile for each examinee, and identifying areas where additional instruction is needed. In three parameter logistic (3PL) IRT framework, when the test information functions (TIFs) of alternative test forms are identical, the test forms can be treated as weakly parallel forms (Samejima, 1977). Most of the existing methods build multiple forms by using the constrained combinatorial optimization approach to produce similar TIFs. Chen (2005) proposed to use Monte Carlo approach based on the 3PL IRT model to assemble equivalent forms by mimicking the joint distribution of the discrimination and difficulty parameters of the reference form. The purpose of this research is twofold: (1) Analyze the items of the math subset of the Basic Competence Test for Junior High School Students (BCTJHS) based on a cognitive diagnosis model to find out the skills required in order to answer each question correctly;(2) to further develop an algorithm based on the Monte Carlo approach for generating multiple information-rich tests using the Cognitive Diagnostic Index that can provide similar diagnostic information. A simulated data and a real data from the math subset of the BCTJHS in Taiwan will be analyzed based on the cognitive model of deterministic input noisy “and” gate model (DINA). Four different Monte Carlo item selection methods will be proposed, and the integer linear programming method will be used as a baseline to evaluate the effectiveness of the Monte Carlo simulation approach.