以屬性認知診斷指標改善認知診斷模型組卷品質- 以國中數學科基本學力測驗為例

Abstract

近期由於認知診斷測驗模型的發展，使得測驗可以提供每一個學生在各方面能力精通 與否的剖析圖，並得知學生學習狀況的強項與弱項。自動化組卷也是近期急速發展的 研究領域，但大部分現有的組卷方法，均屬於有限的組合最佳化的方法。Henson及 Douglas (2005)是第一個將自動組卷法應用於編製認知診斷測驗。Chen (2011) 採用 蒙地卡羅模擬法，根據Henson及Douglas (2005)所提出的認知診斷指標(CDI)來編製 多個平行題本的認知診斷測驗。然而認知診斷指標(CDI)僅提供題目層級的訊息，並非 屬性層級，如此將使具有高認知診斷指標(CDI)的題目，可能產生部分屬性測量性不 佳的情況。此外，原本的模型假設屬性間是獨立無相關，因此在估計屬性參數時較不 符合實際狀況以致產生估計學生屬性熟悉程度較不正確。本研究的目的有二:(一) 應 用屬性層級的認知診斷指標來改進認知診斷組卷的品質，以期各平行測驗題本對考生 能力能提供更精確的診斷資訊。 (二) 將台灣國民中學基本學力測驗的數學科的考試 資料，根據認知診斷模型對題目所欲測量的數學能力來做分析。本研究將使用一個模 擬題庫及台灣國民中學基本學力測驗的數學科的考試資料，根據認知診斷模型的DINA 模型分析，並提出三個不同的抽樣與分類方法，並以整數線性規劃方式為基礎來比較。

The recent development of Cognitive Diagnosis Models (CDMs) makes it possible to offer a skill mastery profile for each examinee, and identifying the strengths and weaknesses of student learning. Automated test assembly (ATA) is also an emerging research area in educational assessment, but the mainly application are based on traditional single score assessment. Henson and Douglas (2005) was the first study using Automated Test Assembly (ATA) techniques to build CDM-based forms. Most of the existing methods build multiple forms by using the constrained combinatorial optimization approach. Chen (2011) proposed to use Monte Carlo approach based on a CDM model to assemble equivalent forms based on a Cognitive Diagnostic Index (CDI) proposed by Henson and Douglas (2005). However, CDI is an item-level information index, and does not reveal the attribute-level information. Thus, poor measurement may be found on some attributes even though an item has high CDI value. Besides, the correlations among attributes in Chen (2011) were uncorrelated and thus the estimation of student’s mastery level of attributes will not be accurate. The purpose of this research is twofold: (1) to improve the algorithm based on the Monte Carlo approach for generating multiple information-rich tests using the Attribute level Cognitive Diagnostic Index that can provide similar diagnostic information; (2) Analyze the items of the math subset of the Basic Competence Test for Junior High School Students (BCTJHS) based on a cognitive diagnosis model to find out the skills required in order to answer each question correctly. Three different Monte Carlo item selection methods compared with the integer linear programming method to evaluate the effectiveness of the Monte Carlo simulation approach.