建構實作評量測驗產生器之研究

Abstract

實作評量是根據受測者實際完成一項特定任務所作的評量，因為實作評量重視受測者實作的能力，所以很多認證考試都需要受測者通過相關的實作評量測驗。軟體操作技能檢定就是一個例子，藉由觀察受測者實際操作軟體來完成特定任務的操作過程，來評量受測者是否具備特定的軟體操作技能。然而，老師以人工的方式評量是花費龐大而且耗時。隨著電腦網路的快速發展，如何利用電腦化測驗的優勢，來輔助老師檢測軟體操作技能是我們主要的研究目標。根據我們的觀察，學生操作軟體來完成技能檢定中設計的特定任務，一般都會設定起始操作畫面，經由一些的操作動作，來檢測是否完成任務的操作需求。在這篇論文中，我們以一個有限狀態機(FSM)來描述軟體的操作流程，將FSM中的狀態(State)定義成描述軟體的操作畫面，而轉換函數(Transition)則定義成描述受測者的操作動作。以這樣的概念，我們提出了Functional Specification Language，即可方便的透過正規文法(Regular grammar)來描述軟體的操作流程，並使用parser generator來產生對應的實作評量試題。老師可以藉由修改正規文法來設計實作評量試題中的檢測項目。我們利用此一方法來實作MS Word的軟體操作試題，最後我們請教學專家來確認這些試題是否能檢測受測者特定的軟體操作技能，而他們認為設計出來的試題是正確且容易被了解的。

Performance-based testing (PBT) is usually used to assess the examinee’s procedural knowledge, the knowledge of knowing how, by performing some real world tasks. Many software skill certification exams have integrated PBT as a part of their exam to certify the examinee’s software operating skill, where the examinee needs to perform a sequence of actions on specific software to achieve the required results. Traditionally, the evaluation of the examinee’s software operating skills which only can be manually done by the teacher is time-consuming and costly. With our observation, using software to perform a sequence of actions to complete the task seems like a navigation process from the starting point of the software run-time state to get the required results, which can be modeled as a Finite State Machine (FSM), where the current state of FSM represents the software run-time status, and the transitions of FSM represent the actions the examinee can perform. Once the examinee performs an action in a certain state, the corresponding state transition will be triggered to move from the current state to the next state and then the PBT tester will visualize the next software run-time status. Based on this concept, a set of regular grammar, called the Functional Specification Language (FSL), is defined to describe the software run-time status and transitions of the PBT tester according to the functionality of specific software. Thus, a parser generator can be applied to generate the corresponding PBT tester based on the given FSL and the related action routines. To evaluate the proposed scheme, several experiments have been done to show the correctness, reusability, and expressive power of the scheme.