人機介面及視窗應用程式之通用橋接介面系統

Abstract

摘要 視窗環境下，傳統方法要讓已開發完成的應用程式，具有介面控制能力，以達到操控應用程式之目的，一般常用的方法，是在應用程式設計時，將介面控制系統之控制功能程式，利用函數呼叫以及模組化之方式，直接寫入並包裝在單一的應用程式□。在如此設計開發方式下，存在以下幾個問題：1、使用者必須要對控制介面及程式語言，具備相當豐富的知識背景，才有可能設計出具有人機介面控制功能的應用程式。2、如果要□現有之應用程式加上人機介面控制功能，設計者必須擁有此應用程式的原始碼，才有可能為此應用程式新增或修改介面控制功能；若沒有應用程式原始碼，我們將很難為應用程式新增或修改介面控制程式。3、即使擁有程式原始碼，設計者亦必須針對應用程式的原理、架構及技術重新加以分析，才有可能撰寫出適合的介面控制程式。如此開發過程顯得非常沒有彈性及效率，且將造成相當大的困難及人力、時間之浪費。基於以上原因，本論文針對軟體工程方法學的應用研究，開發出一個具有通用架構的視覺化介面平台，做為介面控制裝置與應用程式間的橋樑，使用者只需經由此橋樑，針對應用程式之控制元件做些簡易的描述設定，不需要撰寫任何程式，便可將介面控制裝置與視窗應用程式銜接，輕易地將原先不具有介面控制能力的視窗應用程式，設計成為具有人機介面控制功能的視窗應用程式。 本視覺化通用介面系統研究包括兩大部分，第一：我們提出視覺化通用介面架構觀念，可將開發的視覺化通用橋樑介面，銜接一般的介面控制與視窗應用程式，並且提供使用者簡易之介面操作及指令設定。使用者可以在視窗之任意座標上設定可控制圖格物件（Square Object），並附予每個圖格物件一個名稱。只要在視窗應用程式之相對應位置上設定此圖格物件，然後利用語音將控制命令送至開發之通用橋樑介面(Generic Interface Bridge, GIB)，再經由GIB介面解析輸入之指令後，模擬操作滑鼠或鍵盤控制此圖格物件，如此便可達到利用語音來操作視窗應用程式之目的。為了增加操作的彈性及擴充性，我們採用巨集指令（Macro Command）來定義及組合控制指令，使得單一巨集指令可以連續執行數個語音命令，如此可增快指令下達之速度，並可避免因指令過長，造成輸入過程中雜訊進入而影響辨識結果。第二：我們想用PDA手機，透過網路操作遠端PC或數位電視上的多媒體視窗應用程式。然而想要讓手機具有操作遠端多媒體應用程式功能，需要在手機及多媒體應用程式中，撰寫複雜的程式才可以達到。因此我們採用提出之視覺化通用介面方法，並結合發展之程式碼剖析器(Parser Generator)，解析特定裝置上的多媒體應用程式，經由通用橋樑介面的操作及應用系統元件之描述設定，程式開發者可以在不需要撰寫手機介面控制程式下，自動且快速地在手機上產生具有控制多媒體應用程式功能的介面環境，達到以手機遙控多媒體應用程式的目的。 為了展現提出方法的適用性及可行性，在論文展示範例中，我們將原先不具有語音或遙控功能之視窗應用程式，在不撰寫任何程式語言之情形下，用簡單、快速、有效率之方法與我們研發的通用橋樑介面結合，經由語音或PDA手機輸入控制命令，再由橋接介面系統解析，最後達到輕易控制視窗應用程式之目的。

Abstract In a Windows environment, the commonly used traditional method, which allows developed window application programs with Human Computer Interaction (HCI) control ability, is directly to write the control procedures into the application programs while using low-order designing formula to package procedures into single application system. To apply such devising method, the designer must possess certain knowledge about application system designing and programming in order to devise an application system with HCI control functions. Particularly when the design is completed, it is relatively difficult to revise or add any system functions to it without the primitive code. Three major problems may exist under such development of HCI control procedures. First, system designers must be equipped with abundant knowledge about the design of HCI and programming languages in order to design an application with HCI function. Second, if we want to design an application, which lacked interaction ability before, we need to obtain the primitive code of the particular application due to the difficulty in modifying new programs without the code. Third, even if we have obtained the code, we need to re-analyze the entire structure of application in order to write a suitable control program. These tasks will leave the designer with much trouble and seemingly resulting in less flexibility and efficiency. In this thesis we will emphasize on the research of Software Engineer Methodology to develop a visual generic interface bridge (GIB) system. Under this GIB system, designers devise the application system with speech or remote HCI control functions in a much easier and efficient manner with the need for defining only some parameters of objects in an application environment whereas the user are not required to write any program code. The research on GIB system consists of two parts: “Integration of GIB and HCI,” and “GIB-based Application Interface (GAI) generation”. Under the GIB system, we propose the concept of GIB system in connection with HCI and applications. The GIB provides a visual operating interface in which designers draw controllable square objects at any corresponding position on the windows and name each square object. Subsequently, we use speech function to operate actions for mouse and keyboards corresponding to the square object; implying that we can easily control the application with speech as well. In order to increase the operation of applications with more flexibility and expandability, we may use macro command to define and combine the control commands. One macro command may combine with several control commands; this implies that noise effects between long commands can be avoided, making the application control more flexible. Under the GAI system, we use PDA cellular phone to control application programs on the PC in a convenient manner. Nonetheless this process is not simple due to many complicated procedures in and between these systems must be written. For this reason, we use proposed concept of GIB system and adding program parser method, under which the PDA can easily connect with HCI and multimedia applications to achieve the goal of controlling via simple interface operating and setting of AP’s environment. In regards to demonstration of the feasibility and suitability of the proposed method, we put in practice of developing a GIB system to be used as the bridge between HCI interface and window applications. In the following examples, we literally manipulate the connection of a generic visual interface with application program in a very simple, fast, and effective manner. Speech or remote control HCI ability is implemented more easily and efficiently throughout this GIB system without the need for writing any program code.