協同式軟體測試架構之探討

Abstract

軟體測試是一件耗費人力的工作，軟體測試過程中，往往需要測試人員、開發人員與使用者的參與，充份的溝通，才能順利的進行。近年來，群眾外包(crowdsourcing)慢慢形成風潮，「三個臭皮匠，勝過一個諸葛亮」，利用群體的多樣性，提供解決問題的知識或創意創新的點子，藉此完成企業的外包工作。網路上有相當多的人力資源，但是要如何有效運用這些沒有組織的人力來完成企業的工作，變成群眾外包中相當重要的課題。本論文提出一個協同式的軟體測試方法架構，包含三個階段，測試階段、監控階段與診斷階段。在測試階段，利用群眾外包環境的特性，將協同測試問題轉化成整數線性規劃問題(ILP)，稱為群眾外包協同測試方法(crowdsourcing-based collaborative testing approach)。考量群眾外包的特性，此方法測試階段主要是將協同測試問題轉化成ILP問題，我們發現協同測試是一個NP-Complete問題。因此，我們提出一個啟發性的演算法，並利用啟發式演算法，採用四種貪婪式策略求解。監控階段則是在測試標的上佈置監控器，藉以監控測試過程中的變化。診斷階段利用監控結果，找出可能存在的系統瓶頸與問題。為了實現我們的演算法，本論文也開發了一個雛形工具，協同測試系統(Collaborative Testing System)，簡稱為COTS。實驗結果發現，我們的方法可以在有限的時間內，產生符合效益的有效解，結果近似於ILP工具產生的最佳解。

Software testing processes generally are labor-intensive processes and often involve substantial collaboration among testers, developers, and even users. However there is considerable human resource capacity on the Internet, such as in social networks, expert communities, or internet forums, called “crowd”. Therefore, how to effectively utilize crowd resources to support collaborative testing becomes an interesting but challenging topic. By considering features of crowdsourcing, we propose a framework for collaborative approach of software testing. This approach contains testing phase, monitoring phase, and diagnosis phase. In testing phase, the collaborative testing problem in the “crowd” environment is an NP-Complete job assignment problem and can be formulated as an Integer Linear Programming (ILP) problem. We transform the original problem into an ILP problem and solve the ILP by heuristic strategies. Although the optimal solution of the ILP problem can be obtained by package tools, the computational complexity makes it infeasible for solving large-scale instances. Our idea is to use the greedy approach with four heuristic strategies to solve the problem, called crowdsourcing-based collaborative testing approach. In monitoring phase, we acquire knowledge of domain experts and deploy monitors to monitor the testing system. In diagnosis phase, the monitoring results are analyzed to identify the bottleneck and problem in testing system. Besides, we implement the prototype system, COllaborative Testing System, COTS. The experimental results show that our proposed heuristic algorithms can produce approximate solutions of good quality in an acceptable time.