提升嵌入式系統圖形化介面測試之精確度

Abstract

基於省電與成本考量，嵌入式系統所配有的運算資源通常有限，要將現有的圖形界面測試技術套用在嵌入式系統軟體開發流程上是非常困難的。隨著智慧型手持裝置(如手機、平板)的普及，在嵌入式系統上進行自動化圖形化介面測試的需求也大幅提高，但現有的測試工具在使用時會有自動化不精確的問題。我們基於現有的測試工具Sikuli以遠端測試架構為基礎實作了圖形化介面自動化測試工具SPAG，以提升測試事件重播的精確度。SPAG將遠端手持式裝置的畫面顯示在本機端，讓使用者可以透過本機端畫面操作手持式裝置並錄製自動化測試項目。我們設計批次事件傳送與智慧等待兩種方法並實作在SPAG中，用以改善滑動等手勢的重播精確度，同時避免待測程式延遲造成的測試失敗。SPAG會自動評估待測程式是否產生延遲，並等待額外的時間讓待測程式完成反應，以提高測試指令的完成機率。基於輔助功能技術，SPAG能錄製流程中自動偵測待測程式狀態的改變，產生驗證指令，並在測試時自動驗證手持式裝置之狀態。我們選定一常用自動化測試工具monkeyrunner來比較測試精確度。在指定待測系統負荷工作量的情況下，SPAG自動化測試的精確度仍然達到90%以上。若依照一般測試流程估計，SPAG 將可以比monkeyrunner少用11%~71%的測試回數。

The embedded systems are built with limited computation resources. Therefore, it is hard to apply conventional GUI testing tools on embedded systems. However, with the popular of smart handheld device, e.g. smart phone and pads, more software run on handheld devices and rising the need of software testing on handheld device. Current testing tools on embedded systems all have defect on reproducing GUI operations, so we design a Smart Phone Automated GUI testing tool (SPAG) based on Sikuli, in order to reduce testing defects. We design Batch Event and Smart Wait method to reproduce GUI operations accurately and ensure the device under test (DUT) can fully process all test operations. We also design the Hybrid Verifying method based on accessibility technology to generate most verify operations and verify them automatically. We use an experiment to compare our SPAG with monkeyrunner on test accuracy and time efficiency, in which SPAG archives 90% testing accuracy with all types of workload while monkeyrunner only has test accuracy of 27%~88%. We use the second experiment to analyze the contribution of our methods. The testing result shows our Smart Wait method brings more improvement on test accuracy.