運用程式碼覆蓋範圍分類程式失誤狀況

Abstract

隨著軟體產業的蓬勃發展，若要維繫良好的軟體品質，必然要能找出軟體錯誤的原因。然而軟體日趨複雜，人工方式除錯成本高昂，已難以應付龐大的程式碼。尋求降低漏洞測試成本、提高除錯效率，成為相關自動化工具發展的最大需求。 自動化的測試工具近年來發展迅速，現存有許多自動化漏洞測試工具，包括Smartfuzz [1]、BFF [2]和FOE [3]等。然而自動化工具中，錯誤分類這個環節的研究日漸趨緩，因此本論文提出一種新的錯誤分類方法，有別於傳統錯誤分類法、基於除錯器的歷史呼叫堆疊(stack trace)為基準，我們提出運用程式碼覆蓋範圍(code coverage)、來達成錯誤分類的需求，。 在評估方面，我們以實際程式(real program)與測試程式(test program)為目標程式，在觀察程式行為後，設計出覆蓋整個原始碼可能路徑的輸入資料。在分類方法上，則是以gcov [4]分析測資涵蓋率，當目標程式輸入測試資料時，若程式發生失誤，則透過分析程式的gcov檔案進行錯誤分類。我們同時針對同筆測試資料經傳統方法分類，比較兩者的分類結果，顯現過去分類方法的缺陷，並探討透過以程式碼覆蓋範圍為基準的方法，改善缺陷、使分類結果更精準。

Software is getting complicated due to the changing needs and flourishing development of software industry. To better improve software quality, we need to find the major reasons which cause the program crash. However, debugging by software developer is not an efficient method, especially in large software. Many automated tools are developed to enhance the fault localization efficiency and reduce the maintenance cost. Several automated tools include smartfuzz, BFF and FOE. Most of the researches are focused on improving software testing process, and the primary triage method is based on the stack trace hash, and is unchanged for a long time. Therefore, we propose a new triage method based on code coverage. We use real programs and special test methods as our target. After observing program behavior, we produce different input data sets for all possible paths. Our triage method is designed by analyzing the gcov coverage results, on every time the input causes the program crash. For the same crash input, we also use traditional stack trace hash method to contrast the flaws with our proposed method. Our experiment results reveal that our proposed method based on code coverage exhibits better triages in terms of number of unique bugs and correct classifications of faults.