利用指令軌跡的相似度進行惡意軟體分群

Abstract

每天都有大量的惡意軟體產生，但這之中有許多惡意程式都是透過既有的惡意程式所改寫而來。這些藉由改寫而來的惡意程式通常都具有類似的行為，透過惡意軟體分群可以將相似的惡意程式歸納到相同的群組。惡意程式分析人員可以利用歸納出來的惡意程式群組加速對未知惡意程式的分析。本論文提出一個完整的分群機制，可以歸納具有相似攻擊行為的惡意程式到相同的群組。為了避免惡意程式使用隱匿程式碼的技術躲避分析，所以我們利用動態的方式在程式執行過程中進行程式追蹤。此外，為了避免惡意程式利用變造系統程式呼叫的內容來混淆追蹤，我們將追蹤的內容目標鎖定在CPU所運行的記憶體指令上。我們引用Taint技術來過濾掉惡意程式呼叫系統函式庫所產生的大量指令，避免真實的惡意程式指令遭到稀釋。藉由惡意程式兩兩間的指令比對，取得惡意程式彼此的相似度。最後再利用分群演算法透過先前取得的惡意程式相似度將相似的惡意程式納到同一群組。根據最後實驗分析的結果，我們有很高的機率將不相似的惡意程式分到不同的群組，並且發現現有的惡意程式分類的子群體。這些子群體的指令關聯性不大，卻同樣可以達成相同的攻擊。

Although a large number of malicious programs are created every day, most of them mutate from existing ones. These mutant malware programs may seemingly appear differently, but actually act with similar behavior patterns. By clustering these malware programs into the same cluster, the malware analysis effort can be reduced significantly. In this paper, we propose a clustering approach to malware classification by comparing instruction trace similarity of binary programs being tested. We take advantage of dynamic analysis to trace malware instructions at runtime. Our method can discover malware in disguise by using techniques such as polymorphism or code injection. By tracing malware instructions, our scheme ensure that the detection mechanism cannot be circumvented or sabotaged by malicious API tampering. The taint technique we adopted can filter massive instructions created by normal system library as noise to the malware analysis. Collected instruction traces are then compared to measure their similarity so that the clustering can be performed. The results demonstrate that our system is able to cluster malware with similar codes, and can recognize new malware which is undetected by anti-virus tools.