利用三階段行為分析來偵測和分類已知與未知的惡意程式

Abstract

惡意軟體已嚴重危害到網際網路的安全，近年來已有許多惡意程式防治方案被提出。為了達到高偵測率及低時間損耗，本論文提出一個三階段行為分析技術來偵測和分類惡意程式。前兩階段用於惡意程式偵測，第三階段用於分類惡意程式。我們採取兩種不同方式的偵測機制，藉由兩種偵測機制的混合使用，可有效改善偵測準確度並加速偵測流程。在第一階段，我們利用GFI沙盒系統和類神經網路為每一個程式算出惡意程度的值。在第二階段，我們先從惡意程式所產生系統呼叫序列中找出所有共同子字串，再套用貝式機率模型留下惡意行為，再利用這些惡意行為作字串比對來偵測。在第三階段，我們先定義惡意程式類別向量，透過餘弦相似定理計算出該樣本的相似度，再以最高相似度那個向量所代表的類別來做分類。本論文提出之兩階段偵測與分析方法對於惡意軟體之偵測不僅可達到3.6%的漏判率與6.8%的誤判率，且能正確分類超過85.8%的已知型態的惡意程式，此外，本方法在整體效能上亦可減少大量的執行時間。

In recent years, many anti-malware solutions have been proposed. To improve both detection accuracy and time efficiency for known and even unknown malware, we propose a three-phase behavior-based malware detection and classification approach, with a fast detector in the 1st-phase to filter most programs, a slow detector in the 2nd-phase, and then a classifier at the 3rd to tell the malware type. The fast detector runs programs in a sandbox to extract external behaviors fed into a trained artificial neural network (ANN) to evaluate their maliciousness, while the slow detector extracts and matches internal behaviors, i.e., the longest common substring (LCS) of system call sequences, fed into a trained Bayesian model to calculate their maliciousness. In the 3rd-phase, we define malware type vectors consisting of internal behaviors, and calculate the cosine similarity to classify malware. The experimental results show that the integrated 2-phase detection performs significantly better than any 1-phase detection alone in both detection accuracy and time efficiency. The proposed 2-phase detection scheme can achieve 3.6% in FNR and 6.8% in FPR. Besides, this approach can distinguish the known types malware from unknown samples with an accuracy of 85.8%.