基於系統呼叫之整合式惡意程式行為偵測

Abstract

惡意程式可以泛指病毒、木馬程式、蠕蟲等會對使用者電腦造成危害的軟體，由於目前網路的發達以及一些自動化惡意程式產生工具的散播，使得新型態惡意程式的成長和變種速度加快，他們常常會經由加密、多型、變型等方法來混淆其特徵值，使得使用傳統特徵比對方法的防毒軟體無法即時偵測，因此一些以即時偵測為目標、基於系統呼叫的動態偵測方法被提出，這些動態方法針對惡意程式的檔案和登錄檔存取行為做偵測，但是其缺點是只有監測單一程序的行為，無法得知程序之間的互動情形。本論文中為了改善這些缺點，提出了結合自我參考複製行為、金字塔結構、以系統呼叫組成的有限狀態機和存活意圖的動態偵測方法，在綜合這些方法後可以有效的監控系統中所有程式的行為，進而能夠偵測所有可能存在的惡意程式，這些方法各自擁有不同的權重，在執行偵測之後以加權的方式算出總分，以超越門檻與否決定是否是惡意程式。本論文在虛擬機器中實驗了20種最近的惡意程式，並且監控其系統呼叫，在分析這些紀錄後對於惡意程式的偵測率達85%，另外在一般使用者日常經常執行的不同類型應用程式中則沒有誤報產生。

Malware refers to the software that can harmuser’s computer such as virus, trojan and worm. The growth and metamorphic rate of new types of malware have been accelerated due to the de- velopment of the internet and the widespread of automated tools. Malware usually use encryp- tion, polymorphic and metamorphic to achieve the obfuscation of their signature, causing the traditional signature-based antivirus cannot detect them in real-time. Therefore, some system call based methods have been proposed in order to achieve real-time detection. These methods focus on monitoring the file and registry access behavior of malware. The behavior of each process is examined separately, resulting in that they have no ability to monitor the interaction between processes. In this thesis, we propose an integrated approach including self-reference replication, GSR pyramidal structure, finite state automata formed by system calls and survival intent as the dynamic detection method to improve these defects. After combining these meth- ods, we can monitor the behavior of every process in the system effectively, and thus be able to detect any possible existence of malware. Each of these methods has been given different weights, the sum of weights can be used to determine the detection result if it exceeds the pre- defined threshold. We test 20 recent malware samples in virtual machines and monitored all system calls. After analyzing these system calls and applying our method, the detection rate is up to 85%. Besides the malware sample, the false positive rate is 0% in the detection consist of different types of benign samples which are frequently used by normal users.