A framework for quantitative evaluation of parallel control-flow obfuscation

Abstract

Software obfuscation is intended to protect a program by thwarting reverse engineering. Several types of software obfuscation have been proposed, and control-flow obfuscation is a commonly adopted one. In this paper, we present a framework to evaluate parallel control-flow obfuscation, which raises difficulty of reverse engineering by increasing parallelism of a program. We also define a control flow graph of a program and some atomic operators for obfuscating transformations. The proposed framework comprises three phases: parsing, formalization and evaluation. A program is first parsed to a control flow graph. Then, we formalize a parallel control-flow obfuscating transformation based on our atomic operators. By selecting target code blocks in the control flow graph and applying obfuscating transformations to the target code blocks, the original program is then obfuscated. In the third phase, we define a measure to calculate the program complexity. The measure can be considered as a degree to which an obfuscating transformation can confuse a human trying to understand the obfuscated program. Such a measure can also be used as the base of the potency metric to estimate the capability of the obfuscated program against reverse engineering. Our novel framework helps efficiently examine a control-flow obfuscating transformation in a systematic manner and helps select an appropriate obfuscating transformation among a number of candidates to better protect a program. (C) 2012 Elsevier Ltd. All rights reserved.