A fast method for frequent pattern discovery with secondary memory

Abstract

Data mining technology has been widely studied and applied in recent years. Frequent pattern mining is one important technical field of such research. The frequent pattern mining technique is popular not only in academia but also in the business community. With advances in technology, databases have become so large that data mining is impossible because of memory restrictions. In this study, we propose a novel algorithm for Fast mining with Secondary Memory, abbreviated as FSM-Mining, to help improve this situation. FSM-Mining saves a part of the information that is not stored in the memory, and through the use of mixed hard disk and memory mining we are able to complete data mining with limited memory. The results of empirical evaluation under various simulation conditions show that FSM-Mining delivers excellent performance in terms of execution efficiency and scalability.