Browsing by Author "Chung, Sheng-Hao"

Now showing 1 - 6 of 6

A Disk-Based Mining Algorithm for Frequent Pattern Discovery from Big Data in Distributed Computing Environments
(2016-11)
Lin, Kawuu W.
;
Chung, Sheng-Hao
;
Hsiao, Chun-Yuan
;
Lin, Chun-Cheng
;
Chen, Pei-Ling
In distributed computing environments, frequent pattern mining by a multi-computing node can greatly improve mining efficiency. However, the drawback of memory limitations may cause interruption in the kernel and computing nodes when recursively building a frequent pattern (FP) tree or an FP-growth algorithm. In this paper, we propose disk-based FP-tree generation and node-based clustering mechanisms to solve the insufficient memory problem. Results from empirical evaluations show that the proposed method delivers excellent scalability.
A Disk-Based Mining Algorithm for Frequent Pattern Discovery from Big Data in Distributed Computing Environments
(2016-11-01)
Lin, Kawuu W.
;
Chung, Sheng-Hao
;
Hsiao, Chun-Yuan
;
Lin, Chun-Cheng
;
Chen, Pei-Ling
In distributed computing environments, frequent pattern mining by a multi-computing node can greatly improve mining efficiency. However, the drawback of memory limitations may cause interruption in the kernel and computing nodes when recursively building a frequent pattern (FP) tree or an FP-growth algorithm. In this paper, we propose disk-based FP-tree generation and node-based clustering mechanisms to solve the insufficient memory problem. Results from empirical evaluations show that the proposed method delivers excellent scalability.
A Distributed Algorithm for Fast Mining Frequent Patterns in Limited and Varying Network Bandwidth Environments
(2019-05-01)
Lin, Chun-Cheng
;
Li, Wei-Ching
;
Chen, Ju-Chin
;
Chung, Wen-Yu
;
Chung, Sheng-Hao
Data mining is a set of methods used to mine hidden information from data. It mainly includes frequent pattern mining, sequential pattern mining, classification, and clustering. Frequent pattern mining is used to discover the correlation among various sets of items within large databases. The rapid upward trend in data size slows the mining of frequent patterns. Numerous studies have attempted to develop algorithms that operate in distributed computing environments to accelerate the mining process. FLR-mining (Fast, Load balancing and Resource efficient mining algorithm) is one of the fastest methods of mining with efficient consideration of load balancing and resources. FLR-mining can automatically determine the appropriate number of computing nodes. However, FLR-mining and existing methods assume that the network bandwidth is constant. In practical distributed and many-task computing systems, this assumption fails because there are packet collisions caused by many mining tasks that run in a simultaneous manner. Therefore, a method that can consider the varying network bandwidth is necessary. In this study, we propose a method that can rapidly mine frequent patterns under the varying network bandwidth. The proposed method can also determine the appropriate number of computing nodes to efficiently utilize computing resources and achieve load balancing. Through empirical evaluation, the proposed method is shown to deliver excellent performance in terms of execution efficiency and load balancing.
A fast and distributed algorithm for mining frequent patterns in congested networks
(2016-03)
Lin, Kawuu W.
;
Chung, Sheng-Hao
;
Lin, Chun-Cheng
;
工業工程與管理學系
;
Department of Industrial Engineering and Management
With advances in technology, frequent pattern mining has been used widely in our daily lives. By using this technology, one can obtain interesting or useful information that would help one make decisions and apply judgment. For example, marketplace managers mine transaction data to obtain information that can help improve services, understand customer buying habits, determine a suitable scheme for placement of goods to increase profits, or for medical and biotechnology applications. However, the rate at which data is generated is very rapid, leading to problems caused by Big Data. Therefore, many researchers have studied distributed, parallel and cloud computing technology to select the best among them. However, data mining uses multiple computing nodes, which requires the transmission of a considerable amount of data in a network environment. The available network bandwidth is limited when many different tasks are being transmitted at the same time and many servers are working in the same network segment. This results in poor transmission, causing severe transfer delay, either internal or external to the network. Thus, we propose the fast and distributed mining algorithm for discovering frequent patterns in congested networks (FDMCN) algorithm, which is based on CARM. The main purpose is to reduce FP-tree transmission such that only a portion of the information is required for mining using computing nodes. The results of empirical evaluation under various simulation conditions show that the proposed method FDMCN delivers excellent performance in terms of execution efficiency and scalability when compared with the PSWS algorithm.
A fast and low idle time method for mining frequent patterns in distributed and many-task computing environments
(2018-12-01)
Lin, Chun-Cheng
;
Chung, Sheng-Hao
;
Chen, Ju-Chin
;
Yu, Yuan-Tse
;
Lin, Kawuu W.
Association rules mining has attracted much attention among data mining topics because it has been successfully applied in various fields to find the association between purchased items by identifying frequent patterns (FPs). Currently, databases are huge, ranging in size from terabytes to petabytes. Although past studies can effectively discover FPs to deduce association rules, the execution efficiency is still a critical problem, particularly for big data. Progressive size working set (PSWS) and parallel FP-growth (PFP) are state-of-the-art methods that have been applied successfully to parallel and distributed computing technology to improve mining processing time in many-task computing, thereby bridging the gap between high-throughput and high-performance computing. However, such methods cannot mine before obtaining a complete FP-tree or the corresponding subdatabase, causing a high idle time for computing nodes. We propose a method that can begin mining when a small part of an FP-tree is received. The idle time of computing nodes can be reduced, and thus, the time required for mining can be reduced effectively. Through an empirical evaluation, the proposed method is shown to be faster than PSWS and PFP.
A fast method for frequent pattern discovery with secondary memory
(2017-01-01)
Lin, Kawuu W.
;
Chung, Sheng-Hao
;
Chen, Ju-Chin
;
Huang, Sheng-Shiung
;
Lin, Chun-Cheng
Data mining technology has been widely studied and applied in recent years. Frequent pattern mining is one important technical field of such research. The frequent pattern mining technique is popular not only in academia but also in the business community. With advances in technology, databases have become so large that data mining is impossible because of memory restrictions. In this study, we propose a novel algorithm for Fast mining with Secondary Memory, abbreviated as FSM-Mining, to help improve this situation. FSM-Mining saves a part of the information that is not stored in the memory, and through the use of mixed hard disk and memory mining we are able to complete data mining with limited memory. The results of empirical evaluation under various simulation conditions show that FSM-Mining delivers excellent performance in terms of execution efficiency and scalability.