Development of Big Data Multi-VM Platform for Rapid Prototyping of Distributed Deep Learning
檔案
本條目沒有附加檔案。
日期
2018-01-01
類型
系所
期刊名稱
BIG DATA - BIGDATA 2018
卷期與頁次
第 10968 卷,頁 182–193
ISSN
0302-9743
語言
英文
摘要
The present study utilizes VirtualBox virtual environment technology to develop the personal big data multi-VM platform with four-node Spark and Hadoop cluster that can effectively replicate and provide an environment for developers to easily design and implement the Spark and Hadoop Map/Reduce programming. Before running their Big Data and deep learning applications in physical multi-node Spark and Hadoop Cluster, developers can conduct Map/Reduce programing simply on the proposed multi-VM platform, which is exactly the same as the physical one. To demonstrate its capability and applicability, this study utilizes the deep learning application as an example for function illustration. In this study, the big data multi-VM platform provides the rapid prototyping of distributed deep learning by using a cutting-edge framework TensorFlowOnSpark (TFoS) for AI developers. To look into deep insight, this study performs the deep-learning benchmark in different types of cluster systems including the multi-node big data VM platform, physical standalone system and the physical small-cluster system. The results indicate that InputMode. SPARK can get 3.3 times faster than InputMode. TENSORFLOW on the big data VM platform and even achieve 6.1 times faster on the physical server.
所屬典藏
會議論文引用此條目
Wu, C.-H., Chuang, C.-N., Chang, W.-Y., & Tsai, W.-F. (2018). Development of Big Data Multi-VM Platform for Rapid Prototyping of Distributed Deep Learning. BIG DATA - BIGDATA 2018, 10968, 182–193. https://doi.org/10.1007/978-3-319-94301-5_14
引用文字依 APA 第 7 版格式自動產生,使用前請再核對。
分享
永久網址:https://ir.lib.nycu.edu.tw/handle/11536/150723