Development of Big Data Multi-VM Platform for Rapid Prototyping of Distributed Deep Learning

檔案

本條目沒有附加檔案。

至出版者網站查看(DOI) ›

日期

2018-01-01

期刊名稱

BIG DATA - BIGDATA 2018

卷期與頁次

第 10968 卷,頁 182–193

ISSN

0302-9743

語言

英文

摘要

The present study utilizes VirtualBox virtual environment technology to develop the personal big data multi-VM platform with four-node Spark and Hadoop cluster that can effectively replicate and provide an environment for developers to easily design and implement the Spark and Hadoop Map/Reduce programming. Before running their Big Data and deep learning applications in physical multi-node Spark and Hadoop Cluster, developers can conduct Map/Reduce programing simply on the proposed multi-VM platform, which is exactly the same as the physical one. To demonstrate its capability and applicability, this study utilizes the deep learning application as an example for function illustration. In this study, the big data multi-VM platform provides the rapid prototyping of distributed deep learning by using a cutting-edge framework TensorFlowOnSpark (TFoS) for AI developers. To look into deep insight, this study performs the deep-learning benchmark in different types of cluster systems including the multi-node big data VM platform, physical standalone system and the physical small-cluster system. The results indicate that InputMode. SPARK can get 3.3 times faster than InputMode. TENSORFLOW on the big data VM platform and even achieve 6.1 times faster on the physical server.

所屬典藏

會議論文
顯示完整詮釋資料 統計資料

引用此條目

Wu, C.-H., Chuang, C.-N., Chang, W.-Y., & Tsai, W.-F. (2018). Development of Big Data Multi-VM Platform for Rapid Prototyping of Distributed Deep Learning. BIG DATA - BIGDATA 2018, 10968, 182–193. https://doi.org/10.1007/978-3-319-94301-5_14

BibTeX RIS

引用文字依 APA 第 7 版格式自動產生,使用前請再核對。

分享

永久網址:https://ir.lib.nycu.edu.tw/handle/11536/150723