平行檔案系統的設計與製作

Abstract

由於近幾年來，處理器與記憶體效能進步的速度遠快於輸出入裝置效 能的改善， I/O子系統遂成為電腦系統的效能提昇的障礙。為了改 善這個問題，許多平行系統開始朝製作平行檔案系統的方向發展，以提昇 I/O子系統的效率。我們設計製作了一個可移植的平行檔案系統，這個檔 案系統架構在一個可移植的MPI環境上，並提供一個MPI-IO規格的使用者 程式庫。檔案以資料分割的方式分佈於多個資料伺服器上，並利用平行存 取來增加I/O效能。我們的平行檔案系統允許使用者彈性定義檔案的邏輯 分割，指定檔案分佈方式，讓多個處理程序同時平行存取一個檔案，並能 保持存取動作sequential consistency的語意。目前我們的檔案系統的製 作是以工作站群組為實驗平台，將多部Sun工作站以Ethernet連結，利用 MPI系統建立一個平行環境。 Processor speed and memory capacity have been rapidly increasing in computer systems, but the disk speed has not been improved at the same rate. Due to the limitation of mechanical operation. In order to improve the performance of I/O subsystem most recent researches focus on parallel disk schemes that use several independent disks in parallel to aggregate disk bandwidth. Parallelfile system exploit I/O performance by using parallel diskscheme as well as provide parallel interface for users. This paper presents a MPI-IO-based parallel file system(MPFS).MPFS is base on a MPI, Message Passing Interface standandard, parallel environment.Due to the lack of MPI in I/O interface, we use MPI- IO, a parallel file I/O interface for MPI,as the user interface.MPFS provides users to define physical and logical partition of files. Files are striped over multiple dataservers. User can specify the physical partition of files where and how MPFS should putthe file data.MPFS uses MPI datatype to implement dynamic user-defined logical partition that has more flexibility and expressiveness withno moving the physical position of file.Many processes can access a file inparallel which is stored over multiple data servers. MPFS provides sequential consistency for independent access operations. We implement MPFS in a group of workstations that are connected with Ethernet. Due to the portability of MPFS, We can port MPFS to high speed network machines or MP in future.