MPFS平行檔案系統的製作

Abstract

在計算能力與I/O速度差距更大的平行系統中﹐I/O子系統成為電腦系統的 效能提昇主要的障礙。為了提高I/O子系統的效能﹐許多平行系統開始朝 製作平行檔案系統的方向發展﹐在平行檔案系統中﹐檔案以資料分割的方 式分佈於多個資料伺服器上﹐使用者程式可以利用平行存取來提高I/O頻 寬﹐增加I/O效能﹐但是現有的平行檔案系統大多是在專屬的機器平台上 製作。為了解決平行環境下發展的程式移植性不佳､開發成本高的問題﹐ 四十幾個單位組織聯合制訂了MPI(Message Passing Interface)標準﹐我 們製作的MPFS平行檔案系統﹐架構在可移植性高的MPI環境上﹐根據MPI- IO規格製作使用者程式庫﹐提供程式設計者使用容易且高效能的存取函式 。MPFS除了有高度的移植性之外﹐系統並允許使用者彈性定義檔案的邏輯 分割﹐指定檔案分佈方式與其他系統功能的設定﹐可以讓使用者依照應用 程式的行為調整出最佳的系統設定。目前MPFS檔案系統的製作以工作站群 組為實驗平台﹐以100Mbit/s Fast Ethernet相連結﹐已經可以移植到 FreeBSD及SunOS的作業系統上。 The performance of CPU and memory system has been increasing dramatically in recent years, while I/O sub-systems have not been improving at the same rate and becomes the performance bottleneck of modern computer systems specially in parallel computing systems. One way to improve I/O system bandwidth is to stripe file data across several disks or data servers so that file access can be carried out in a parallel fashion. Moreover, a parallel file interface is needed so that a programmer can effectively exploit the parallel I/O bandwidth in multi- processor systems. Most of the existing parallel file systems were designed for dedicated platform with low portability. This thesis presents the design of MPFS, a MPI-based parallel file system. MPI-IO is an extension of MPI to support a flexible user-defined logical file partition and a rich set of file access functions. Our MPFS implementation includes full supports of a high-level interface to describe the partitioning of file data among processes, a collective interface describing data transfers between process and file, asynchronous I/O operations allowing computation to be overlapped with I/O, shared file pointer among a group of processes, and limited support of hints to specify physical data layout across data servers. MPFS is now built on a workstation cluster connected by 100Mb/s fast Ethernet with FreeBSD operating system and the implementation can be ported to SunOS. Performance measurements of 64 MB file access are also performed on our system. The measurements show that the network bandwidth is the system bottleneck when more than one data server are utilized.