嵌入式網路通訊裝置評比技術與工具之研發---總計畫(I)

Abstract

「嵌入式系統」與「自由軟體」是目前網路通訊產業的兩大底層基礎命脈，也是工研院交大 網路測試中心(NBL，由總計劃主持人 – 林盈達教授所成立)成立五年來所深耕的兩大重點。 從超過100 次測試服務的經驗中了解到廠商對於測試工具的需求，因此NBL 也開始在兩年 前投入了測試工具開發(主要透過自由軟體)，合作經驗包含與Cisco 開發Seucirty applicances 測試工具以及與聯發科技開發WiFi phone 的測試工具。不過，若只能針對整個系統進行黑箱 測試依舊無法滿足廠商對於釐清產品問題的需要，還必須提供灰/白箱測試、應用效能測試、 耗電測試、元件效能測試(ex.儲存裝置)、甚至是分析與診斷機制。 對於這些新的需求，我們以總計畫來補強目前NBL 熱門送測產品(i.e. WiFi phone, Wireless SIP gateway, Security appliances)所需要的黑箱測試工具、以子計畫一來完成應用效能評比技 術與工具、以子計畫二來完成耗電分析與評比工具、以子計畫三來完成核心系統與通訊行為 分析與評量工具、以子計畫四來完成儲存系統效能分析與評比工具。在計子畫一裡利用總計 畫所提供的測試設備與流量，來設計出各種不同的測試項目，藉以觀察並評估不同的環境條 件對系統效能的影響，在子計畫二與子計畫四選擇Power 與Flash作為測試的原因在於這兩 種元件是目前NBL 熱門送測產品中較重要的部分，而且這兩種元件目前較無合適的測試工 具。總計畫與四項子計畫之間關係緊密，總計畫提供一共通的測試平台產生traffic pattern 餵 進子計畫開發工具的環境裡，子計畫二與子計畫四需要共同開發power consumption 的模型， 四個子計畫間又必須合作研發kernel instrument 的技巧。 本總計畫為期兩年，延續本年度的成果，共通測試技術與平台可以提供不同層級的流量，例 如: packet level、connection level、session level，不同種類的流量，例如: SIP、RTP、HTTP、 FTP、SMTP、MSN、P2P，不同數量與速度的流量，不同的傳輸介面，例如: WLAN、Ethernet、 Analog，目前已經可以進行測試以及提供各子計畫產生背景流量，本年度已完成的規格訂定 及架構設計將可在第一年結束時完成第一個版本並交由NBL 應用，與服務廠商委託測，同 時在本年度也將利用公開測試活動以及年度road show，除了將與廠商、學界等研討相關的 測試方法，將本計畫所設計的測試工具進行推廣，同時也將程式公開原始碼，第二年結束時 將測試工具產品化、客製化，協助廠商將工具移轉到其內部使用。

Embedded systems and open source are two underlying technological driving forces in networking and communications industry nowadays. They are also two keys that Network Benchmarking Laboratory (NBL), founded by the PI of this proposal, Prof. Ying-Dar Lin, has been deeply involved in since the establishment in 2002. Realizing the testing requirements from over 100 testing projects, NBL has been involved in developing test tools (mostly from open source) since two years ago. The cooperation includes developing testing tools of security appliances with Cisco and tools of WiFi phones with MediaTek. However, only “black-box” testing on the end-systems. is insufficient to identify the product problems for most vendors, who also demand gray-box/white-box testing, user-experience performance testing, power consumption testing, component testing(ex. storage devices), and even the analysis and diagnosis mechanisms. It will be very helpful to vendors when we offer such a complete testing coverage and diagnosis skills. Four projects are ready to meet these new requirements. The grand project is to enhance the black-box testing for products often tested at NBL, including WiFi phone, Wireless SIP gateway and security appliances. The sub-project 1 is to develop technologies and tools for analyzing and benchmarking user-experience performance, the sub-project 2 is for analyzing and benchmarking power consumption, the sub-project 3 is for analyzing and benchmarking kernel systems and communication behaviors, and the sub-project 4 is for storage systems. Base on the equipment and network traffic provided by the grand project, the sub-project 1 is trying to design more advanced and userful test items.We choose power and Flash memory as component testing in the sub-project 2 and the sub-project 4 because they are components of products often tested at NBL and there are no suitable testing tools for them so far. The five projects are closely related. The grand project provides common testing environment to generate various types of traffic patterns into the environment of each sub-project. The sub-project 2 and the sub-project 4 shall work out a common power consumption model. All sub-projects shall work out the kernel instrument technologies. This proposal has three targets: development of tools, establishment of common testing environment, and deployment of tools. The testing tools to be developed are Wireless LAN Mobility Test System (WMTS), Detection Rate Automation (DRA), IPS Evasion, and In-Lab Live Testing (ILLT). The common testing environment provides network traffic in various levels (packet level, connection level, and session level), in various types (e.g., SIP, RTP, HTTP, FTP, SMTP, MSN, P2P), in various capacities and rates, and from various interfaces (e.g., WLAN, Ethernet, Analog). For the deployment of tools, there are three ways to do in NBL: private testing, public testing, and road show. At the end of the first year, we shall finish the first version of tools and hand over NBL for operations on real products. Besides, the source code will be released to the pulic through Open Foundry and the website of NBL. At the end of the final year, we shall turn our tools into product quality packages, and then customize and transfer them to vendors for internal use.