嵌入式網路通訊裝置評比技術與工具之研發---總計畫

Abstract

「嵌入式系統」與「自由軟體」是目前網路通訊產業的兩大底層基礎命脈，也是工研院交大網路測試中心(NBL，由總計劃主持人 – 林盈達教授所成立)成立五年來所深耕的兩大重點。從超過100次測試服務的經驗中了解到廠商對於測試工具的需求，因此NBL也開始在兩年前投入了測試工具開發(主要透過自由軟體)，合作經驗包含與Cisco開發Seucirty applicances測試工具以及與聯發科技開發WiFi phone的測試工具。不過，若只能針對整個系統進行黑箱測試依舊無法滿足廠商對於釐清產品問題的需要，還必須提供灰/白箱測試、應用效能測試、耗電測試、元件效能測試(ex.儲存裝置)、甚至是分析與診斷機制。 對於這些新的需求，我們以總計畫來補強目前NBL熱門送測產品(i.e. WiFi phone, Wireless SIP gateway, Security appliances)所需要的黑箱測試工具、以子計畫一來完成應用效能評比技術與工具、以子計畫二來完成耗電分析與評比工具、以子計畫三來完成核心系統與通訊行為分析與評量工具、以子計畫四來完成儲存系統效能分析與評比工具。在計子畫一裡利用總計畫所提供的測試設備與流量，來設計出各種不同的測試項目，藉以觀察並評估不同的環境條件對系統效能的影響，在子計畫二與子計畫四選擇Power 與Flash作為測試的原因在於這兩種元件是目前NBL熱門送測產品中較重要的部分，而且這兩種元件目前較無合適的測試工具。總計畫與四項子計畫之間關係緊密，總計畫提供一共通的測試平台產生traffic pattern餵進子計畫開發工具的環境裡，子計畫二與子計畫四需要共同開發power consumption的模型，四個子計畫間又必須合作研發kernel instrument的技巧。 總計畫的目標有三項: 發展測試工具、建立共通測試技術與平台、推廣工具，要發展的測試工具包括: Wireless LAN Mobility Test System、Detection Rate Automation、IPS Evasion以及In-Lab Live Testing，共通測試技術與平台可提供不同層級的流量，例如: packet level、connection level、session level，不同種類的流量，例如: SIP、RTP、HTTP、FTP、SMTP、MSN、P2P，不同數量與速度的流量，不同的傳輸介面，例如: WLAN、Ethernet、Analog，工具推廣的部分會有三類型的方法，其中包含了委託測試服務、公開測試活動以及年度road show。預計第一年結束時完成測試工具的規格訂定、架構設計以及實驗結果、第二年結束時完成測試工具的第一個版本並且交由NBL應用，同時也將程式開放源碼、第三年結束時將測試工具產品化、客製化，協助廠商將工具移轉到其內部使用。

Embedded systems and open source are two underlying technological driving forces in networking and communications industry nowadays. They are also two keys that Network Benchmarking Laboratory (NBL), founded by the PI of this proposal, Prof. Ying-Dar Lin, has been deeply involved in since the establishment in 2002. Realizing the testing requirements from over 100 testing projects, NBL has been involved in developing test tools (mostly from open source) since two years ago. The cooperation includes developing testing tools of security appliances with Cisco and tools of WiFi phones with MediaTek. However, only “black-box” testing on the end-systems. is insufficient to identify the product problems for most vendors, who also demand gray-box/white-box testing, user-experience performance testing, power consumption testing, component testing(ex. storage devices), and even the analysis and diagnosis mechanisms. It will be very helpful to vendors when we offer such a complete testing coverage and diagnosis skills. Four projects are ready to meet these new requirements. The grand project is to enhance the black-box testing for products often tested at NBL, including WiFi phone, Wireless SIP gateway and security appliances. The sub-project 1 is to develop technologies and tools for analyzing and benchmarking user-experience performance, the sub-project 2 is for analyzing and benchmarking power consumption, the sub-project 3 is for analyzing and benchmarking kernel systems and communication behaviors, and the sub-project 4 is for storage systems. Base on the equipment and network traffic provided by the grand project, the sub-project 1 is trying to design more advanced and userful test items.We choose power and Flash memory as component testing in the sub-project 2 and the sub-project 4 because they are components of products often tested at NBL and there are no suitable testing tools for them so far. The five projects are closely related. The grand project provides common testing environment to generate various types of traffic patterns into the environment of each sub-project. The sub-project 2 and the sub-project 4 shall work out a common power consumption model. All sub-projects shall work out the kernel instrument technologies. This proposal has three targets: development of tools, establishment of common testing environment, and deployment of tools. The testing tools to be developed are Wireless LAN Mobility Test System (WMTS), Detection Rate Automation (DRA), IPS Evasion, and In-Lab Live Testing (ILLT). The common testing environment provides network traffic in various levels (packet level, connection level, and session level), in various types (e.g., SIP, RTP, HTTP, FTP, SMTP, MSN, P2P), in various capacities and rates, and from various interfaces (e.g., WLAN, Ethernet, Analog). For the deployment of tools, there are three ways to do in NBL: private testing, public testing, and road show. At the end of the first year, we shall finish tool specification, architecture design and experimental results. At the end of the second year, we shall finish the first version of tools and hand over NBL for operations on real products. Besides, the source code will be released to the pulic through Open Foundry and the website of NBL. At the end of the final year, we shall turn our tools into product quality packages, and then customize and transfer them to vendors for internal use.