嵌入式網路通訊裝置評比技術與工具之研發---子計畫一：嵌入式網路通訊裝置應用效能評比技術與工具之研發(中心分項)(I)

Abstract

嵌入式系統平台已被大量應用在商用及家用產品上如PDA 和WiFi Phone，但是對於嵌入式系統網 路裝置在使用者端的應用效能上卻缺乏可用的測試工具，廠商在開發產品之後無法精確得到其效 能，當產品有關於效能方面的問題也無從得知可以改善的方式，為此，開發「嵌入式系統網路通訊 裝置應用效能評比工具」，除了要提供黑箱測試數據外，更應設計對於開發者可進行加強以及偵錯 的灰箱及白箱測試，協助開發者清楚了解產品在使用者端所遇到的缺點以及相關改進的方法。 本計畫欲開發的項目，是針對目前熱門的嵌入式系統網路通訊產品，進行相關網路應用服務效能評 比(如: Throughput, Latency, Session Capacity, Session Rate, Voice Quality)，利用NBL現有的工具以及 Open Source 軟體，以Repeatable、Scalable、Automatic、Integrated 等四個條件為前提，進行1. 測 試計畫及方法之研發, 2. 對測試設備無依賴性之測試工具研發, 3. 整合型測試工具研發，配合總計 畫提供的共通測試平台，提供各種不同層級的背景流量，更能貼近實際上使用者端使用的環境。 本計畫為期兩年，延續本年度的成果，針對相關的通訊協定：SSL VPN, SIP, RIP，針對相關的網路 介面：Ethernet, Wireless，針對相關的API對應的程式語言：Java, TCL等，配合現有的測試工具對 各項待測產品進行測試以及資料收集，並且已初步開發SSL VPN Tunnel Capacity及Integrated Voice Quality Testing 等兩項測試工具，預計將於第一年就研究的成果進行測試計畫以及測試工具軟體開 發，初步進行待測物測試(包含黑箱、灰箱及白箱測試)，同時驗證開發工具的正確性和可靠性，並 開放原始碼；第二年將測試項目及參數標準化，將整合測試自動化，針對各項產品進行測試並公佈 測試報告，並將測試工具產品化、客製化。

Embedded System has been used for business and home appliance widely. However, there is no congruent tool for benchmarking embedded network and communication devices for user-experience performance. Developers have no idea to improve products’ performance according to the feedback from users. So, it is necessary to develop tools and technologies to benchmark the embedded network and communication devices for user-experience performance. Besides the black-box testing technologies and tools, we should provide gray-box and white-box tests for developers to enhance and debug the devices. This proposal will focus on the popular embedded system network products to benchmark the related network application functions like Throughput, Latency, Session Capacity, Session Rate, Voice Quality. There are three objects of this subproject: 1. Research and Development of Test Plans, 2. Development of Test Utility Independent Test Tools and 3. Development Integrated Test Tools by using the available instruments and open source software on the understanding that repeatable, scalable, automatic and integrated. The grand project provides the common testing environment for us in generating background traffic to make the test network becomes real and trustworthy. In the plan of test utilities development, we target on three testing tools, which are「Real Application Throughput and Session Capacity」, 「SSL VPN Tunnel Capacity」, and 「Integrated Voice Quality」. For the first year, we will study the embedded system network devices and the related network protocols. At the same time, benchmark the applications, gather the report data and learn the APIs provided by the instruments, and design new test plans. After the first year, we will execute new designed test plans and start to develop our test tools. We will begin to do the basic black-box, gray-box and white-box test, by using the prototype of our new tools. And we have to validate and open it. Finally, we shall normalize the test items and parameters, automate the test plan, benchmark the products and open the test report, and then turn our tools into product quality packages in the second year.