嵌入式系統測試實驗室---基準評比指標設計---總計畫及子計畫一：嵌入式系統測試實驗室---耗時評比指標設計

Abstract

嵌入式系統測試實驗室: 基準評比指標設計--總計畫 在國科會嵌入式網路通訊裝置評比技術與工具之研發的計畫支持下，我們與 資策會合作於2011 年5 月成立資策會交大嵌入式測試實驗室(Embedded Benchmarking Lab, EBL)，並且發展出一系列測試方法及工具，將其整合成EBL Test Suite v1.0。本計畫延續這項努力，有別於過去兩年較專注在對待測物原始碼插 碼進行剖析(profiling)與最佳化(optimization)，本年度較專注在不需原始碼的 評比(benchmarking)或剖析，如此比較能被進行初期合作的廠商接受。 整體計畫目標是提供各種測試方法與工具徹底貫穿各層次的評比與剖析。總 計畫的目標是整合子計畫成果於共通測試平台，並負責開發嵌入式裝置自動化控 制技術。子計畫一及子計畫二主要提供效能及耗能剖析工具，以top-down 的方式 檢視所有軟體及系統效能及耗能。子計畫四及子計畫五也會從硬體面向以 bottom-up 的觀點來檢視檔案系統讀寫、記憶體存取之效能。而軟、硬體的效能最 終將反映在使用者操作之使用經驗品質(Quality of Experience, QoE)上，因此子計畫 三也會同步開發使用者介面評測服務。我們將整合子計畫對軟、硬體效能、耗能 剖析資訊做為最佳化參數，這些結果將有助於嵌入式系統提供CPU 排程、快取記 憶體配置、電源管理、I/O 排程以及快取緩衝演算法等控制，進而提升嵌入式系統 效能。 對於測試方案的推廣，在國際化方面 EBL 和嵌入式自由軟體聯盟(Open Embedded Software Foundation, OESF)的會員工業技術研究院(ITRI)共同合作， 針對Android 裝置制定測試方法及開發測試套件。在國內推廣方面，EBL 也和台灣 嵌入式產業聯盟(Taiwan Embedded Industry Alliance , TEIA)合作，建立Testing SIG，負責收集業界的各種測試需求，讓EBL 可以提供更多符合業界的測試方案來 協助嵌入式產業提升競爭力。結合NBL 的成功經驗，EBL 也會以三種模式進行推廣， 包含委託測試服務、公開測試活動以及年度road show。

Embedded Benchmarking Lab: Baseline Benchmark Design - Grand Project Under the support of the past NSC EBL project, with III we co-founded III-NCTU Embedded Benchmarking Lab (EBL) in May 2011. We consolidated a series of test methodologies and tools into EBL Test Suite v1.0. Compared to the past two years with more efforts on profiling and optimization with source codes of devices under test (DUTs), this project focuses more on source-code-free benchmarking and profiling, which might facilitate the initial cooperation with vendors. The overall objective is to provide benchmarking and profiling methodologies and tools to cover all layers. The grand project aims to integrate results from subprojects to form common testbeds, and automate the process of testing DUTs. Subprojects 1 and 2 develop benchmarking and profiling techniques on performance and power consumption, in a top-down manner. Subprojects 4 and 5 examine the performance of the underlying file system access and memory usage, in a bottom-up manner. As the end result of hardware/software performance would be reflected on the quality of experience (QoE) of user operations, subproject 3 develops benchmarking and profiling techniques of user interface (UI). We shall integrate the results of benchmarking and profiling from subprojects to help optimizing overall system performance by CPU scheduling, flash memory allocation, power management, I/O scheduling, and cache management. To promote the developed test solutions, we collaborate internationally with Open Embedded Software Foundation (OESF) and its member Industrial Technology Research Institute (ITRI) to define Android test suites. Locally we collaborate with Taiwan Embedded Industry Alliance (TEIA) to setup Testing SIG to collect the needs from the industry. Based on the experiences from NBL, EBL does market communications through private testing, public testing, and annual roadshow.