應用於行動裝置並結合圖像目標物追蹤與手部追蹤之雲端互動式擴增實境系統

Abstract

近年來，擴增實境(Augmented Reality (AR))被視為有前景的技術。它能將相機拍攝到的現實景象與影片、圖像及三維(Three-Dimensional (3-D))物件等虛擬資訊結合，主要由樣式辨認來實現，其中包含特徵點辨識、特徵點描述及特徵點配對。隨著行動平台的快速發展，AR已被應用在行動裝置上，例如Android。互動式AR是一種人機互動(Human-Cmputer Interaction (HCI))的技術，它能讓使用者利用手與虛擬資訊互動。本論文提出了應用於行動裝置之雲端互動式擴增實境系統(cloud based Mobile Augmented Reality Interactive System (MARIS))，其中利用了基於特徵點之均值偏移演算法、基於手部模型之粒子濾波器及基於水平集之輪廓演進來實現圖像目標物追蹤(MARIS-I)與手部追蹤(MARIS-H)。MARIS-I能計算出圖像目標物之粗略位置。MARIS-H中包含指尖追蹤與手背追蹤兩種追蹤模式。由於MARIS-I與MARIS-H需要大量的運算資源，我們利用雲與端的架構來減輕裝置端運算複雜度、資料儲存及記憶體的需求。我們將MARIS示範在繪本中且能延伸並應用至數位學習系統或互動式應用中。

In recent years, augmented reality (AR) is considered to be a promising technology that combines virtual information such as videos, images, and three-dimensional (3-D) objects with a real camera view. AR is realized by pattern recognition, which is composed of feature detection, feature description, and feature points matching. With the rapid development of mobile device, AR can be implemented on mobile platforms such as Android. Interactive AR is a kind of human-computer interaction (HCI) which can let the user interact with virtual information on display by his hand. In this thesis, we proposed a cloud based Mobile Augmented Reality Interactive System (MARIS) that utilizes feature based mean-shift algorithm, hand model based particle filter algorithm, and level-set based contour evolution to realize image target tracking (MARIS-I) and hand tracking (MARIS-H). MARIS-I can estimate the image target's location. Two tracking modes such as fingertip tracking and back of hand tracking are included in MARIS-H. Because the combination of MARIS-I and MARIS-H needs large computing resources, we also exploit device/cloud architecture to decrease the requirement of computational complexity, data storage, and memory at device side. The proposed MARIS are demonstrated in a picture book to verify the application in digital learning system and can be further extended to other interactive applications.