以視覺為基礎之桌上型多點觸控人機介面

Abstract

以視覺為基礎之下，我們提出一個可在投影畫面上操作的桌上型多點觸控人機介 面。此系統架構主要由一台架設於桌面正上方的投影機在桌面產生影像，並透過 一對可見光攝影機取像。考量此系統的實用性與可攜性，我們限制這對攝影機只 安裝於桌面與投影機之間。目前以視覺為基礎的各種多點觸控系統，大多都避免 環境光源的變化或操作中的影像改變所造成背景的變化。不同於過去的做法，我 們同時考慮環境光源的變化及投影畫面對背景與前景的干擾，因此在凌亂的動態 背景下，如何精準地決定觸控點位置並判定觸控點是否碰觸投影面，是我們主要 討論的課題。在觸控點的偵測上，我們專注於手指指尖的偵測，為了對抗凌亂的 動態背景及亮度變化，我們採用Histogram of Oriented Gradients(HOG)來偵測物體。此外為了提高觸控點位置判定的精準度，我們加入形狀模型(Shape Model)詳細地描述物體外型。最後為了決定觸控點是否碰觸投影面，我們藉由一個已知的平面透過Planar Homography 建立兩台攝影機之間的視差(parallax)，利用攝影機之間對於觸控點的視差來判定觸控點是否碰觸了投影面。

In this these, we present a vision-based system to enable multi-touch interactions on projected images using only a pair of visible-light cameras. The system is composed mainly of an overhead image projector and a pair of cameras installed between the projector and the image surface for practicability and portability. Due to the difficulty of object detection in cluttered images, most existing systems in related works have tried to avoid the influence of dynamic backgrounds. Unlike these works, we use visible-light cameras to detect objects in the projected images. How to determine whether the fingertips have touched the projection surface and how to determine precisely the positions of the touch points in cluttered backgrounds are the main topics of the proposed system. In our approach, we use Histogram of Oriented Gradients (HOG) features to detect fingers in dynamic backgrounds under illumination variations. Furthermore, we use a shape model that describes the detailed contour of fingers to provide precise positions of the touch points. Finally, by projecting the detected fingertips from each camera onto a common plane and then observing the difference between the projected points, we can decide whether the touch points have contacted the surface or not.