以擴增實境與影像分析技術作書庫尋書與其應用

Abstract

每當人們走進一個圖書相關的環境，例如: 書店、圖書館或是書庫等等，人們可能會迷失了方向或是不知道想找的書的所在位置。一般來說，他們會選擇去詢問工作人員來指引方向。而本研究提出了一個應用於圖書相關環境的室內導覽系統，該系統結合了電腦視覺和擴增實境的技術，並採用主從式系統和行動裝置，提供更直覺的擴增實境介面，供使用者使用。系統主要有兩個部分: 在圖書相關的環境中找書以及顯示書的資訊，兩者皆透過擴增實境的方式呈現。 首先，在找書方面，我們會先在環境中創造出許多「充滿信號的藝術編碼影像」，這些影像裡都藏有相對於整體環境的位置資訊。透過手機來拍攝這些影像，傳送至伺服端，伺服端會解析這張影像並抽取出藏在影像當中的位置資訊，視為使用者目前所在的位置。而當使用者要找書的時候，系統會依照書名找出書的所在位置並將此訂為目的地，接著，伺服端會幫使用者規劃出一條最短路徑。之後，使用者所持的手機會顯示擴增實境式的導引資訊，清楚的告訴使用者下一個影像的位置，一步步導引使用者直到到達目的地的書架前。 關於顯示圖書資訊方面，當使用者到達目的地的書架前之後，使用者對著書架上的所有書拍張照片，我們必須找出使用者所要的書在書架上的位置，為此我們將使用書背影像辨識。為了完成這個目標，我們得事先建立一個書背圖資料庫。首先，我們會使用影像處理的方式來切割出每一本書背圖，接著，利用OCR (Optical character recognition)的技術來連結書背圖資料庫與原有的書名資料庫。等資料庫創建好後，本研究採用加速型穩健特徵(speeded-up robust feature, SURF)比對演算法，此方法是利用伺服器端事先建立的書背圖資料庫與書架前所拍的照片進行影像比對及辨識。找出指定的書之後，系統會將書所有有關的資訊以擴增實境的方式呈現在使用者的行動裝置上。 上述方法的實驗結果良好，顯示出本研究所提系統與方法確實可行。

When people visit a book-related space, such as a bookstore, a library, or a stack room, they might get lost or have no idea about how to reach a desired location or a book. In this study, an indoor guidance system based on augmented reality (AR) and image analysis techniques for the application of book search in a book-related space by the use of a mobile device is proposed. The system has two main functions: book search in the book-related space and book information introduction, both being based on AR techniques. At first, for book search, in order to guide a user to walk along a planned path, a set of signal-rich-art code images is created and attached at proper locations in the book-related space for use as landmarks in the guidance process. In each code image, the position information of a prominent spot or corner in the environment is embedded. The proposed system can extract the information embedded in each signal-rich-art code image from a captured version of the code image obtained with a mobile-device camera, and analyze the user’s location at each visited spot using the extracted information. When the user wants to search a book, the system finds out by text search the book shelf on which the book is located, takes it as a destination, and plans a path from the user’s location to the destination. Then, the system generates AR arrows and displays them on the user’s mobile-device screen to guide the user to visit the path nodes one by one. Specifically, at each node, the user takes an image of the landmark for the system to locate the user and generate accordingly the next arrow for the user to follow until reaching the destination which is a bookcase. About AR-based book information introduction, when the user reaches the destination book shelf where the searched book is located, the user takes an image of the book shelf on which, in addition to the desired book, there are others. Recognition of the desired book in the acquired image is thus required. For this, book-spine image recognition is adopted. In order to accomplish this goal, a book-spine image database is constructed in advance. At first, image processing techniques are adopted to segment book-spine images. Then, a knowledge-based OCR technique is proposed to construct a book-spine image database which is augmented with book titles. With the database ready, a method is proposed to recognize the book spine in the book shelf image by SURF matching against the book-spine image database. Finally, for AR-based book information introduction, after the book spine of the searched book is recognized, the book information is overlaid onto the book-shelf image shown on the mobile device for the user to inspect. Good experimental results are also included to show the feasibility of the proposed system and methods. Precision measures and statistics are shown as well to indicate the system’s effectiveness in handling real conditions in book search applications.