運用聲音感測推論對話關係於智慧型手機網路

Abstract

近年來，智慧型手機為合作式計算與社交族群推論提供了一個前途大有可為的平台。智慧型手機上的麥克風是一項非常普及的感測器且被用於聲音感測。許多個人日常生活的資訊和活動都可以藉由分析所記錄的聲音資料來推論。在這篇文章中，我們考慮到一項非常重要的社交互動資訊，也就是對話夥伴們的推論。我們在智慧性手機上設計了一個有效率的對話推論系統，它可以即時且快速得推論手機使用者的對話夥伴們。考慮到對話片段的連續性與重疊性，我們提出了兩個全新的推論方法，用來辨別鄰近的說話者們之間的對話關係。透過無線通訊的介面，智慧型手機們可以互相的合作並且藉由分析聲音的資料來計算每個人說話的片段。對話的說話片段包括了使用者講話的時間點以及他/她講了多久。此外，為了改善我們推論的準確度，我們的系統也計算說話者們的情緒並且使用對話族群情緒的一致性來進行推論。為了評估我們的系統，我們從電影片段中以及現實生活中收集了二到九個說話者們的對話資料。結果顯示出我們的系統不論是在安靜的室內或者吵雜的室外都達到了前途大有可為的效能，除此之外，我們也將整個系統實作在安卓智慧型手機上來驗證我們的方法在目前裝置上運行的可行性。

Recently, smartphones provide a promising platform for cooperative computation and inference among social groups. Microphone is a common sensor on a smartphone that can be used for acoustic sensing. A lot of personal daily contexts and activities may be inferred by analyzing the recorded acoustic data. In this work, we consider the inference of conversation partners, which is an important context of social interaction. We design an efficient smartphone-based conversation inference system, which can quickly derive the conversation partners of a phone user in a real-time manner. By considering the continuity and overlap of speeches, we propose two novel inference methods to distinguish the conversational relationship among co-located speakers. Via direct wireless communications, smartphones cooperatively conduct speaker turn recognition by processing acoustic data. The speaker turn recognition of a conversation consists of when a user speaks and how long he/she speaks. Furthermore, to improve inference accuracy, our system also derives speakers’ emotion of utterances and uses the consistency of emotions of a conversation group to make inference. To evaluate our system, we collect conversation data from movie clips and real life with the number of speakers ranging from 2 to 9. The result shows that our system achieves promising performance in both quiet indoor and noisy outdoor environments. In addition, we have also demonstrated a prototype on Android smartphones to verify the feasibility of our approach from off-the-shelf devices.