Realizing the Real-time Gaze Redirection System with Convolutional Neural Network

檔案

本條目沒有附加檔案。

至出版者網站查看(DOI) ›

日期

2018-01-01

期刊名稱

PROCEEDINGS OF THE 9TH ACM MULTIMEDIA SYSTEMS CONFERENCE (MMSYS'18)

卷期與頁次

頁 509–512

語言

英文

摘要

Retaining eye contact of remote users is a critical issue in video conferencing systems because of parallax caused by the physical distance between a screen and a camera. To achieve this objective, we present a real-time gaze redirection system called Flx-gaze to post-process each video frame before sending it to the remote end. Specifically, we relocate and relight the pixels representing eyes by using a convolutional neural network (CNN). To prevent visual artifacts during manipulation, we minimize not only the L2 loss function but also four novel loss functions when training the network. Two of them retain the rigidity of eyeballs and eyelids; and the other two prevent color discontinuity on the eye peripheries. By leveraging the CPU and the GPU resources, our implementation achieves real-time performance (i.e., 31 frames per second). Experimental results show that the gazes redirected by our system are of high quality under this restrict time constraint. We also conducted an objective evaluation of our system by measuring the peak signal-to-noise ratio (PSNR) between the real and the synthesized images.

所屬典藏

會議論文
顯示完整詮釋資料 統計資料

引用此條目

Hsu, C.-F., Chen, Y.-C., Wang, Y.-S., Lei, C.-L., & Chen, K.-T. (2018). Realizing the Real-time Gaze Redirection System with Convolutional Neural Network. PROCEEDINGS OF THE 9TH ACM MULTIMEDIA SYSTEMS CONFERENCE (MMSYS'18), 509–512. https://doi.org/10.1145/3204949.3209618

BibTeX RIS

引用文字依 APA 第 7 版格式自動產生,使用前請再核對。

分享

永久網址:https://ir.lib.nycu.edu.tw/handle/11536/150966