Development of Novel Lip-Reading Recognition Algorithm

Abstract

Total laryngectomy is a common treatment for patients with advanced laryngeal and hypopharyngeal cancer, but it is also a result from the loss of the natural voice and directly affects the basic communication functions in daily life. Reconstructing the basic communication function is an important issue for these patients after total laryngectomy surgery. Recently, the image processing technique for lip-reading recognition has been widely developed and applied in various kinds of applications. It is also one of the possibly alternative approaches to reconstructing the basic communication function for these patients after total laryngectomy surgery. Although many human lip-reading recognition methods have been developed to detect lip contour precisely, detecting pronouncing lip contour effectively is still a difficult challenge. In this paper, a novel lip-reading recognition algorithm was proposed to recognize English vowels from the lip contour when speaking. Here, several criteria for detecting the mouth region of interest (ROI) were designed to reduce the error rate of detecting the mouth ROI and lip contour. Moreover, several lip parameters, including the width, height, contour points, area, and the ratio (width/height) of lips, were used to recognize the lip contour and English vowels when speaking. The advantages of the proposed method are that it could detect the mouth ROI automatically, reduce the influence of individual differences, such as the individual lip shape or makeup effect, and it also could perform a good performance without pretraining. Finally, the performance of lip-reading recognition under different backgrounds and individual differences was also tested, and the accuracy of the proposed algorithm on lip-reading recognition was over 80%.