Browsing by Author "Liu, Jiaying"

Now showing 1 - 5 of 5

AI plus Multimedia Make Better Life?
(2018-01-01)
Cheng, Wen-Huang
;
Liu, Jiaying
;
Kankanhalli, Mohan S.
;
El Saddik, Abdulmotaleb
;
Huet, Benoit
FashionOn: Semantic-guided Image-based Virtual Try-on with Detailed Human and Clothing Information
(2019-01-01)
Hsieh, Chia-Wei
;
Chen, Chieh-Yun
;
Chou, Chien-Lung
;
Shuai, Hong-Han
;
Liu, Jiaying
The image-based virtual try-on system has attracted a lot of research attention. The virtual try-on task is challenging since synthesizing try-on images involves the estimation of 3D transformation from 2D images, which is an ill-posed problem. Therefore, most of the previous virtual try-on systems cannot solve difficult cases, e.g., body occlusions, wrinkles of clothes, and details of the hair. Moreover, the existing systems require the users to upload the image for the target pose, which is not user-friendly. In this paper, we aim to resolve the above challenges by proposing a novel FashionOn network to synthesize user images fitting different clothes in arbitrary poses to provide comprehensive information about how suitable the clothes are. Specifically, given a user image, an in-shop clothing image, and a target pose (can be arbitrarily manipulated by joint points), FashionOn learns to synthesize the try-on images by three important stages: pose-guided parsing translation, segmentation region coloring, and salient region refinement. Extensive experiments demonstrate that FashionOn maintains the details of clothing information (e.g., logo, pleat, lace), as well as resolves the body occlusion problem, and thus achieves the state-of-the-art virtual try-on performance both qualitatively and quantitatively.
Garment Detectives: Discovering Clothes and Its Genre in Consumer Photos
(2019-01-01)
Hidayati, Shintami C.
;
Hua, Kai-Lung
;
Tsao, Yu
;
Shuai, Hong-Han
;
Liu, Jiaying
Clothing image analysis has shown its potential for use in a wide range of applications such as personalized clothing recommendation. Given a consumer photo, this paper addresses the problem of finding clothes and recognizing the genre of that clothes. This problem is very challenging due to large variations of uncontrolled realistic imaging conditions. To tackle these challenges, we formulate a novel framework by integrating local features of multi-modality as the instances of the price-collecting Steiner tree (PCST) problem to discover clothing regions, and exploiting visual style elements to discover the clothing genre. The experimental results show that our fully automatic approach is effective to identify irregular shape of clothing region, and it significantly improves the accuracy of clothing genre recognition for images taken in unconstrained environment.
LR3M: Robust Low-Light Enhancement via Low-Rank Regularized Retinex Model
(2020-01-01)
Ren, Xutong
;
Yang, Wenhan
;
Cheng, Wen-Huang
;
Liu, Jiaying
;
電子工程學系及電子研究所
Noise causes unpleasant visual effects in low-light image/video enhancement. In this paper, we aim to make the enhancement model and method aware of noise in the whole process. To deal with heavy noise which is not handled in previous methods, we introduce a robust low-light enhancement approach, aiming at well enhancing low-light images/videos and suppressing intensive noise jointly. Our method is based on the proposed Low-Rank Regularized Retinex Model (LR3M), which is the first to inject low-rank prior into a Retinex decomposition process to suppress noise in the reflectance map. Our method estimates a piece-wise smoothed illumination and a noise-suppressed reflectance sequentially, avoiding remaining noise in the illumination and reflectance maps which are usually presented in alternative decomposition methods. After getting the estimated illumination and reflectance, we adjust the illumination layer and generate our enhancement result. Furthermore, we apply our LR3M to video low-light enhancement. We consider inter-frame coherence of illumination maps and find similar patches through reflectance maps of successive frames to form the low-rank prior to make use of temporal correspondence. Our method performs well for a wide variety of images and videos, and achieves better quality both in enhancing and denoising, compared with the state-of-the-art methods.
Switch Mode Based Deep Fractional Interpolation in Video Coding
(2019-01-01)
Xia, Sifeng
;
Yang, Wenhan
;
Hu, Yueyu
;
Cheng, Wen-Huang
;
Liu, Jiaying
Fractional interpolation is a significant technology in motion compensation of video coding. It generates sub-pixel level reference samples in inter prediction to facilitate temporal redundancy removal between video frames. Recently, some methods explore to introduce the deep learning technique for fractional interpolation and have obtained better compression results. However, existing deep learning based methods still treat fractional interpolation as a traditional interpolation problem but fail to adjust it to the motion compensation scenario. In this paper, we design a switch mode based deep fractional interpolation method to introduce integer pixels of different positions to the interpolation of sub-pixel position samples. By switching between integer pixels of different positions, our method can infer the sub-pixels with smaller variations and achieve better fractional interpolation results. Consequently the motion compensation performance can be further improved. Experimental results have also verified the efficiency of the switch mode based deep fractional interpolation. Compared with High Efficiency Video Coding, our method achieves 2.8% bit saving on average and up to 6.2% bit saving under low-delay P configuration.