Browsing by Author "Cheng, Wen-Huang"

Now showing 1 - 20 of 20

A Cloud-based Intelligent Skin and Scalp Analysis System
(2018-01-01)
Huang, Wen-Shiung
;
Hong, Bing-Kai
;
Cheng, Wen-Huang
;
Sun, Shih-Wei
;
Hua, Kai-Lung
The love of beauty is an essential part of all healthy human nature. Not only do women pay great attention to facial care, but in recent years, men's consumption in this area has also grown year by year. In facial care, in addition to sunscreen, skin care, tattoos and other chemical-based skin care products, mechanical skin washing machine has also become one of the most popular beauty appliances in recent years. In order to learn the effectiveness of various face care products and tools, in this paper, we have proposed an intelligent system that integrates face washing, wireless cameras, smart phones, and cloud image analysis functionality to allow users to obtain product recommendations. The proposed system is easy to carry and provides seven analysis functionalities, such as skin color, pigmentation, skin texture, wrinkles, pores, texture analysis, and would produce an in-depth analysis report. The proposed system is not only helpful for personal beauty care, but also beneficial for professional medical clinics and makeup companies. The collected big data will also be utilized as a reference for future cosmetics, and potential medical and biotechnology company product development and marketing.
ADAPTING SEMANTIC SEGMENTATION OF URBAN SCENES VIA MASK-AWARE GATED DISCRIMINATOR
(2019-01-01)
Lin, Yong-Xiang
;
Tan, Daniel Stanley
;
Cheng, Wen-Huang
;
Hua, Kai-Lung
;
電子工程學系及電子研究所
Training a deep neural network for semantic segmentation relies on pixel-level ground truth labels for supervision. However, collecting large datasets with pixel-level annotations is very expensive and time consuming. One workaround is to utilize synthetic data where we can generate potentially unlimited data with their corresponding ground truth labels. Unfortunately, networks trained on synthetic data perform poorly on real images due to the domain shift problem. Domain adaptation techniques have shown potential in transferring the knowledge learned from synthetic data to real world data. Prior works have mostly leveraged on adversarial training to perform a global aligning of features. However, we observed that background objects have lesser variations across different domains as opposed to foreground objects. Using this insight, we propose a method for domain adaptation that models and adapts foreground objects and background objects separately. Our approach starts with a fast style transfer to match the appearance of the inputs. This is followed by a foreground adaptation module that learns a foreground mask that is used by our gated discriminator in order to adapt the foreground and background objects separately. We demonstrate in our experiments that our model outperforms several state-of-the-art baselines in terms of mean intersection over union (mIoU).
AI plus Multimedia Make Better Life?
(2018-01-01)
Cheng, Wen-Huang
;
Liu, Jiaying
;
Kankanhalli, Mohan S.
;
El Saddik, Abdulmotaleb
;
Huet, Benoit
BeautyGlow: On-Demand Makeup Transfer Framework with Reversible Generative Network
(2019-01-01)
Chen, Hung-Jen
;
Hui, Ka-Ming
;
Wang, Szu-Yu
;
Tsao, Li-Wu
;
Shuai, Hong-Han
As makeup has been widely-adopted for beautification, finding suitable makeup by virtual makeup applications becomes popular. Therefore, a recent line of studies proposes to transfer the makeup from a given reference makeup image to the source non-makeup one. However, it is still challenging due to the massive number of makeup combinations. To facilitate on-demand makeup transfer, in this work, we propose BeautyGlow that decompose the latent vectors of face images derived from the Glow model into makeup and non makeup latent vectors. Since there is no paired dataset, we formulate a new loss function to guide the decomposition. Afterward, the non-makeup latent vector of a source image and makeup latent vector of a reference image and are effectively combined and revert back to the image domain to derive the results. Experimental results show that the transfer quality of BeautyGlow is comparable to the state-of-the-art methods, while the unique ability to manipulate latent vectors allows BeautyGlow to realize on-demand makeup transfer.
DRESSING FOR ATTENTION: OUTFIT BASED FASHION POPULARITY PREDICTION
(2019-01-01)
Lo, Ling
;
Liu, Chia-Lin
;
Lin, Rong-An
;
Wu, Bo
;
Shuai, Hong-Han
Accurate analysis of fashion trends is crucial. However, existing predictive algorithms of fashion popularity are restricted to be feasible on the coarse style level but not a finer item level. That is, they are only predictive in the future popularity of a given type of fashion styles (e.g., Rocker), but cannot be precisely down to a particular outfit look chosen by individuals. This paper thus proposes the first solution directly aimed at predicting the fine-grained fashion popularity of an outfit look by taking social media as the learning source. Particularly, a deep temporal sequence learning framework is developed and the proposed framework is evaluated on a real dataset of 380,000 street fashion images collected from the fashion website lookbook.nu. The experimental results show that our proposed framework outperforms the state-of-the-art approaches, with a relative increase of 11.51% to 27.62% (MSE metric) and 7.02% to 32.61% (CSE metric) in the prediction accuracy.
FashionOn: Semantic-guided Image-based Virtual Try-on with Detailed Human and Clothing Information
(2019-01-01)
Hsieh, Chia-Wei
;
Chen, Chieh-Yun
;
Chou, Chien-Lung
;
Shuai, Hong-Han
;
Liu, Jiaying
The image-based virtual try-on system has attracted a lot of research attention. The virtual try-on task is challenging since synthesizing try-on images involves the estimation of 3D transformation from 2D images, which is an ill-posed problem. Therefore, most of the previous virtual try-on systems cannot solve difficult cases, e.g., body occlusions, wrinkles of clothes, and details of the hair. Moreover, the existing systems require the users to upload the image for the target pose, which is not user-friendly. In this paper, we aim to resolve the above challenges by proposing a novel FashionOn network to synthesize user images fitting different clothes in arbitrary poses to provide comprehensive information about how suitable the clothes are. Specifically, given a user image, an in-shop clothing image, and a target pose (can be arbitrarily manipulated by joint points), FashionOn learns to synthesize the try-on images by three important stages: pose-guided parsing translation, segmentation region coloring, and salient region refinement. Extensive experiments demonstrate that FashionOn maintains the details of clothing information (e.g., logo, pleat, lace), as well as resolves the body occlusion problem, and thus achieves the state-of-the-art virtual try-on performance both qualitatively and quantitatively.
FIT-ME: IMAGE-BASED VIRTUAL TRY-ON WITH ARBITRARY POSES
(2019-01-01)
Hsieh, Chia-Wei
;
Chen, Chieh-Yun
;
Chou, Chien-Lung
;
Shuai, Hong-Han
;
Cheng, Wen-Huang
The image-based virtual try-on system has raised research attention recently, but it still requires to upload an image of a user with the target pose. We present a novel learning model, Fit-Me network, to seamlessly fit in-shop clothing into a person image and simultaneously transform the pose of the person image to another given one. The proposed Fit-Me network helps users not only save the time used to change clothes physically but also provide comprehensive information about how suitable the clothes are. By facilitating the arbitrary pose transformation, we can generate consecutive poses to help users get more information for deciding whether to buy the clothes or not from different aspects.
FUZZY PERSONALIZED SCORING MODEL FOR RECOMMENDATION SYSTEM
(2019-01-01)
Yang, Chao-Lung
;
Hsu, Shang-Che
;
Hua, Kai-Lung
;
Cheng, Wen-Huang
;
電子工程學系及電子研究所
In this research, we aim to propose a data preprocessing framework particularly for financial sector to generate the rating data as input to the collaborative system. First, clustering technique is applied to cluster all users based on their demographic information which might be able to differentiate the customers' background. Then, for each customer group, the importance of demographic characteristics which are highly associated with financial products purchasing are analyzed by the proposed fuzzy integral technique. The importance scores across items and customers are generated either on customer groups and individuals. The analysis shows the proposed method is able to differentiate customers based on their demographic and purchasing behaviors. Also, the generated rating matrix can be directly used for collaborative filtering model.
Garment Detectives: Discovering Clothes and Its Genre in Consumer Photos
(2019-01-01)
Hidayati, Shintami C.
;
Hua, Kai-Lung
;
Tsao, Yu
;
Shuai, Hong-Han
;
Liu, Jiaying
Clothing image analysis has shown its potential for use in a wide range of applications such as personalized clothing recommendation. Given a consumer photo, this paper addresses the problem of finding clothes and recognizing the genre of that clothes. This problem is very challenging due to large variations of uncontrolled realistic imaging conditions. To tackle these challenges, we formulate a novel framework by integrating local features of multi-modality as the instances of the price-collecting Steiner tree (PCST) problem to discover clothing regions, and exploiting visual style elements to discover the clothing genre. The experimental results show that our fully automatic approach is effective to identify irregular shape of clothing region, and it significantly improves the accuracy of clothing genre recognition for images taken in unconstrained environment.
LR3M: Robust Low-Light Enhancement via Low-Rank Regularized Retinex Model
(2020-01-01)
Ren, Xutong
;
Yang, Wenhan
;
Cheng, Wen-Huang
;
Liu, Jiaying
;
電子工程學系及電子研究所
Noise causes unpleasant visual effects in low-light image/video enhancement. In this paper, we aim to make the enhancement model and method aware of noise in the whole process. To deal with heavy noise which is not handled in previous methods, we introduce a robust low-light enhancement approach, aiming at well enhancing low-light images/videos and suppressing intensive noise jointly. Our method is based on the proposed Low-Rank Regularized Retinex Model (LR3M), which is the first to inject low-rank prior into a Retinex decomposition process to suppress noise in the reflectance map. Our method estimates a piece-wise smoothed illumination and a noise-suppressed reflectance sequentially, avoiding remaining noise in the illumination and reflectance maps which are usually presented in alternative decomposition methods. After getting the estimated illumination and reflectance, we adjust the illumination layer and generate our enhancement result. Furthermore, we apply our LR3M to video low-light enhancement. We consider inter-frame coherence of illumination maps and find similar patches through reflectance maps of successive frames to form the low-rank prior to make use of temporal correspondence. Our method performs well for a wide variety of images and videos, and achieves better quality both in enhancing and denoising, compared with the state-of-the-art methods.
Reversible AMBTC-Based Data Hiding With Security Improvement by Chaotic Encryption
(2019-01-01)
Wang, Hsiang-Ying
;
Lin, Hsin-Ju
;
Gao, Xiang-Yun
;
Cheng, Wen-Huang
;
Chen, Yung-Yao
Social networking and cloud computing are being extensively used, and in this era, the frequency of sending information or images to each other is increasing. The prevention of private information leakage during communication over the Internet has become a concern in the past decades. Several data protection methods, such as cryptographic, watermarking, and steganography techniques, have been proposed to protect private data. In this paper, an embedding method is proposed based on an absolute moment block truncation coding (AMBTC)-compressed image. High and low mean tables are extracted from a compressed image and are divided into non-overlapping blocks. An adaptive variable N-bit bit plane truncation image embedding method is proposed to embed the secret data in each block. In this method, at the receiver end, the secret data are extracted, and the original AMBTC image could be recovered by recalling the stored peak and zero points. In addition, a chaotic encryption scheme is integrated into the proposed system to improve robustness against security vulnerability. The results show that the proposed method has superior performance and higher payload compared with the reference methods.
SEGMENTING HEPATIC LESIONS USING RESIDUAL ATTENTION U-NET WITH AN ADAPTIVE WEIGHTED DICE LOSS
(2019-01-01)
Liu, Yu-Cheng
;
Tan, Daniel Stanley
;
Chen, Jyh-Cheng
;
Cheng, Wen-Huang
;
Hua, Kai-Lung
We propose a novel network architecture called Residual Attention U-Net (ResAttU-Net) for segmenting hepatic lesions. Our model incorporates residual blocks that can extract more complex features as compared with traditional convolutional layers combined with a skip-connection attention module that learns to focus on the relevant features for the task of hepatic lesions segmentation. Moreover, we train our model using an adaptive weighted dice loss that prioritizes the pixels of the tumor class over the pixels of the background class. We evaluate our model on the MICCAI Liver Tumor Segmentation (LiTS) benchmark dataset. Our experimental results show that our method significantly improves upon several state-of-the-art baselines for hepatic lesion or liver tumor segmentation.
SMP Challenge: An Overview of Social Media Prediction Challenge 2019
(2019-01-01)
Wu, Bo
;
Cheng, Wen-Huang
;
Liu, Peiye
;
Liu, Bei
;
Zeng, Zhaoyang
"SMP Challenge" aims to discover novel prediction tasks for numerous data on social multimedia and seek excellent research teams. Making predictions via social multimedia data (e.g. photos, videos or news) is not only helps us to make better strategic decisions for the future, but also explores advanced predictive learning and analytic methods on various problems and scenarios, such as multimedia recommendation, advertising system, fashion analysis etc. In the SMP Challenge at ACM Multimedia 2019, we introduce a novel prediction task Temporal Popularity Prediction, which focuses on predicting future interaction or attractiveness (in terms of clicks, views or likes etc.) of new online posts in social media feeds before uploading. We also collected and released a large-scale SMPD benchmark with over 480K posts from 69K users. In this paper, we define the challenge problem, give an overview of the dataset, present statistics of rich information for data and annotation and design the accuracy and correlation evaluation metrics for temporal popularity prediction to the challenge.
SPATIALLY-AWARE DOMAIN ADAPTATION FOR SEMANTIC SEGMENTATION OF URBAN SCENES
(2019-01-01)
Lin, Yong-Xiang
;
Tan, Daniel Stanley
;
Cheng, Wen-Huang
;
Chen, Yung-Yao
;
Hua, Kai-Lung
It is very expensive and time consuming to collect a large enough dataset with pixel-level annotations to train a semantic segmentation model. Synthetic datasets are common alternatives for training segmentation models, however models trained on synthetic data do not necessarily perform well on real world images due to the domain shift problem. Domain adaptation techniques address this problem by leveraging on adversarial training to align features. Prior works have mostly performed global feature alignment. They do not consider the positions of objects. However, objects in urban scenes are highly correlated with their spatial locations. For example, the sky will always appear on top while cars will usually appear in the middle of the image. Based on this insight, we propose a spatial-aware discriminator that accounts for the spatial prior on the objects in order to improve the feature alignment. We demonstrate in our experiments that our model outperforms several state-of-the-art baselines in terms of mean intersection over union (mIoU).
Switch Mode Based Deep Fractional Interpolation in Video Coding
(2019-01-01)
Xia, Sifeng
;
Yang, Wenhan
;
Hu, Yueyu
;
Cheng, Wen-Huang
;
Liu, Jiaying
Fractional interpolation is a significant technology in motion compensation of video coding. It generates sub-pixel level reference samples in inter prediction to facilitate temporal redundancy removal between video frames. Recently, some methods explore to introduce the deep learning technique for fractional interpolation and have obtained better compression results. However, existing deep learning based methods still treat fractional interpolation as a traditional interpolation problem but fail to adjust it to the motion compensation scenario. In this paper, we design a switch mode based deep fractional interpolation method to introduce integer pixels of different positions to the interpolation of sub-pixel position samples. By switching between integer pixels of different positions, our method can infer the sub-pixels with smaller variations and achieve better fractional interpolation results. Consequently the motion compensation performance can be further improved. Experimental results have also verified the efficiency of the switch mode based deep fractional interpolation. Compared with High Efficiency Video Coding, our method achieves 2.8% bit saving on average and up to 6.2% bit saving under low-delay P configuration.
Unlocking Author Power: On the Exploitation of Auxiliary Author-Retweeter Relations for Predicting Key Retweeters
(2020-03-01)
Wu, Bo
;
Cheng, Wen-Huang
;
Zhang, Yongdong
;
Cao, Juan
;
Li, Jintao
Retweeting is a powerful driving force in information propagation on microblogging sites. However, identifying the most effective retweeters of a message (called the "key retweeter prediction" problem) has become a significant research topic. Conventional approaches have addressed this topic from two main aspects: by analyzing either the personal attributes of microblogging users or the structures of user graph networks. However, according to sociological findings, author-retweeter dependencies also play a crucial role in influencing message propagation. In this paper, we propose a novel model to solve the key retweeter prediction problem by incorporating the auxiliary relations between a tweet author and potential retweeters. Without loss of generality, we formulate the relations from four relational factors: status relation, temporal relation, locational relation, and interactive relation. In addition, we propose a novel method, called "Relation-based Learning to Rank (RL2R)," to determine the key retweeters for a given tweet by ranking the potential retweeters in terms of their spreadability. The experimental results show that our method outperforms the state-of-the-art algorithms at top-k retweeter prediction, achieving a significant relative average improvement of 19.7-29.4 percent. These findings provide new insights for understanding user behaviors on social media for key retweeter prediction purposes.
Urban Multimedia Computing: Emerging Methods in Multimedia Computing for Urban Data Analysis and Applications Guest Editors' Introduction
(2020-07-01)
Min, Weiqing
;
Tian, Yonghong
;
Huang, Zi
;
Cheng, Wen-Huang
;
El Saddik, Abdulmotaleb
VEHICLE DETECTION IN THERMAL IMAGES USING DEEP NEURAL NETWORK
(2018-01-01)
Chang, Chin-Wei
;
Srinivasan, Kathiravan
;
Chen, Yung-Yao
;
Cheng, Wen-Huang
;
Hua, Kai-Lung
In today's world, it becomes critical for a self-driving car to detect the vehicles irrespective of it being a day or night. We propose a real-time vehicle detection using a sequence of night-time thermal images. Moreover, the thermal images have the capability of retaining even the minuscule vehicle details in a dim environment. For an efficient vehicle detection, the thermal image dataset collected during the dusk and night is used for training purposes. Subsequently, the contrast enhancement and sharpening of these images are performed using the Thermal Feature Enhancement (TFE). Then the concatenated images are supplied as the input to allow the model to learn more effectively. Besides, we also propose an improved convolution network model entitled as the Thermal Image Only Looked Once (TOLO) model for vehicle detection. Additionally, we propose a method called as Low Probability Candidate Filter (LPCF) to compensate the probability of not-easy-to-detect vehicles. Our proposed method produces better results for the F1-measure in comparison with existing methods.
What Dress Fits Me Best? Fashion Recommendation on the Clothing Style for Personal Body Shape
(2018-01-01)
Hidayati, Shintami Chusnul
;
Hsu, Cheng-Chun
;
Chang, Yu-Ting
;
Hua, Kai-Lung
;
Fu, Jianlong
Clothing is an integral part of life. Also, it is always an uneasy task for people to make decisions on what to wear. An essential style tip is to dress for the body shape, i.e., knowing one's own body shape (e.g., hourglass, rectangle, round and inverted triangle) and selecting the types of clothes that will accentuate the body's good features. In the literature, although various fashion recommendation systems for clothing items have been developed, none of them had explicitly taken the user's basic body shape into consideration. In this paper, therefore, we proposed a first framework for learning the compatibility of clothing styles and body shapes from social big data, with the goal to recommend a user about what to wear better in relation to his/her essential body attributes. The experimental results demonstrate the superiority of our proposed approach, leading to a new aspect for research into fashion recommendation.
ZipNet: ZFNet-level Accuracy with 48 x Fewer Parameters
(2018-01-01)
Antioquia, Arren Matthew C.
;
Tan, Daniel Stanley
;
Azcarraga, Arnulfo
;
Cheng, Wen-Huang
;
Hua, Kai-Lung
With the introduction of Convolutional Neural Networks, models for image classification achieve higher classification accuracy. Based on the pattern of the design of CNN architectures, increasing the number of layers equates to a higher classification accuracy, but also increases the number of parameters and model size. This negatively affects the model training time, processing time, and memory requirement. We develop ZipNet, a CNN architecture with a higher classification accuracy than ZFNet, the winner of ILSVRC 2013, but with 48.5x smaller model size and 48.7x fewer parameters. The classification accuracy of ZipNet is higher than the performance of ZFNet and SqueezeNet on all configurations of the Caltech-256 dataset with varying number of training examples.