Browsing by Author "Wang, Yu-Shuen"

Now showing 1 - 20 of 24

Adversarial Colorization Of Icons Based On Structure And Color Conditions
(2019-01-01)
Sun, Tsai-Ho
;
Lai, Chien-Hsun
;
Wong, Sai-Keung
;
Wang, Yu-Shuen
;
交大名義發表
We present a system to help designers create icons that are widely used in banners, signboards, billboards, homepages, and mobile apps. Designers are tasked with drawing contours, whereas our system colorizes contours in different styles. This goal is achieved by training a dual conditional generative adversarial network (GAN) on our collected icon dataset. One condition requires the generated image and the drawn contour to possess a similar contour, while the other anticipates the image and the referenced icon to be similar in color style. Accordingly, the generator takes a contour image and a man-made icon image to colorize the contour, and then the discriminators determine whether the result fulfills the two conditions. The trained network is able to colorize icons demanded by designers and greatly reduces their workload. For the evaluation, we compared our dual conditional GAN to several state-of-the-art techniques. Experiment results demonstrate that our network is over the previous networks. Finally, we will provide the source code, icon dataset, and trained network for public use.
ADVERSARIAL GENERATION OF DEFENSIVE TRAJECTORIES IN BASKETBALL GAMES
(2018-01-01)
Chen, Chieh-Yu
;
Lai, Wenze
;
Hsieh, Hsin-Ying
;
Wang, Yu-Shuen
;
Peng, Wen-Hsiao
In this paper, we present a method to generate realistic trajectories of defensive players in a basketball game based on the ball and the offensive team's movements. We train on the NBA dataset a conditional generative adversarial network that learns spatio-temporal interactions between players' movements. The network consists of two components: a generator that takes as input a latent noise vector and the offensive team's trajectories to generate defensive trajectories; and a discriminator that evaluates the realistic degree of the generated results. Our system allows players and coaches to simulate how the opposing team will react to a newly developed offensive strategy for evaluating its effectiveness. Experimental results demonstrate the feasibility of the proposed algorithm.
BasketballGAN: Generating Basketball Play Simulation Through Sketching
(2019-01-01)
Hsieh, Hsin-Ying
;
Chen, Chieh-Yu
;
Wang, Yu-Shuen
;
Chuang, Jung-Hong
;
交大名義發表
We present a data-driven basketball set play simulation. Given an offensive set play sketch, our method simulates potential scenarios that may occur in the game. The simulation provides coaches and players with insights on how a given set play can be executed. To achieve the goal, we train a conditional adversarial network on NBA movement data to imitate the behaviors of how players move around the court through two major components: a generator that learns to generate natural player movements based on a latent noise and a user sketched set play; and a discriminator that is used to evaluate the realism of the basketball play. To improve the quality of simulation, we minimize 1.) a dribbler loss to prevent the ball from drifting away from the dribbler; 2.) a defender loss to prevent the dribbler from not being defended; 3.) a ball passing loss to ensure the straightness of passing trajectories; and 4) an acceleration loss to minimize unnecessary players' movements. To evaluate our system, we objectively compared real and simulated basketball set plays. Besides, a subjective test was conducted to judge whether a set play was real or generated by our network. On average, the mean correct rates to the binary tests were 56.17 %. Experiment results and the evaluations demonstrated the effectiveness of our system.
Chess Evolution Visualization
(2014-05-01)
Lu, Wei-Li
;
Wang, Yu-Shuen
;
Lin, Wen-Chieh
;
資訊工程學系
;
Department of Computer Science
We present a chess visualization to convey the changes in a game over successive generations. It contains a score chart, an evolution graph and a chess board, such that users can understand a game from global to local viewpoints. Unlike current graphical chess tools, which focus only on highlighting pieces that are under attack and require sequential investigation, our visualization shows potential outcomes after a piece is moved and indicates how much tactical advantage the player can have over the opponent. Users can first glance at the score chart to roughly obtain the growth and decline of advantages from both sides, and then examine the position relations and the piece placements, to know how the pieces are controlled and how the strategy works. To achieve this visualization, we compute the decision tree using artificial intelligence to analyze a game, in which each node represents a chess position and each edge connects two positions that are one-move different. We then merge nodes representing the same chess position, and shorten branches where nodes on them contain only two neighbors, in order to achieve readability. During the graph rendering, the nodes containing events such as draws, effective checks and checkmates, are highlighted because they show how a game is ended. As a result, our visualization helps players understand a chess game so that they can efficiently learn strategies and tactics. The presented results, evaluations, and the conducted user studies demonstrate the feasibility of our visualization design.
Content Aware Texture Compression
(2015-11-01)
Chiu, Yu-Cheng
;
Wang, Yu-Shuen
;
資訊工程學系
;
Department of Computer Science
Efficient and high quality texture compression is important because computational powers are always limited, especially for embedded systems such as mobile phones. To access random texels instantly, down sampling and S3TC are the most commonly used methods due to the fixed compression ratio at every local region. However, the methods are content oblivious, which uniformly discard texel information. Therefore, we present content aware warp to reduce a texture resolution, where homogeneous regions are squeezed more to retain more structural information. We then relocate texture coordinates of the 3D model as the way of relocating texels, such that rendering the model with our compressed texture requires no additional decompression cost. Our texture compression technique can cooperate with existing methods such as S3TC since the compression strategies are independent. By reducing texture resolution in a content aware manner, followed by texture compression using S3TC, as the experiments show, we achieve higher compression ratios without degrading visual quality significantly.
Court Reconstruction for Camera Calibration in Broadcast Basketball Videos
(2016-05)
Wen, Pei-Chih
;
Cheng, Wei-Chih
;
Wang, Yu-Shuen
;
Chu, Hung-Kuo
;
Tang, Nick C.
We introduce a technique of calibrating camera motions in basketball videos. Our method particularly transforms player positions to standard basketball court coordinates and enables applications such as tactical analysis and semantic basketball video retrieval. To achieve a robust calibration, we reconstruct the panoramic basketball court from a video, followed by warping the panoramic court to a standard one. As opposed to previous approaches, which individually detect the court lines and corners of each video frame, our technique considers all video frames simultaneously to achieve calibration; hence, it is robust to illumination changes and player occlusions. To demonstrate the feasibility of our technique, we present a stroke-based system that allows users to retrieve basketball videos. Our system tracks player trajectories from broadcast basketball videos. It then rectifies the trajectories to a standard basketball court by using our camera calibration method. Consequently, users can apply stroke queries to indicate how the players move in gameplay during retrieval. The main advantage of this interface is an explicit query of basketball videos so that unwanted outcomes can be prevented. We show the results in Figs. 1, 7, 9, 10 and our accompanying video to exhibit the feasibility of our technique.
Enhancing the Realism of Sketch and Painted Portraits With Adaptable Patches
(2018-02-01)
Lee, Yin-Hsuan
;
Chang, Yu-Kai
;
Chang, Yu-Lun
;
Lin, I-Chen
;
Wang, Yu-Shuen
Realizing unrealistic faces is a complicated task that requires a rich imagination and comprehension of facial structures. When face matching, warping or stitching techniques are applied, existing methods are generally incapable of capturing detailed personal characteristics, are disturbed by block boundary artefacts, or require painting-photo pairs for training. This paper presents a data-driven framework to enhance the realism of sketch and portrait paintings based only on photo samples. It retrieves the optimal patches of adaptable shapes and numbers according to the content of the input portrait and collected photos. These patches are then seamlessly stitched by chromatic gain and offset compensation and multi-level blending. Experiments and user evaluations show that the proposed method is able to generate realistic and novel results for a moderately sized photo collection.
Focus plus Context Metro Maps
(2011-12-01)
Wang, Yu-Shuen
;
Chi, Ming-Te
;
交大名義發表
;
National Chiao Tung University
We introduce a focus+context method to visualize a complicated metro map of a modern city on a small displaying area. The context of our work is with regard the popularity of mobile devices. The best route to the destination, which can be obtained from the arrival time of trains, is highlighted. The stations on the route enjoy larger spaces, whereas the other stations are rendered smaller and closer to fit the whole map into a screen. To simplify the navigation and route planning for visitors, we formulate various map characteristics such as octilinear transportation lines and regular station distances into energy terms. We then solve for the optimal layout in a least squares sense. In addition, we label the names of stations that are on the route of a passenger according to human preferences, occlusions, and consistencies of label positions using the graph cuts method. Our system achieves real-time performance by being able to report instant information because of the carefully designed energy terms. We apply our method to layout a number of metro maps and show the results and timing statistics to demonstrate the feasibility of our technique.
Generating Defensive Plays in Basketball Games
(2018-01-01)
Chen, Chieh-Yu
;
Lai, Wenze
;
Hsieh, Hsin-Ying
;
Zheng, Wen-Hao
;
Wang, Yu-Shuen
In this paper, we present a method to generate realistic defensive plays in a basketball game based on the ball and the offensive team's movements. Our system allows players and coaches to simulate how the opposing team will react to a newly developed offensive strategy for evaluating its effectiveness. To achieve the aim, we train on the NBA dataset a conditional generative adversarial network that learns spatio-temporal interactions between players' movements. The network consists of two components: a generator that takes a latent noise vector and the offensive team's trajectories as input to generate defensive team's trajectories; and a discriminator that evaluates the realistic degree of the generated results. Since a basketball game can be easily identified as fake if the ball handler, who is not defended, does not shoot the ball or cut into the restricted area, we add the wide open penalty to the objective function to assist model training. To evaluate the results, we compared the similarity of the real and the generated defensive plays, in terms of the players' movement speed and acceleration, distance to defend ball handlers and non-ball handlers, and the frequency of wide open occurrences. In addition, we conducted a user study with 59 participants for subjective tests. Experimental results show the high fidelity of the generated defensive plays to real data and demonstrate the feasibility of our algorithm.
Imitating Popular Photos to Select Views for an Indoor Scene
(2019-10-01)
Su, Rung-De
;
Liao, Zhe-Yo
;
Chen, Li-Chi
;
Tung, Ai-Ling
;
Wang, Yu-Shuen
Selecting informative and visually appealing views for 3D indoor scenes is beneficial for the housing, decoration, and entertainment industries. A set of views that exhibit comfort, aesthetics, and functionality of a particular scene can attract customers and facilitate business transactions. However, selecting views for an indoor scene is challenging because the system has to consider not only the need to reveal as much information as possible, but also object arrangements, occlusions, and characteristics. Since there can be many principles utilized to guide the view selection, and various principles to follow under different circumstances, we achieve the goal by imitating popular photos on the Internet. Specifically, we select the view that can optimize the contour similarity of corresponding objects to the photo. Because the selected view can be inadequate if object arrangements in the 3D scene and the photo are different, our system imitates many popular photos and selects a certain number of views. After that, it clusters the selected views and determines the view/cluster centers by the weighted average to finally exhibit the scene. Experimental results demonstrate that the views selected by our method are visually appealing.
Immersive VR Environment for Architectural Design Education
(2017-01-01)
Tsou, Chia-Hung
;
Hsu, Ting-Wei
;
Lin, Chun-Heng
;
Tsai, Ming-Han
;
Hsu, Pei-Hsien
In tradition architectural design education, instructor and students share and discuss their ideas through drawings and models. Physical models are hard to be modified during discussion, and it is difficult to view the interior spaces. Drawings, including plans, elevations and sections, are generally used as a medium for the purpose of presentation and discussion. We propose an integrated system to support architectural design education, in which multiple users can not only explore and discuss 3D designs in a VR environment, but also manipulate 3D elements. This system also integrates procedural 3D modeling, providing users with the capability of modifying 3D designs in real-time through adjustments of parameters.
Interactive Metro Map Editing
(2016-02-01)
Wang, Yu-Shuen
;
Peng, Wan-Yu
;
資訊工程學系
;
Department of Computer Science
Manual editing of a metro map is essential because many aesthetic and readability demands in map generation cannot be achieved by using a fully automatic method. In addition, a metro map should be updated when new metro lines are developed in a city. Considering that manually designing a metro map is time-consuming and requires expert skills, we present an interactive editing system that considers human knowledge and adjusts the layout to make it consistent with user expectations. In other words, only a few stations are controlled and the remaining stations are relocated by our system. Our system supports both curvilinear and octilinear layouts when creating metro maps. It solves an optimization problem, in which even spaces, route straightness, and maximum included angles at junctions are considered to obtain a curvilinear result. The system then rotates each edge to extend either vertically, horizontally, or diagonally while approximating the station positions provided by users to generate an octilinear layout. Experimental results, quantitative and qualitative evaluations, and user studies show that our editing system is easy to use and allows even non-professionals to design a metro map.
Interactive Visual Analysis for Vehicle Detector Data
(2015-06-01)
Chen, Yi-Cheng
;
Wang, Yu-Shuen
;
Lin, Wen-Chieh
;
Huang, Wei-Xiang
;
Lin, I-Chen
Visualization of vehicle detection (VD) data is essential because the data play an important role in traffic control and policy development. Most previous works focus on visualizing trajectories obtained from global positioning system (GPS), which are detailed but less representative. In contrast, VD data report the traffic statistic at each sensing site during a time span, including speed, flow, and occupancy of each lane, which contain comprehensive traffic information for analysis. In this work, we visualize three-year VD data of freeways in Taiwan. The visualization depicts the traffic situation at a site over time using a color-coded chart that extends from left to right over time. The charts are vertically stacked and horizontally aligned according to VD\'s located mileage and data time, respectively, to provide global insight. Our system allows semantic zoom, which changes the chart appearance in a continuous manner, to enable macro- and micro- scopic visualizations. Analysts can explore events that span an area with different sizes and that persist a time span with various lengths. To ensure the feasibility of our visualization, before the system design, we conducted a study with experts who work in the national freeway bureau and the institute of transportation of Taiwan. We also showed our results to the experts after the prototype system was built. The feedback shows that our VD data visualization is helpful to traffic control and policy development.
Interactive Visual Analysis of Browsing and Authoring Behaviors in Web Media
(2015)
Kuan, Yen-Ting
;
Tai, Ming-Hung
;
Kuan, Hsuan-Hao
;
Ho, Tan-Chi
;
Wang, Yu-Shuen
Web media is becoming popular nowadays for people to read news or articles. It is interesting and important for the web media providers to learn the browsing behaviors of their readers and authors. In this paper, we present a visual analytics system for discovering the browsing and authoring behaviors in web media based on users web page clickstream data. Our system provides multiple views of visualization with user interaction, including a scatter plot providing an overview to help analysts locate data of interest, and a force-directed graph for investigating relations between data entities. Interesting behaviors have been explored and found by using the proposed multiple views system.
iVRNote: Design, Creation and Evaluation of an Interactive Note-Taking Interface for Study and Reflection in VR Learning Environments
(2019-01-01)
Chen, Yi-Ting
;
Hsu, Chi-Hsuan
;
Chung, Chih-Han
;
Wang, Yu-Shuen
;
Babu, Sabarish V.
In this contribution, we design, implement and evaluate the pedagogical benefits of a novel interactive note taking interface (iVRNote) in VR for the purpose of learning and reflection lectures. In future VR learning environments, students would have challenges in taking notes when they wear a head mounted display (HMD). To solve this problem, we installed a digital tablet on the desk and provided several tools in VR to facilitate the learning experience. Specifically, we track the stylus' position and orientation in the physical world and then render a virtual stylus in VR. In other words, when students see a virtual stylus somewhere on the desk, they can reach out with their hand for the physical stylus. The information provided will also enable them to know where they will draw or write before the stylus touches the tablet. Since the presented iVRNote featuring our note taking system is a digital environment, we also enable students save efforts in taking extensive notes by providing several functions, such as post-editing and picture taking, so that they can pay more attention to lectures in VR. We also record the time of each stroke on the note to help students review a lecture. They can select a part of their note to revisit the corresponding segment in a virtual online lecture. Figures and the accompanying video demonstrate the feasibility of the presented iVRNote system. To evaluate the system, we conducted a user study with 20 participants to assess the preference and pedagogical benefits of the iVRNote interface. The feedback provided by the participants were overall positive and indicated that the iVRNote interface could be potentially effective in VR learning experiences.
Look at Me! Correcting Eye Gaze in Live Video Communication
(2019-06-01)
Hsu, Chih-Fan
;
Wang, Yu-Shuen
;
Lei, Chin-Laung
;
Chen, Kuan-Ta
;
資訊工程學系
Although live video communication is widely used, it is generally less engaging than face-to-face communication because of limitations on social, emotional, and haptic feedback. Missing eye contact is one such problem caused by the physical deviation between the screen and camera on a device. Manipulating video frames to correct eye gaze is a solution to this problem. In this article, we introduce a system to rotate the eyeball of a local participant before the video frame is sent to the remote side. It adopts a warping-based convolutional neural network to relocate pixels in eye regions. To improve visual quality, we minimize the L2 distance between the ground truths and warped eyes. We also present several newly designed loss functions to help network training. These new loss functions are designed to preserve the shape of eye structures and minimize color changes around the periphery of eye regions. To evaluate the presented network and loss functions, we objectively and subjectively compared results generated by our system and the state-of-the-art, DeepWarp, in relation to two datasets. The experimental results demonstrated the effectiveness of our system. In addition, we showed that our system can perform eye-gaze correction in real time on a consumer-level laptop. Because of the quality and efficiency of the system, gaze correction by postprocessing through this system is a feasible solution to the problem of missing eye contact in video communication.
Motion Manipulation for Characters in Videos
(2015)
Chu, Hung-Kuo
;
Yeh, Che-Hua
;
Tang, Nick C.
;
Tyan, Hsiao-Rong
;
Wang, Yu-Shuen
Multimedia content creation and manipulation have attracted great attention in recent years due to the popularity of mobile image and video capturing devices. In our daily life, the most common subject appearing in the captured content are people. Hence, to create images and videos by manipulating appearances or motions of the human character inside these media becomes an important research issue. In this paper, we propose a novel idea that requires the fusion of video and audio intelligence. We develop a system whose inputs are a monocular video captured by a stationary RGBD sensor and a music clip. In the context of a human character performing a music conducting motion, the output is a re-targeted video where the motion of performer is manipulated according to the emotional cues extracted from the music clip. To achieve this goal, our system is decomposed into three major stages. First, one needs to access to the geometric and appearance information pertaining to meaningful and representative targets in the video sequence. Second, a systematic way is needed to reliably classify and identify important emotions from the music. Third, to complete the emotion transfer, one has to manipulate the video targets based on the extracted music emotions.
Realizing the Real-time Gaze Redirection System with Convolutional Neural Network
(2018-01-01)
Hsu, Chih-Fan
;
Chen, Yu-Cheng
;
Wang, Yu-Shuen
;
Lei, Chin-Laung
;
Chen, Kuan-Ta
Retaining eye contact of remote users is a critical issue in video conferencing systems because of parallax caused by the physical distance between a screen and a camera. To achieve this objective, we present a real-time gaze redirection system called Flx-gaze to post-process each video frame before sending it to the remote end. Specifically, we relocate and relight the pixels representing eyes by using a convolutional neural network (CNN). To prevent visual artifacts during manipulation, we minimize not only the L2 loss function but also four novel loss functions when training the network. Two of them retain the rigidity of eyeballs and eyelids; and the other two prevent color discontinuity on the eye peripheries. By leveraging the CPU and the GPU resources, our implementation achieves real-time performance (i.e., 31 frames per second). Experimental results show that the gazes redirected by our system are of high quality under this restrict time constraint. We also conducted an objective evaluation of our system by measuring the peak signal-to-noise ratio (PSNR) between the real and the synthesized images.
Scalable and Coherent Video Resizing with Per-Frame Optimization
(2011-07-01)
Wang, Yu-Shuen
;
Hsiao, Jen-Hung
;
Sorkine, Olga
;
Lee, Tong-Yee
;
交大名義發表
The key to high-quality video resizing is preserving the shape and motion of visually salient objects while remaining temporally-coherent. These spatial and temporal requirements are difficult to reconcile, typically leading existing video retargeting methods to sacrifice one of them and causing distortion or waving artifacts. Recent work enforces temporal coherence of content-aware video warping by solving a global optimization problem over the entire video cube. This significantly improves the results but does not scale well with the resolution and length of the input video and quickly becomes intractable. We propose a new method that solves the scalability problem without compromising the resizing quality. Our method factors the problem into spatial and time/motion components: we first resize each frame independently to preserve the shape of salient regions, and then we optimize their motion using a reduced model for each pathline of the optical flow. This factorization decomposes the optimization of the video cube into sets of sub-problems whose size is proportional to a single frame's resolution and which can be solved in parallel. We also show how to incorporate cropping into our optimization, which is useful for scenes with numerous salient objects where warping alone would degenerate to linear scaling. Our results match the quality of state-of-the-art retargeting methods while dramatically reducing the computation time and memory consumption, making content-aware video resizing scalable and practical.
Spatially and Temporally Optimized Video Stabilization
(2013-08-01)
Wang, Yu-Shuen
;
Liu, Feng
;
Hsu, Pu-Sheng
;
Lee, Tong-Yee
;
資訊工程學系
Properly handling parallax is important for video stabilization. Existing methods that achieve the aim require either 3D reconstruction or long feature trajectories to enforce the subspace or epipolar geometry constraints. In this paper, we present a robust and efficient technique that works on general videos. It achieves high-quality camera motion on videos where 3D reconstruction is difficult or long feature trajectories are not available. We represent each trajectory as a Bezier curve and maintain the spatial relations between trajectories by preserving the original offsets of neighboring curves. Our technique formulates stabilization as a spatial-temporal optimization problem that finds smooth feature trajectories and avoids visual distortion. The Bezier representation enables strong smoothness of each feature trajectory and reduces the number of variables in the optimization problem. We also stabilize videos in a streaming fashion to achieve scalability. The experiments show that our technique achieves high-quality camera motion on a variety of challenging videos that are difficult for existing methods.