Browsing by Author "Liao, Hong-Yuan Mark"

Now showing 1 - 20 of 28

A language modeling approach to atomic human action recognition
(2007)
Liang, Yu-Ming
;
Shih, Sheng-Wen
;
Shih, Arthur Chun-Chieh
;
Liao, Hong-Yuan Mark
;
Lin, Cheng-Chung
Visual analysis of human behavior has generated considerable interest in the field of computer vision because it has a wide spectrum of potential applications. Atomic human action recognition is an important part of a human behavior analysis system. In this paper, we propose a language modeling framework for this task. The framework is comprised of two modules: a posture labeling module, and an atomic action learning and recognition module. A posture template selection algorithm is developed based on a modified shape context matching technique. The posture templates form a codebook that is used to convert input posture sequences into training symbol sequences or recognition symbol sequences. Finally, a variable-length Markov model technique is applied to learn and recognize the input symbol sequences of atomic actions. Experiments on real data demonstrate the efficacy of the proposed system.
A Local Feature-based Human Motion Recognition Framework
(2009)
Lai, Yu-Chun
;
Liao, Hong-Yuan Mark
;
Lin, Cheng-Chung
;
Chen, Jian-Ren
;
Luo, Y. -F. Peter
In this paper, we propose a local feature-based human motion analysis framework Instead of using traditional analysis methods to characterize the global structure of human motion, we extract features directly from local regions that contain motion. To implement the above concept, we adopt the rules of visual attention theory, which assert that a human motion can be described simply by a set of local features comprised of spatial relationships rather than human postures. We select two kinds of features to represent the local variation of a human motion. First, we extract the long-term movement trend of the motion. The second feature is actually a set of rough features derived by sampling multi-scale moving edges. The two types of features are considered together during the recognition process. Our experiments demonstrate that the proposed approach can achieve very good recognition results.
Automatic Music Video Generation Based on Simultaneous Soundtrack Recommendation and Video Editing
(2017-01-01)
Lin, Jen-Chun
;
Wei, Wen-Li
;
Yang, James
;
Wang, Hsin-Min
;
Liao, Hong-Yuan Mark
An automated process that can suggest a soundtrack to a user-generated video (UGV) and make the UGV a music-compliant professional-like video is challenging but desirable. To this end, this paper presents an automatic music video (MV) generation system that conducts soundtrack recommendation and video editing simultaneously. Given a long UGV, it is first divided into a sequence of fixed-length short (e.g., 2 seconds) segments, and then a multi-task deep neural network (MDNN) is applied to predict the pseudo acoustic (music) features (or called the pseudo song) from the visual (video) features of each video segment. In this way, the distance between any pair of video and music segments of same length can be computed in the music feature space. Second, the sequence of pseudo acoustic (music) features of the UGV and the sequence of the acoustic (music) features of each music track in the music collection are temporarily aligned by the dynamic time warping (DTW) algorithm with a pseudosong-based deep similarity matching (PDSM) metric. Third, for each music track, the video editing module selects and concatenates the segments of the UGV based on the target and concatenation costs given by a pseudo-song-based deep concatenation cost (PDCC) metric according to the DTW-aligned result to generate a music-compliant professional-like video. Finally, all the generated MVs are ranked, and the best MV is recommended to the user. The MDNN for pseudo song prediction and the PDSM and PDCC metrics are trained by an annotated official music video (OMV) corpus. The results of objective and subjective experiments demonstrate that the proposed system performs well and can generate appealing MVs with better viewing and listening experiences.
Court Reconstruction for Camera Calibration in Broadcast Basketball Videos
(2016-05)
Wen, Pei-Chih
;
Cheng, Wei-Chih
;
Wang, Yu-Shuen
;
Chu, Hung-Kuo
;
Tang, Nick C.
We introduce a technique of calibrating camera motions in basketball videos. Our method particularly transforms player positions to standard basketball court coordinates and enables applications such as tactical analysis and semantic basketball video retrieval. To achieve a robust calibration, we reconstruct the panoramic basketball court from a video, followed by warping the panoramic court to a standard one. As opposed to previous approaches, which individually detect the court lines and corners of each video frame, our technique considers all video frames simultaneously to achieve calibration; hence, it is robust to illumination changes and player occlusions. To demonstrate the feasibility of our technique, we present a stroke-based system that allows users to retrieve basketball videos. Our system tracks player trajectories from broadcast basketball videos. It then rectifies the trajectories to a standard basketball court by using our camera calibration method. Consequently, users can apply stroke queries to indicate how the players move in gameplay during retrieval. The main advantage of this interface is an explicit query of basketball videos so that unwanted outcomes can be prevented. We show the results in Figs. 1, 7, 9, 10 and our accompanying video to exhibit the feasibility of our technique.
Enriching Variety of Layer-wise Learning Information by Gradient Combination
(2019-01-01)
Wang, Chien-Yao
;
Liao, Hong-Yuan Mark
;
Chen, Ping-Yang
;
Hsieh, Jun-Wei
;
交大名義發表
This study proposes to use the combination of gradient concept to enhance the learning capability of Deep Convolutional Networks (DCN), and four Partial Residual Networks-based (PRN-based) architectures are developed to verify above concept. The purpose of designing PRN is to provide as rich information as possible for each single layer. During the training phase, we propose to propagate gradient combinations rather than feature combinations. PRN can be easily applied in many existing network architectures, such as ResNet, feature pyramid network, etc., and can effectively improve their performance. Nowadays, more advanced DCNs are designed with the hierarchical semantic information of multiple layers, so the model will continue to deepen and expand. Due to the neat design of PRN, it can benefit all models, especially for lightweight models. In the MSCOCO object detection experiments, YOLO-v3-PRN maintains the same accuracy as YOLO-v3 with a 55% reduction of parameters and 35% reduction of computation, while increasing the speed of execution by twice. For lightweight models, YOLO-v3-tiny-PRN maintains the same accuracy under the condition of 37% less parameters and 38% less computation than YOLO-v3-tiny and increases the frame rate by up to 12 fps on the NVIDIA Jetson TX2 platform. The Pelee-PRN is 6.7% mAP@0.5 higher than Pelee, which achieves the state-of-the-art lightweight object detection. The proposed lightweight object detection model has been integrated with technologies such as multi-object tracking and license plate recognition, and is used in a commercial intelligent traffic flow analysis system as its edge computing component. There are already three countries and more than ten cities have deployed this technique into their traffic flow analysis systems.
Examplar-Based Object Posture Super-Resolution Using Manifold Learning
(2012)
Ling, Chih-Hung
;
Lin, Chia-Wen
;
Hsu, Chiou-Ting
;
Liao, Hong-Yuan Mark
;
資訊工程學系
This paper proposes a learning-based approach to increase the temporal resolutions of human motion sequences. Given a set of high resolution motion sequences, our idea is first to learn the motion tendency from this learning dataset and then synthesize new postures for the low-resolution sequence according to the learned motion tendency. We summarize the proposed framework in the following steps: (1) Each motion sequence is first projected into a low-dimension manifold space, where the local distance between postures could be better preserved. We then represent each of the projected motion sequences as a motion trajectory. (2) Next, motion priors learned from the HR training sequences are used to reconstruct the motion trajectory for the input sequence. (3) Finally, we use the reconstructed motion trajectory combined with object inpainting technique to generate the final result. Our experimental results demonstrate the effectiveness of the proposed method, and also show its outperformance over existing approaches.
Example-Based Human Motion Extrapolation and Motion Repairing Using Contour Manifold
(2014-01-01)
Tang, Nick C.
;
Hsu, Chiou-Ting
;
Weng, Ming-Fang
;
Lin, Tsung-Yi
;
Liao, Hong-Yuan Mark
We propose a human motion extrapolation algorithm that synthesizes new motions of a human object in a still image from a given reference motion sequence. The algorithm is implemented in two major steps: contour manifold construction and object motion synthesis. Contour manifold construction searches for low-dimensional manifolds that represent the temporal-domain deformation of the reference motion sequence. Since the derived manifolds capture the motion information of the reference sequence, the representation is more robust to variations in shape and size. With this compact representation, we can easily modify and manipulate human motions through interpolation or extrapolation in the contour manifold space. In the object motion synthesis step, the proposed algorithm generates a sequence of new shapes of the input human object in the contour manifold space and then renders the textures of those shapes to synthesize a new motion sequence. We demonstrate the efficacy of the algorithm on different types of practical applications, namely, motion extrapolation and motion repair.
Human motion recognition using clay representation of trajectories
(2006)
Lai, Yu-Chun
;
Liao, Hong-Yuan Mark
;
資訊工程學系
;
Department of Computer Science
In this paper, we propose a novel human motion recognition approach that incorporates two fundamental concepts. First, the Shape Context and a clustering method are used to extract moving articulated parts from a video sequence. Then, we represent the moving articulated parts by trajectories. Our trajectory extraction approach provides good tolerance under various background and lighting conditions. Significantly, landmark point selection is not necessary in our approach, since trajectory generation is based on the extraction of moving articulated parts. The second concept is that the extracted trajectories are seen as forces pushing the articulated parts. Those forces are applied to a claylike deformed material, which is used to represent the trajectories' behavior. The experiment results show that the proposed approach is robust against noise and incorrectly extracted trajectories, such as redundant and missing trajectories.
Human Object Inpainting Using Manifold Learning-Based Posture Sequence Estimation
(2011-11-01)
Ling, Chih-Hung
;
Liang, Yu-Ming
;
Lin, Chia-Wen
;
Chen, Yong-Sheng
;
Liao, Hong-Yuan Mark
We propose a human object inpainting scheme that divides the process into three steps: 1) human posture synthesis; 2) graphical model construction; and 3) posture sequence estimation. Human posture synthesis is used to enrich the number of postures in the database, after which all the postures are used to build a graphical model that can estimate the motion tendency of an object. We also introduce two constraints to confine the motion continuity property. The first constraint limits the maximum search distance if a trajectory in the graphical model is discontinuous, and the second confines the search direction in order to maintain the tendency of an object's motion. We perform both forward and backward predictions to derive local optimal solutions. Then, to compute an overall best solution, we apply the Markov random field model and take the potential trajectory with the maximum total probability as the final result. The proposed posture sequence estimation model can help identify a set of suitable postures from the posture database to restore damaged/missing postures. It can also make a reconstructed motion sequence look continuous.
Learning Atomic Human Actions Using Variable-Length Markov Models
(2009-02-01)
Liang, Yu-Ming
;
Shih, Sheng-Wen
;
Shih, Arthur Chun-Chieh
;
Liao, Hong-Yuan Mark
;
Lin, Cheng-Chung
Visual analysis of human behavior has generated considerable interest in the field of computer vision because of its wide spectrum of potential applications. Human behavior can be segmented into atomic actions, each of which indicates a basic and complete movement. Learning and recognizing atomic human actions are essential to human behavior analysis. In this paper, we propose a framework for handling this task using variable-length Markov models (VLMMs). The framework is comprised of the following two modules: a posture labeling module and a VLMM atomic action learning and recognition module. First, a posture template selection algorithm, based on a modified shape context matching technique, is developed. The selected posture templates form a codebook that is used to convert input posture sequences into discrete symbol sequences for subsequent processing. Then, the VLMM technique is applied to learn the training symbol sequences of atomic actions. Finally, the constructed VLMMs are transformed into hidden Markov models (HMMs) for recognizing input atomic actions. This approach combines the advantages of the excellent learning function of a VLMM and the fault-tolerant recognition ability of an HMM. Experiments on realistic data demonstrate the efficacy of the proposed system.
LINEAR PRODUCTION GAME SOLUTION TO A PTZ CAMERA NETWORK
(2010)
Lai, Yu-Chun
;
Liang, Yu-Ming
;
Shih, Sheng-Wen
;
Liao, Hong-Yuan Mark
;
Lin, Cheng-Chung
Reconfiguring the PTZ parameters of a camera network is an combinatorial optimization problem and computing the optimal solution is very time consuming. Therefore, existing methods can only provide sub-optimal solutions. In this paper, a nonlinear objective function for better utilizing the cameras to track multiple targets is proposed. Furthermore, it is shown that by expanding the unknown parameters and imposing new constraints, the nonlinear objective function can be converted into a linear production game (LPG) problem. Since an LPG possesses an optimal solution which can be evaluated with polynomial time, the proposed method is efficient and accurate. Computer simulations have been conducted and the results show that the proposed method is very promising.
Linear Production Game Solution to a PTZ Camera Network
(2013-05-01)
Lai, Yu-Chun
;
Liang, Yu-Ming
;
Shih, Sheng-Wen
;
Lin, Cheng-Chung
;
Liao, Hong-Yuan Mark
Reconfiguring the parameters of PTZ cameras in a surveillance system is a combinatorial optimization problem. Computing the optimal solution is very time consuming, and existing methods can only provide sub-optimal solutions. In this paper, we propose a non-linear objective function that better utilizes a camera network to track multiple targets. We also show that, by expanding the unknown parameters and imposing new constraints, the non-linear objective function can be converted into a linear production game (LPG) problem. Since an LPG yields an optimal solution that can be evaluated in polynomial time, the proposed method is efficient and accurate. The results of simulations and a real-world experiment demonstrate the proposed method's potential.
Method for automatic key posture information abstraction
(2007-05-10)
Liao, Hong-Yuan Mark
;
Chen, Duan-Yu
The method for automatic key posture information abstraction of this invention comprises the steps of: Abstracting from a series of continuous digitized images spatial features of objects contained in said images; abstracting shape features of said objects using a method of probability calculation; detecting key posture information contained in said series of continuous images using a method of entropy calculation; removing redundant key postures; mating obtained key postures with key posture templates stored in a codebook; and encoding mated key postures.
Motion flow-based video retrieval
(2007-10-01)
Su, Chih-Wen
;
Liao, Hong-Yuan Mark
;
Tyan, Hsiao-Rong
;
Lin, Chia-Wen
;
Chen, Duan-Yu
In this paper, we propose the use of motion vectors embedded in MPEG bitstreams to generate so-called "motion flows", which are applied to perform video retrieval. By using the motion vectors directly, we do not need to consider the shape of a moving object and its corresponding trajectory. Instead, we simply "link" the local motion vectors across consecutive video frames to form motion flows, which are then recorded and stored in a video database. In the video retrieval phase, we propose a new matching strategy to execute the video retrieval task. Motions that do not belong to the mainstream motion flows are filtered out by our proposed algorithm. The retrieval process can be triggered by query-by-sketch or query-by-example. The experiment results show that our method is indeed superb in the video retrieval process.
Motion Manipulation for Characters in Videos
(2015)
Chu, Hung-Kuo
;
Yeh, Che-Hua
;
Tang, Nick C.
;
Tyan, Hsiao-Rong
;
Wang, Yu-Shuen
Multimedia content creation and manipulation have attracted great attention in recent years due to the popularity of mobile image and video capturing devices. In our daily life, the most common subject appearing in the captured content are people. Hence, to create images and videos by manipulating appearances or motions of the human character inside these media becomes an important research issue. In this paper, we propose a novel idea that requires the fusion of video and audio intelligence. We develop a system whose inputs are a monocular video captured by a stationary RGBD sensor and a music clip. In the context of a human character performing a music conducting motion, the output is a re-targeted video where the motion of performer is manipulated according to the emotional cues extracted from the music clip. To achieve this goal, our system is decomposed into three major stages. First, one needs to access to the geometric and appearance information pertaining to meaningful and representative targets in the video sequence. Second, a systematic way is needed to reliably classify and identify important emotions from the music. Third, to complete the emotion transfer, one has to manipulate the video targets based on the extracted music emotions.
Movie scene segmentation using background information
(2008-03-01)
Chen, Liang-Hua
;
Lai, Yu-Chun
;
Liao, Hong-Yuan Mark
;
資訊工程學系
;
Department of Computer Science
Scene extraction is the first step toward semantic understanding of a video. It also provides improved browsing and retrieval facilities to users of video database. This paper presents an effective approach to movie scene extraction based on the analysis of background images. Our approach exploits the fact that shots belonging to one particular scene often have similar backgrounds. Although part of the video frame is covered by foreground objects, the background scene can still be reconstructed by a mosaic technique. The proposed scene extraction algorithm consists of two main components: determination of the shot similarity measure and a shot grouping process. In our approach, several low-level visual features are integrated to compute the similarity measure between two shots. On the other hand, the rules of film-making are used to guide the shot grouping process. Experimental results show that our approach is promising and outperforms some existing techniques. (C) 2007 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
Moving foreground object detection via robust SIFT trajectories
(2013-04-01)
Sun, Shih-Wei
;
Wang, Yu-Chiang Frank
;
Huang, Fay
;
Liao, Hong-Yuan Mark
;
資訊科學與工程研究所
In this paper, we present an automatic foreground object detection method for videos captured by freely moving cameras. While we focus on extracting a single foreground object of interest throughout a video sequence, our approach does not require any training data nor the interaction by the users. Based on the SIFT correspondence across video frames, we construct robust SIFT trajectories in terms of the calculated foreground feature point probability. Our foreground feature point probability is able to determine candidate foreground feature points in each frame, without the need of user interaction such as parameter or threshold tuning. Furthermore, we propose a probabilistic consensus foreground object template (CFOT), which is directly applied to the input video for moving object detection via template matching. Our CFOT can be used to detect the foreground object in videos captured by a fast moving camera, even if the contrast between the foreground and background regions is low. Moreover, our proposed method can be generalized to foreground object detection in dynamic backgrounds, and is robust to viewpoint changes across video frames. The contribution of this paper is trifold: (1) we provide a robust decision process to detect the foreground object of interest in videos with contrast and viewpoint variations; (2) our proposed method builds longer SIFT trajectories, and this is shown to be robust and effective for object detection tasks; and (3) the construction of our CFOT is not sensitive to the initial estimation of the foreground region of interest, while its use can achieve excellent foreground object detection results on real-world video data. (c) 2012 Elsevier Inc. All rights reserved.
Segmentation of human body parts in video frames based on intrinsic distance
(2007)
Lai, Yu-Chun
;
Liao, Hong-Yuan Mark
;
Lin, Cheng-Chung
;
資訊工程學系
;
Department of Computer Science
We propose an intrinsic-distance based segmentation approach for segmenting human body parts in video frames. First, since the human body can be seen as a set of articulated parts, we utilize the moving articulated attributes to identify body part candidate regions automatically. The candidate regions and the background candidate regions are generated by voting and assigned to the spatiotemporal volume, which is comprised of frames of the video. Then, the intrinsic distance is used to estimate the boundaries of each body part. Our intrinsic distance-based segmentation technique is applied in the spatiotemporal volume to extract the optimal boundaries of the intrinsic distance in a video and obtain segmented frames from the segmented volume. The segmented results show that the proposed approach can tolerate incomplete and imprecise candidate regions because it provides temporal continuity. Furthermore, it can reduce over growing in the original intrinsic distance-based algorithm, since it can handle ambiguous pixels. We expect that this research can provide an alternative to segmenting a sequence of body parts in a video.
Simultaneous Tensor Decomposition and Completion Using Factor Priors
(2014-03-01)
Chen, Yi-Lei
;
Hsu, Chiou-Ting
;
Liao, Hong-Yuan Mark
;
資訊工程學系
;
Department of Computer Science
The success of research on matrix completion is evident in a variety of real-world applications. Tensor completion, which is a high-order extension of matrix completion, has also generated a great deal of research interest in recent years. Given a tensor with incomplete entries, existing methods use either factorization or completion schemes to recover the missing parts. However, as the number of missing entries increases, factorization schemes may overfit the model because of incorrectly predefined ranks, while completion schemes may fail to interpret the model factors. In this paper, we introduce a novel concept: complete the missing entries and simultaneously capture the underlying model structure. To this end, we propose a method called simultaneous tensor decomposition and completion (STDC) that combines a rank minimization technique with Tucker model decomposition. Moreover, as the model structure is implicitly included in the Tucker model, we use factor priors, which are usually known a priori in real-world tensor objects, to characterize the underlying joint-manifold drawn from the model factors. By exploiting this auxiliary information, our method leverages two classic schemes and accurately estimates the model factors and missing entries. We conducted experiments to empirically verify the convergence of our algorithm on synthetic data and evaluate its effectiveness on various kinds of real-world data. The results demonstrate the efficacy of the proposed method and its potential usage in tensor-based applications. It also outperforms state-of-the-art methods on multilinear model analysis and visual data completion tasks.
Spatio-Temporal Learning of Basketball Offensive Strategies
(2015)
Chen, Ching-Hang
;
Liu, Tyng-Luh
;
Wang, Yu-Shuen
;
Chu, Hung-Kuo
;
Tang, Nick C.
Video-based group behavior analysis is drawing attention to its rich applications in sports, military, surveillance and biological observations. The recent advances in tracking techniques, based on either computer vision methodology or hardware sensors, further provide the opportunity of better solving this challenging task. Focusing speci fi cally on the analysis of basketball o ff ensive strategies, we introduce a systematic approach to establishing unsupervised modeling of group behaviors. In view that a possible group behavior (offensive strategy) could be of di ff erent duration and represented by dynamic player trajectories, the crux of our method is to automatically divide training data into meaningful clusters and learn their respective spatio-temporal model, which is established upon Gaussian mixture regression to account for intra-class spatio-temporal variations. The resulting strategy representation turns out to be flexible that can be used to not only establish the discriminant functions but also improve learning the models. We demonstrate the usefulness of our approach by exploring its e ff ectiveness in analyzing a set of given basketball video clips.