Browsing by Author "Peng, Wen-Hsiao"

Now showing 1 - 20 of 50

A COMPARATIVE STUDY ON ATTENTION-BASED RATE ADAPTATION FOR SCALABLE VIDEO CODING
(2009)
Tsai, Chia-Ming
;
Lin, Chia-Wen
;
Lin, Weisi
;
Peng, Wen-Hsiao
;
資訊工程學系
We conduct subjective tests to evaluate the performance of scalable video coding with different spatial-domain bit-allocation methods, visual attention models, and motion feature extractors in the literature. For spatial-domain bit allocation, we use the selective enhancement and quality layer assignment methods. For characterizing visual attention, we use the motion attention model and perceptual quality significant map. For motion features, we adopt motion vectors from hierarchical B-picture coding and optical flow. Experimental results show that a more accurate visual attention model leads to better perceptual quality. In cooperation with a visual attention model, the selective enhancement method, compared to the quality layer assignment, achieves better subjective quality when an ROI has enough bit allocation and its texture is not complex. The quality layer assignment method is suitable for region-wise quality enhancement due to its frame-based allocation nature.
A context adaptive bit-plane coder with maximum-likelihood-based stochastic bit-reshuffling technique for scalable video coding
(2006-08-01)
Peng, Wen-Hsiao
;
Chiang, Tihao
;
Hang, Hsueh-Ming
;
Lee, Chen-Yi
;
電子工程學系及電子研究所
In this paper, we propose a context adaptive bit-plane coding (CABIC) with a stochastic bit reshuffling (SBR) scheme to deliver higher coding efficiency and better subjective quality for fine granular scalable (FGS) video coding. Traditional bit-plane coding in FGS algorithm suffers from poor coding efficiency and subjective quality. To improve coding efficiency, our CABIC constructs context models based on both the energy distribution in a block and the spatial correlations in the adjacent blocks. Moreover, it exploits the context across bit-planes to save side information. To improve subjective quality, our SBR reorders the coefficient bits by their estimated rate-distortion performance. Particularly, we model transform coefficients with Laplacian distributions and incorporate them into the context probability models for content-aware parameter estimation. Moreover, our SBR is implemented with a dynamic priority management that uses a low-complexity dynamic memory organization. Experimental results show that our CABIC improves the PSNR by 0.5 similar to 1.0 dB at medium and high bit rates. While maintaining similar or even higher coding efficiency, our SBR improves the subjective quality.
A Parametric Window Design for OBMC with Variable Block Size Motion Estimates
(2009)
Chen, Yi-Wen
;
Wang, Tse-Wei
;
Tseng, Yu-Cheng
;
Peng, Wen-Hsiao
;
Lee, Suh-Yin
This paper addresses the problem of adapting overlapped block motion compensation (OBMC) windows for use with variable blocksize motion estimates. We tackle the problem by using a parametric window design, which expresses, based on a statistical motion model, the optimal weights as a function of the distances between the predicted pixel and its nearby block centers. The formula enables both prediction weights and prediction order to be adapted on a pixel-by-pixel basis. Extensive experiments have been conducted using JM 12.4. Compared with conventional block motion compensation, our scheme shows a bitrate saving of 18%, (5% on average) while maintaining the same or even higher PSNR (0.1 dB). It also provides a competitive advantage to variable block size motion compensation. Additionally, a hybrid of the two techniques achieves a further bitrate reduction of 13%. The result, nevertheless, provides only a lower bound on what is achievable since both motion estimation and mode decision were accomplished without considering OBMC. Further improvement is expected by incorporating iterative methods.
A rate-distortion optimization model for SVC inter-layer encoding and bitstream extraction
(2008-12-01)
Peng, Wen-Hsiao
;
Zao, John K.
;
Huang, Hsueh-Ting
;
Wang, Tse-Wei
;
Huang, Lin-Shung
The Scalable Video Coding (SVC) standard enables viewing devices to adapt their video reception using bitstream extraction. Since SVC offers spatial, temporal, and quality combined scalability, extracting proper bitstreams for different viewing devices can be a non-trivial task, and naive choices usually produce poor playback quality. In this paper, we propose a two-prong approach to achieve rate-distortion (R-D) optimal extraction of SVC bitstreams. For SVC encoding, we developed a set of adaptation rules for setting the quantization parameters and the inter-layer dependencies among the SVC coding layers. A well-adapted SVC bitstream thus produced manifests good R-D trade-offs when its scalable layers are extracted along extraction paths consisting of successive refinement steps. For extracting R-D optimized bitstreams for different viewing devices, we formalized the notion of optimal and near-optimal extraction paths and devised computationally efficient strategies to search for the extraction paths. Experiment results demonstrated that our R-D optimized adaptation schemes and extraction strategies offer significant improvement in playback picture quality among heterogeneous viewing devices. Particularly, our adaptation rules promise R-D convexity along optimal extraction paths and permit the use of steepest-descent strategy to discover the optimal/near-optimal paths. This simple search strategy performs only half of the computation necessary for an exhaustive search. (C) 2008 Elsevier Inc. All rights reserved.
A reconfigurable video embedding transcoder based on H.264/AVC: Design tradeoffs and analysis
(2008)
Li, Chih-Hung
;
Peng, Wen-Hsiao
;
Chiang, Tihao
;
電子工程學系及電子研究所
;
Department of Electronics Engineering and Institute of Electronics
In this paper, we propose a system architecture for H.264/AVC video embedding transcoder (VET). In addition, the proposed platform-based design can seamlessly combine the MW-VET and decoder such that it can be dynamically configured to perform video decoding and transcoding alternatively or simultaneously. Furthermore, we perform the pruned design space exploration on the design of inter/intra prediction and the on-chip data bus width. Our proposed architecture provides a better tradeoff among execution cycles, hardware cost, resource utilization, and video quality because of the reconfigurable processing modules and the hybrid pipelining. As compared to the cascaded pixel domain transcoder that has the highest complexity, our hardware efficient VET can significantly reduce the hardware cost while maintaining similar rate-distortion performance. Finally, the proposed architecture is verified at system level using transaction level modeling (TLM) technique. From the simulation results, the proposed architecture with the best tradeoff configuration can achieve a transcoding rate up to 358 frames per second for SD video source while clocking at 162MHz.
A Synthesis-Quality-Oriented Depth Refinement Scheme for MPEG Free Viewpoint Television (FTV)
(2009)
Chen, Chun-Chi
;
Chen, Yi-Wen
;
Yang, Fu-Yao
;
Peng, Wen-Hsiao
;
資訊工程學系
This paper addresses the problem of refining depth information front the received reference and depth images within the MPEG FT V framework. An analytical model is first developed to approximate the per-pixel synthesis distortion (caused by depth-image compression) as a function of depth-error variances, intensity variations, ground-truth depth and virtual camera locations. We then follow the model to detect unreliable depth pixels by inspecting intensity gradients and to refine their values with a candidate-based block disparity search. Additional side information is transmitted to make both operations robust against compression effects. Experimental results show that our scheme offers an average PSNR improvement of 1.2 dB over MPEG FTV and consistently outperforms the state-of-the-art methods. Moreover, it can remove synthesis artifacts to a great extent, producing a result that is very close in appearance to the ground-truth view image.
Adding selective enhancement in scalable video coding for region-of-interest functionality
(2006)
Peng, Wen-Hsiao
;
Chiang, Tihao
;
Hang, Hsueh-Ming
;
交大名義發表
;
National Chiao Tung University
In this paper, we propose a lossless, graceful, and arbitrary-shaped region-of-interest (ROI) functionality for the scalable video coding in MPEG-4 Part 10 Amd. 1. Specifically, we propose a prioritized block coding scheme and a layer remapping technique. Within an enhancement layer, the prioritized block coding reshuffles the transform coefficients for ROI. Moreover, the layer remapping technique prioritizes different ROI among enhancement layers. For graceful and arbitrary-shaped properties, additional syntax are coded at the levels of slice, layer, and macroblock. To minimize overhead, an efficient representation for priority information is proposed. Experimental results show that our schemes can offer the ROI functionality over a wide range of bit rates while maintaining coding efficiency.
Advances in the scalable amendment of H.264/AVC
(2007-01-01)
Huang, Hsiang-Chun
;
Peng, Wen-Hsiao
;
Chiang, Tihao
;
Hang, Hsueh-Ming
;
交大名義發表
To support clients with diverse capabilities, ISO/IEC MPEG and ITU-T form a Joint Video Team (JVT) to develop a scalable video coding (SVC) technology that uses single bitstream to provide multiple spatial, temporal, and quality (SNR) resolutions, thus satisfying low-complexity and low-delay constraints. It is an amendment of the emerging standard H.264/AVC and it provides an H.264/AVC-compatible base layer and a fully scalable enhancement layer, which can be truncated and extracted on-the-fly to obtain a preferred spatio-temporal and quality resolution. An overview of the adopted key technologies in the SVC and a comparison in coding efficiency with H.264/AVC are presented.
ADVERSARIAL GENERATION OF DEFENSIVE TRAJECTORIES IN BASKETBALL GAMES
(2018-01-01)
Chen, Chieh-Yu
;
Lai, Wenze
;
Hsieh, Hsin-Ying
;
Wang, Yu-Shuen
;
Peng, Wen-Hsiao
In this paper, we present a method to generate realistic trajectories of defensive players in a basketball game based on the ball and the offensive team's movements. We train on the NBA dataset a conditional generative adversarial network that learns spatio-temporal interactions between players' movements. The network consists of two components: a generator that takes as input a latent noise vector and the offensive team's trajectories to generate defensive trajectories; and a discriminator that evaluates the realistic degree of the generated results. Our system allows players and coaches to simulate how the opposing team will react to a newly developed offensive strategy for evaluating its effectiveness. Experimental results demonstrate the feasibility of the proposed algorithm.
All about Structure: Adapting Structural Information across Domains for Boosting Semantic Segmentation
(2019-01-01)
Chang, Wei-Lun
;
Wang, Hui-Po
;
Peng, Wen-Hsiao
;
Chiu, Wei-Chen
;
交大名義發表
In this paper we tackle the problem of unsupervised domain adaptation for the task of semantic segmentation, where we attempt to transfer the knowledge learned upon synthetic datasets with ground-truth labels to real-world images without any annotation. With the hypothesis that the structural content of images is the most informative and decisive factor to semantic segmentation and can be readily shared across domains, we propose a Domain Invariant Structure Extraction (DISE) framework to disentangle images into domain-invariant structure and domain-specific texture representations, which can further realize image-translation across domains and enable label transfer to improve segmentation performance. Extensive experiments verify the effectiveness of our proposed DISE model and demonstrate its superiority over several state-of-the-art approaches.
An Interframe Prediction Technique Combining Template Matching Prediction and Block-Motion Compensation for High-Efficiency Video Coding
(2013-08-01)
Peng, Wen-Hsiao
;
Chen, Chun-Chi
;
資訊工程學系
;
Department of Computer Science
This paper introduces an interframe prediction technique that combines two motion vectors (MVs) derived respectively from template and block matching for overlapped block motion compensation (OBMC). It has a salient feature of not having to signal the template MV, while achieving a prediction performance close to that of bi-prediction. We begin by studying template matching prediction (TMP) from a theoretical perspective. Based on two signal models, the template MV is shown to approximate the pixel true motion around the template centroid, through which we explain why TMP generally outperforms SKIP prediction but is inferior to block-based motion compensation in terms of prediction performance. We then approach the problem of finding another MV to best complement the template MV from both deterministic and statistical viewpoints, the latter leading to the search of its optimal sampling location in the motion field. The result is a search criterion with OBMC window functions forming a geometry-like motion partitioning when the template area is straddled on the top and to the left of a target block. Generalizations to adaptive template design, multihypothesis prediction and motion merging are made to explore the complexity and performance trade-offs. Extensive experiments based on the HM-6.0 software show that the best of them, in terms of compression performance, achieves 1.7-2.0% BD-rate reductions at a cost of 26% and 39% increases in encoding and decoding times, respectively.
Analysis of Template Matching Prediction and Its Application to Parametric Overlapped Block Motion Compensation
(2010)
Wang, Tse-Wei
;
Chen, Wen
;
Peng, Wen-Hsiao
;
資訊工程學系
;
Department of Computer Science
Template matching prediction (TMP), which estimates the motion for a target block by using its surrounding pixels, has been observed to perform efficiently in inter-frame coding. In this paper, we expose, from a more theoretical viewpoint, the factors that determine the prediction efficiency of TMP. It is shown that the motion estimate found by template matching tends to be the motion associated with the template centroid and that TMP consistently outperforms SKIP prediction, but hardly competes with block motion compensation (BMC) unless both the motion and intensity fields are less random or have high spatial correlation. We also demonstrate how template and block motion estimates can jointly be applied in a parametric overlapped block motion compensation (OBMC) framework to further improve temporal prediction. Preliminary results show that combining TMP with OBMC can yield 2-16% reductions in mean-square prediction error, as compared with the single use of OBMC. The gain is even higher (18%) when the performance is compared with that of the standard BMC.
Analytical Mode-Dependent Rate and Distortion Models for H.264/SVC Coarse Grain Scalability
(2012)
Wu, Chung-Hao
;
Tseng, Yu-Chen
;
Peng, Wen-Hsiao
;
電子工程學系及電子研究所
;
Department of Electronics Engineering and Institute of Electronics
This paper presents analytical rate and distortion models to estimate the rate-distortion (R-D) behavior of different coding modes for H.264/SVC coarse grain scalability (CGS) by jointly considering the effects of quantization, motion-compensated prediction, and motion partitioning structures. Based on the forward channel model and two statistical signal models, we first derive a pair of mode-dependent rate and distortion models for coding modes in base layer. Then these models are extended to account for the coding modes with inter-layer residual prediction in enhancement layer, especially for CGS. Preliminary results show that the proposed rate and distortion models for both layers have the potentiality to estimate the R-D behavior of real data.
BI-PREDICTION COMBINING TEMPLATE AND BLOCK MOTION COMPENSATIONS
(2011)
Lee, Chung-Lin
;
Chen, Chun-Chi
;
Chen, Yi-Wen
;
Wu, Mu-Hsuan
;
Wu, Chung-Hao
This paper introduces a bi-prediction scheme with only a motion overhead as for unidirectional prediction. It combines motion vectors found by template and block matchings with the overlapped block motion compensation (OBMC). An optimal window function is designed based on a model-based framework. Additionally, the concept of adaptive motion merging is incorporated to enable a template-matching-free implementation. Three algorithms featuring different performance and complexity trade-offs are implemented using the TMuC-0.9_HM software and tested with the common test conditions. Relative to the anchor, the best of them achieves an average BD-rate saving of 2.2%, with a minimum of 0.2% and a maximum of 4.7%.
DEEP REINFORCEMENT LEARNING FOR VIDEO PREDICTION
(2019-01-01)
Ho, Yung-Han
;
Cho, Chuan-Yuan
;
Peng, Wen-Hsiao
;
資訊工程學系
;
Department of Computer Science
This paper introduces a hybrid video prediction scheme that combines the classic parametric overlapped block motion compensation (POBMC) technique with neural networks. Most learning-based video prediction methods rely on a black-box-like model for either direct generation of future video frames or estimation of a dense motion field. The model complexity often increases drastically with frame resolution. Departing from pure black-box approaches, this paper leverages the theoretically-grounded POBMC in a reinforcement learning framework to estimate a sparse motion field for future frame warping. Two neural networks are trained to identify critical points in the motion field for motion estimation. We train our model on 10k unlabeled frames in KITTI dataset and achieve the state-of-the-art SSIM score of 0.923 on CaltechPed and an average SSIM scroe of 0.856 on Common Intermediate Format (CIF) standard sequences.
Design of memory sub-system in H.264/AVC decoder
(2007)
Li, Chih-Hung
;
Chang, Chang-Hsuan
;
Peng, Wen-Hsiao
;
Hwang, Wei
;
Chiang, Tihao
In this paper, we present the memory sub-system of a H.264/AVC decoder designed for High profile and Level 4. Our design incorporates a synchronization buffer as a pre-cache buffer. We investigate the efficiency of DRAM access and power dissipation when the buffer is designed at different granularities. Statistical results show that the granularity of larger block size has higher memory efficiency; less access cycles and power dissipation. However, the granularity of 8x8 block size provides better trade-off among cost, efficiency, power, and real-time requirement.
Design of memory sub-system with constant-rate bumping process for H.264/AVC decoder
(2007-02-01)
Li, Chih-Hung
;
Peng, Wen-Hsiao
;
Chiang, Tihao
;
資訊工程學系
;
電子工程學系及電子研究所
In this paper, we propose an efficient memory sub-system and a constant-rate bumping process for a H.264/AVC decoder conforming to High profile@ Level 4. To efficiently utilize the throughput of external DRAM a synchronization buffer is employed as a bridge for reformatting the read/write data exchanged between the on-chip hardware and the off-chip DRAM In addition, we optimize the issues of read/write commands and adaptively enable the auto-precharge function by monitoring the motion information of a submacroblock. Furthermore, a regulation buffer with size comparable to the decoded picture buffer is created to ensure a constant output rate of decoded pictures for any conformed prediction structures. Along with other modules, the proposed scheme is verified at system level using transaction level modeling (TLM) technique. Statistical results show that synchronization buffer of larger block size provides higher memory efficiency, less access cycles and power dissipation. However, the granularity of 8x8 block size provides better trade-off among cost, efficiency, power, and real-time requirement.
DESIGN SPACE EXPLORATION OF AN H.264/AVC-BASED VIDEO EMBEDDING TRANSCODER USING TRANSACTION LEVEL MODELING
(2008)
Li, Chih-Hung
;
Peng, Wen-Hsiao
;
Chiang, Tihao
;
電子工程學系及電子研究所
;
Department of Electronics Engineering and Institute of Electronics
In this paper, we perform the design space exploration for an H.264/AVC video embedding transcoder. Specifically, the design space is pruned for the sub-modules including inverse transform, inter and intra. prediction, and deblocking filter with various microarchitecture designs, processing order, memory hierarchy, and granularity of synchronization. In addition, we propose an efficient deblocking filter suitable for 8x8 block pipeline. Compared to the traditional designs, our proposed deblocking filter reduces memory requirement, processing latency, and access frequency to the local memory. The synthesized logic gate count is only 8K using the 0.18 um technology with the maximum frequency of 162 MHz. For rapid exploration, all the design alternatives are simulated with higher level of abstraction using the transaction level modeling to explore 160 design combinations. Our simulation results provide an extensive tradeoff analysis among processing speed, cost, and utilization. Besides, the cost-normalized hardware utilization where the cost of each sub-module weights its associated utilization assists the system designers to keep a balance across different modules.
Discriminatively-learned Global Image Representation Using CNN as a Local Feature Extractor for Image Retrieval
(2015-01-01)
Ku, Wei-Lin
;
Chou, Hung-Chun
;
Peng, Wen-Hsiao
;
資訊工程學系
;
Department of Computer Science
This work introduces an image retrieval framework based on using deep convolutional neural networks (CNN) as a local feature extractor. Motivated by the great success of CNN in recognition tasks, one may be tempted to simply adopt the output of CNN as a global image representation for retrieval. This straightforward approach, however, has proved deficient, because it can be vulnerable to various image transformation attacks. To address this issue, we propose to treat CNN as a local feature extractor, and a local image patch selection mechanism is developed to extract discriminative patches by observing their objectness responses, aspect ratios, relative scales, and locations in the image. The criterion is given by a learned posterior probability indicating how likely the image patch in question will find a correspondence in another similar image. In addition, the CNN's weight parameters are specifically adapted by a contrastive loss function to suit retrieval tasks. Extensive experiments on typical retrieval datasets confirm the superiority of the proposed scheme over the state-of-the-art methods.
Fast Bi-Directional Prediction Selection in H.264/MPEG-4 AVC Temporal Scalable Video Coding
(2011-12-01)
Lin, Hung-Chih
;
Hang, Hsueh-Ming
;
Peng, Wen-Hsiao
;
資訊工程學系
;
電子工程學系及電子研究所
In this paper, we propose a fast algorithm that efficiently selects the temporal prediction type for the dyadic hierarchical-B prediction structure in the H.264/MPEG-4 temporal scalable video coding (SVC). We make use of the strong correlations in prediction type inheritance to eliminate the superfluous computations for the bi-directional (BI) prediction in the finer partitions, 16 x 8/8 x 16/8 x 8, by referring to the best temporal prediction type of 16 x 16. In addition, we carefully examine the relationship in motion bit-rate costs and distortions between the BI and the uni-directional temporal prediction types. As a result, we construct a set of adaptive thresholds to remove the unnecessary BI calculations. Moreover, for the block partitions smaller than 8 x 8, either the forward prediction (FW) or the backward prediction (BW) is skipped based upon the information of their 8 x 8 partitions. Hence, the proposed schemes can efficiently reduce the extensive computational burden in calculating the BI prediction. As compared to the JSVM 9.11 software, our method saves the encoding time from 48% to 67% for a large variety of test videos over a wide range of coding bit-rates and has only a minor coding performance loss.