Browsing by Author "Hsiao, Hsu-Feng"

Now showing 1 - 20 of 24

A DEPTH REFINEMENT ALGORITHM FOR MULTI-VIEW VIDEO SYNTHESIS
(2010)
Shih, Hsin-Chia
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
With the recent progress of display, capture device, and coding technologies, multi-view video applications such as stereoscopic video, free viewpoint TV (FTV), and free viewpoint video (FVV) have been introduced to the world with growing interest. To achieve free navigation of such applications, depth information is required along with the video data. There have been many research activities in the area of depth estimation; however, it still poses us great challenge to estimate accurate depth map. In this paper, we propose a depth refinement algorithm for multi-view video synthesis. The proposed algorithm classifies the pixel-wise depth map into two categories, one is reliable and the other is unreliable, followed by the depth refinement algorithm for those pixels with unreliable depth values. Except for the depth refinement algorithm, we also propose a reliable weighted view interpolation algorithm. At last, the refined depth map is evaluated by the quality of the synthesized view.
A max-min fairness congestion control for layered streaming of scalable video
(2006-09-01)
Hsiao, Hsu-Feng
;
Hwang, Jenq-Neng
;
資訊工程學系
;
Department of Computer Science
In a best-effort networking environment, efficient and fair congestion control is highly desired for every traffic flow to share the bandwidth appropriately. This paper proposes a congestion control algorithm for user datagram protocol rate-based layered streaming of scalable video, e.g., 3-D wavelet based scalable video streaming, which provides a variety of video bit rates. This proposed congestion control mechanism, as an extension of explicit control protocol that is a newly proposed congestion control. protocol believed to be superior to transport control protocol, accommodates both window-based and rate-based flows to the heterogeneous network environment which can include wired and wireless channels. This paper further introduces the notion of reserved packet length so that the traffic of layered video can better share the bandwidth of a network by taking account of the max-min fairness with other traffic.
A Weighted Low-rank Matrix Approximation Based Template Matching Scheme for Inter-frame Prediction
(2012)
Huang, Zhi-Long
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
In the field of video coding, inter-frame prediction plays an important role in improving compression efficiency. This task is achieved by finding a predictor for each block such that the residual data can be close to zero as much as possible. For recent video coding standards, motion vectors are required for a decoder to locate the predictors during video reconstruction. Block matching algorithms are usually utilized in the stage of motion estimation to find such motion vectors. For decoder-side inter-frame predictive coding, proper templates are defined and template matching algorithms are used to produce a predictor for each block such that the overhead information to transmit motion vectors can be avoided. However, the conventional criteria of either block matching or template matching algorithms may lead to the generation of worse predictors. To enhance coding efficiency, a weighted low-rank matrix approximation approach for decoder-side inter-frame prediction is proposed in this paper. By finding dominating block candidates as well as their corresponding importance factors, a predictor for each block is formed. Together with mode decision, the coder can switch to a better mode between the motion compensation by block matching and the proposed template matching scheme which can operate at decoder side.
An Error Resilient Multiple Description Video Coder
(2011)
Huang, Yi-Jen
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
The idea of multiple description video coding has been introduced to deal with the issues of bandwidth/path diversity and packet loss due to network congestion and/or error-prone channels which might cause serious quality degradation in video applications such as multimedia streaming and video conferencing services. In this paper, two approaches to description generation are proposed to produce multiple descriptions at higher coding efficiency. One of them is motivated by the multiple description scalar quantizer to reduce the distortion and the other is the coefficient partition in transform domain in order to balance the descriptions better. An estimation mechanism is further proposed to alleviate the drifting problem due to description fluctuation by synchronizing the reference frames at the encoder and the decoder as much as possible. The experiments show that the proposed methods offer substantial improvement at the event of description loss.
Balanced Parallel Scheduling for Video Encoding with Adaptive GOP Structure
(2013-12-01)
Hsiao, Hsu-Feng
;
Wu, Chen-Tsang
;
資訊工程學系
;
Department of Computer Science
Due to the nature of a dynamic group of picture (GOP) structure, parallel scheduling for video encoding becomes challenging. To address this, the balanced frame-level parallel scheduling algorithms are developed. The proposed approaches first determine the frame priority and then the thread priority assignment for scheduling. The concept of the algorithms lies in the analysis of coding complexity, temporal influence, and the required temporal burden to finish coding. To complete the scheduling with the dynamic GOP structure, a block-based abrupt and gradual scene change detection algorithm is also proposed to determine the GOP structure adaptively. The experiments show that the scheduling performance is close to the optimal. In addition, the concept of batch processing is incorporated so that the required buffer can be reduced.
Cross-Layer Error Recovery Scheme for Data Transmission over 4G Wireless Networks
(2015)
Yang, Wen-Chih
;
Chen, Ming-Dao
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
In a video streaming application, quality of experience for a user can be affected by various factors. The video quality can be improved if the effective throughput can be increased at an acceptable packet loss rate and low latency. To reduce the packet loss rate over error-prone channels, a cross-layer error control scheme is developed for heterogeneous data transmission over wireless channels in this paper. The error control scheme utilizes both link-layer automatic repeat request and application-layer channel coding. The automatic repeat request is served as providing a basic level of loss recovery without introducing intolerable delay, while the application-layer channel coding is based on the previously developed layer-aligned multipriority rateless codes to satisfy a desired packet loss rate at a minimal bandwidth overhead. The cross-layer architecture of the error control scheme will control the retransmission timers based on the channel conditions while the control parameters of the layer-aligned multipriority rateless codes will be determined to fulfill the heterogeneous requirement of overall packet loss rate required by an application at a minimal cost. The improved performance of the developed cross-layer scheme is confirmed with the experiments using a real-world 4G system.
Depth Estimation and Video Synthesis for 2D to 3D Video Conversion
(2014-07-01)
Han, Chien-Chih
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
With the recent progress of multi-view devices and the corresponding signal processing techniques, stereoscopic viewing experience has been introduced to the public with growing interest. To create depth perception in human vision, two different video sequences in binocular vision are required for viewers. Those videos can be either captured by 3D-enabled cameras or synthesized as needed. The primary contribution of this paper is to establish two transformation models for stationary scenes and non-stationary objects in a given view, respectively. The models can be used for the production of corresponding stereoscopic videos as a viewer would have seen at the original event of the scene. The transformation model to estimate the depth information for stationary scenes is based on the information of the vanishing point and vanishing lines of the given video. The transformation model for non-stationary regions is the result of combining the motion analysis of the non-stationary regions and the transformation model for stationary scenes to estimate the depth information. The performance of the models is evaluated using subjective 3D video quality evaluation and objective quality evaluation on the synthesized views. Performance comparison with the ground truth and a famous multi-view video synthesis algorithm, VSRS, which requires six views to complete synthesis, is also presented. It is shown that the proposed method can provide better perceptual 3D video quality with natural depth perception.
Disocclusion Compensation Using Adaptive Template Matching for View Synthesis
(2016)
Lu, Chung-Ying
;
Zhan, Yi-Xuan
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
In a multi-view application, view synthesis is required to synthesize unavailable views using texture and depth information of the given views. In this paper, a compensation algorithm based on template matching for partial recovery of disocclusion regions is proposed. The shape of a template is dynamically adjusted, and a new reference mode which mimics pushing dominoes is also proposed. To find an appropriate template region, a histogram-based clustering approach is proposed to distinguish the background from the foreground objects. The improved performance is confirmed using subjective and objective evaluation methods.
Dynamic FEC-Distortion optimization for H.264 scalable video streaming
(2007)
Wen, Wei-Chung
;
Hsiao, Hsu-Feng
;
Yu, Jen-Yu
;
資訊工程學系
;
Department of Computer Science
Forward error correction codes have been shown to be a feasible solution either in application layer or in link layer to fulfill the need of Quality of Service for multimedia streaming over the fluctuant channels. In this paper, we propose FEC-distortion optimization algorithms to efficiently utilize the bandwidth for better video quality. The optimization criterions are based on the unequal error protection by taking account of the error drifting problems from both temporal motion compensation and inter-layer prediction of H.264/MTEG-4 AVC scalable video coding. Also, it can adapt to the content-dependent quality contribution of each video frame in a video layer. Lightweight error-concealment is also incorporated with the proposed algorithms for better H.264 SVC streaming. For some applications where either computation might be the bottleneck or the upper bound of non-decodable probability of each video layer is specified, alternative bandwidth allocation algorithm is provided with the trade-off of slight quality degradation.
Expectation Model and Scheduling for Video Streaming
(2018-01-01)
Chiang, Yu-Jui
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
Resource scheduling is important to improve quality of experience for the services of multimedia streaming. In this paper, a scheduling algorithm of modulation and coding schemes for layered video streaming over wireless broadband networks is proposed. We suggest using the effective throughput, defined as the decompressible data rate, to formulate the utility of a streaming service, because of the stronger relationship between the video quality and the effective throughput. In the proposed method, we address the scheduling problem using a designed expectation model. The model is used to estimate the utility according to the expected contribution of coded blocks. Then, we develop a coarse to fine grained resource scheduling algorithm to solve the scheduling problem effectively.
Fast end-to-end available bandwidth estimation for real-time multimedia networking
(2006)
Wang, Sheng-Shuen
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
Dynamic bandwidth estimation serves as an important basis for performance optimization of real-time distributed multimedia applications. The objective of this paper is to develop a bandwidth estimation algorithm for the fast fluctuated Internet. We analyze the relationship between the one way delay and the dispersion of packets train, and propose an available-bandwidth estimation algorithm which makes use of these two features without requiring administrative access to the intermediate routers along the network path. Instead of binary search or fixed-rate bandwidth adjustment of the probing data as described in literature, we use top-down approach to infer available bandwidth robustly and much more efficiently.
Hierarchical Optimization of Cascading Error Protection Scheme for H.264 Scalable Video Streaming
(2011-03-01)
Wen, Wei-Chung
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
Forward error correction (FEC) coding has been shown to offer a feasible solution to fulfill the need for Quality of Service for multimedia streaming over the fluctuant channels, especially in terms of the reduction of end-to-end delay. In this paper, we propose the Dynamic FEC-Distortion Optimization Algorithm to efficiently utilize the network bandwidth for better visual quality by means of hierarchical coding structure with the cascading error protection scheme. The optimization criteria are based on the unequal error protection by taking account of the error drifting problems from both temporal motion compensation and inter-layer prediction of the H.264/MPEG-4 AVC scalable video coding so that the priorities of each video components can be differentiated for the calculation of the distribution of parity packets. It is shown that the cascading error protection scheme makes the hierarchical structure of error erasure code more efficient. Also, the proposed algorithm works particularly well for fast motion videos and the performance does not depend on accurate estimation of packet loss rate.
Layer-Aligned Multipriority Rateless Codes for Layered Video Streaming
(2014-08-01)
Hsiao, Hsu-Feng
;
Ciou, Yong-Jhih
;
資訊工程學系
;
Department of Computer Science
There exists a multitude of techniques, including automatic repeat request and error correction codes, to minimize data corruption when transmitting over error-prone networks. Streaming of multimedia data can usually withstand a certain level of data loss, yet have strict limitations on the latency tolerance. To enable acceptable reliability of transmission and low transmission latency, the channel coding approach is usually more appealing at the cost of additional bandwidth. In this paper, an N-cycle layer-aligned overlapping structure, which is good for layered data, is proposed. Accordingly, layer-aligned multipriority rateless codes were developed with favorable probabilities to control the protection strength for each layer of the streaming data. The major contribution of this paper is the analytical model developed to predict the failure decoding probabilities for each video layer and it is shown to achieve accurate estimation. A prediction model to estimate the expected decompressible video frames was developed for use with the developed codes for streaming scalable videos. By maximizing the number of expected decompressible video frames, the protection strength of the developed codes can then be determined. Simulation results show that the developed codes are good for streaming layered videos, which are difficult to deal with using traditional rateless codes, with or without unequal error protection.
Layered congestion control for scalable video coding based on the efficient bandwidth inference
(2007)
Wang, Sheng-Shuen
;
Hsiao, Hsu-Feng
;
Lee, Suh-Yin
;
資訊工程學系
;
Department of Computer Science
Scalable video coding allows greater granularity adaptation at bit level to the time-vary heterogeneous networks. In this paper, we propose a congestion control algorithm based on the bandwidth estimation techniques. In order to cooperate with H.264/MPEG-4 AVC SVC extension which can achieve fine granularity of scalability, we design an adaptation strategy to smartly switch to an efficient bandwidth inference mode during the channel probing period according to the one way delay and jitter profile so that the subscription decision of SVC layers can be made quickly to better utilize the network resource. In case of the unavoidable network congestion, we unsubscribe scalable video layers according to the recently received throughput instead of only dropping one layer at a time to rapidly accommodate the streaming service to the channels. The simulations show that the proposed algorithm converges fast to the available bandwidth and can adapt to the fluctuated network efficiently.
Nonuniform Overlapped Rateless Codes for Streaming over Error Prone Channels
(2016)
Chang, Yu-Shu
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
To facilitate reliable data transmission over error prone networks, many error correction techniques have been developed to combat data corruption. The reliability requirement of different applications can vary. The multimedia streaming services can usually withstand a certain level of data loss, but they have a strict limitation on the latency tolerance. In this paper, a section-based protection scheme for video streaming built upon the proposed nonuniform overlapped rateless codes is developed. The overlapped structure can achieve lower probability of failure decoding. An analytic model to predict the unsuccessful decoding probability is also proposed, and a coding strategy for streaming is then constructed. The coding efficiency of the proposed method is significantly improved for delay-sensitive streaming applications.
Perceptual Rate Distortion Optimization for Block Mode Selection in Hybrid Video Coding
(2013)
Huang, Chen-Chou
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
Video compression technologies developed recently have taken advantage of a hybrid approach for better coding efficiency. For example, in the MPEG-4 Advanced Video Coding standard and also the new High Efficiency Video Coding, there are numerous possibilities of motion representation and residual coding options in transform domain. The coding parameters, including block modes, motion vectors, and indices of reference frames, are often correlated with each other in spatial and temporal directions. The selection of those coding parameters can affect the coding efficiency greatly. In many of the state-of-the-art solutions, the selection of coding options is based on the minimization of a Lagrangian cost function which is constructed of conventional distortion function and produced data rate. However, the conventional distortion function measured as either sum of absolute difference or mean square error does not often represent the observed deficiency in reconstructed videos. In this paper, a perceptual rate distortion optimization algorithm for block mode selection is developed, and the content-dependent Lagrangian multiplier is then derived. The just-noticeable distortion is used in the proposed method such that perceptual quality can be considered. The simulation results show that the rate-perceptual distortion performance is improved substantially without sacrificing the conventional rate-distortion performance.
Priority-Base Retransmission Scheduling for Automatic Repeat Request over Wireless Networks
(2015)
Chang, Chun-Yi
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
When data is transmitted over error-prone channels, portions of data may be corrupted due to many reasons, including multipath fading, interference, signal attenuation, and shadowing. Even if there is only one bit uncorrectable, the data block or packet will be considered as invalid, and will be discarded. The Automatic Repeat reQuest (ARQ) is an important mechanism to recover lost packets transmitted over either unstable or congested networks. An ARQ mechanism is also implemented in the 4G wireless networks on the link layer, where a receiver can request its sender to retransmit invalid packets. However, retransmission requires additional bandwidth, and it can result in longer delay. For some applications, such as layered video streaming, the utility of each packet of a stream can be different. The QoS requirements of different flows can vary as well. In this paper, a priority-based scheduling method for retransmission is proposed for wireless channels with limited bandwidth resources. For intra-flow traffic, which can be treated as several link-layer flows, the priorities can be assigned to the flows using different QoS classes. A dynamic utility function is developed for the scheduling of invalid packets such that the overall utility can be improved. The amount of resources for retransmission is also regulated dynamically such that each flow can occupy reasonable bandwidth. The improved performance is verified with simulation results.
QoE-Driven Performance Analysis of Cloud Gaming Services
(2014-01-01)
Wen, Zi-Yi
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
With the popularity of cloud computing services and the endorsement from the video game industry, cloud gaming services have emerged promisingly. In a cloud gaming service, the contents of games can be delivered to the clients through either video streaming or file streaming. Due to the strict constraint on the end-to-end latency for real-time interaction in a game, there are still challenges in designing a successful cloud gaming system, which needs to deliver satisfying quality of experience to the customers. In this paper, the methodology for subjective and objective evaluation as well as the analysis of cloud gaming services was developed. The methodology is based on a non-intrusive approach, and therefore, it can be used on different kinds of cloud gaming systems. There are challenges in such objective measurements of important QoS factors, due to the fact that most of the commercial cloud gaming systems are proprietary and closed. In addition, satisfactory QoE is one of the crucial ingredients in the success of cloud gaming services. By combining subjective and objective evaluation results, cloud gaming system developers can infer possible results of QoE levels based on the measured QoS factors. It can also be used in an expert system for choosing the list of games that customers can appreciate at a given environment, as well as for deciding the upper bound of the number of users in a system.
QoS-Driven Optimization for Video Streaming Using Layer-Aligned Multipriority Rateless Codes
(2015)
Hung, Lien-En
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
Forward error correction techniques are commonly used for delay-sensitive video streaming applications. Streaming of compressed videos can usually withstand a certain level of packet loss, depending on the compression methods and error concealment techniques. In this paper, a QoS-driven optimization algorithm is proposed for the streaming of scalable videos using layer-aligned multipriority rateless codes over wideband wireless channels. Layer-aligned multipriority rateless codes provide controllable protection strengths for different layers of a scalable video. The coding is based on the N-cycle layer-aligned overlapping structure where each layer of scalable videos is protected by N windows. The objective of the proposed method in this paper is to decide a coding strategy by minimizing the total data rate for scalable video streaming, while fulfilling user\'s expectation of desired recovery rates of different video layers. The developed QoS-driven optimization algorithm is based on the prediction model of layer-aligned multipriority rateless codes. Its better performance is verified with simulations and real-world experiments over 4G wireless networks.
RGBD Salient Object Detection using Spatially Coherent Deep Learning Framework
(2018-01-01)
Huang, Posheng
;
Shen, Chin-Han
;
Hsiao, Hsu-Feng
;
資訊工程學系
;
Department of Computer Science
In this paper, a learning based salient object detection method for RGBD images is introduced. With the assistance of depth information, the silhouette features of an object can be retrieved primarily, and it can lead to better detection of salient objects. In addition, many recent works still rely on some image post-processing methods to improve their performance. We develop a more efficient end-to-end model with a modified design of loss function used in our training network. The design of the new loss function is to increase the spatial coherence of detected salient objects. From the evaluation results, the proposed approach shows good performance compared with the methods that are considered to be state-of-the-art.