異質無線網路之可擴充性多媒體群播服務---運用在異質無線網路多方視訊廣播服務之可調視訊編解碼最佳化與網路介面設計

Abstract

本計劃之目標為針對整合計畫「異質無線網路之可擴充性多媒體群播應用(Scalable Multimedia Multicasting Service Over Heterogeneous Wireless Networks)」，進行可調視訊 編碼最佳化，與開發可調視訊編碼之網路介面。 隨著視訊編碼與傳輸技術的快速發展，可調性多媒體群播(意指在不同的傳輸通道 中，發送單一可調位元流給予不同使用者)對於遍地即時之多媒體通訊應用而言，已成 為一個可行並實際的做法。可以預期的是，在不久的將來，使用者不僅可以在不同多媒 體平台上觀賞視訊，並且能夠藉由不同網路協定與連結方式來存取伺服器。為了能夠支 援多樣的使用者需求與不斷變化的網路環境，ISO/IEC MPEG 與ITU-T 組織了一個JVT 團隊來制定可調視訊編碼標準技術，其目的是以單一的視訊位元流提供空間上 (Spatial)、時間上(Temporal)以及畫質精細(SNR)可調的特性。目前，可調視訊編碼提供 了一個可完全相容於MPEG-4 AVC/H.264 之基本層，以及一個可以被即時捨棄與切割來 達到不同時間，空間，與畫質解析度之可調層。 儘管可調視訊編碼能夠提供諸多可調性特色，然而卻也在伺服器端(Servers)、媒體 轉接器(Media Relay)與接收端(Receivers)帶來不同的困難與挑戰。對伺服器而言，可調 視訊編碼之高複雜度對於即時應用是一大挑戰。初步分析指出，目前最新的參考軟體即 使在最快的機器上作編碼，其速度距離即時條件仍有1000 倍之差。另外，連接伺服器 端與接收端之媒體轉接器必須持續不斷的隨著不同接收端需求與網路環境作適應性的 位元流調整。因此，為了能夠讓網路封包作即時的應用，伺服器必須適當地在封包內加 入優先權資訊，藉以輔助媒體轉接器對位元流的擷取。儘管目前可調視訊編碼已在網路 擷取層(NAL)中提供簡易的優先權資訊，但是現有方法並無法同時滿足不同接收端之需 求。因此，在此方面仍有許多研究空間。再者，對於低階之可攜式接收器，媒體轉接器 甚至必須提供從可調視訊編碼轉換至MPEG-4 AVC/H.264 編碼之能力。因此，在媒體轉 接器上，另一個值得探討的問題是如何兼顧轉碼複雜度與轉碼品質。而另一方面在接收 端，其主要議題則在於如何根據影像特性與撥放器能力，有效地利用已接收之位元流作 最佳解碼，並在錯誤發生的狀況下提供優雅解碼品質。 根據上述之問題，在本計畫中我們將研究並開發適用在異質無線網路環境下之可擴 充性多媒體群播應用相關技術。同時所提之技術將在展示系統與最新的參照軟體中驗 證。以下是各年度之計畫目標：第一年 -- (1)發展快速演算法，改進目前可調視訊編解 碼之速度，(2)根據可調視訊之傳輸需求，發展網路介面；第二年 -- (3)根據可調視訊編 碼標準與MPEG-4 AVC/H.264標準兩者之語法定義，發展轉碼技術，以及(4)最佳化位元 流擷取技術；第三年 -- (5)提供可調視訊解碼錯誤容忍與(6)錯誤隱藏機制。

This subproject aims at optimizing the scalable extension of MPEG-4 AVC/H.264 and developing associated network interface for the scalable multicasting service over heterogeneous wireless networks. With the rapid advances of video coding and transport technologies, the scalable multimedia multicasting ─ the ability to deliver multimedia data streams from a single encoding source to a wide range of receivers through different communication channels ─is now a feasible and practical solution to pervasive real-time multimedia communication and entertainment. It can be foreseen that the users in the near future can not only watch the live videos on various multimedia platforms, but also access the services by different types of connections. In order to support clients with diverse capabilities and varying network conditions, ISO/IEC MPEG and ITU-T form a Joint Video Team (JVT) to develop a scalable video coding (SVC) technology that uses single bitstream to provide multiple spatial, temporal, and quality (SNR) resolutions. It provides a H.264/AVC-compatible base layer and a fully scalable enhancement layer, which can be truncated and extracted on-the-fly to obtain a preferred spatial-temporal and quality resolution. While the SVC offers many attractive features, it comes with different challenges at servers, media relays, and receivers. At servers, the highly computational complexity of SVC encoder puts up a barrier in the real-time applications. The preliminary analysis has shown that the up-to-date reference software is 1000x slower than real-time when executed on the state-of-the-art machines with combined scalability enabled. Besides, as the bridges connecting servers and receivers, the media relays need to constantly adapt the SVC bitstream for matching device capabilities and network conditions. To allow the packets be parsed in a real-time manner, servers must embed the priority information to facilitate the bitstream extraction. Although the current algorithm provides simple priority identification in the SVC network abstraction layer (NAL), it cannot simultaneously fulfill the requirements of different devices, and thus, leaving plenty of research spaces for further improvement. In addition, the media relay may sometimes perform the transcoding from SVC to MPEG-4 AVC/H.264 for low-complexity portable devices. Thus, how one can minimize the complexity while maintaining the quality of transcoding is another issue at media relays. On the other hand, at receivers, the problem is how to smartly utilize the received bitstream according to the video characteristic and display capability so as to optimize the decoded quality and provide graceful degradation in the presence of packet loss. In terms of the problems raised, in this project, we will accordingly investigate and develop related technologies, and validate the proposed schemes with the up-to-date reference software and in the demonstration system. The yearly goals of this project include: Year 1 -- (1) design fast algorithms for SVC encoder and (2) develop network interfaces for the transport of SVC. Year 2 -- (3) develop SVC to MPEG-4 AVC/H.264 transcoder based on syntax direct mapping and (4) provide rate-distortion optimized bitstream extraction scheme. Year 3 – (5) offer SVC decoder with graceful degradation and (6) propose error concealment scheme for SVC.