Browsing by Author "Alajaji, F"

Now showing 1 - 9 of 9

A note on the Poor-Verdu upper bound for the channel reliability function
(2002-01-01)
Alajaji, F
;
Chen, PN
;
Rached, Z
;
電信工程研究所
;
Institute of Communications Engineering
In an earlier work, Poor and Verdu established an upper bound for the reliability function of arbitrary single-user discrete-time channels with memory. They also conjectured that their bound is tight for all coding rates. In this note, we demonstrate via a counterexample involving memoryless binary erasure channels (BECs) that the Poor-Verdu upper bound is not tight at low rates. We conclude by examining possible improvements to this bound.
A rate-distortion theorem for arbitrary discrete sources
(1998-07-01)
Chen, PN
;
Alajaji, F
;
電信工程研究所
;
Institute of Communications Engineering
A rate-distortion theorem for arbitrary (not necessarily stationary or ergodic) discrete-time finite-alphabet sources is given, This result, which provides the expression of the minimum epsilon-achievable fixed-length coding rate subject to a fidelity criterion, extends a recent data compression theorem by Steinberg and Verdu.
Csiszar's cutoff rates for arbitrary discrete sources
(2001-01-01)
Chen, PN
;
Alajaji, F
;
電信工程研究所
;
Institute of Communications Engineering
Csiszar's forward beta -cutoff rate (given a fixed beta > 0) for a discrete source is defined as the smallest number Ro such that for every R > R-0, there exists a sequence of tired-length codes of rate R with probability of error asymptotically vanishing as e(-n beta (R-R0)). For a discrete memoryless source (DMS), the forward beta -cutoff rate is shown by Csiszar [6] to be equal to the source Renyi entropy. An analogous concept of reverse beta -cutoff rate regarding the probability of correct decoding is also characterized by Csiszar in terms of the Renyi entropy. In this work, Csiszar's results are generalized by investigating the beta -cutoff rates for the class of arbitrary discrete sources with memory. It is demonstrated that the limsup and liminf Renyi entropy rates provide the formulas for the forward and reverse beta -cutoff rates, respectively. Consequently, new fixed-length source coding operational characterizations for the Renyi entropy rates are established.
Csiszar's cutoff rates for the general hypothesis testing problem
(2004-04-01)
Alajaji, F
;
Chen, PN
;
Rached, Z
;
電信工程研究所
;
Institute of Communications Engineering
In [6], Csiszar established the concept of forward beta-cutoff rate for the error exponent hypothesis testing problem based on independent and identically distributed (i.i.d.) observations. Given beta < 0, he defined the forward beta-cutoff rate as the number R-0 greater than or equal to 0 that provides the best possible lower bound in the form beta(E - (R) over bar (0)) to the type 1 error exponent function for hypothesis testing where 0 < E < R-0 is the rate of exponential convergence to 0 of the type 2 error probably. He then demonstrated that the forward beta-cutoff rate is given by where D1/(1-beta)(Xparallel to(X) over bar) denotes the Renyi alpha-divergence [19], alpha > 0, alpha not equal 1. Similarly, for 0 < beta < 1, Csiszar also established the concept of reverse beta-cutoff rate for the correct exponent hypothesis testing problem. In this work, we extend Csiszar's results by investigating the forward and reverse beta-cutoff rates for the hypothesis testing between two arbitrary sources with memory. We demonstrate that the lim inf Renyi a-divergence rate provides the expression for the forward beta-cutoff rate. We also show that if the log-likelihood large deviation spectrum admits a limit, then the reverse beta-cutoff rate equals the liminf a-divergence rate, where alpha and 1/1-beta and 0 < beta < beta(max), where beta(max) is the largest beta < 1 for which the lim inf 1/1-beta-divergence rate is finite. For beta(max) less than or equal to beta < 1, we show that the reverse cutoff rate is in general only upper-bounded by the lim inf Renyi divergence rate. Unlike in [4], where the alphabet for the source coding cutoff rate problem was assumed to be finite, we assume arbitrary (countable or continuous) source alphabet. We also provide several examples to illustrate our forward and reverse beta-cutoff rates results and the techniques employed to establish them.
General formulas for Csiszar's source coding cutoff rates
(2000)
Chen, PN
;
Alajaji, F
;
電信工程研究所
;
Institute of Communications Engineering
In this work, Csiszar's fixed-length source coding beta -cutoff rates are investigated for the class of arbitrary discrete sources with memory. It is demonstrated that the limsup and liminf Renyi entropy rates provide the formulas for the forward and reverse beta -cutoff rates, respectively. Consequently, new fixed-length source coding operational characterizations for the Renyi entropy rates are established.
Generalized source coding theorems and hypothesis testing: Part I - Information measures
(1998-05-01)
Chen, PN
;
Alajaji, F
;
電信工程研究所
;
Institute of Communications Engineering
Expressions for epsilon-entropy rate, epsilon-mutual information rate and epsilon-divergence rate are introduced. These quantities, which consist of the quantiles of the asymptotic information spectra, generalize the inf/supentropy/information/divergence rates of Han and Verdu. The algebraic properties of these information measures are rigorously analyzed, and examples illustrating their use in the computation of the E-capacity are presented. In Part II of this work, these measures are employed to prove general source coding theorems for block codes, and the general formula of the Neyman-Pearson hypothesis testing type-II error exponent subject to upper bounds on the type-I error probability.
Generalized source coding theorems and hypothesis testing: Part II - Operational limits
(1998-05-01)
Chen, PN
;
Alajaji, F
;
電信工程研究所
;
Institute of Communications Engineering
In light of the information measures introduced in Part I, a generalized version of the Asymptotic Equipartition Property (AEP) is proved, General fixed-length data compaction and data compression (source coding) theorems for arbitrary finite-alphabet sources are also established, Finally, the general expression of the Neyman-Pearson type-II error exponent subject to upper bounds on the type-I error probability is examined.
On the optimistic capacity of arbitrary channels
(1998)
Chen, PN
;
Alajaji, F
;
電信工程研究所
;
Institute of Communications Engineering
A formula for the optimistic capacity of arbitrary channels is established. It is shown to equal the supremum, over all input processes, of the input-output zero-sup-information rate. A general expression for optimistic epsilon-capacity is also provided.
Optimistic Shannon coding theorems for arbitrary single-user systems
(1999-11-01)
Chen, PN
;
Alajaji, F
;
電信工程研究所
;
Institute of Communications Engineering
The conventional definitions of the source coding rate and or channel capacity require the existence of reliable codes for all sufficiently large block lengths. Alternatively, if it is required that good codes exist for infinitely many block lengths, then optimistic definitions of source coding rate and channel capacity are obtained. In this work, formulas for the optimistic minimum achievable fixed-length source coding rate and the minimum epsilon-achievable source coding rate for arbitrary finite-alphabet sources are established, The expressions for the optimistic capacity and the optimistic epsilon-capacity of arbitrary single-user channels are also provided. The expressions of the optimistic source coding rate and capacity are examined for the class of information stable sources and channels, respectively, Finally, examples for the computation of optimistic capacity are presented.