Browsing by Author "Wang, YJ"

Now showing 1 - 7 of 7

A second-order learning algorithm for multilayer networks based on block Hessian matrix
(1998-12-01)
Wang, YJ
;
Lin, CT
;
電控工程研究所
;
Institute of Electrical and Control Engineering
This article proposes a new second-order learning algorithm for training the multilayer perceptron (MLP) networks. The proposed algorithm is a revised Newton's method. A forward-backward propagation scheme is first proposed for network computation of the Hessian matrix, H, of the output error function of the MLP. A block Hessian matrix, H-b, is then defined to approximate and simplify H. Several lemmas and theorems are proved to uncover the important properties of H and H-b, and verify the good approximation of H-b to H; H-b preserves the major properties of H. The theoretic analysis leads to the development of an efficient way for computing the inverse of H-b recursively. In the proposed second-order learning algorithm, the least squares estimation technique is adopted to further lessen the local minimum problems. The proposed algorithm overcomes not only the drawbacks of the standard backpropagation algorithm (i.e. slow asymptotic convergence rate, bad controllability of convergence accuracy, local minimum problems, and high sensitivity to learning constant), but also the shortcomings of normal Newton's method used on the MLP, such as the lack of network implementation of H, ill representability of the diagonal terms of H, the heavy computation load of the inverse of H, and the requirement of a good initial estimate of the solution (weights). Several example problems are used to demonstrate the efficiency of the proposed learning algorithm. Extensive performance (convergence rate and accuracy) comparisons of the proposed algorithm with other learning schemes (including the standard backpropagation algorithm) are also made. (C) 1998 Elsevier Science Ltd. All rights reserved.
A second-order learning algorithm for multilayer networks based on block Hessian matrix
(1998-12-01)
Wang, YJ
;
Lin, CT
;
電控工程研究所
;
Institute of Electrical and Control Engineering
This article proposes a new second-order learning algorithm for training the multilayer perceptron (MLP) networks. The proposed algorithm is a revised Newton's method. A forward-backward propagation scheme is first proposed for network computation of the Hessian matrix, H, of the output error function of the MLP. A block Hessian matrix, H-b, is then defined to approximate and simplify H. Several lemmas and theorems are proved to uncover the important properties of H and H-b, and verify the good approximation of H-b to H; H-b preserves the major properties of H. The theoretic analysis leads to the development of an efficient way for computing the inverse of H-b recursively. In the proposed second-order learning algorithm, the least squares estimation technique is adopted to further lessen the local minimum problems. The proposed algorithm overcomes not only the drawbacks of the standard backpropagation algorithm (i.e. slow asymptotic convergence rate, bad controllability of convergence accuracy, local minimum problems, and high sensitivity to learning constant), but also the shortcomings of normal Newton's method used on the MLP, such as the lack of network implementation of H, ill representability of the diagonal terms of H, the heavy computation load of the inverse of H, and the requirement of a good initial estimate of the solution (weights). Several example problems are used to demonstrate the efficiency of the proposed learning algorithm. Extensive performance (convergence rate and accuracy) comparisons of the proposed algorithm with other learning schemes (including the standard backpropagation algorithm) are also made. (C) 1998 Elsevier Science Ltd. All rights reserved.
Multi-gigabit serial link transmitter- off-chip and on-chip
(2005)
Jou, SJ
;
Lin, CH
;
Chen, CN
;
Wang, YJ
;
Hsiao, JY
Multi-Gbps serial link transmitter for both off-chip and on-chip transmission are presented. For off-chip transmission, a new pre-emphasis design methodology and circuits for a 4/2 PAM transmitter over cable are proposed. A test chip of transmitter with pre-emphasis, PLL circuit and on-chip termination resistors is implemented using tsmc 0.18 urn CMOS process. The measurement results of 1015 Gbps (4/2 PAM) are carried out over 5 meter (m) long cable and is in agreement with our analysis and simulation results. For on-chip transmission, SerDes based serial link architecture is used in on-chip application. Using tsmc 0.13 um CMOS process, the operation speed and power consumption are 5 Gbps and 3.2 mW respectively with the interconnect area is half of parallel architecture.
Partition of a set of integers into subsets, with prescribed sums
(2005-12-01)
Chen, FL
;
Fu, HL
;
Wang, YJ
;
Zhou, JQ
;
應用數學系
A nonincreasing sequence of positive integers < m(1), m(2),(...), m(k)> is said to be n-realizable if the set I-n = {1, 2,(...), n} can be partitioned into k mutually disjoint subsets S-1, S-2,(...), S-k such that Sigma(x is an element of Si) x = m(i) for each 1. <= i <= k. In this paper, we will prove. that a nonincreasing sequence of positive integers < m(1), m(2),(...),m(k)> is n-realizable under the. conditions that Sigma(i=1)(k) m(i) = ((n+1)(2)) and m(k-1) >= n.
Recurrent learning algorithms for designing optimal controllers of continuous systems
(2000-09-01)
Wang, YJ
;
Lin, CT
;
電控工程研究所
;
Institute of Electrical and Control Engineering
This paper proposes the recurrent learning algorithm for designing thr controllers of continuous dynamical systems in the optimal control problems. The designed controllers are in the form of unfolded recurrent neural networks embedded with physical laws coming from the classical control techniques. The proposed learning algorithm is characterized by its double-forward-recurrent-loops structure for solving both the temporal recurrent and the structure recurrent problems. The first problem is resulted from the nature of general optimal control problems, where the objective functions are often related to (evaluated at) some specific (instead of all) time steps or system states only, causing missing Learning signals at some time steps or system states. The second problem is due to the high-order discretization of the continuous systems bg the Runge-Kutta method that we perform to increase the control accuracy. This discretization transforms the system into several identical subnetworks interconnected together, like a recurrent neural network expanded in the time axis. Two recurrent learning algorithms with different convergence properties are derived; the first- and second-order learning algorithms. The computations of both algorithms are local and performed efficiently as network signal propagation. We also propose two new nonlinear controller structures for two specific control problems:1) two-dimensional (2-D) guidance problem and 2) optimal PI control problem. Under the training of the proposed recurrent learning algorithms, these two controllers can be easily tuned to be suboptimal for given objective functions. Extensive computer simulations have shown the optimization and generalization abilities of the controllers designed bg the proposed learning scheme.
Runge Kutta Neural Network for identification of continuous systems
(1998)
Wang, YJ
;
Lin, CT
;
電控工程研究所
;
Institute of Electrical and Control Engineering
This paper proposes Range Kutta Neural Networks (RKNNs) for identification of continuous-time nonlinear systems.(1) These networks are constructed according to the Runge Kutta approximation method. The RKNNs can thus precisely model continuous-line systems and do long-term prediction of system state trajectories. Since the RKNNs model continuous-time systems, they can incorporate available continuous relationship (physical laws) of the identified systems into their structures directly. Also, they are insensitive to the size of sampling interval in prediction. We also show theoretically the superior generalization and long-term prediction capability of the RKNNs over the normal neural networks. A class of novel recursive least square (RLS) algorithms, called nonlinear recursive least square (NRLS) learning algorithms, are developed for the RKNNs. Computer simulations demonstrate the proved properties of the RKNNs.
Runge-Kutta neural network for identification of dynamical systems in high accuracy
(1998-03-01)
Wang, YJ
;
Lin, CT
;
電控工程研究所
;
Institute of Electrical and Control Engineering
This paper proposes the Runge-Kutta neural networks (RKNN's) for identification of unknown dynamical systems described by ordinary differential equations (i.e., ordinary differential equation or ODE systems) in high accuracy. These networks are constructed according to the Runge-Kutta approximation method. The main attraction of the RKNN's is that they precisely estimate the changing rates of system states (i.e., the right-hand side of the ODE (x) over dot = f(x)) directly in their subnetworks based on the space-domain interpolation within one sampling interval such that they can do long-term prediction of system state trajectories. We show theoretically the superior generalization and long-term prediction capability of the RKNN's over the normal neural networks. Two types of learning algorithms are investigated for the RKNN's, gradient-and nonlinear recursive least-squares-based algorithms. Convergence analysis of the learning algorithms is done theoretically, Computer simulations demonstrate the proved properties of the RKNN's.