标题: 张量式类神经网路应用于多通路资料分类
Tensor Neural Networks for Multi-way Data Classification
作者: 包苡廷
Bao, Yi-Ting
简仁宗
Chien, Jen-Tzung
电信工程研究所
关键字: 多通路资料;张量分解;类神经网路;深层学习;模式识别;影像辨识;multi-way data;tensor factorization;neural network;deep learning;pattern classification;image recognition
公开日期: 2015
摘要: 在机器学习与讯号处理的领域中,人们对于实用资讯系统中到处可见的多通路资料(multi-way or multi-channel data)的分析与研究兴趣与日俱增,使得张量分解(tensor factorization)与分类成为重要的研究议题。在传统的类神经网路(neural network)中,分类器是利用一组输入向量(input vector)进行模型训练,使用训练出来的模型进行测试资料的预测与判断。过去多通道的资料常被展开成高维度(high-dimensional)的向量来对模型进行训练。但不同通路上相邻(neighboring)时间(temporal)与空间(spatial)的关联资讯也遗失在分类器的训练过程中,使得分类的效能受到了限制,另外也需要更大量的参数模型来建构并表示复杂的资料型态与特性。本篇论文发展全新的张量式类神经网路分类器(tensor classification network)透过张量分解(tensor factorization)与类神经网路分类的结合,它可以萃取出多通道的特征(multi-way feature)并进行分类。张量式类神经网路有效整合塔克拆解(Tucker decomposition)与传统单通路(one-way)类神经网路分类器,传统类神经网路的仿射变换(affine transformations)因此被张量转换(tensor transformation)所取代。我们延伸传统以向量为主之单通路类神经网路成为具一般化特性之张量式多通路类神经网路,透过张量空间中输入张量(input tensor)到潜在张量(latent tensor)的映射,让多通道的时空资讯能精简的保留在训练出来的张量参数中,并且发展张量式倒传递演算法来有效率的建立张量式类神经网路。具高效能的张量式映射使得我们可以取得一个非常简洁的分类器,同时训练计算所花的时间也较传统类神经网路更为快速。在本篇研究中,对于张量式类神经网路与向量式类神经网路及张量拆解的比较也进行了探讨。在影像辨识评估的实验中发现,相对于向量式类神经网路,张量式类神经网路可以达到同等甚至是更好的正确率但却只需要极少的参数量,同时,训练的计算成本也降低很多。
The growing interests in multi-way or multi-channel data analysis have made the tensor factorization and classification a crucial issue in the areas of signal processing and machine learning. Conventionally, the neural network (NN) classifier is estimated from a set of input vectors or one-way observations. The multi-way observations are unfolded as the high-dimensional vectors for model training. As a result, the classification performance is constrained because the correlation or neighboring information in temporal or spatial domains among different ways is lost in the trained NN classifier. More parameters are required to learn the complicated data structure from multiple ways, trials or channels. This study presents a new tensor classification network (TCN) which combines tensor factorization and NN classification for multi-way feature extraction and classification. The proposed TCN can be viewed as a generalization of NN classifier for multi-way data classification where Tucker decomposition and nonlinear operation are performed in each hidden unit. Using this approach, the affine transformation in conventional NN is replaced by the tensor transformation. We generalize from vector-based NN classifier to tensor-based TCN where the multi-way information in temporal, spatial or other domains is preserved through projecting the input tensors into latent tensors. The projection over tensor spaces is efficiently characterized so that a very compact classifier could be achieved. The proposed TCN does not only construct a compact model but also reduce the computation time in comparison with the traditional NN classifier. The tensor error backpropagation algorithm is developed to efficiently establish a tensor neural network. Experimental results on image recognition over different datasets demonstrate that TCN could attain comparable or even better classification performance but with very few parameters and the reduced computation cost when compared with the traditional NN classifier.
URI: http://140.113.39.130/cdrfb3/record/nctu/#GT070260246
http://hdl.handle.net/11536/127260
显示于类别:Thesis