Constructing hysteretic memory in neural networks

Abstract

Hysteresis is a unique type of dynamic, which contains an important property, rate-independent memory. In addition to other memory-related studies such as time delay neural networks, recurrent networks, and reinforcement learning, rate-independent memory deserves further attention owing to its potential applications. In this work, we attempt to define hysteretic memory (rate-independent memory) and examine whether or not it could be modeled in neural networks. Our analysis results demonstrate that other memory-related mechanisms are not hysteresis systems. A novel neural cell, referred to herein as the propulsive neural unit, is then proposed. The proposed cell is based on a notion related the submemory pool, which accumulates the stimulus and ultimately assists neural networks to achieve model hysteresis. In addition to training by backpropagation, a combination of such cells can simulate given hysteresis trajectories.