Reconstruct Dynamic Systems from Large-Scale Open Data

Abstract

With the prosperity of e-commerce, on-line vendors use recommendation systems in different fields. Classic recommendation algorithms are designed assuming that data is stationary and will not change over time. However, since the scale and variability of data are growing gradually, these methods will encounter the issues of the memory deficient and the out-of-date model, which degrade the recommendation accuracy intensively. In addition, retraining the whole model for every new arrival record results in high complexity. In this paper we propose a light-weight adaptive updating method to overcome these issues. Comparing with the explicit feedback recommendation, which asks the customers to express their opinions on the recommended items, the implicit feedback recommendation is easier to collect and non-intrusive way. However, the dynamic time-variant system with implicit feedback has not been seen in the literature. In this paper, we propose a real-time incremental updating algorithm (RI-SGD) to deal with time-variant systems based on the implicit feedback. We compare our method with methods that retraining the whole model and show that our method costs less than 1% of the retraining time with a competitive accuracy.