A 3D predictive visual tracker for tracking multiple moving objects with a stereo vision system

Abstract

This paper presents a 3D feature-based visual tracker for tracking multiple moving objects by using a predictor that first partitions 3D features into different common-motion clusters and then predicts the motion of each cluster with Kalman filters. The 3D features are computed from a sequence of stereo images by combining two 2D temporal matching modules and one stereo correspondence module. To partition the 3D features into different common-motion clusters, we propose a RANSAC-based clustering method by using rigid body consensus which assumes that all the extracted 3D features on a rigid body have the same 3D motion. By using the motion estimates obtained with the RANSAC-based method as the measurements, we are able to use linear Kalman filters to predict the motion of each cluster, and then, to predict the next position of each 3D feature. Preliminary experiments showed that the proposed 3D predictive visual tracker can serve as a robust 3D feature tracker for an active stereo vision system.