Tracking Through Occlusions

Dr McElory Hoffmann (PhD), Emmerentia Jacobs (MSc), Eugene Pretorius (MSc), Robbie Vos (MSc), M Pistorius (MSc), HB de Villiers (MSc), Pieter Hotzhausen (MSc)


Tracking is one of the fundamental problems of computer vision with a large number of applications from surveillance to gesture recognition, radar, and much else besides.

Tracking a large, rigid object against a uniform background is not that hard, but tracking small deformable objects against cluttered backgrounds is hard. Tracking through occlusions is challenging.


Since the general problem is hard, the emphasis has shifted towards tracking known objects for which the deformations can be learned. If only shape information is used, you are probably doing something akin to Active Contours. Although this is sufficiently robust for many applications, there is more information that can be exploited, namely texture of the object. Since this can also change during the motion, typical due to lighting changes, the changes should also be learned. Combining shape and texture in a specific way, is known as Active Appearance Models (AAMs).


AAMs are more robust but does not handle occlusions particularly well. The first video shows a typical break-down.

An occlusion basically means that one does not receive any observations of the object during the occlusion. One approach is to combine a Particle Filter with the AAM. It provides estimates of the expected observations while the occlusion lasts, based on all the observations and motion up to that time. When observations again become available after the occlusion, these are then used to update the estimates calculated during the occlusion. Provided the estimates and resumed observations do not differ too much (are within the observation error bounds), the system is able to recover as shown in the video.

Here is an example where the recovery after occlusion breaks down right at the end.

Courses offered
Indigenous Plants