Anomalous Behavior Detection

New environments and threats require real-time adaptive analytic tools to continuously process multi-sensor data, extract useful information and proactively alert security personnel.

The challenge is immense – thousands of sensors streaming data, millions of assets and people to protect and a constantly evolving threat profile.

Human monitoring is effective in recognizing threatening events, but costly, inefficient and fatiguing. As a result of resource limitations or monitoring fatigue, many events go undetected. Until recently, state-of-the-art computer processing of security video has only been capable of detecting behavior based on prior knowledge of the environment and the exact anticipated threat. These systems exhibit a devolving effectiveness (probability of detection) by being either over-trained or by becoming out of date with the monitored environment.

SIG's research has addressed the development of algorithms for adaptive processing of multi-sensor data, employing feedback (Lifelong Learning) to optimize the linkage between observed data and sensor control. This multimodal adaptive system is applicable for intelligence, surveillance, and reconnaissance (ISR) in general environments, addressing base and port security, as well as urban and suburban sensing during wartime and peace-keeping operations. Of significant importance for current and anticipated DHS activities, SIG technology is designed to detect asymmetric threats, with the goal of recognizing unusual behavior or activities. Technologies and systems developed through our research have been designed for semi-automated scene awareness, with the objective of recognizing behavior that appears atypical for a given object class (e.g. atypical object motion, and dynamic characteristics of people and vehicles).

Technology

Probability Modeling - The foundation for SIG's Analytic Framework is based on the use of probability theory to consistently describe information at all levels of the framework. SIG’s novel approach applies probability model(s) to every pixel in an image to derive background and foreground color models, the association of pixels to shapes, the association of shapes to objects and ultimately to determine whether the observed behavior for a specific object has a low likelihood of being consistent with previously observed behavior exhibited by similar objects in the scene.

Probability modeling is also the key to the other elements in the Analytic Framework which include Multi-hypothesis and Adaptive Learning. Multi-hypotheses tracking is a ground-breaking concept which enables the Analytic Framework to simultaneously account for many possible instantiations of objects, trajectories and behaviors, each with different likelihoods based on training data and/or prior knowledge in real-time. Most other video analytic approaches force a maximum likelihood fit after each frame and purge all remaining data and evidence. What if a probabilistic approach determines that in a video frame an object being tracked is nearly equally likely to be following one of three tracks? Traditional approaches pick one track or abandon the data in hopes that the next frame is more informative. Using SIG's multi-hypothesis technology retains information in multiple frames, calculating multiple possible tracks and utilizing all available data in an informative way. This approach enables better event detection performance with fewer false alarms.

Adaptive Learning is integrated throughout the Analytic Framework and allows the system to operate when the environment, objects and behavior are unknown, a priori. Normal behavior is learned from the observed environment, objects and their behaviors. Anomalous behavior by a specific class of object [behavior which has never previously been observed] may be allowable, but is cued to the analyst for labeling and is re-incorporated in the lifelong learning of the system. Analyst cueing is integral to Adaptive Learning and effectively adds the "human-in-the-loop" to exploit understanding and contextual information. Classes of objects (people, vehicles, etc.) are also identified and labeled through the learning process.

Sensor Management Agent (SMA) is the focal point for risked-based optimal decisions. The SMA integrates the feedback link with the "Analyst-in-the-Loop" and drives optimal classification performance for a given sensing cost. The SMA can, for example, control PTZ video sensors, managing them to optimally collect more information about a scene or object of interest while balancing the value of new information against a specific probability of anomalous behavior in the temporarily un-observed scene. Sensor fusion for "global" tracking of objects and behaviors is also performed in the SMA.

The SIG Analytic Framework has proven to be very robust in dynamic environments (motion in the background), with static and dynamic occlusions and with low resolution poor quality video.