Mistral Solutions shares an insight on the role of Artificial Intelligence

Installation and use of CCTV Cameras for security & surveillance is a no-brainer. Cameras are considered a fundamental commodity for setting up any surveillance infrastructure, but at the same time, 24x7 monitoring of hundreds or thousands of video feeds by operators doesn’t serve the purpose of providing proactive surveillance and quick response to breaches.

Software-based Video Content Analytics (VCA) provides a certain level of reprieve by raising real-time alerts for a few standard breaches like left baggage, motion detection, etc., but the in-accuracy and false-positives far outweighed the potential benefits, to an extent that most of the operators disable these analytics to avoid the innumerable false alarms.

About Video Content Analytics (VCA)

With the advent of Artificial Intelligence (AI) and Deep Neural Networks (DNN), VCA software is being trained to detect, identify, and distinguish various objects in video by exposing them to a large number of tagged examples. In addition to AI-based object classification, computer vision algorithms are also being used to extract data such as absolute speed and size, direction, colour, path, and area. This data can then be searched to concentrate the video analytics effort on relevant information.

In the last decade, with the availability of a significant amount of data and increased computational power, experts have been able to take the theoretical ideas of deep learning and put them to practical use, specifically in the domain of computer vision.

AI in Video Content Analytics

VCA engine accepts the raw video stream and converts it to a comprehensible format The objective of VCA software is to analyse the video stream, one frame at a time, and create a structured database of information out of the unstructured video data. The VCA engine accepts the raw video stream and converts it to a comprehensible format. It then processes the same using computer vision & deep learning technology. As part of this processing, it performs the following critical tasks:

• Object Detection
• Object Segmentation
• Object Tracking
• Object Recognition
• Object Classification

In addition to the above operations, the various object attributes like timestamp, colour, size are also extracted and saved as part of the metadata. Deep learning classification & recognition algorithms are used here to ensure higher accuracy. This metadata is then processed to perform various kinds of analytics.

Face detection, recognition & alert

Accurate face detection and recognition are very critical to law enforcement agencies. It helps in identifying people of interest and is also helpful in post-incident investigations. Broadly, some of the benefits of Facial Recognition application are:

• Automatic attendance
• Automatic recognition of authorized individuals or re-identification of unknown people
• Automatic alert for blacklisted/barred people or no-go zone breach
• Customizable MIS reports (alerts / movements / area-access / area-usage)

Precise face recognition rapidly pinpoints people of interest in real-time using digital images extracted from the video, external image sources and pre-defined watchlists.

Unique face features are extracted and coded into a feature vector that represents a specific face. This feature vector is stored in the database and is used to compare it to the watchlist when faces are searched for.  With the advancement of AI-based deep learning algorithms, FR Systems can now be trained with DNN models having a large set of sample faces. In addition, the advancement of GPU technology has ensured that facial recognition can be done at a large scale and in real-time.

Traffic & road safety

AI technology has enabled VCA applications to detect traffic violations accurately and automatically. The availability of a large set of video data and computational resources has enabled the respective DNN models to be trained effectively. Here are some of the VCA uses cases for Traffic & Road Safety:

• No-Helmet and Triple riding detection
• Wrong way Driving or Illegal turn detection
• No-Parking violation detection
• Licence Plate Detection
• Stop-Line Crossing detection
• No-Seatbelt or Mobile Usage detection
• Over-Speeding Detection

Object Tracking

Object tracking facilitates tracking a vehicle in case of a hit-and-run or to track a person who left a suspicious package During post-incident analysis, object tracking facilitates tracking a vehicle in case of a hit-and-run or to track a person who may have left a suspicious package at the incident site. Using computer vision algorithms, once the object in a frame is detected and segmented, it can then be matched against a set of defined categories which could be a car, bike, truck, man/woman with a cap, jacket, or backpack etc.

The VCA software can be trained to identify these categories by using DNN models. Once the object of interest is detected and matched, the object segmentation defines the pixels used by the object and the movement of those pixels across the video frames can be tracked from multiple CCTV Cameras, thereby giving the entry/exit route of the object.

Video forensics

AI-based deep learning can also help in solving crimes if captured on CCTV cameras. Machine learning techniques can be used for colour conversion, regeneration, and comparison between two video backgrounds, which will help forensic teams to identify vehicles or objects during the post-incident investigation.

AI-based machine learning algorithms can help in other forensic activities such as:

• Vehicle model detection
• 3D face reconstruction
• Video enhancement by Image Super-resolution
• Video De-hazing and noise reduction
• Licence Plate De-hazing
• Predictive Image searching

Conclusion

Artificial Intelligence is the next evolution in Video Analytics. Owing to the advent of high-performance GPU hardware, Deep learning-based AI techniques are being widely adopted by various VCA software OEMs. This improves the detection accuracy without increasing the hardware cost exponentially. For end-users, it greatly reduces the workload of security staff and brings significant benefits by detecting unusual incidents and solving a lot of video forensic problems.

Moreover, it enables them to use the massive amount of CCTV video data generated for system training purposes instead of getting overwritten over a period. In the future, the quality of detection will continue to improve thereby improving the adoption of AI in Security and Surveillance.