Gaps in video from dropped video frames are a common problem. A web search on the phrase “dropped video frames” turns up links oriented towards gamers, YouTube videographers and even the world of video surveillance, discussing why frame drops happen and providing advice for coping with them.
On Windows computers, video frame drops are a fact of life. Much of the advice can help reduce the number of frame drops, though never completely eliminate them. If you are an amateur filmmaker, and you drop key video frames, you can always do another “take.”
Not so in the 24/7 world of security surveillance. You never know which few seconds of video might be critical in determining what really happened, who was really there and who could be held liable.General use IT products like a Windows computer are architected to be event-driven and time-shared
Causes of video frame drops in surveillance systems
To begin to address this problem, it’s important to have a better understanding of the major causes of video frame drops in a video surveillance system.
Many video surveillance systems stream the output from the video cameras over a network to an off-the-shelf computer such as a Windows machine, which acts as the video server and runs your Video Management Software (VMS). The video server stores all the video files to racks of disk drives (disk drives being an affordable choice for the large amount of storage needed in a multi-camera system). Let’s focus on these major sources of video frame drops:
- Video servers dropping incoming video frames when various system tasks compete for resources
- Disk heads in your disk drives that can’t write the incoming video files fast enough to keep up with the continuous stream of data
Many integrators who use off-the-shelf Windows computers will ensure the computer is dedicated to being the video server
Video processing overload
General IT products like a Windows computer are architected to be event-driven and time-shared. They strive for fairness and are designed to share the processing capacity so that no single task is starved. Instead, all tasks get a smaller proportion of processing capacity.
This means new events, like a keystroke or automatic start of a common “background” task or system service, can interrupt and delay ongoing tasks like the handling of your incoming video streams.
In the networked world, that is usually a good choice, since the network protocols will recover by resending the data and the latency that adds is not a problem. This does not work with video. Your cameras are continuing to send new video, so even if you could resend, this would simply bog down the computer still further and cause even more frames to drop.
If the CPU, memory or disk handling required by both the new event and your ongoing video processing exceeds the capacity of the computer, its default way of dealing with the overload is to drop data coming in over its network connections.
| Disk drives are optimised for reading and writing data quickly, generally in a variety of block sizes depending on what is being stored |
Dropped video frame testing
Recently, we started monitoring and testing for dropped video frames on off-the-shelf Windows video servers. In our tests, just clicking on a Windows disk management command resulted in almost 4,000 frames being dropped, an 8-second gap in the video record – and as we all know, a lot can happen in 8 seconds.
Even we were surprised by how often and how consistently frames were dropped, coinciding with various demands on the servers from things like background tasks. Almost always, these drops happened as a cluster, multiple seconds of contiguous video simply gone, a much more serious issue than dropping a few random frames here or there.The best solution is to use a machine and operating system that are optimised to be part of a video surveillance system
Many integrators who use off-the-shelf Windows computers will ensure the computer is dedicated to being the video server, and they will try to tweak the Windows operating system to reduce the number of background tasks, like removing any third-party antivirus programs or performance programs. This reduces the number of video frame drops but does not come close to eliminating all of them.
Video surveillance system solution
The best solution is to use a machine and operating system that are optimised to be part of a video surveillance system, designed to funnel resources as needed to handling the incoming video streams versus attempting to share resources evenly across all tasks or dropping incoming video data when the system is overloaded. By giving priority to the incoming video data and being sized correctly for the number and type of cameras, these systems are much more robust and less prone to dropping frames.
Optimising storage for surveillance
The previous sections looked at how video frames get dropped before your VMS even sees them. However, frames can go missing on the other side of the system when the storage system cannot keep up with the incoming video data.
Most systems use some form of disk storage; because to store all the video from multiple cameras running 24/7 can require a massive amount of storage – frequently into the terabytes – and disk storage is still affordable on this scale.
Disk drives are optimised for reading and writing data quickly, generally in a variety of block sizes depending on what is being stored. If you think about a database, various data records are written once, read many times as people look up data and modified (written again). Storing the video output from multiple cameras breaks this model in several ways:
- The block sizes for video data are larger than typical data files
- The blocks are written once and very seldom read, so it is very write-intensive
- New files from multiple cameras are continuously coming in to be written
When you need high performance and reliability from a system that faces heavy demands, off-the-shelf may simply not be good enough
Avoid missing key video
All these factors keep the disk head extremely busy and lead to longer seek times. Essentially, the disk head cannot write fast enough and cannot keep up with the incoming stream of video data to be stored. The result is that some video frames get dropped.
By using disk systems that are optimised for video storage, you can improve your odds. Solutions that improve head seek time, consolidate multiple video files into one file to reduce the number of writes needed and find other ways for the disk head to work more efficiently can greatly reduce the chances that frames might be dropped.
The off-the-shelf components that are frequently used for video surveillance systems are general purpose machines that do a decent job of supporting a small system with a minimal number of cameras. However, even in a small system with only 3 to 4 cameras, our testing has shown that video frame drop is a given, due partly to Windows’ bias to share resources among tasks and default to dropping incoming data/video streams when busy, and partly to off-the-shelf disk drives that cannot keep up with the write-intensive nature of storing video feeds.
These issues are due to the size and nature of video files, but are not unique to surveillance systems. In the film industry, they too use custom machines as their video servers to handle incoming streams from their digital cameras and store those streams to custom storage hardware. When you need high performance and reliability from a system that faces heavy demands, off-the-shelf – though less expensive initially – may simply not be good enough. Only you can determine how much tolerance you have to the risk of missing key video sequences, and whether your potential liability and the risk of lawsuits is increased beyond an acceptable measure by a surveillance system that captures less than 100% of the action.