What do users want from their surveillance system? Ask any security professional, and you will hear answers like these: sharp, high quality images; widespread camera coverage; real-time analytics; fast search and retrieval of video footage with minimal, if any, intervention from people in the information technology department.
In other words, they want the tools they need to do their job. Delivering those tools requires an underlying technology infrastructure that includes a storage file system—preferably, one designed for video applications.
How do you know if one storage solution is better suited for video than another? What makes a file system ideal for video applications? There are several factors to think about.
Optimising for write operations
What type of work do you expect the storage system to perform? It’s important to know because some systems are designed to handle certain types of work better than others.
Traditional information technology (IT) workloads consist of high volumes of transactions. Files are written to disk and then many read operations are performed. File systems that are optimised for read operations perform well for this type of workload, but they are not ideal for surveillance systems.
Surveillance workloads are different from typical transaction-based workloads. Streaming video and sensor data, from a network of hundreds or even thousands of multifunction cameras, flows continuously. Video management software (VMS) is used to manage the process—from image acquisition through visualisation, analysis, and storage—while performing real-time analytics and metadata generation.
According to IHS, adoption of network cameras is moving quickly, and unit shipments of HD cameras are increasing |
In storage terms, that means the file system should be optimised for write operations, not read operations. To perform well, the file system must be capable of ingesting streaming video from many cameras concurrently and performing simultaneous write operations without creating input/output bottlenecks.
Multiple tiers of video storage
How much storage capacity is needed to support a surveillance system? It depends on how many cameras are installed, what type of cameras they are, and how long the video must be retained.
High-definition (HD) cameras require more storage than analogue cameras. Thanks to dropping camera prices, more organisations are replacing older cameras with digital units and expanding their surveillance coverage. According to IHS, adoption of network cameras is moving quickly, and unit shipments of HD cameras are increasing. By the time we reach 2019, according to IHS, 2.5 exabytes of surveillance data will be produced every day.
Since storage costs can account for as much as 60% of the budget for a surveillance system, the storage configuration must balance capacity, performance, and cost. That’s best accomplished by implementing multiple tiers of storage where older video files can be stored on a lower cost medium such as tape. The way files are managed between tiers, however, can make a big difference when working with video.
Managing video data storage
What method does the file system use to manage data and storage? The advantage of a multi-tier storage architecture is that you can make better use of high-performance disk by moving older files off to tape, lowering your storage costs, but still be able to retrieve them later if needed.
Many information technology-centric storage solutions use a process called information lifecycle management (ILM) to manage data and storage. With ILM, usage policies are created for files. When the policy conditions are met, the file is moved to another tier of storage using an archive or data mover application.
| Thanks to dropping camera prices, more organisations are replacing older cameras with digital units and expanding their surveillance coverage |
This process is not ideal when working with video. High-resolution cameras generate large files. When ILM policy conditions are met, the video files are copied across the same ingest network to another tier, which impacts system performance. To lessen the impact, the data often is saved to a lower resolution (a process called grooming) before it is moved.
Using the ILM method, if a file resides on archive storage, it must be restored back to the VMS system before it can be searched. That adds complexity and time. Plus, if the file has been groomed, the image quality is not as clear and sharp because of the lower resolution.
File systems for video surveillance
When security professionals need to access recorded video, they want an easy way to search, find, and view it at the highest image quality possible, regardless of where the files are stored. Working in the timeline is important to security professionals. They want to perform their job from within the VMS application, not bounce between the VMS and an archive or data mover application to restore files.
Using the ILM method, if a file resides on archive storage, it must be restored back to the VMS system before it can be searched |
File systems designed specifically for video applications meet these criteria. Like ILM, multiple tiers of storage are incorporated, and usage policies dictate when files are moved between tiers. The method used to manage data and storage, however, is better suited for video files.
These file systems can write multiple copies of the files at the same time. For example, when a file is written to disk it is simultaneously written to a tape device residing on a separate network. That way, when the usage policy is met, the data doesn’t need to be groomed and copied across the ingest network; the copy already resides on tape, so the storage management software just releases the blocks on disk, avoiding any performance issues.
Choosing file systems for video performance
In addition, file systems designed for video applications are integrated with the VMS software and don’t require a separate archive or data mover application to move files between tiers. From the user’s perspective, they see one drive letter—a single name space—that represents all storage, instead of separate drive letters for each tier. That makes searching files easier. Plus, if the file that a user wants resides on tape, it doesn’t have to be restored before it can be searched. The user works within the VMS system without worrying about where the file is located, and the storage management software does the rest.
Evaluating a file system to determine if it will perform well for video applications requires digging into the details. You need to examine the architecture and understand how the file system operates – but the results are worth the effort.