Video surveillance advancements lead to data storage challenges

The LTO Program discusses how organisations are struggling to balance the benefit of video surveillance with high costs and increasing legal parameters

We live in a world where there is an increasing need for video surveillance for crime prevention and public security and safety. Video surveillance is being used in greater quantity and with higher quality expectations in airports, cities and workplaces around the globe. In fact, the average person is reportedly caught on a surveillance recording more than 75 times per day.

From security camera footage to body cameras, this system of surveillance is also producing gigantic amounts of data daily. A day of video surveillance, for example, is estimated to be collecting more than 500 petabytes (PB) of data. And this number is increasing. According to a recent IHS Technologies report, future estimates show that by 2019 data will grow to over 2,500 petabytes daily. Organisations and governments are struggling to balance the benefit of video surveillance with the high costs, resources and increasing legal parameters being developed.

How is video surveillance used today?

Body cameras and surveillance footage have been in the news nearly every day over the past few years, and have played a huge role in many high profile cases. For example, the role of video in law enforcement was invaluable in bringing the 2013 Boston Marathon bombers to justice. Following the 2014 shooting of Michael Brown in Ferguson, Missouri, law enforcement agencies across the country integrated body cameras into their organisations, adding to the existing use of patrol car cameras. And with tighter security in airports, many airports are installing close to 20,000 cameras throughout their terminals. However, footage of fatal shootings, for example, is often cut or edited, and occasionally lost due to poor storage handling or lack of security. New regulations are being implemented to combat these issues, calling for surveillance videos to be retained for longer periods of time and in some cases indefinitely.

But where is all of this video going? Who is responsible for maintaining that critical footage over long periods of time, and ensuring the data is safe and easy to access when needed? What is the true cost of this system? And what does it mean for the future of the video surveillance industry as more and more video is captured? The main challenges facing organisations grappling with the increase of video surveillance are the amount of video being captured, how to store it for infinite amounts of time and the cost to maintain this storage.

How much video is there?

As with most industries, data collection is used to retain data for legal purposes and future analytics. When thinking about the industry verticals that use video surveillance, the amount of data is endless. Let’s look at one example – airports. Thinking about the 20,000 cameras at a typical international airport, and assuming the airport is using 15 frame NTSC video capture, that’s 5.3 PB of data per day! Even if the data frame rate is reduced to 3 frames per second, that’s still 1 PB of data daily and 30 PB of data each month. There’s no questioning the enormous amount of data being captured.

Now let’s think about how long this video needs to be stored. Airport guidelines require any “event” to be stored for seven or more years! An “event” includes theft, reported injuries and any conflicts. These “events” happen anywhere between 20 to 40 times per day. For the purposes of estimating the amount of data these “events” produce, let’s assume the following - if each “event” is ten minutes long and captured on at least ten cameras, that’s about 18 GB of data that needs to be kept for seven plus years. With 20 to 40 of these occurrences happening daily, that’s 360-720 GB per day!

Current data collection practices tend to be locally stored on-site or sent to cloud storage. In many video surveillance environments, it is common to use local DVR or NAS storage systems.

This large amount of daily data is an insurance of sorts for the airport and sometimes individuals involved in the incident and some feel that the benefits of having this video surveillance justifies the cost of managing the data.

What is the true cost of storage?

In every industry, video surveillance helps protect an organisation from liability. But what is the cost for this insurance? Where should industry leaders turn to get the most out of this investment? Should they choose disk, cloud or tape?

Let’s take a look at the costs of storing all this video data. Assuming that only ten percent of the video will ever be looked at, the total cost of ownership has been calculated below (Figure 1).

Figure 1: Total Cost of Ownership Long Term (cloud data was pulled based on the public pricing provided by both Google and Amazon)

In this comparison, the Object Storage JBOD is given the benefit of the lowest cost open compute type storage. This comparison also takes into account having management of tape and object storage technology on-site, which increased the cost, but gives a more realistic representation of pricing.

It is clear that tape technology is the lowest cost storage solution for digital data. With the introduction of the Linear Tape File System (LTFS) the system is easier to use than ever. In relation to the surveillance industry, LTFS allows data to be viewed in the same way that it is seen on a disk system. Working with LTO tape technology, LTFS allows tape to be self-describing and improves archive management. Utilising data from an LTO tape solution is now as simple as dragging and dropping the files.

In order to create savings for consumers and lower company bottom lines, surveillance and security industry professionals need to embrace low cost solutions for increasing storage needs. With legal requirements mounting, it is imperative that a lower cost solution be implemented to keep data secure, easily accessible and stored for long periods of time.

How can the industry adapt to this new wave of data?

The video surveillance and security industry are looking at increasing data needs and with it comes new challenges and regulations. Industry leaders are going to need to address this data wave head first with sustainable, long-term solutions. Video surveillance is the security solution of the future and will continue to be used to protect and serve communities globally.

By The LTO Program is a consortium of representatives setting standards and specifications for Linear Tape-Open storage technology, overseen by Hewlett Packard Enterprise, IBM and Quantum.