The Storage Challenge
- By Steven Titch
- Feb 01, 2009
By Lee Caswell’s reckoning, the petabyte, or 1 quadrillion bytes, of storage purchased by the Silverton Hotel and Casino in Las Vegas is more than what most Fortune 500 companies will buy in five years.
Caswell, co-founder and chief marketing officer at Pivot3, a Spring, Texas-based supplier of storage management systems, said the Silverton Hotel’s requirements spotlight the unique challenge video surveillance presents to storage management.
Such challenges will only intensify as surveillance operations turn to megapixel and high-resolution cameras, which generate images that contain far more data than conventional cameras. At the same time, disk read/write patterns in surveillance can befuddle IT professionals because they are radically different from most other IT applications. Even so, the latest IT storage models, particularly shared-storage architectures, bring enormous efficiency and savings to surveillance environments.
Pivot3, along with competitors such as Intransa Inc., San Jose, Calif., and Bosch Security Systems, Fairport, N.Y., are targeting the security market with managed storage solutions adapted for the segment’s unique requirements. Their opportunities stem in large part from the way IP networking has fragmented the functionality once contained within a DVR box. Processing once done on a DVR is now done on a server. Software has migrated to video management systems. The final question is where to place the storage on the network. The use of redundant arrays of independent disks is one answer, although it is increasingly seen as a stop-gap measure. RAID is analogous to adding an external hard drive to a PC. It’s good for managing a growing amount of information but still has limits.
RAIDs also don’t solve reliability issues. Storage is arguably the most vulnerable component in an IT system. Disk drives rely on mechanics—tiny motors, spindles or other moving parts that are more prone to failure than electronic components. Consider that disk failure is the most common fault that kills a PC.
RAIDs also tend to be inefficient. Because they handle storage from either one camera or a small group of cameras, they force the user to estimate the amount of required capacity. As a result, RAID users tend to overprovision disk space to ensure enough capacity, said Bud Broomhead, CEO of Intransa, which includes Wynn Resorts and Joy Hotel Group as clients.
Plus, RAIDs take up additional floor space, consume a large percentage of system power and generate much of its heat. Most of the “Green IT” initiatives being promoted by firms such as IBM, Microsoft and Sun are directed at reducing and consolidating storage devices.
Network Attached
DVRs and RAIDS are direct-attached storage solutions. IT departments are turning more to network-attached storage solutions, in which storage media sits on the network as a node. This allows it to work as a shared resource in which any number of devices can write to it and retrieve data from it.
The aggregation of storage allows IT departments to engineer greater reliability and redundancy. Just as when passenger jets went from four engines to two, they were made more efficient, redundant and failsafe, Broomhead said, shared storage systems also are being hardened because storage consolidation is occurring on a far larger scale than the 2:1 ratio in commercial aviation. When hundreds of DVRs can be consolidated into a single point of storage, redundancy techniques can be used that would be prohibitively expensive if attempted individually box by box, Broomhead said.
Several NAS architectures exist, from storage area networks— a common method—to newer models such as virtualization, wherein sequential information is broken up and stored across any number of servers in a wide area network.
One Day, 1TB
Engineering the right storage solution for surveillance, however, must account for its unique requirements. First, surveillance systems demand huge amounts of space.
“Most storage area networks are built to handle 3 to 4 terabytes. We’re putting in 10 times that amount in seven days,” Caswell said. Just one megapixel camera can generate as much as 1 TB in one day, he said.
The declining cost of storage also puts new solutions within reach. Today, $5 buys the same capacity $100 bought five years ago. A terabyte drive now costs $300. Five years ago, a gigabyte drive cost $1,000. “The ability to scale has gone up as the cost has come down,” Caswell said.
Reduced storage costs also generate a proportional savings from consolidation. SANs dynamically allocate space. Devices can use what they need, meaning storage space is allocated more efficiently. This feature is especially useful with cameras, which can have a wide variation in storage requirements.
A video stream, after all, is information contained within the images. The brighter the background, the greater the movement and the amount of panning and zooming all add to the information being collected. “These affect the rate at which you use up storage,” said Bob Banerjee, product marketing manager for IP video products at Bosch Security Systems. “On one camera, the same storage capacity can get you 30 days, while on another, it’s 12 days, even though the bandwidth is the same.”
'Gnarly Profile'
The biggest difference between surveillance storage applications and more conventional IT data storage is in the management of the read-to-write ratio.
Storage use patterns, Broomhead said, usually fit two profiles: a series of random but small reads and writes across a large database, or large sequential reads and writes involving individual files. The first case applies to data most people deal with in day-to-day business. Database updates and changes, file creation, editing and deletion, and Internet use follow this pattern. They usually involve small amounts of data and can be anywhere in the database. Read-to-writes are generally balanced, ranging from a ratio of 50-50 to a ratio of 60-40.
The second case, large sequential reads and writes, applies to movie downloads, CT scans and other high-volume reads and writes that generally consist of one file or group of files. The application is not random and is fairly balanced. While there is a large amount of data, it is either being recorded or read. Simultaneous read-write is rare.
Most storage vendors optimize their systems to either of these two profiles, Broomhead said. Surveillance, however, fits neither. “It’s a gnarly profile,” he said. It is random, like the first case, but involves large, sequential files, like the second. Readwrites also are highly asynchronous, with a ratio of 98-to-2. “And those 2 percent of reads happen at the same time—when there’s an incident,” Broomhead said.
Intransa offers a patented storage management mechanism that normalizes the randomness of the input/out stream and handles the spike in reads when it occurs. The feature is part of its Video Storage Administrator that permits users to configure a SAN to local requirements, provide efficient storage and file paths and support dynamic expansion through a “wizard”-type interface. Pivot3 provides similar management capabilities through its RAIGE storage management software.
Storage is Strategic
Even though the cost of storage has been decreasing, it still accounts for half the cost of the overall system. That’s where centralization and consolidation become so important, Caswell said. Not only do they turn hundreds of DVRs and RAIDs into one big pipe for storage, users get more storage capacity per dollar as well as an IP-based infrastructure that allows for the choice of best-in-class storage, network equipment, cameras and software.
The problem, however, is that security operations tend to design the surveillance system first and deal with storage issues second, when they need to be addressed in tandem, a routine approach with IT. “In the IT world, people specialize in storage,” Broomhead said. “Storage has an impact on the performance of the organization. Storage is strategic and must be treated as such.”
As misunderstandings about storage issues make up a large share of the conflicts between security and IT departments, Broomhead advises security and surveillance directors to learn all they can about storage solutions, particularly the benefits of consolidation, clustering and virtualization.
“If I were talking to the video surveillance community, I would have a discussion on storage,” Broomhead said. “IT is strategic to the enterprise. Storage is strategic to IT.”
This article originally appeared in the February 2009 issue of Network-Centric Security.