One of the important benefits of tape emulation, or VTL technology, is that backup software sees the disk as tape with which backup software has enjoyed a long partnership. However some of the key problems that companies face after the backup to a disk-as-disk system is completed include:
Data protection vendors are working to meet these challenges though it is only now that companies are starting to see the results of these behind-the-scenes efforts. One such initiative is Symantec's NetBackup OpenStorage API that Symantec announced back in November 2006. Its purpose: provide customers with more flexibility in managing backup data stored on disk-based backup solutions.
When Symantec announced its OpenStorage API, it envisioned treating "disk as disk" so companies could take full advantage of the options that new disk-based storage systems afforded them. Specific features that the adoption of the OpenStorage API on these system would enable included granting NetBackup permission to manage replication between storage systems as well as the copying of data from disk to tape (on disk-based storage system that supported this feature).
The upside for disk-based backup providers like Quantum is that the NetBackup OpenStorage API provided them a base line to develop code as it developed replication and disk-to-tape copy features for its disk-based storage systems. By coding to the standards detailed in Symantec's NetBackup OpenStorage API, Quantum would have the assurance that any company using NetBackup could backup to its DXi Series disk systems and manage its replication and disk-to-tape copy features without the NetBackup catalog loosing track of what data resides where.
This brings us to today. Though Symantec made this announcement over 18 months ago, only now are we starting to see some of the fruits of its OpenStorage API announcement. During the recent Symantec Vision Conference, Quantum did a live demonstration of the integration between its DXi7500 enterprise disk-based backup system and NetBackup 6.5. Using the OpenStorage API plug-in, and including key management capabilities through NetBackup. Ultimately, this development will allow users to manage duplicate backup images on multiple DXi systems as well as copy data from disk to tape using the DXi7500. The management, cataloging, and tracking of data that is moved or copied by the DXi7500 are completely controlled by NetBackup regardless if the DXi7500 moves data from system to system or off to to tape.
When Quantum announced its new DXi7500, it took aim at the enterprise data center by offering expanded features, such as a policy-based de-duplication methodology, simple scalability and high availability. However enterprise companies still need to centrally manage and track their backup data and many still turn to Symantec NetBackup to perform this task. In fact, according to Quantum, it has tens of thousands of NetBackup customers using its disk and tape systems. So now that this integration between these two technology is nearing completion, companies can jointly use NetBackup and Quantum's OpenStorage API software option to take another step forward in centralizing the management of their backup data regardless of where the data resides or on what type of media.
]]>The problem that enterprise companies face is how do they best manage data once it is backed up to disk? This may seem like an intuitive question for enterprise companies to ask at the outset before purchasing a disk-based solution but this happens less frequently than one may think.
Look at this from the perspective in which most enterprise companies find themselves. The pain that backup has caused has been so acute for so long that just the mere prospect of quickly solving it can lead to an uninformed buying decision.
Unfortunately, buying a disk-based backup solution is just like buying anything else: If you purchase it without fully examining what your backup environment looks like and how the disk-based backup solution will fit into it, it may not end up being the best decision for your environment. This is relevant because the problems that can surface when using disk-based backup can become just as acute as using tape is now.
One specific area of concern that enterprise companies especially need to consider prior to implementing is the management of data backed up in remote and branch offices (ROBOs). Issues that can arise in these circumstances include:
It's when one starts to put all of these pieces together (disk-based backup, deduplication, replication, ROBOs and tape) and view them from an enterprise perspective that the backup picture once again gets much more convoluted. In these cases, enterprise companies need to think more strategically about what disk-based backup solution they introduce. While they may correctly assume that any disk-based backup solution will improve backup times, the list of solutions that can address the new problems that disk-based backup creates becomes very short, very fast.
Enterprise companies with ROBOs that are starting to ask thesequestions should consider Quantum's DXi series of disk-based backup solutions: the DXi3500, DXi5500 and DXi7500. While I will more directly address the aforementioned issues in forthcoming blog entries, here are three of the DXi's key value propositions to consider now:
Disk-based backup is "the" right solution for companies to bring in to address their current backup problems but not every disk-based backup solution is right for every company. No where does this statement hold truer than in enterprise companies that have ROBOs, multiple backup software products and different application recovery requirements.
Enterprise companies may be in a rush to bring resolution to their age-old problem of backup pain by using disk-based backup. However the complexities and questions that they need to answer prior to implementing disk-based backup go well beyond just faster, successful backups. Rather, companies need to think about how they will manage data after it is successfully backed up and, in this respect, Quantum's DXi series brings a compelling solution to the table.
]]>Now that this agreement is out in the open, the real question becomes what does Quantum hope to gain from this relationship? On the surface, it appears this agreement puts EMC and Quantum at loggerheads in the rapidly growing space of deduplication. While EMC and Quantum will use different hardware, the software that drives their respective disk systems will be based on Quantum's technology. In this respect, much of the functionality found in the software will be the same, including the policy-based deduplication I detailed in an earlier blog, though Quantum is putting more emphasis on features such as direct tape creation given its continued focus on integration of disk and tape resources within the enterprise.
The stage for this announcement was actually set some time ago. Quantum's current relationship with EMC was cultivated through a long-standing relationship in which EMC would resell ADIC tape libraries. It was this channel collaboration which helped to lay the groundwork for this week's deduplication announcement.
EMC's adoption of Quantum's deduplication software first and foremost validates Quantum's deduplication technology and lends credibility to it longer term. While Quantum believes that the deduplication software behind its DXi platform is solid, on a larger scale the company has struggled somewhat to gain traction and mindshare in the deduplication space. So having EMC put Quantum's deduplication software through its paces against other competitors and then select it provides Quantum with some needed validation around its deduplication approach.
Partnering with EMC also should help Quantum prosper over the long haul. Deduplication is hot right now and, in many respects, a "green field" opportunity (it is estimated that less than 10% of businesses have implemented some form of disk-based backup with deduplication). But Quantum questions whether vendors that rely strictly on deduplication can continue to meet customers' needs longer term.
Quantum does not see deduplication becoming a standard, per se, as there will always be some different technology approaches that will work better in certain customer environments. However customers want some assurance that the deduplication technology they select will be well-supported into the future. EMC offering Quantum's deduplication technology should help reassure customers that Quantum's form of deduplication will achieve broad market acceptance now and into the future.
In terms of competing against EMC in customer accounts, Quantum does not necessarily foresee that as an issue either. Quantum does not anticipate crossing paths on a regular basis with EMC when selling these systems and, in those circumstances when it does occur, they will work that out on a case-by-case basis. Quantum makes the point that if EMC and Quantum had anticipated fighting tooth and nail in the general market place, this deal probably never would have happened. So the fact that they both signed off on this agreement indicates their mutual desire to work together.
The new relationship, or maybe better termed "renewed" relationship, between EMC and Quantum gives Quantum's deduplication technology the immediate credibility that it was seeking. EMC is one of the giants, if not "THE" giant in disk-based storage systems and incorporating Quantum's deduplication into its systems gives EMC a deduplication software platform that it can use to scale across the enterprise. By signing this agreement, EMC has made a significant bet on Quantum's deduplication approach and indicates that it believes that Quantum has its deduplication house in order. Now all Quantum needs to do is execute.
]]>In a previous blog entry, I took a look at some of the challenges created by deduplicating and replicating backup data from remote offices to a home office and then restoring it again at the remote office. In this blog entry, I'll examine how the architecture of Quantum's DXi backup portfolio addresses these specific challenges by offering greater flexibility with more deduplication choices.
One of the more agonizing decisions that some companies face when looking to implement the same deduplication scheme across the enterprise is quantifying which version of deduplication to use: inline or post-processing. From a purist's viewpoint, inline (deduplicating data as it is ingested) is sometimes viewed as the best approach since data is deduplicated immediately as it is ingested. However scaling an inline deduplication architecture to satisfy enterprise ingest throughput levels (500 Mbps or greater) during peak backup times becomes problematic at best. In environments where these high ingest levels exist, post-processing (deduplicating the data after it is stored to disk) is generally viewed as the best approach.
Quantum's implementation of deduplication across its DXi products takes a unique approach in this respect. Rather than placing its DXi family firmly in either the "inline" or "post-processing" camps, the DXi3500/5500 appliances use Quantum's adaptive approach to deduplicate data as the data is ingested while the new DXi7500 offers a choice of deduplication approaches based on user policies. This strategy allows users to match specific deduplication approached with different backup jobs to maximize both performance and disk resource utilization.
Let's first take a high level look at how Quantum's DXi3500 and DXi5500 appliances deduplicate data using its "Adaptive" deduplication:
The Adaptive deduplication process is available on all of Quantum's DXi products but, depending on which model a customer is using, the choices that a customer has available to it differ. For instance, the DXi3500 and DXi5500 appliances only offer adaptive deduplication and deduplication is always enabled whereas using the DXi7500 system companies can configure deduplication as "Adaptive," "Fully Deferred" or simply turn deduplication off based on each backup job's requirements.
Quantum gave the DXi7500 these additional options since it is intended for deployment in enterprise environments that may need to satisfy short backup windows; backups that contain a high percentage of new data where deduplication is less effective; the requirement for the fast creation of new tapes exists; or when rapid restores are needed.
However that is only part of the story. Each of Quantum's DXi products also supports the creation of logical partitions. The DXi3500 can support up to 8 partitions; the DXi5500 can support up to 16 partitions and the DXi7500 can support up to 64 partitions. Each partitions companies can configure as either NAS or as a virtual tape library (VTL).
Using this option, remote offices can configure partitions for their specific backup environment. If attaching a partition to a LAN, the remote office can configure it as a NAS to facilitate LAN-based backups. However if a remote office happens to have an iSCSI SAN in place, companies can configure another partition as a VTL for improved backup performance. The good news is that regardless of how the partition is configured or how many partitions exist, all of the data is globally deduplicated across all of the partitions on the DXi. (This is only the first step in Quantum's end-to-end de-duplication and replication strategy which I'll dissect further in an upcoming blog.)
The DXi7500 also supports all of these configuration options for partitions plus it offers one more deduplication option - "fully deferred". Using fully deferred deduplication on specific partitions, companies can deploy the DXi7500 in high performance backup environments since companies can direct backups to the "fully deferred" or post-processing partition. These different deduplication options on the DXi7500 coupled with the availability of partitions on all of its models give companies a number of powerful new approaches to deduplicate and replicate data across the enterprise.
In a future blog entry, I'll examine more deeply how the DXi family achieves a form of global deduplication and what options they offer to replicate deduplicated data between appliances located at remote offices, central data centers and disaster recovery sites.
]]>Though Quantum announced the DXi7500 some time ago, today marks general availability of the DXi7500 that it views as meeting today's enterprise concerns. There are a number of features that Quantum includes within the DXi7500 to scale into the enterprise including:
One of the more innovative features of the DXi7500 is its adaptive approach to deduplication. It appears that when Quantum was looking at how to best approach deduplication for the enterprise, it did not to take sides in the "inline" versus "post- processing" deduplication debate and instead gave the customer the flexibility to choose which approach best suits their needs.
This is particularly valuable to companies that have a mix of backup jobs with both low and high performance characteristics and that need more than one alternative when deduplicating backup data. Other products only offer one choice for deduplicating data - "inline" or "post-processing" - but not both. Using Quantum's adaptive deduplication approach, however, companies can match the form of deduplication to the characteristics of their backup job or even the nature of the data contained in the backup jobs. Configured this way, the DXi7500 can match the deduplication approach to the requirements of specific backup jobs.
Matching the deduplication approach to the type of backup job, or even data within the backup job, delivers faster ingest speeds during the backup window. However, because the DXi7500 also gives companies the option to create partitions and assign specific deduplication methods to these partitions, companies can designate which partitions use what specific type of deduplication. In those circumstances where companies expect moving backup data to tape immediately or backups contain a high percentage of new data, companies can take advantage of its "fully deferred" deduplication policy that postpones the deduplication until the backup is fully complete.
Quantum also looks to give companies a compelling reason to use the DXi7500 as the target for receiving replicating data from DXi3500s and DXi5500s as well as using the DXi7500 as a foundation in building a company's disaster recovery (DR) strategy. By deploying DXi3500s or DXi5500s at remote offices, companies can replicate data from these appliances back to a central DXi7500 in their primary data center. Once the data is replicated and centrally stored on the DXi7500, companies can then optionally place a DXi7500 at their disaster recovery site and then replicate data from the DXi7500 in their home office to this secondary site.
The DXi7500 also provides a mechanism to copy data to removable media (tape) for those companies that do not plan to replicate data to a remote site or have long term archiving or compliance requirements. The DXi7500 can either use its own software to copy data from disk on tape or companies can optionally use Symantec's NetBackup 6.5 that recognizes the DXi7500 and can manage the migration of data from disk to tape.
Of course, a final prerequisite when positioning a storage system like the DXi7500 at the enterprise core is to address enterprise concerns about high availability and reliability. The DXi7500 provides dual RAID controllers, dual redundant power and cooling, and hot replaceable components. Dual-node DXi7500 systems eliminate all single points of failure by using dual DXi controllers, active-active failover for all hardware, and cluster-aware software components that fail-over as needed.
Quantum is one of the first deduplication vendors to make the jump from the midrange market with its DXi3500 and DXi5500 backup appliances into the enterprise space with its DXi7500. On the surface, Quantum appears to have put all of the features into the DXi7500 in order to succeed: a highly available and reliable system, careful navigation of the deduplication debate through adaptive and fully deferred duplication and the use of the same replication software across its midrange and enterprise appliances.
This should work in Quantum's favor near and long term. Despite the fact that the Quantum is somewhat late to market with an enterprise level deduplication system, deduplication is still just getting started, not tailing off. By introducing one of the first core-to-edge global deduplication and replication schemes in the market, Quantum should find ample opportunities for companies that are eagerly looking for a single vendor to meet all of their disk-based data protection and disaster recovery needs.
]]>Bringing backup data from remote and branch offices back to a home office is a particularly thorny problem that enterprises continue to face. Directly sending nightly full, incremental or differential backup jobs over a wide area network (WAN) connection back to the home office can saturate the WAN link and cause backups to exceed backup windows and result in failed backups. However the current procedure of backing up data to disk or tape at the remote site perpetuates the problem of how to most efficiently and securely transmit backup data back to the home office or disaster recovery site.
Placing disk-based backup appliances with data deduplication technology at the remote or branch office can be a good first step towards resolving this problem. Bt placing these appliances at the remote site companies keep backup traffic on their local LAN and off of the corporate WAN and backups complete more quickly and within the backup window. Further, the deduplication feature reduces the amount of backup data stored on the appliance so it can keep more data for longer periods of time.
Still, adding disk-based backup and deduplication to remote sites doesn't solve the challenge of consolidating business data within the core data center (home office) or ensuring all corporate data is secure for disaster recovery/business continuance purposes. In this case, one needs to ensure that the offering provides software that replicates data from the remote office back to the central office.
For companies with only one remote office or that just need to replicate data to a disaster recovery site, a number of appliances provide replication to like appliances at the remote site. Since the data is deduplicated before it is replicated, the amount of data that needs to be replicated from the remote office to the home office is minimized and only net new data needs to be replicated back to the target site. This also lessens the network bandwidth requirements and administrators can configure the replication to occur during periods of low network activity.
The difficulty that arises is when companies have multiple or global remote offices and each has a deduplicating backup appliance. If the deduplicating backup appliance only supports a one-to-one replication configuration, the company may need to purchase enough backup deduplicating appliances for the home office to match the number of deduplicating appliances that they have in the remote offices. This approach is very costly, consumes additional data center resources (power, cooling and floor space) and quickly becomes management intensive, especially in remote offices without adequate IT support.
Companies in these situations need to identify vendors that offer a data deduplication architecture that supports a many-to-one replication model and that can also globally deduplicate the data once it receives the data from the remote sites. By supporting a many-to-one configuration, companies only need one deduplicating backup appliance to receive the data from all of its remote sites. The global deduplication feature is needed since it can further reduce the amount of data that companies need to transmit and store on the deduplicating backup appliance at the home office.
This is the architecture that Quantum has adopted for its DXi3500 and DXi5500 appliances and its scalable DXi7500 system to facilitate the deduplication, replication and optimal storage of backup data across an enterprise. In the next blog entry, I'll take a look at how Quantum's DXi-Series supports this architecture and what configuration options this platform provides.
]]>A key challenge that small and midsize businesses (SMBs) face is quantifying what benefits that they will derive from implementing deduplication in their environment. SMBs may have some idea about much data they backup on a daily, weekly or monthly basis to tape. However converting the numbers into meaningful ROI data that SMBs can use when sizing and purchasing a deduplicating disk appliance like Quantum's DXi3500 or DXi5500 is another matter altogether.
Multiple factors come into play when deciding between a DXi3500 or DXi5500 and forecasting what type of deduplication benefits they might expect. Some examples include:
It is for these types of reasons that Quantum's field marketing and sales organization has developed what it refers to as its regional solutions specialists or "Tiger" teams to help businesses determine what size DXi deduplicating appliance is the best fit for its customer environments. When dealing with Quantum customers, their mission is to ensure and verify every customer has a positive experience with Quantum's deduplicating appliances. To do this, a core tenant of their responsibilities is doing the front end analysis that includes customer interviews and site surveys.
During this process, they also educate both the customer account and local sales teams about how to best implement deduplication and set realistic expectations. Quantum's CTO, Jeff Tofano, says that some of the feedback he has gotten back so far from these "Tiger" teams indicates that a lot of customers are struggling with what to expect from deduplication. Putting these teams in the field has helped to dispel certain misperceptions about the deduplication ratio that customers think they might otherwise achieve. "20:1 and 30:1 is a ubiquitous number that is often cited as an expected deduplication ratio but feedback from the field tells us that this number can vary a lot," says Tofano.
Quantum recognizes that it is playing catch-up in the deduplication space but it is taking the appropriate steps to compete and differentiate itself. Quantum's "Tiger" teams are one of those differentiators. By putting qualified sales personnel and engineers in front of customers at the very beginning of the sales process, setting realistic expectations and providing their customers with the right DXi from the outset, Quantum has a greater degree of confidence that its consultative customer approach and its ability to solutions sell disk, tape and management can deliver on client expectations and its clients have had their expectations set according to facts gathered from their environment.
]]>In the first part of this two-part series, I examined tape's evolving role in data protection, what events are occurring that are prompting companies to need to encrypt data stored to tape and how Quantum is evolving its lines of disk and tape products to meet these need demands. This second part takes a look at what the impact and overhead is of managing encrypting data stored on tape and what steps the industry in general and Quantum specifically is taking to address this.
Quantum's larger strategy is to provide solutions that keep data on both disk and tape is reflective of the approach that most large storage vendors are taking. More users want to use disk as their initial target for backup data to keep backup and restore times to a minimum. However they want to keep tape in the mix for their business's archiving, long term data retention and offsite recovery requirements that keep their storage costs low while giving them the flexibility to move data offsite. As companies move data from disk to tape, the onus is on more squarely on companies to encrypt data to ensure it is not compromised as a result of laws like California's SB 1386.
Companies can choose from a number of places to encrypt data: backup software, encryption appliances or on the tape drive. Quantum's disk and tape products support the either of the first two options though Quantum includes encryption capabilities on its LTO-4 tape drives as well as the option to encrypt data as it is being replicated between the company's DXi-Series disk systems. Of the three, encrypting backup data using the tape drive seems to be gaining some industry momentum as the most logical place to encrypt data. Backup software may encrypt data regardless if the backup data is stored to disk or tape and introduces overhead on the host server. Alternatively, an encryption appliance introduces more cost into the backup equation. By using LTO-4 tape technology, companies get encryption for "free" since encryption is included with the tape drive and the overhead associated with encrypting data by the tape drive is minimal (typically 1 - 2%).
The ease in which companies can implement encryption using an LTO-4 tape drive has one immediate advantage. Quantum's Senior Product Manager, Robert Callaghan, finds that many companies that start to encrypt data using a tape drive can immediately come into compliance with new state statutes and avoid some of the litigation issues that some companies already have to address. "If data on a tape cartridge is encrypted using 256-bit encryption and the tape is lost or misplaced, in most cases the company is off the hook," says Callaghan.
Yet the more pressing question is not which method should companies choose to encrypt data but, "How do companies generate and manage the encryption keys that are used to encrypt and decrypt the data?" The obstacle here is that there is no industry standard way to generate or manage encryption keys long term. Though there is a standard before the IEEE committee (IEEE 1619.3), Callaghan says that it will take some time before all of the participating parties come to agreement on a common key management standard.
In the meantime, users have one of three general methods they can use to generate and manage keys: a common key, user generated keys or cryptographically generated keys. The common key is the simplest to manage but also the least secure. If this key is compromised or hacked, all of the data encrypted with that key is essentially compromised. User generated keys are equally problematic. Though they are more random, they may not be sufficiently complex and users are responsible for tracking what data was encrypted with what key and when. Neither of these options are sustainable or reliable models for most corporations.
The third option of using cryptographically generated protected keys is emerging as the best solution today for enterprises to centrally manage encryption keys until encryption key management standards are ratified. In Quantum's case, the Quantum Encryption Key Manager (Q-EKM) gives enterprises the flexibility to encrypt data and centrally manage the data. While using Q-EKM does not guarantee that the encryption keys will be in compliance with whatever standard emerges (nor does using any other vendors), since Quantum is represented on the IEEE committee, it is logical to assume that Quantum will provide a roadmap for ensuring that the encryption keys it generates will remain manageable long term using current and future standards.
]]>The trend away from tape as a primary target for backup is undeniable. Every size company from small and home offices (SOHOs) to the largest enterprise is in some way examining how to more effectively use disk in their backup strategy. This does not mean companies are abandoning the use of tape. In fact, a majority of organizations still rely on tape as their primary means of data protection. However, with the increasing popularity of disk-based backup, companies are repurposing tape for their offsite data storage and longer term data retention needs. As they do so, new requirements for tape encryption and encryption key management are emerging.
The need to encrypt data was more clearly brought to light by a recent class action suits filed against TJX Companies, the parent company of TJ Maxx department stores, and Fidelity National Information Services (NYSE: FIS). The suit against TJX files for damages for the pain and suffering endured by credit and debit card customers as a result of security breach while the suit filed against FIS made no claim for damages but claimed negligence, invasion of privacy and breach of implied contract.
In the case of FIS, the suit alleges that FIS failed to immediately make the internal data breach publicly known and that prior to the public announcement by FIS, a California resident started noticing an influx of direct marketing and promotional offers as well as phone calls to his home. While the data compromised at FIS was the result of a senior database administrator misappropriating confidential consumer information, it is easy to extrapolate how these types of lawsuites can apply to companies who loose or cannot account for missing tapes. These types of cases are prompting more companies to act now and encrypt any data stored to tape.
I recently had conversations with two individuals at Quantum Corporation to discuss these trends in data protection and how Quantum is responding to them. Steve Whitner, Quantum's Disk Systems Product Marketing Manager, agrees that a transition is occurring from using tape as a primary target for backup to disk, with disk becoming the predominant way companies initially store their backup data. Though Quantum offers tape products, it also offers disk-based solutions, ranging from its entry and midrange DXi3500 and DXi5500 appliances to its high end DXi7500 system. Each of these disk-based products supports block-level deduplication and, in the case of the DXi7500, it supports the replication of data from the smaller DXi3500 and DXi5500 appliances to the DXi7500.
Since disk still costs more than tape on a raw per GB basis even with deduplication, companies need to find a balance between using disk and tape. Whitner said that in one particular case, all data that the client wrote to tape had to be written in non-rewritable format and stored for six months. Then at the end of the six months, the client needed to destroy the tape. By storing all of this data to disk and deduplicating it, the client was able to save on tape media costs to the tune of $200,000 annually, according to Quantum.
However the majority of users will not experience such a dramatic hard dollar savings by switching from tape to disk. If anything, the cost of using a disk system with deduplication will at best match what it costs companies to store and manage data on tape. Companies will see their biggest savings coming in the form of soft dollars in terms of how much time their staff spends troubleshooting tape backups, how they're more quickly restoring data and managing daily offsite tape rotations.
Tape's best fit now for many companies will likely take place in its new role of long term data retention and offsite data storage. Many companies will find it more economical to keep storing data on tape for these purposes since the growth of data in most companies' production data stores still exceeds the ability of companies to backup and store this data.
However once the data moves from disk to tape, the new problem that emerges is the risk of compromised data. One of tape's largest benefits - its portability - is also becoming a liability in this day and age since, theoretically, anyone with the right tape hardware and software can restore the data on a lost or misplaced tape. This could result in a scenario comparable to what TJX Companies is experiencing now so more companies are looking to encrypt data as they store it to tape to mitigate this risk. How Quantum supports encrypting data stored on tape and performs the encryption key management will be the focus of the next blog entry.
]]>