File virtualization: over 1 billion served.

Storage systems are bulging at the seams with file-based data, and it is the kind of data that is exceptionally challenging to house and manage. Some organizations are looking for ways to manage this data better using more and more file servers and storage devices; others are trying to figure out how to manage it at all. And the growth-management problem is happening even as end-users are increasing their file-access demands.

Historically, file archiving "point products" made management more efficient. But they address just the first part of the problem: They manage only migration and retention policies of inactive information.

A crucial component for improving management of inactive and active file-based data is global file virtualization. Global file virtualization insulates clients and applications from the impact of changes to the IT infrastructure and dramatically lowers the total cost of ownership. It provides holistic management-file archiving for inactive files, management of active data, and ongoing optimization of capacity and performance across systems and storage tiers. File virtualization allows data to flow freely across information targets, and it gives organizations a way to decrease costs while maximizing the value derived from this file-based information.

Simplifying file management is critical, especially for business users and IT organizations that are dealing with file counts that easily range into the billions.

Best Practices for File Virtualization in the Enterprise

Global file virtualization must manage information movement transparently and support active archiving of files from Network Attached Storage (NAS). That's how the underlying IT costs of storing and managing files are reduced. File virtualization also must recognize stub files and offer global namespace functionality at the granular file level. Virtualization increases overall file storage efficiency and significantly reduces the time and cost of managing information in storage networks.

Some more must-haves: Policy-based management is vital to global file virtualization. The policy engine should have flexible rule support for characteristics like file type, location, age, and size. The policy engine also must automate file movement and retrieval across storage tiers. And it must be able to manage orphan files, file versioning, and stub file restoration. Global file virtualization must take advantage of technologies and procedures such as snapshot technology, backup/recovery, and anti-virus protection. Also important is the ability to support multi-protocol discovery, tight file security, and simplified network integration.

The above best practices are needed in any size organization. But enterprise-scale environments face even more considerations. In enterprise environments, global file virtualization requires high scalability and throughput with no bottlenecks. To achieve this objective, the file virtualization solution must be capable of integrating into leading IP switches rather than be based on a proprietary switch. The virtualization solution should also be able to perform sophisticated filtering and event management depending on the operation. It shouldn't be required to process all of the storage I/Os.

File virtualization must also be simple to deploy into existing environments without reconfiguring mount points, deploying agents, or changing backup and recovery procedures. It should also include capabilities such as rollback and pause/restart capabilities for transaction environments.

Seven Things That File Virtualization Must Do

Global file virtualization must integrate seven specific capabilities for optimizing file storage.

Why Content Addressed Storage (CAS) Is Critical

To get the highest return from their information assets, business users are demanding 24x7 access to both active and inactive files. To address this requirement, file virtualization needs to make use of a highly effective tier of storage, one that offers online access to a virtually infinite repository of files without also incurring the corresponding burden of expense. Content Addressed Storage (CAS) is that optimum tier.

CAS provides fast, disk-based access to inactive files. It computes a digital fingerprint (an address) for the file and from the file. This content address assures authenticity: If the contents of the file change at all, a different digital fingerprint is then computed. The content address is the only thing an application needs to know in order to access the file; it is a location-independent identifier that eliminates the traditional burden placed on applications and users to manage the physical and logical placement of files within storage.

Content addresses ensure that a file is stored only once, regardless of how many people or applications request that it be saved. If 10 applications ask for the same file to be stored, the same content address (digital fingerprint) is computed 10 times, and the CAS system realizes that 10 different requests are being made to store the same file. Instead of storing the file 10 times, CAS stores it once, giving each application a pointer to that common object. It is by using these "use-specific" pointers (metadata files) that each application or user can specify a unique retention period specific to that use of the file.

In this way, the same file could be requested to be stored 100 different times or more-100 different times with 100 different requested retention time periods. Yet the file is only stored once, and CAS provides 100 pointers to the single instance of the file. This approach dramatically reduces the number of necessary archive storage system purchases compared with tape, optical, or traditional disk solutions.

By enforcing an application's retention and disposition policies on files intrinsic in storage, CAS ensures that no file can be accidentally or purposely deleted before its retention period expires. This capability alone provides substantial value to organizations that are trying to address compliance requirements, internal governance, and important day-to-day business issues.

Another key aspect of CAS is its ability to self-configure, self-manage, and self-heal. Between its ability to reduce the quantity of storage needed to store content and its ability to reduce management complexity, CAS is the optimum target for archiving files in a global file virtualization solution.

Summary

Global file virtualization enables enterprises to create highly flexible, heterogeneous environments that provide fast access to information, dramatically lower the time and costs associated with file and file system management, and fully support information lifecycle management (ILM) strategies. Global file virtualization is not limited to file servers or NAS. It must also extend file management functionality to CAS, and it must optimize management of active and inactive files alike.

Jack Norris is Director of Virtualization Marketing at EMC.

www.emc.com

RELATED ARTICLE: HEAT Index: file virtualization top of the heap

TheInfoPro is an independent research firm in New York City that each year produces a Storage Technology HEAT Index. This index identifies the popularity of storage technologies by showing the results of interviews with Fortune 1000 IT users about their planned spending and implementation plans. ILM-related strategies including data migration, archiving, classification, and encryption topped recent annual storage HEAT indexes, but the most recent one shows file virtualization as the new top technology for the Fortune 1000. According to TheInfoPro's Robert Stevenson, managing director of the storage sector, the file virtualization buzz results from the fast growth of installed NAS in the data center and long project timelines required for block virtualization.

Capacity management  Finds and resolves capacity issues and balances
                     storage utilization.
File management      Automatically identifies and archives static files
                     using policies, including policies for metadata-
                     based retention settings. The file management
                     process lowers TCO and streamlines backup and
                     recovery.
Global namespace     Enables multiple file systems to appear as a
  management         single virtual file system. Centrally manages
                     industry-standard Windows and Unix namespace
                     services across distributed environments.
Migration and        Enables administrators to move files
  consolidation      transparently without affecting end users or
                     applications.
Performance          Identifies and addresses performance bottlenecks
  management         and hotspots.
Synchronous IP       Synchronously replicates selected directories
  replication        between locations to protect critical data.
Tiered storage       Uses frequency-of-access data to identify
  management         unstructured data and relocate it to lower-cost,
                     near-line storage.