Lifecycle management for storage consolidation.

Storage consolidation one of the most important areas of investment storage professionals in 2003 and 2004. It's estimated that three out of four enterprises today are currently engaged a server and storage consolidation project. This is a reflection of the current economic climate as many IT departments view consolidation as a "no brainer" method to reduce costs and increase operation efficiencies.

Nonetheless, storage consolidation projects should approached methodically--with judicious use of upfront planning and painstaking project management. There must also be cooperation and collaboration between functional groups that manage and administer applications, systems, networks and end-user systems. Further, lines of business must buy into the project since minimizing business disruption will be their primary concern

That's where lifecycle management of the data and information to be consolidated comes into the picture. The concept behind the data lifecycle is that data has different values at different points time, and the objective is match data to the most appropriate storage based on its value to the organization. Bur lifecycle management is not a turnkey product that can be purchased; rather it is an approach to developing and implementing methodologies to achieve one's desired storage and data management objectives, such as storage consolidation.

According to Steve Kenniston, senior analyst at Enterprise Storage Group (ESG) analyst firm, "Information Lifecycle Management (ILM) is a combination of people, process and technologies to help IT professionals ensure that the proper information asset lives on the proper storage asset at every moment in time during its life. ILM involves an assessment of the environment to determine what data and storage resources exist, organization and classification of resources based on business requirements, automated data migration and placement of data to the most appropriate storage asset based on the information's relative value to the organization."

Lifecycle management is emerging as an essential discipline for streamlining the storage consolidation process from initial planning to automated migration of data and information resources over their useful life.

Distributed Servers Create Abundant Problems

Today, many large companies have hundreds to thousands of servers throughout the enterprise. Even to deploy one new application, it is not uncommon for IT departments to purchase several individual servers for test, pilot and production phases of the project.

These servers, whether they are for business applications, databases, file/print, development or Web serving, all carry individual purchase, software license, and maintenance costs with them. It is easy to see how costs can get out of control in large environments.

Another problem with distributed server farms it that many organizations develop small islands of storage where utilization levels are low (20% of capacity or less). This is because storage administrators often overprovision file storage to avoid the high cost of system downtime that is needed to install additional storage components. Therefore, new applications that are deployed with their own server cannot take advantage of this excess capacity.

These servers require ongoing administration attention for tasks such as application management, patch management, US upgrades, backup, and system tuning. As more and more servers are added to the network, IT administrators find it increasingly difficult to keep up with the demand and are forced into fire-fighting mode. This situation is not only inefficient and expensive, but it also puts servers and critical applications at risk.

The Solution: Storage Consolidation

IT Managers are realizing that a tremendous effort is required to locate, backup, repair and manage hundreds of distributed storage devices. To overcome these challenges, IT departments are consolidating their disparate server and storage resources to NAS and SAN deployments for centralize and scalable networked file storage.

Most enterprises see consolidation as an effective way for IT to reduce the total cost of ownership of storage. Serve and storage consolidation help maximize utilization of IT personnel, facility and system resources. It enables centralized management of resource and improves system reliability and availability. It increases the number of terabytes that can b administered by a single person. It can also save valuable data center floor space an deliver cost savings through lower maintenance and service costs, fewer hardware components, and fewer software licenses.

However, whether you plan is to introduce a SAN, a NAS filer, or both, the migration and retirement of legacy servers presents many new challenges for the storage administrator. For instance, on of the most difficult initial steps is corralling all the data. This involves collecting detailed information about the environment, such as identifying which devices to consolidate, determining utilized and available disk space, locating printers being shared on the LAN, and validating application and database versions. Once this inventory process is complete, IT administrators are then faced with the challenge of how to plan and implement the consolidation in order to minimize disruption to the organization.

Overcome Challenges with Lifecycle Management

Lifecycle management provides both the technology an methodology required to overcome the challenges associate with storage consolidation, IT administrators are able to gain control of the consolidation process from initial planning to the deployment phases. Lifecycle management fur consolidation involves a series of four steps, which include assessment, classification automation and evaluation.

As an example, let's consider a Top 200 law firm--A B, & C--that currently has large distributed file server an direct-attached storage farm that is under-utilized and difficult to manage. They have decided to consolidate the mass amounts of data on the servers and storage to their high-end NAS storage and wherever possible, they would like to utilize the cost-effective ATA storage now available to enhance the financial savings of the consolidation project. Minimizing business disruption is a must, since the firm is currently engaged in two very high-profile cases.

Assessment

In the assessment phase of life cycle management, the 17 administrators inventory all the existing data and storage resources at both the individual departments and the corporate data center. They identify all the servers that will be consolidate and gather detailed information such as manufacturer make/model, source and target volumes for data migration total disk space (available and used), versions of operating systems and applications, and print shares. Additionally, the administrators gather information about the users and the data, such as space consumption by file types and directories, file age distribution and data usage patterns by department.

Historically, this type of data collection was done manually using custom scripts. These scripts are not only painstakingly difficult to develop and implement but must be manually updated as changes occur to the environment. They also do not address issues such as local administration rights being required to identify disk space usage. Fortunately, Storage Resource Management tools are available today that can greatly simplify the data collection process and, in some cases, do not require installation of server agents. Within hours, administrators are able to summarize the pertinent information for the consolidation in customizable reports.

Once this information is gathered, administrators are able to determine the overall scope of the consolidation. They know how many server volumes will need to be centralized, how much network-attached storage capacity will be required at the corporate datacenter, and what files and directories are least/most utilized by the law firm. At this stage, the IT administrators are now ready to begin the next phase of the lifecycle management process for consolidation.

Classification

In the classification phase, the IT administrators organize and prioritize their storage resources based on attributes, such as storage type (i.e., DAS, NAS), cost per megabyte, capacity, or make/model. There are lifecycle management solutions available today that logically group storage volumes according to criteria such as these. For instance, A, B, & C groups

their DAS volumes based on utilized capacity and make/model in order to prioritize which servers should be centralized first. Similarly, they create groups for the low-end NAS devices based on cost (i.e., ATA disk-based RAID) and high-end NAS groups based on performance and availability characteristics. These resource groups will represent the source and targets for data migration during the consolidation process itself.

The IT administrators will also present the reports generated on data usage to the law firm's partners in order to collaborate on how critical the data is to the business. The administrators will create custom file groups using attributes such as file type, extension, age, size, last modified/accessed times, and directory paths in order to logically organize the data that will be consolidated. A low-value file group might include directories and files for settled lawsuits, while a high-value file group would include the discovery and litigation data for the two unsettled high-profile cases.

A major benefit of the file resource groups is the administrators' ability to organize and prioritize their data resources according to organization-specific business rules, simplifying their data migration policies without worrying about identifying and managing all individual files or directories.

Data Migration

Once all the appropriate data and storage resources groups are defined and the department heads have agreed on a plan for where their data should reside after the consolidation, data management policies must be established to determine where (on which NAS filer) data should be located. Automated Data Migration software is very useful in this step; however, it can also be done using administrator written scripts or traditional hierarchical storage management (HSM) software.

A key objective for the IT administrators is to minimize end-user impact during the consolidation. To accomplish this, they first create policies or scripts to migrate all the low-value data from the file servers and DAS storage to the low-end NAS devices. The migration polices are scheduled to run during off-peak hours to minimize impact to network bandwidth. Additionally, the data migration should be completely transparent to the attorneys and their applications. An automated data migration policy can leave behind a file container that includes all the file property information and takes up only one cluster (usually 1-72 kilobytes) of space.

If the attorneys need to access the migrated files again, the migration policy automatically and immediately recalls the files to their original location and re-classifies this data as high-value. The next step is to implement the data movement policies or scripts on the rest of the data to complete the server and storage consolidation. This remaining 30% of the data includes the high-value files and the remaining file containers left behind after the previous migration. To minimize disruption, the move policies are executed on a department-by-department basis.

The move policies or scripts must ensure that all file containers are rejoined with the files already migrated to the target NAS volumes and all file attributes (i.e., owner, last modified/accessed, etc.) are properly recreated. This ensures at the end of the consolidation, the data files on the NAS servers are a perfect replica of the source files originally on the department servers. The administrators are now able to shut down or re-deploy the distributed file servers. Figure 1 provides a high-level illustration of the data migration and movement policies.

[FIGURE 1 OMITTED]

Evaluation

In the evaluation phase, the administrators monitor and report on the progress of the storage consolidation throughout the entire process. For instance, they track the changes in storage resource utilization over time and monitor the status and effectiveness of policy or script operations in order to appropriately make adjustments to file and volume groups, migration thresholds, and policy schedules, as needed. Furthermore, this lifecycle management solution does not need to stop at enhancing the server and storage consolidation process, it can continue to be a valuable solution for managing and optimizing the data and storage resources on the consolidated server(s) throughout their lifecycle.

The lifecycle management process for storage consolidation enables IT administrators to overcome the many challenges of consolidation and offers numerous benefits including improved consolidation planning, minimal user disruption, continuous monitoring and reporting of the IT environment, and automated, policy-based data migration and movement to ensure optimization on the new consolidated environment.

www.arkivio.com

Glenn Rhodes is director of product marketing at Arkivio, Inc. (Mountain View, CA)