SAN Change Management: a tutorial.

Top storage analysts and a growing number of IT organizations at Fortune 1000 companies have anointed SAN Change Management software as must-have technology for effectively managing and growing Storage Area Networks (SANs). This article will:

* Define SAN Change Management

* Describe drivers fueling interest in the technology

* Detail benefits attained by a large financial institution

* Explain how organizations can utilize the SAN Change Management to reduce the risks, time and resources in managing storage networks

SAN Change Management automates two separate and essential change processes. One process is proactive by predicting the impact of planned changes prior to implementation. The other is reactive by continuously monitoring and validating the health of a SAN across each and every access path. Monitoring identifies problems--such as a missing path, a cabling error, or LUN masking mistake--that jeopardize application availability and security. This process troubleshoots problems through deterministic root-cause analysis and suggests solutions.

At a number of large companies, SAN Change Management software has replaced today's error-prone manual methods and cumber-some spreadsheet comparisons--woefully inadequate at preventing SAN problems and finding and fixing problems that escape proactive detection. To validate the impact of a zone change, for example, the storage administrator must compare and correlate data from the HBA details spreadsheet, the zoning database and the storage masking spreadsheet. Too much IT is spent managing changes and change-induced problems.

Drivers toward SAN Change Management

Adopters of SAN Change Management recognize that older methods cannot possibly attain high application availability--a priority for CIOs. Another driver toward SAN Change Management: 60-to-80 percent of problems in SANs--including the most challenging to remedy--result directly from SAN changes, according to industry estimates. Basic day-to-day actions cause SAN errors: Someone from the cabling group disconnects the wrong cable without your awareness. A member of the storage team makes a typing error when adding LUN masking for a new volume, but you only learn of the mistake after downtime occurs. A storage administrator is required to shut down a switch, but doesn't fully understand its impact on the SAN.

Adding more people is not an answer given the sheer complexity of managing and scaling SANs. A 100-server SAN requires maintenance of more than 20,000 actual access paths and 100,000 potential logical paths. To be successful, a storage manager must understand how a single change in such a SAN can affect several thousand paths. From this example, it becomes clear that humans cannot fully understand the impact of changes throughout the maze of access paths, their relationships and inter-dependencies.

IT organizations that utilize SAN Change Management software:

* Identify and repair errors faster

* Assess the risk of upcoming SAN changes

* Monitor the implementation to ensure correct execution

* Gain better control of their storage networks through automated, best-practice change processes that coordinate activity among dispersed groups responsible for making changes.

SAN Change Management identifies and analyzes errors and vulnerabilities before, during and after change implementation. First, the software establishes a baseline of the existing SAN environment, identifying discrepancies between findings and business rules. On an ongoing basis, the software continuously monitors and validates the health of a SAN throughout changes, growth, migrations and consolidations. Through SAN Change Management, storage administrators and groups responsible for changes regain most of their weekends and nights, when most changes are implemented.

"We've gone from putting out fires to enabling our storage staff to focus on more strategic activities that can help the business," said Robert Shinn, a principal at State Street Global Advisors (SSgA), which successfully employs SAN Change Management software.

Primary components of a SAN Change Management solution include:

* Path management to understand all physical and logical relationships between an application and its data.

* Continuous monitoring and troubleshooting to pinpoint problems and identify violations that stray from policies and best practices.

* Deterministic root-cause analysis to accelerate problem resolution. (Gartner, Inc. calls root-cause analysis "the Holy Grail in IT change management.")

* Automated planning and simulation of end-to-end access paths to predict the impact of changes before implementation.

* Capture of change history for audit, regulatory compliance and a step-by-step understanding of change events.

By automating the SAN Change Management process, users gain an independent "check-and-balances" layer that guards the functioning of Storage Resource Management (SRM) and provisioning tools and continuously validates all planned and unplanned changes.

SAN Change Management ensures a stable foundation that empowers all other storage management investments, according to Steve Duplessie, founder and senior analyst at Enterprise Strategy Group: "You can't utilize your storage products if your infrastructure is fraught with holes and errors." Duplessie maintains that companies "have been putting the cart before the horse" by investing first in SRM over the SAN Change Management foundation.

According to Gartner, improving IT Change Management processes is one of the best investments an enterprise can make: "Companies that don't properly manage IT changes lose time, money and efficiency, and subject the entire business to undo risk."

ROI from SAN Change Management

A number of Fortune 1000 companies have documented that SAN Change Management has prevented application downtime, accelerated troubleshooting of problems, and dramatically increased coordination between groups responsible for changes.

In particular, State Street Global Advisors (SSgA), offers a detailed ROI analysis from its application of SAN Change Management. The world's largest institutional asset-management company with $1.3 trillion in assets, SSgA has nearly doubled in size in the last three years. During that period, its storage network environment grew tremendously in size and complexity. The company's IT organization recognized the challenges and importance of providing SAN reliability in order to support the business, deliver greater value to share-holders and bring greater return on clients' investments: "We make extremely large block trades and stability in our storage is essential," Shinn said. "We can't have applications down any longer than people can hold their breath."

SSgA's use of SAN Change Management software has met business requirements that the company had set:

* Enable SAN growth without increasing operational resources

* Reduce storage deployment time for business applications

* Reduce over-deployment of SAN capital assets 

* Increase SAN stability by anticipating, reducing and quickly resolving problems and disruptions. For example, the software will evaluate a plan to add a switch or alter LUN masking, apply simulation to determine the impact of that change, identify steps required in the proper sequence to avoid problems, and validate that the change was made correctly.

A detailed ROI analysis showed a fast payback in operational efficiency alone (about seven months). ROI did not include benefits derived from improved application availability. SSgA determined that automation of its SAN Change Management process would yield the following benefits:

* A $543,000 savings annually in SAN change process operations

* A $295,000 savings annually in SAN problem resolution

* A 55 percent reduction in storage deployment time

* Preventing 35 percent of problems from ever occurring

* An 80 percent reduction in SAN-induced business disruptions

* SAN growth without an increase in operational resources

* An increase in IT infrastructure stability

"A large amount of outages are caused by changes that aren't deployed properly," Shinn said. "[SAN Change Management] avoids many problems and outages in the first place, reduces our time to react to others, deploys new storage faster and helps us meet ever-changing business needs."

Recommendations for Success

Companies are urged to consider the following in planning and deploying SAN Change Management:

* An effective SAN Change Management solution must implement quickly and produce immediate results. For example, one of the worlds largest ISP's installed its SAN Change Management software to protect a very large SAN in 24 hours. Within that time frame, storage architects gained a complete assessment of their SAN that showed discrepancies with business rules--such as violations in unauthorized paths, redundancy, and host ports access, and vulnerabilities in LUN sharing, zoning and clustering. These results were used for immediate SAN stabilization and cleanup.

* To reduce the risk of employing new technology, SAN Change Management software must be completely non-invasive--agent-less, read only and operating out of band. Organizations should weigh the cost of overhead that the technology introduces into their environments.

* SAN Change Management must be able to simulate changes to determine the impact of any change on a SAN prior to implementation, analyzing the change impact on application availability, performance and security.

* SAN Change Management must perform deterministic root-cause analysis to troubleshoot problems quickly.

* SAN Change Management must continuously validate changes by correlating functions from various devices with business rules, detecting violations and vulnerabilities in near real time.

* SAN Change Management must improve coordination among dispersed groups responsible for implementing changes. "We've improved our cross-organization change management processes and dramatically reduced phone calls to our technology people in the middle of the night," Shinn said. "We now have a standard change process and vocabulary that is understood across all the different groups, that's much more effective at avoiding and pinpointing problems. And we've freed up resources for things that are of great value to the business."

* SAN Change Management must capture complete change history, for audit, regulatory compliance and step-by-step understanding of change events. Users can then easily roll back to the last steady state and identify where problems surfaced and what caused them. For example, SAN Predictive Change Management software can provide a simple interface that shows green check marks and red alerts to mark change steps performed correctly and incorrectly. Not only can storage administrators see at a glance what has changed, but can view anything that has been impacted by a change.

[ILLUSTRATION OMITTED]

* SAN Change Management must accelerate the change process to keep up with the pace of business. Without SAN Change Management, it becomes enormously costly, if not impossible, for storage administrators to work through a once changed cycle before having to make another change.

Mistakes that Storage Managers should avoid:

* Assume that the SRM reporting applications will solve your Change Management problems.

* Decide on a Change Management product without having it installed, up and running and showing value in your environment.

* Fail to take into account the cost and risk of deploying the product. For example, how much will it cost to deploy and maintain agents on all hosts in the data center?

* Assume you will always be able to have best storage people available to extend your team to support change and growth. (There comes a point when throwing more bodies at a problem will not help--when the complexity of the SAN outstretches human ability to view and understand how changes are impacted by a maze of end-to-end access paths, their relationships and interdependencies.)

Best practice checklist:

* Is the set of required functionality for SAN Change management fully supported by the product?

* Does the vendor's roadmap for the SAN Change Management product extend over time in order to perform Change Management across the entire data center?

* Does the vendor provide a customer-customizable ROI model that demonstrates a solid return based on savings in operational expenses alone (not including downtime)?

An organization should consider SAN Change Management when its SAN reaches a size that manual methods are no longer effective at managing changes and growth. Generally, a SAN should have at least 250 switch ports. Organizations should also consider SAN Change Management to supply the solid storage infrastructure to support a migration or consolidation of equipment.

In the next three years, solving change management problems across the data center will become a CIO priority. Given that SANs are a complex, strategic piece of the IT infrastructure, SAN Change Management will continue to become a growing importance.

Steve Feldman is marketing director at Onaro, Inc. (Boston, MA)

www.onaro.com