SAN Community Ignores Legacy Systems At Its Peril.

Bridging the data gap between open systems and mainframe storage

It is estimated that over 70% of mission-critical data is stored on mainframes, yet the Storage Area Network (SAN) community has all but ignored legacy systems. Today's SANs equal open systems. No solutions exist that are specifically optimized to address interconnectivity between open systems storage and mainframe storage, despite the fact that mainframes are an essential business resource for the largest revenue producing companies in the world.

E-commerce is often associated with Unix and Windows NT systems. What is often overlooked is that mainframes are an integral part of the e-commerce supply chain. Telephone companies, shipping companies, credit card companies, banks, etc. all have room after room of mainframes supporting e-commerce. Mainframes are involved in online transactions, from before a customer clicks on the BUY icon up until the product arrives at the customer's door. If "data" is synonymous with "business," then interconnecting open systems data storage and mainframe data storage is a requisite part of the new economy.

The explosive growth of Internet-generated information has created a dramatically higher level of data storage requirements within large organizations. Experts believe that data storage is growing by over 100% annually. A key application driving this growth is data warehousing. According to Marc Farley, author of Building Storage Networks, data warehousing has been "the main impetus behind the desire to transfer data between mainframes and open systems servers."

Interconnecting open systems and mainframe storage isn't easy. Their data structures, protocols, and fundamental architectures are very different. In the case of interoperability, both environments are double-edged swords: their strength can be their weakness.

The nature of open systems, well, is that it's open. The premise of the open systems environment is that multiple vendors are building compatible products that interoperate, allowing customers to have a common set of applications and devices from competitive sources. The reality is that vendors develop products that may or may not have some degree of compatibility, depending on how much the market drives them to do so. It is staggering when you think about the scope of an open systems environment that includes chips, processors, boards, interface cards, communication protocols, cabling, operating systems, infrastructure devices, and applications with thousands of vendors developing, innovating, and delivering products to the market. It's no wonder that open systems can be unreliable and unstable.

Mainframes, on the other hand, are closed and intentionally proprietary. As a result, the mainframe environment is extremely stable and reliable, as very few vendors control how everything works end-to-end. The mainframe world is able to achieve superior reliability and performance because operating systems, communication protocols, hardware, and system components are all tightly integrated.

What is being done today to bridge the data gap between open systems and mainframe storage? In order to connect these two disparate worlds, companies have been forced to engineer around inefficiencies. There are technologies that allow data to be moved between open systems and mainframe storage, but they do not provide advanced functionality, intelligence, or management features and they are prohibitively slow.

There are various ways that mainframe and open systems data can be accessed today. Over the standard IP network, data may be transferred by all standard means, including FTP and NFS. Additionally, there are several vendors that provide middleware to move and transform data between these two environments. There are also devices that support SCSI, Fibre Channel (FC), and ESCON interfaces, but these products have no intelligence and provide only physical connectivity. Another solution is to utilize storage subsystems that support SCSI, FC, and ESCON interfaces. Although the storage subsystem has software that performs the data transformation, the system running the software is not integrated into the storage subsystem and is actually a separate process. This causes latency and limits functionality. Additionally, storage subsystems are vendor dependent, which restricts customers' options.

There are major problems with all of these solutions. Each one requires host-based software to perform its tasks. These host-based software solutions have proven to be slow and cumbersome, severely impacting overall business productivity. The hosts are production systems and any additional processing impacts their performance. Moving large volumes of data goes from being impractical to impossible. Additionally, most of these solutions provide no online presence on the data path. In order to share data, the mainframe needs to actively open a new data path. This is done on a point in time basis and not continuously. This means that the data is old as soon as it is stored.

None of these approaches optimizes or enhances performance. Today, companies have moved from gigabytes of stored data to terabytes of stored data. Predications are that stored data will soon be entering the petabyte range. The current solutions do not have the ability to move and manage this amount of data. The result is a direct negative impact on an organization's revenue. The consequences could be severe.

Finally, these solutions offer very little intelligence, scalability, or robustness. They may be adequate for raw file transfer, but the ability to move volumes of data, multiple simultaneous replications, online backups, etc. between platforms does not exist.

Intelligent SAN Gateways

Since data is the essential element for businesses, controlling the data is paramount. Open systems data and mainframe data have core differences and the only way to effectively connect these two worlds is to intelligently manipulate the data through a gateway. At the same time, a high performance solution is necessary since the amount of data is growing exponentially. A balance is needed since intelligence and performance are requisite. The problem is, whenever intelligence is injected into a process, latency is introduced. Performance without intelligence offers a limited solution. What is required is an intelligent SAN gateway that sits in the middle of the data path that not only interconnects mainframes and open systems storage on an interface level, but also on a data level. The intelligent SAN gateway should be vendor independent and support multiple platforms.

Let's first examine the bandwidth issues. There is a paradigm shift between open systems and mainframe architecture. Open systems uses a few fat pipes and mainframes use many thin pipes. On the open systems side, FC can move data at 1Gbit. ESCON only supports approximately 17MB of bandwidth, creating the bottleneck. The intelligent SAN gateway would address this issue by aggregating ESCON channels to create fatter pipes. This solution would effectively take multiple ESCON channels and create a virtual channel that provides much higher bandwidth. The intelligent SAN gateway needs to be scalable with the ability to add additional ports and, therefore, provide as much bandwidth as required (see Fig).

A new interconnect technology, FICON (Fibre Connection) is becoming available for mainframe storage. While FICON provides key benefits, including increased bandwidth, major issues still exist that need to be resolved before it can be fully deployed. Existing mainframe systems cannot upgrade to FICON, so companies will need to put in bridges in order to connect ESCON and FICON. While new mainframes have FICON channels installed, at this time there are no control units that support this new interconnect technology. The intelligent SAN gateway needs to support the existing infrastructure and adapt to the new technology as it becomes available.

Processing is another issue that needs to be addressed. The responsibility needs to be taken away from the host and storage subsystems and put on an outboard system. The proposed intelligent SAN gateway would be capable of moving a great deal of data and act on it in real time. To achieve this without impacting performance, the intelligent SAN gateway will need to be equipped with a great deal of horsepower. This could be accomplished with multiple RISC processors on a per port basis.

The placement of the intelligent SAN gateway is also a crucial issue. The gateway needs to sit in the middle of the data path in order to intercept and interpret data as it transverses open systems and mainframe environments. Placing the gateway anywhere else would limit its ability to have the access and control of the data required to be effective.

Intelligence needs to be provided so that the data can be transformed on the fly between mainframe and open systems storage. The platform itself would have embedded intelligence working with lightweight agents residing on various hosts and storage systems. Such a device should extend beyond basic file transfers and raw data conversion. The intelligent SAN gateway should perform advanced functions and provide volume transfers, volume remapping, online backups, point-in-time copy, snapshot copy, asynchronous and synchronous mirroring, and other applications.

There is an elephant in the room and everyone is ignoring it. True heterogeneous enterprise SANs will never exist unless the issue of interconnecting mainframe storage is addressed. Basic connectivity and applications are grossly insufficient for today's requirements, let alone the future. The time has come for intelligent, robust, and powerful solutions that will solve major interoperability, performance, and scalability issues found in today's storage area networks.

Tony Asaro is the director of business development at SANgate Systems, Inc.