Optimizing network storage processing performance: programmable multi-processors and high-speed I/O enable new levels of flexibility and performance for network storage systems.

Storage system architects today face a dilemma. Overall data proliferation coupled with rapid emergence of new technologies such as IP storage and storage virtualization have created the need for faster, higher performance storage network solutions. Today's Storage Area Networks (Fibre Channel-based, iSCSI, or FCIP), and Network Attached Storage (NAS) system designs, often do not deliver enough performance, flexibility, or scalability to keep pace with growing demands for capacity and throughput.

To address next generation storage processing requirements, a new category of powerful, programmable multi-processors with high levels of I/O and memory bandwidth is required; moving forward, these processor solutions will play a critical role in enabling new levels of performance and flexibility for network storage systems.

Technical Challenges

Storage system designers and architects face a number of critical technical challenges today:

* Processing performance

* Flexibility

* I/O and memory bottlenecks

* Fixed power and space constraints

Processing Performance. Storage systems must be able to handle more complex packet processing than traditional networking systems. Storage protocol mediation requires termination and regeneration of each protocol; furthermore, packets must be reformatted from one protocol to the other (e.g. IP, SCSI). In more advanced storage applications, the actual implementation and processing of the upper layers of the protocol (SCSI) may have to be done at the processor level, making this a highly compute-intensive task.

To address the need for high performance processing in the data plane (in-line) and the control plane (exception path), system designers need to evaluate not only the speed, or frequency, of the processor solution, but also the headroom and intelligence on the chip to support features such as TCP checksum and deep packet examination and manipulation.

Furthermore, since storage applications are especially sensitive to latency considerations, integration at the silicon level becomes key. In a multi-chip solution, there is added latency each time a task gets passed to another chip. Higher levels of on-chip integration can speed up overall system communication, thus reducing response time and improving overall processing performance.

Flexibility. While new trends such as iSCSI and FCIP are exciting, these specifications are still evolving and are not yet finalized. To support new protocols and emerging standards in a constantly evolving industry environment, storage equipment vendors will require processor solutions that can be programmed and updated easily using widely available software architectures and tools. In this way, system vendors can do software-based field upgrades to boost performance and deliver new features and thereby maximize the "time in market" of their system solutions.

I/O and Memory Bottlenecks. With processor clock speeds reaching frequencies of 1GHz and higher, system designers are facing a new problem: these high performance processors need equally fast access to high-speed I/O and memory to sustain wire-speed performance. For example, the PCI bus is widely used today as the primary I/O interface for chip to chip interconnect within a system. However, the PCI bus bandwidth becomes quickly saturated when needing to sustain very high data rates, especially with added peripherals such as Gigabit Ethernet PCI cards. Processor solutions that can integrate high-speed network I/O interfaces on-chip become very attractive.

Storage processing is also bound by memory considerations--not only is support for large external memory (DRAM) required, but there is also a need for high-speed access to memory to minimize latency and improve response time for data requests. Next-generation systems will need to address memory bandwidth to keep pace with advances in system I/O and overall processing performance.

Space and Power Limitations. In the datacenter, there are severe space and power constraints. System architects must balance system size and power consumption considerations with performance in developing next-generation system designs. Processor solutions that optimize power and performance, while maintaining a high level of integration are key.

A New Category of Processors

A new generation of MIPS-based processors from Broadcom delivers the industry's highest performance at low power while achieving a high level of integration. These processors offer a promising combination of high performance and flexibility ideally suited to address the needs of next-generation storage systems.

Broadcom's SiByte processors integrate high-performance cores that can scale up to 1GHz, with high-speed I/O interfaces and peripherals. The first member of the SiByte family, the Broadcom BCM 1250, features two MIPS64 cores that scale from 600 MHz to 1GHz, up to 128Gbps bus bandwidth, a high-speed memory subsystem that supports up to peak 50Gbps memory bandwidth, and up to 30Gbps total I/O bandwidth tightly integrated onto a 60 million transistor chip. Key interfaces include three integrated Gigabit Ethernet MACs, PCI, HyperTransport, and generic I/O. (Figure 1). The BCM1250 is targeted at mid to high-end storage applications including SAN routers, gateways, directors, switches, and NAS appliances. Recently, the single-core BCM1125 was announced to provide OEM vendors with a powerful alternative for lower cost devices such as HBAs or TCP offload engines.

[FIGURE 1 OMITTED]

The SiByte processors have a number of advantages over both general-purpose processors and traditional network processors for storage system applications. Unlike most general-purpose processors to date, they have the high integration and intelligence on chip to support in-line packet processing at wire speed rates ranging from 1Gbps to 2.4Gbps. However at the same time, the SiByte processors have a high degree of programming flexibility, one of the key benefits of taking advantage of a well-understood general purpose architecture such as MIPS. For example, the two cores on the BCM1250 are fully software programmable and can be partitioned to manage tasks as required--one core may handle iSCSI protocol processing while the other terminates TCP traffic, or handles exception processing.

Programming is made easier by the ready availability of software tools. Since the SiByte processors leverage the standard MIPS architecture, developers can use existing third party MIPS tools, such as compilers, assemblers, and debuggers. In addition, Broadcom provides firmware, as well as software drivers for the most popular operating systems including VxWorks, Linux, and NetBSD. This comprehensive software development platform provides high programming flexibility and allows customers to develop their own software applications such as virtualization protocols, storage management, load balancing, and SCSI or InfiniBand command translations on top of the SiByte platform.

Integrated I/O

SiByte processors are designed for maximum I/O throughput, with three 10/100/1000 Ethernet MACs, a 32-bit/66MHz PCI host bridge, HyperTransport Host Bridge, SMBus, GPIO, flash interface, interrupts, and DMA all integrated onto the single chip. These integrated I/O functions eliminate the need for a separate system controller, and provide flexibility in design. The Gigabit Ethernet MACs can also be configured as three 8bit or two 16-bit Packet FIFO interfaces. Two serial ports are available to use as UARTs for console ports or as synchronous interlaces. The HyperTransport and PCI interfaces enable customers to build high bandwidth, multiprotocol solutions using Gigabit Ethernet, Fibre Channel, iSCSI, and InfiniBand or any other emerging protocols.

Hyper Transport for High-Speed Data Communication

The HyperTransport Host Bus provides a high-speed chip-to-chip interface for connecting coprocessors, such as encryption engines, peripherals, or multiple SiByte processors. With peak data transfer rate of 6.4GB/sec, the HyperTransport specification provides better than an order of magnitude increase in bus transaction throughput over existing bus architectures such as PCI, PCIX, and AGP.

Typical Storage Applications for Advanced Multi-Processors

Highly integrated network processor solutions such as the SiByte family of processors can be implemented in various applications to address the need for high speed, high capacity storage in data centers, including SAN and NAS. One example, shown in Figure 2, is a line card in a SAN switch capable of performing storage protocols (TCP, NFS, CIFS) on one processor while processing iSCSI on the other. For data entering through the Gigabit Ethernet interfaces, non-encrypted packets are processed by the first BCM1250, while encrypted packets are passed to the second BCM1250 and terminated. The BCM5840 delivers security encryption and authentication for iSCSI packets. The network and storage protocols could also be distributed across both BCM1250s.

[FIGURE 2 OMITTED]

Combining multiple processors and I/O with an industry-leading level of integration, Broadcom's SiByte processors are an example of a new generation of programmable processors that storage system architects can use today to achieve new heights of performance, flexibility, and scalability. They enable development of compact, lower power systems that provide greater capacity and throughput, thus addressing the seemingly conflicting challenges of today's storage system design.

Kim Chan is the product line manager at Broadcom Corp. (Irvine, CA).

www.broadcom.com