Managing The Internet's New Terabit Core.

Service providers will build the Internet's new core from an emerging class of terabit-speed routers, but, for those designing and planning this next generation infrastructure, router performance is just one of the factors they must consider. To succeed, a core network solution must meet a long list of requirements and high on this list is sophisticated network management capabilities.

A December 1998 Yankee Group study estimates that operations management expenses account for 38 percent of a service provider's total cost of network ownership. Included in this expense are the costs for operation, installation, and maintenance of the network. Clearly, the network management tools offered by networking vendors have a significant influence on this very large cost item. Network management issues will become more prominent as Internet traffic grows rapidly and the networks handling it become more complex. The U.S. Commerce Department predicts that 44 percent of American companies will have e-commerce Web sites by the year 2000, up from 10 percent in 1998. With the Internet accounting for a rapidly increasing share of the world's communications traffic, the world's largest network operators and some well-financed new entrants are in a high stakes race to win a share of this new Internet core.

To support this growing traffic load, service providers are rebuilding their core networks with a new generation of routers operating at terabit speeds. Speed is important because a core network cannot be successful unless it can forward IP data at terabit rates. Yet, there are many other key attributes that contribute to the value of the services delivered or that reduce the cost of service delivery. Network management is foremost among these. Network management has long been a significant consideration when selecting networking gear, but, for the new Internet core, router management will be even more critical because:

* If IP networks are to deliver the services currently carried by the circuit switched infrastructure, then the router management systems must become as reliable, effective, and secure as those of circuit switched networks

* Existing router management models are completely inadequate to handle the unprecedented volume of management data, configuration complexity, and capacity planning functions that the new Internet core will demand

* Network performance statistics are key to effectively distributing traffic and optimizing network performance using traffic engineering tools. Manageability must be built in to the router

Many have left network management as an item on a to-do list because developers of terabit routers have invested most of their technical resources on high packet forwarding performance. After all, management doesn't become critical until the networks get big. However, unless developers build manageability into a product before the first line of code is written or the first ASIC gate is etched, they will never reach performance and scalability goals. The demands placed on today's Internet require sophisticated management tools for routing as never before.

Evolving Demands On Internet Services

The new Internet core and the current Internet were built under very different circumstances. Today's Internet traffic may consist of e-commerce and mission critical applications--it is no longer just an alternative form of communication or information as in the past. That is why service providers handling today's Internet traffic must be able to guarantee high-speed connectivity and reliability in the face of intense global competition; it will not be sufficient merely to provide high capacity.

Previously, operators assured service quality by over-provisioning the network, but this tactic has become too expensive. Network operators must also offer high value differentiated services with the lowest possible cost of delivery. Network management tools that efficiently and reliably route traffic will be paramount. The following factors will be particularly important in shaping the new Internet core:

* Demand for utility-class services

* Emerging supply chain business structures

* Differentiated services for business customers

* Unprecedented growth and change

* A growing technical staff famine

* Cost efficiency/network optimization

While these factors contribute to the shape of the new Internet core, service providers will also have to deal with end users who demand a higher level of service than ever before.

Demand For Utility-Class Service

Through its early growth, the Internet has provided a "best effort" service level that would be considered intolerable for traditional voice or data applications. Internet users are demanding that this service quality gap be closed and, with intense competition among service providers, the users will get their wish. The new Internet must deliver the absolute reliability and service consistency of the telephone network. It must also be fast and easy to fulfill the escalating demand for service by new subscribers.

To meet dynamic demands, providers must have management tools that allow them to configure, troubleshoot, and implement complex networks quickly, reliably, and cost-effectively. As deregulation and service convergence spur ever-greater competition among communications service providers, a new chain of service suppliers is emerging. Many service providers structure their Internet core networks as stand-alone businesses and are offering services to external, as well as internal customers.

For example, national backbone operators are selling services to small, regional operators that create customized service packages to address local market niches. This supply chain business structure will place new demands on core network operators. Service level agreements will become vital at the core network level, as backbone operators structure their relationships with the smaller service providers, and managing those networks will help insure that those agreements can be met. Competing downstream providers sharing the same core network will require distinctive service profiles from the backbone to maintain differentiation and meet their business goals.

Business Needs For Differentiated Services

With increasing competition, the industry is moving rapidly towards differentiated services. Enterprise customers are anxious to move their virtual private networks to a lower cost Internet backbone, but they cannot compromise on service quality. To compete successfully for this business, a range of services must be defined according to customer needs, rather than the limitations of the network. While service level agreements are defined at the access level, they cannot be offered without corresponding service levels from the network core and, in the future, service providers will need to use techniques such as Diff-Serv and MPLS to meet the service levels that customers demand.

The carrier networks that will handle these enterprise customers must be built to handle tens of millions of users, which creates a significant scalability challenge. Each physical interface on a terabit router may support several layers of virtual interfaces and carry thousands of separate traffic flows. Network management will be the only way to handle this traffic effectively. With terabit routers that support hundreds of physical interfaces, the scaling challenge takes on a new dimension.

For example, a SONET physical interface may carry hundreds of ATM virtual circuits, each of which has multiple queues to handle DiffServ and a separate IP address (Fig 1). This makes configuration, fault isolation, and performance monitoring functions extremely data intensive. Most router management systems simply do not scale to meet these demands.

Unprecedented Growth And Change

Few new technologies have experienced the kind of rapid market penetration seen by the Internet, but the Internet is not only expanding in size; it is becoming much more complex. The Internet is giving birth to a vast array of services and applications and, as these emerge, grow, compete, and (in some cases) die, the characteristics of the traffic mix will change. Flow rates, packet sizes, burstiness, mission-criticality, and tolerance for latency and jitter, these and other characteristics will evolve in ways that are virtually impossible to predict. Not only must the Internet core grow with increasing traffic, it must also be adaptable as traffic characteristics change. Network management tools will enable providers to adapt to rapid change without bringing the network down.

Growing Technical Staff Famine

The pool of trained technical staff simply cannot keep pace with demand considering the rapid growth of the Internet and related services and applications. Technical staff are getting harder and harder to find and, as they become scarcer, they also command higher salaries and many change jobs more often. Even when companies can find staff, the direct and indirect costs to attract, hire, and retain people are higher than ever. The greater the skill level, the more acute the problem. In the light of this growing staff famine, it becomes essential to use staff as efficiently and effectively as possible. Offloading skilled staff by using automated systems or by lower level staff has a direct and increasing economic benefit.

With intensifying competition comes the pressure to improve the cost of operation. Service providers will stake out a range of market positions, some competing with a least-cost, no-frills service offering and others tailoring their services to particular market segments. But the lessons of long distance telephone deregulation in the U.S. show that all operators will be pressured to maintain the lowest cost in their market. Internet service providers will be faced with some of the same facilities-based cost structure as telephone network operators, including personnel, fiber route miles, and equipment depreciation.

However, optimizing the use of these resources is completely different. Internet routing protocols can distribute traffic unevenly, causing congestion in some segments of the network, while other segments go under-utilized. Routing and switching equipment require different skill sets and operational procedures.

When selecting terabit routers for the new Internet core, network operators should not just be concerned with the performance of the router, but also with the ability of the entire system to meet the evolving demands on Internet service providers. One of the most critical contributions to system performance comes from the network management system (Fig 2). Network management systems are complex and, unlike packet forwarding performance, cannot be compared, based on a single parameter.

But the comparison is nonetheless critical. When evaluating routers, network operators should be wary of claims of management capabilities "coming in the future." Unless support for a promised capability is specifically designed into the architecture of the router, it is unlikely the promise will be delivered. The bottom line is that the winners in the race for the new Internet core will not necessarily be the service providers with the fastest routers. The winners will be those service providers that successfully provision and operate utility-class services; that meet the varying and demanding business needs of their customers; and manage and administer their services efficiently and effectively. Router management will make a key contribution to their success.

Carl Blume is the director of product marketing at IronBridge Networks (Lexington, MA).

10 Questions to ask when choosing a terabit router

Implementation

1. Does management scale to support multiple terabit speed nodes?

2. Can the router manager be integrated with other management applications?

Operation

3. Can faults be rapidly isolated, accurately diagnosed and speedily resolved?

4. Can service be monitored in terms of the Service Level Agreement (SLA) commitments made to customers?

Administration

5. Can the network be engineered and capacity planned?

6. Is management access flexible, yet controllable and secure?

Maintenance

7. Are software upgrades non-disruptive and absolutely reliable?

Provisioning

8. Can customers be provisioned with service levels they need?

9. Is Provisioning non-disruptive and absolutely reliable?

Staffing

10. Are the demands on technical staff minimized?