Analysis of end-user services and their potential load on the network.

ABSTRACT

Packet traffic on a network is generated by a wide variety of processes. With today's multi-taking operating system it is not unusual for a single computer to have several processes (or windows) running concurrently. Each process in turn may be using different protocols with in the TCP/IP protocol suite and each of these protocols presents a possible different profile of packet intensity. This paper will collect packet traffic from different types of packet traffic profiles such as: a single TCP session, multiple TCP sessions, multiple TCP sessions with background packet management traffic, multiple TCP sessions supporting GUI traffic and analyze their intensity and the potential ramifications to network managers.

Keywords: Computer Networks, End-User Services, Statistical Analysis, Network performance analysis.

1. INTRODUCTION

The past decade could be classified as the "decade of connectivity"; in fact it is commonplace for computers to be connected to a LAN, which in turn is connected to a WAN which provides an Internet connection. On an application level this connectivity provides access to data that even five years earlier was unavailable to the general population.

This growth has not occurred without problems, however. The number of users and the complexity/size of their applications continue to mushroom. Many networks are over subscribed in terms of bandwidth especially during peak usage periods. Often network growth was not planned for and these networks suffer from poor design. Also the explosive growth has often necessitated that crisis management be employed just to keep basic applications running. Whatever the source of the problem it is clear that proactive design and management strategies need to be employed to optimize available networking resources (Fortier & Desrochers, 1990).

2. BACKGROUND

Proactive management requires feedback about how efficiently the network is running. It is fairly easy to take past performance data and analyze it to ascertain how efficient the network has been in the past, but much more difficult to forecast future network performance. This is due in part to the dynamic nature of network traffic. Theoretically, "chunks" of data (packets) traversing the network are suppose to arrive according to a poison distribution. Recently, the validity of this model has been open to question. It may have been adequate for some of the first single tier, single protocol networks, but lacks validity in today's hierarchically complex multi-protocol networks. A number of studies confirm that the expected inter-arrival distribution of packets is not exponential as would be expected in the classical M/M/1 model (Krzenski, 1998, Partridge, 1993, Vandolore, Babic & Jain, 1999, Custer, Robinson & Richardson 1999, Custer, Sohn, Robinson & Safonov, 2003). Also, the sizes of the packets themselves can vary. For example, some packets are very small such as management packets that perform address resolution (about 60 bytes). Interactive data transfer packets (telnet or secure shell) may average around (150 bytes), whereas FTP packets may average in 1,000 byte range. From an efficiency perspective the goal is to maximize packet size, not all applications will support the increasing of packet size simply because they have less to send than the maximum packet size or they need an immediate response (such as in a login request the password must be processed before other packets from that session can be processed). Together the packet inter-arrival rate and packet size distribution define the expected workload on a network at any given time. If the workload exceeds the capacity then the network could lock up completely preventing the transmission of data. However, a more common occurrence is for increased workload to cause end user work station delays to reach an unacceptable level. There are basically two strategies to cope with this problem: increase the capacity of the network or reduce or manage the load by application scheduling (Walker, 2000). Regardless of the approach taken some idea of present and future workload requirements is needed.

One method to secure that data is to log packet traffic and analyze it in regard to present and future trends. Although this method can be effective it is very time consuming and requires a high level of expertise (Guster, Robinson & Safonov, 2005). Typically for this method to be cost effective a fairly large network enterprise is required. However, for smaller companies that are using network resources, is there a method that could provide baseline information in a less costly manner? Guster et al, 2004 suggested that network traffic could be broken down into the load generated by each of its protocol components. Although less exacting than the log file analysis the method can reveal basic trends in network usage if the number of users and the type of application they run is known. For example, what if there are 20 users on the system running secure shell and 40 running a web browser. Through protocol analysis it is known that secure shell requires .5Mbs per user and the web browser requires .25Mbs. Simple mathematics tells us that this combination of users will require 20mbs. The remainder of the paper deals with calculating these bandwidth per user session values.

3. METHODOLOGY

The goal was to evaluate raw packet traffic, but to do so in the frame work of its components. The majority of application based network communication techniques use the TCP protocol to form a virtual connection between two devices. The transfer of packets between these two devices is often termed a session. It is possible to have multiple sessions taking place on a given network simultaneously, each with its own unique data transfer pattern. The complexity of the packet workload profile is further complicated by the fact that there are other application related protocols (such as UDP) and management related protocols (such as ARP and ICMP) are also generating packets that compete for bandwidth with the TCP sessions. The intensity and the amount of data transferred can also be influenced by the type of service used. Telnet, secure shell and FTP tend to be text in nature where as http is graphical in nature. So therefore one would suspect that usage parameters can not be based on the protocol alone, but must be generated based on the protocol/service selected by the user. Therefore, the analysis undertaken herein will be broken into two parts, one that deals with TCP and text based services and one part that focuses on TCP and GUI based services.

3.1 TCP Text Based Services

To gain some type of understanding about the characteristics of TCP text based services traffic four data sets were recorded from a live network for this study. The first data set contained 5000 packets all from a single TCP session and no management traffic. The second data set contained 5000 packets from three simultaneous TCP sessions and no management traffic. The third session contained 5000 packets from 5 TCP sessions and ICMP traffic. The fourth session contained 10,000 packets from 7 TCP sessions, UDP application traffic and ARP/ICMP management packets. The data was collected using TCPDUMP a common shareware packet sniffing program that allows control of which type of packets are selected by programming on its command line.

3.2 TCP GUI Based Services

To gain some type of understanding about the characteristics of TCP GUI based services traffic two data sets were recorded from a live network for this study. The first data set contained 12000 packets from 64 TCP browser sessions and no management traffic. The second data set contained 12000 packets from 128 TCP browser sessions and no management traffic. The data was collected using TCPDUMP a common shareware packet sniffing program that allows control of which type of packets are selected by programming on its command line.

4. RESULTS

4.1 TCP Text Based Services

An intensity plot for all text based data sets is displayed in Figure 1. A visual inspection reveals that the one session data set, spikes at the less than 1 second and at the 2.9 second inter-arrival rate. Whereas, the 3 session data set has a major spike at less than 1/2 second and two minor spikes at .5 and 2.5 seconds. The 5 session data set has one major spike at less than 1/10 of a second and several smaller spikes following. The 7 session data set is so intense that all observations fall into the less that .7 category.

[FIGURE 1 OMITTED]

The descriptive statistics for all four data sets are listed below in Table 1. Where IATIME=packet inter-arrival time in seconds, PS=packet size, ThruPut=throughput in bytes/second, and Intensity=number of packets arriving per second. As would be expected as the number of sessions is increased the IAtime gets smaller. Packet size was relatively stable among the sessions in the 250-500 byte range. Whereas throughput was in the 250-350 byte range for the 1,3 and 5 session level, but a massive 202,959 bytes at the 7 session level. The intensity levels were similar in that the 1, 3 and 5 session data sets and were between 7 and 9 packets/sec, while the 7 session data set exhibited an intensity of 354 packets/sec.

Although the descriptive statistics provide useful information there is a certain degree of variance within each variable. Table 2 provides startup values and values once a steady state was reached for throughput and intensity. It is interesting that in the case of the first three data sets the initial values went high to low and in the largest data set went low to high. This may be because a larger data set of 10,000 packets (vs 5,000 for the other data sets) was collected. Another explanation may be that the arrival profile of the management traffic skewed the output in this way. What ever the case it is clear that within each data set a warm up period is needed before a steady state is reached.

Therefore, the graphic representation of throughput and intensity levels was constructed with a warm-up of 10 packets. A series of 8 graphs (Figures 2-9) follows.

[FIGURES 2-9 OMITTED]

In the case of the 1, 3 and 5 session data sets the activity spiked high and then settled down to a relatively consistent much lower value. In the case of the 7 session data set the activity started low and reached a relatively high consistent value. It is also interesting to note how long it took to collect the 5,000 packet samples. For the one session data set it was 4,136 seconds, while the 3 session data set took only 1,395 seconds. The five session data set was able to collect 5,000 packets in 728 seconds while the 7 session data reached 5,000 packets in only about 15 seconds (28 seconds for 10,000 packets).

An analysis of the packet sizes was also performed. Because within the Ethernet architecture packets can vary from ~50 to ~1500 bytes the complexity of the analysis was reduced by breaking the packet sizes into the following groups. Figures 10-13 depict the inter-arrival times within each packet group by data set. Packet Size Group (PSGroup) number corresponds:

PSGroup = 1 if Packet Size < 100

PSGroup = 2 if 100 <= Packet Size < 500

PSGroup = 3 if 500 <= Packet Size < 1000

PSGroup = 4 if Packet Size >= 1000

[FIGURES 10-13 OMITTED]

It is clear in the 1 and 3 session data sets that the small group dominates. However, in data sets 5 and 7 the volume and complexity of the traffic increases and the other packet size groups become closer in magnitude to PS group1.

4.2 TCP Gill Based Services

An intensity plots for both data sets are displayed in Figure 14. The values are multiplied by 1,000 to provide a more pronounced visual pattern. A visual inspection reveals that both the 64 and 128 sessions have about the same frequency of occurrence within each bar chart interval.

[FIGURE 14 OMITTED]

However, a line plot of each session (Figures 15 and 16) reveals that there some pauses in the 64 session, but almost a continuous stream for the 128 session. This would explain the same basic inter- arrival pattern when the line is loaded and the difference in session times. More specifically it took 4.44 seconds for the 64 session to trap 12,000 packets, but only 2.94 seconds for the 128 session to trap 12,000 packets.

[FIGURES 15-16 OMITTED]

The descriptive statistics for both data sets are listed below in Table 3. As in Table 1, IATIME=packet inter-arrival time in seconds, PS=packet size, ThruPut=throughput in bytes/second, and Intensity=number of packets arriving per second. Again, as the number of sessions is increased the IAtime gets smaller. Packet size was relatively stable among the sessions in the 750 byte range. Whereas throughput was in the 1.8 million byte range for the 64 session level and increased to the 2.4 million byte range at the 128 session level. The intensity levels also increased from about 2400 to 3300 packets per second.

As before with the text based data the descriptive statistics provide useful information, but there is a certain degree of variance within each variable. Table 4 provides startup values and values once a steady state was reached for throughput and intensity. It is interesting to note that in theory doubling the number of workstations would double the intensity. In practice it comes close once steady state is reached (2699 versus 4079). However, the throughput only increases by 50%. This may be explained by the fact that the network and the server process may limit how quickly packets arrive and are processed. Based on the variance of values observed it is clear that within each data set a warm up period is needed before a steady state is reached.

Therefore, the graphic representation of throughput and intensity levels was constructed with a warm-up of 10 packets. A series of 4 graphs (Figures 17-20) follows. In all cases the patterns are similar; a lot of variance at the beginning and then a slowly increasing jagged line. One would expect that if a larger sample was analyzed that the slight rise would dissipate.

[FIGURES 17-20 OMITTED]

An analysis of the packet sizes was also performed. Because within the Ethernet architecture packets can vary from ~50 to ~1500 bytes the complexity of the analysis was reduced by breaking the packet sizes into the following groups. Figure 21 depicts the inter-arrival times within each packet group by data set. Interestingly, the distribution is similar except the mean for the small group (< 100) is greater for the 64 session data. This can be explained if one assumes the pauses that occurred were followed by a small packet. There were no packets in the 500-999 size range.

[FIGURE 21 OMITTED]

5. CONCLUSIONS AND RECOMMENDATIONS

5.1 TCP Text Based Services

From the data collected it was clear that there was not a linear increase in packet intensity or throughput when additional sessions were added. In fact, in one instance the throughput mean was less for the 5 session data set than it was for the 3 session data set. This fact indicates that even with in similar sessions there is still significant variance. Perhaps the most important observation was the impact of non-TCP management traffic. Without this traffic one would have expected the 7 session data to exhibit a throughput mean in the 5,000 to 10,000 range when in reality it was 202,959. Similarly, in packet intensity one would have expected a mean in the 15 to 25 range and a value of 354 was observed. Therefore, this management "background noise" must be considered when undertaking capacity planning. If this value remained constant on the link the bandwidth to support it would be approximately 1.5Mbs whereas the 7 TCP sessions only required about .05Mbs. (Note: calculations based on the mean throughput converted to bits / 1000000 and rounded to two places) Therefore, if the values obtained from this data were transferable then one could forecast bandwidth needs as a fixed 1.5Mbs overhead value plus an additional .05Mbs for each additional active 7 TCP user session group. Although it is a rough estimate it is much better than no objective data at all. However, the majority of traffic in many networks is graphical rather than text based and therefore exhibits a higher degree of intensity. That intensity level is addressed in the next section.

5.2 TCP GUI Based Services

As was the case in the text based data, the GUI data collected also did not exhibit a linear increase in packet intensity or throughput. When additional sessions were added it increased from 1.7 to 2.4 million bytes from the 64 to 128 session level. When the same experiment was run at the 256 session level the throughput only increased to 2.5 million bytes. Furthermore, at the 512 session level the throughput remained at the 2.5 million byte level which indicates that the network bandwidth was saturated at the 256 session level. Similarly, the session times follow this pattern. To trap the 12,000 packet sample it took 4.4 seconds at 64 sessions, 2.94 seconds at 128 sessions, 2.37 seconds at 256 sessions and 2.37 seconds at 512 sessions. Perhaps one of the most interesting findings is the fact that the 100Mbs line maxed out at a throughput level of only 2.5 million bytes. If converted to bits it accounts for only about 20% or 20Mbs of a 100Mbs line. However, the 100Mbs line was run through a hub which made it easy to use a packet sniffer to collect the packet data required for this study. This topology meant that the line discipline was half-duplex which reduced the theoretical capacity of the 100Mbs line in half. Therefore, what accounts for the missing 30Mbs of bandwidth? At 100mbs it is possible to pump out a lot of packets in a hurry. However, the protocol stacks can seldom operate at the same transfer rate (bf3Net, 2003). This study used MMWW which uses TCP (transport control protocol) which is often limited to the 40mbs range. In the case of our data collection methodology which used half-duplex one would expect about 20Mbs which is close to the observed value.

5.3 Comparison of TCP GUl and Text Based Services

It is interesting to compare the average throughput. As would be expected the mean throughputs collected from the WWW applications were larger than those collected from the text based applications. Generally speaking, the mean for the text data was about 1,000 Bps per user. Whereas, the mean value for the WWW traffic was 18,750 Bps per user. Knowing these values could simplify forecasting network load requirements, but these values are only applicable to the author's network. However, the basic process is transferable to other networks. The basic procedures can be used to collect data elsewhere and then that data WOULD be representative of the workload profiles on that network. So it is more likely that the process rather than the actual data obtained herein would be transferable. Based on the magnitude of the data obtained it is clear that a typical LAN running at 100Mbs per second would not be challenged at the 7 session text level, in fact it would take ~ 1,000 sessions (at .05Mbs per 7 sessions) to reach the 50% utilization level (and maybe the true capacity based on the protocol stack limitations). The ramifications for a WAN are more severe. Just the management overhead will fill the typical WAN link (a T1 1.544Mbs). A good share of this management traffic does not need to leave the LAN and care should be taken to filter it out. Therefore, one would expect the capacity of the T1 based on the mean data herein to be about 210 text based user sessions. However, the graphic data revealed the required bandwidth per user to be .15Mbs. This value would mean that about 10 users could be supported by a T1 line which is lower than some past observations (Guster, Safonov, Hall & Sundheim, 2003) which placed this value in the 30-40 range. However the data collected here was quite intense and the 30-40 sessions is probably more realistic for moderately loaded networks

Another interesting observation is the large number or small packets < 100 bytes for both the text and graphical data. A good share of these are TCP packets with small payloads (and hence the push flag was set) that needed to be read immediately so therefore, the maximum buffer space could not be used. This is simply the overhead one must expect if one is using an interactive service such as SSH (secure shell) or WWW. In reality it has become a trade off that is widely accepted.

Although the data herein offered some interesting observations there is still much research needed if the concept of forecasting based upon network traffic's protocol components is to be validated. Certainly, more research with larger datasets generated from more user sessions is needed. Furthermore, data should be collected from several different networks to ascertain if there is any transferability across networks. However, the methods suggested here provide a quick and inexpensive alternative to companies wishing to monitor their current network capacity and forecast their future networking needs.

REFERENCES:

bf3Net performance figures and memory requirements (2003). Ethernet data links. http://www.windmill-innovations.com/products/bf3Net/bf3Net testing.htm.

Fortier, P.J. & G. R. Desrochers (1990). Modeling and Analysis of Local Area Networks, CRC Press

Guster, D., Naumovska, V., Robinson, D. & P. Safonov (2004). "Evaluation of Network Packet Traffic Based on its Protocol Related Components", A paper presented at the International Academy of Business and Economics Annual Conference in Las Vegas, October 17-20

Guster, D., Robinson, D., & M. Richardson (1999). Application of the Power Law Process in Modeling the Inter-arrival Times of Packets in a Computer Network. Proceedings of the 30th Annual Meeting of the Midwest Decision Sciences Institute, Springfield, IL, April 22-24

Guster, D., Robinson, D. & P. Safonov (2005) "Packet Inter-Arrival Distributions in Computer Network Workloads", Encyclopedia of Information Science and Technology

Guster, D., Sohn, C., Robinson, D. & P. Safonov (2003). "A Comparison of Asynchronous Transfer Mode (ATM) and High Speed Ethernet and the Network Design Implications to a Business Organization", Journal of Information Technology and Decision Making. 2(4)

Guster, D., Safonov P., Hall C., & R. Sundheim (2003). "Using Simulation to Predict Performance Characteristics of Mirrored Hosts Used to Support WWW Applications". Issues in Information Systems. 4 (2)

Krzenski, K. (1998). Analysis of the Predictive Process Request-Response Modeling in a Hypermedia Environment. Masters Thesis, St. Cloud State University

Vandolore, B., Babic, G. & R. Jain (1999). Analysis and Modeling of Traffic in Modern Data Communications Networks. A paper submitted to the Applied Telecommunication Symposium, 1999

Partridge, C. (1993). The End of Simple Traffic Models. (Editor's Note), IEEE Network, 7(5)

Walker, J. (2000). Testing and Tuning QoS for Network Policies. Technical Paper. Net IQ Corporation

Dennis Custer, St. Cloud State University, St. Cloud, Minnesota, USA

Richard Sundheim, St. Cloud State University, St. Cloud, Minnesota, USA

Paul Safonov, St. Cloud State University, St. Cloud, Minnesota, USA

Dr. Dennis Guster earned his Doctorate at the University of Missouri-St. Louis in 1981. Currently he is a Professor of Business Computer Information Systems and Director of the Business Commuter Research Laboratory at St. Cloud State University, Minnesota, USA. His research interests include: network performance analysis, distributed processing, computer security and data communications/networking.

Dr. Richard Sundheim earned his Ph.D. in Statistics from Purdue University in 1979. Currently he is a Professor of Business Computer Information Systems at St. Cloud State University. His research interests include: data analysis, SAS programming, data mining and network performance analysis.

Dr. Paul Safonov earned his Ph.D. in Applied Mathematics at the Russian Academy of Sciences in the Institute of Control Sciences, Moscow in 1995. Currently he is an Assistant Professor of Business Computer Information Systems at St. Cloud State University, Minnesota, USA. He also was Visiting Scientist and Professor in Germany, Poland, Brazil, France, and lately worked as Senior Researcher in Free University of Brussels, Belgium.

TABLE 1
DESCRIPTIVE STATISTICS

                                Session 1

Variable       N         Mean      Std Dev      Minimum      Maximum

IAtime       4999    0.8274357    1.3315810    0.0000130    2.9988450
PS           4999  353.5263053  316.6704883   54.0000000      1462.00
ThruPut      4999      2374.15     95949.72       403.28   6602150.37
Intensity    4999    7.0142982  309.0674990    1.1776086     21505.38

                                Session 3

Variable       N         Mean      Std Dev      Minimum      Maximum

IAtime       5000    0.2790851    0.6675132            0    2.2885720
PS           5000  332.6480000  262.4563475            0      1462.00
ThruPut      4999      3181.64     94301.89      1145.60   6463158.18
Intensity    4999    9.4314894  302.8317349    3.5128300     21052.63

                                Session 5

Variable       N         Mean      Std Dev      Minimum      Maximum

IAtime       5000    0.1458692    0.2131440            0    0.9979120
PS           5000  262.7812000  241.8233104            0      1462.00
ThruPut      4999      1972.08      2834.88  387.9990583    201365.19
Intensity    4999    7.8179034   48.1734744    6.5393100      3412.97

                                Session 7

Variable       N         Mean      Std Dev      Minimum      Maximum

IAtime      10000    0.0028033    0.0060075            0    0.0610511
PS           9989  562.9782761  531.2389257            0      1490.00
ThruPut      9999    202959.63      7552.46      5838.69    241992.89
Intensity    9999  354.9261255    8.1791791   98.9607748  391.7855745

TABLE 2
MAGNITUDE OF INITIAL AND STEADY STATE VALUES

Data Set       Throughput     Intensity

1 session    800,000 / 427    2,500/1.2
3 session    800,000 / 1191   2,250/3.5
5 session    3,000 / 1,800      20/6.8
7 session   10,000 / 201,250   160/352

TABLE 3
DESCRIPTIVE STATISTICS

                               Session 64

Variable      N         Mean      Std Dev       Minimum      Maximum

IAtime     11998    0.0003702    0.0074742  1.000000E-06    0.4060010
PS         11998  741.4245708  716.0962957         54.00      1514.00
ThruPut    11998   1783898.09    231193.21     136458.02   3225492.03
Intensity  11998      2408.33  311.3048910       1259.45      5597.01

                               Session 128

Variable      N         Mean      Std Dev       Minimum      Maximum

IAtime     11999    0.0002451    0.0013173     0.0000010    0.0567020
PS         11999  736.4300358  715.6988421    54.0000000      1514.00
ThruPut    11999   2463784.50    504162.18     141975.31   4664767.23
Intensity  11999      3356.87  682.3373597       1677.26      6291.97

TABLE 4
MAGNITUDE OF INITIAL AND STEADY STATE VALUES

Data Set        Throughput      Intensity

64 session   145882 / 2000156  4705 / 2699
128 session  175141 / 3003212  5649 / 4079