Intellon^® Powerpacket^TM Networking Performance Analysis

December 10, 2001

by

Bryan Davis

and

Jian Zhang

Introduction

With the exponential growth of the Internet, and the advent of new technologies that use the Internet, such as voice over IP (VoIP), videoconferencing, Netmeeting® and multiplayer games, there is a need for networks that have different performance requirements than conventional file transfer and web browsing applications. These applications are referred to as real-time applications, because the user able to communicate with a distant person or machine, and receive a response almost immediately. Real-time applications generally require a small network delay that is the same for all of its packets, and for the most part, they require a constant data throughput. On the other hand, the data throughput required by these applications is small compared to most file transfer applications, with the exception of high-resolution video conferencing. General-purpose networks cannot be optimized for these requirements, because they still must support the conventional applications such as file transfer and web browsing, and possibly at the same time. Two LAN technologies in particular have the ability to support many new applications, most of which will be real-time: Wireless LANs based on IEEE 802.11b or Bluetooth^®, and power-line LAN based on the Homeplug^® standard.

The Intellon^® Corporation manufactures network communications devices based on the Homeplug standard that enable Ethernet-compatible devices to communicate over standard 120-Volt electrical power lines used in homes and businesses. In order to simultaneously support the different requirements posed by real-time and conventional applications, Homeplug uses a packet prioritization scheme, which gives packets labeled as real-time to have priority when competing for network space over other packets. In this project we are studying the effectiveness of this priority scheduling of improving the performance of Intellon based local area networks.

Background

Network Traffic Characteristics

Network traffic can be classified into two broad categories: delay-insensitive (or conventional) data traffic, and delay-sensitive (or real-time) data traffic. The main difference between the two is that real-time data traffic consists of a data stream whose packets must arrive in order, with a small delay (and thus packet retransmissions are not useful), whereas conventional data traffic consists of a block of data that can be transferred in any order and delay is of less concern.

Real-Time Data Traffic

Real-time traffic typically consists of either one-way or two-way streaming audio/video, time-ordered signals. In the case of one-way streaming, the real-time requirements are not as strict in the sense that a large delay can be tolerated. This is because there is no real-time feedback from the receiver of the signal. In this case, the application layer can create a large buffer to smooth out the differences in the packet transmit times, and if the buffer is large enough, can even request retransmission of lost packets. For two-streaming applications, such as voice conversations or videoconferencing, the real-time requirements are very strict. We will classify the two-way applications as real-time, and one-way streaming applications as conventional.

Digitized voice is generated as a continuous data stream with a certain bit rate. If the voice is uncompressed telephone quality, then the bit rate is a constant 64 kbps. Digitized voice would typically be compressed to a rate of between 5 kbps – 13 kbps. Some compression technologies create variable bit rate voice for further compression. For video conferencing, the rate can vary much more considerably depending on the video refresh rate and picture quality. The data rate in this case can vary from ~20 kbps for jerky, low-resolution web-cam videos to ~1.5 Mbps for high-quality television-like videoconferences. These signals have a compression delay associated with them, and they can also be variable rate.

The important requirement of the performance of the network used for two-way voice streaming is the delay. Studies have shown (reference some book here) that a delay of less than 150 ms is hardly noticeable, whereas a delay of 300 ms causes a perception of significantly decreased quality. Compression adds typically around 20 ms to the delay (depending on the algorithm used). Also, the delay must be constant, variance in the delay, called jitter, causes quality-of-service problems. The application layer can reduce the jitter (if it can be predicted), by buffering the packets to all have the delay of the largest packet delay. Thus, the maximum packet delay should be less than around 200 ms. Of course, this requirement isn’t strict, but it can be used as a general guideline measuring acceptable performance.

A broadband network requires that data be sent at the bit rate of the network, which hopefully is faster than the rate of the real-time data stream. Also, the network requires that the data be packetized and appended with network control information. This requires that we accumulate enough data to fill a packet, and then send the packet all at one time. This creates a delay that is equal to the size of the packet (in bits) divided by the stream data rate called packetization delay. Thus an 8 kbps stream using 50 byte packets would incur a (8 bits/byte)*(50 bytes) / (8000 bits per second) = 50 ms delay. The network will take considerably less time to send the packet than the length of time represented in the packet. This means that there will be a periodic sending of real-time packets on the network followed a period of no real-time packets being sent for this data stream. This situation is shown in the following figure:

P

P

P

I

I – Inter-packet Arrival Time R – Real-Time Data Rate L – Line Data Rate

P – Payload Packet Length

The four parameters are related by the following equation:

If the line data rate L is much larger than the real-time data rate R, then the formula becomes

The efficiency of the network protocol is defined as the payload data length over the network data length and is given by , where O is the overhead associated with each packet. Thus the efficiency h is given by,

And from before, the packetization delay = I. Since R and O are usually given by the application, this gives us one free parameter, I, which will determine our delay and efficiency. As I decreases, our packetization delay decreases, but our efficiency decreases as well. The optimum value of I depends on the total delay in other parts of the network, the real-time and conventional loads on the network, the importance of delay for the real-time application, and the importance of throughput for other applications on the network.

Another requirement is packet error. Since the data cannot, in general, be retransmitted because of the delay requirement, lost packets cannot be recovered. Fortunately, voice and video data is much more tolerant of lost data than is, say, a computer program. Thus for most real-time applications, a certain amount of data can be lost, without a large degradation in quality. The higher the compression used, the more packet errors will degrade the quality. A typical value given for acceptable bit error rate for compressed voice is 0.1%.

Conventional Data Traffic

Conventional data traffic is not temporally ordered like real-time traffic, and it exists in its entirety on one network node, and gets transferred in its entirety to another node. In a real data network, conventional data traffic will be sent at full rate at some times and at a zero rate at other times. These intervals in general will have a stochastic nature. We can break the model into two situations: intervals when data traffic is being sent at full-rate and intervals when no data traffic is being sent. When there are multiple users on the network, the unused portion of time decreases, but the throughput during the used time stays the same (it is always at maximum). The worst-case scenario is then when the data is being sent at full speed. Therefore in this project, we model conventional traffic as a data stream continuous duration that uses as much network time as the network will allow. Also, for conventional traffic, we want to maximize the throughput instead of minimize delay. For a small bit error rate, this will direct us to use the largest packet size available so as to minimize the percentage of network control overhead in the packet.

Conventional Data Parameters

Data will be modeled as a continuous stream of long packets being sent with no inter-packet delay, but at the lowest priority. The packet length is 1518, the inter-packet delay is 0, and the priority is CA0.

Typical Real-Time Traffic Parameters

At 8 kbps, we want to minimize the packetization delay, while not wasting bandwidth on network overhead. If we choose a packetization delay of 30 ms, then we have a total of 30 ms (packetization delay) + 20 ms (compression delay) = 50 ms of delay with no contention on the network. This leaves us with ~200 ms for contention delay and, if this is an Internet call, the delay associated with the Internet and the delay at the other caller’s network.

The inter-packet delay can then be calculated using formula (1). If we assume that the network data rate is much larger than the voice data rate then the inter-packet delay is 30 ms, approximately the same as the packetization delay. The packet payload length is then 8 kbps * 30 ms = 240 bits = 30 bytes. If we assume we are using RTP, we must add Ethernet, IP, UDP, and RTP overhead = 26+20+8+12 = 66 bytes, thus the packet length is 96 bytes. The priority is CA3.

Videoconferencing Parameters

Videoconferencing will be modeled the same as voice, except that the data rate will be higher. Again we choose a packetization delay of 30 ms. Still under the assumption that the videoconferencing data rate is much lower than the network data rate, we have an inter-packet delay of 30 ms. The packet payload length is then 200 kbps * 30 ms = 6000 bits = 750 bytes. When we add the protocol overhead, we get an overall packet length of 816 bytes. Again, the priority is CA3.

Data Parameter Summary

Traffic Type	Packet Length	Inter-packet Delay	Priority
Conventional Data	1518 bytes	0 ms	CA0
Voice	96 bytes	30 ms	CA3
Videoconference	816 bytes	30 ms	CA3

Priority Scheduling

The Homeplug specification allows for four different priorities for packet scheduling on the medium-access-control (MAC) sub-layer of the network. These priorities are from highest to lowest: CA3, CA2, CA1 and CA0. The packets are mapped to one of these priorities based on the priority tagging method outlined in the IEEE 802.1D specification called VLAN tagging.

IEEE 802.1D User Priority Class Descriptions – Derived from 801.2D (1998) – Section H.2.2

User Priority	Application Class
A	Network Control—characterized by a “must get there” requirement to maintain and support the network infrastructure
B	“Voice”—characterized by less than 10 ms delay, and hence maximum jitter (one-way transmission through the LAN infrastructure of a single campus).
C	“Video” or “Audio” —characterized by less than 100 ms delay.
D	Controlled Load—important business applications subject to some form of “admission control,” be that pre-planning of the network requirement at one extreme to bandwidth reservation per flow at the time the flow is started at the other.
E	Excellent Effort—or “CEO’s best effort,” the best-effort type services that an I information services organization would deliver to its most important customers.
F	Best Effort—LAN traffic, as we know it today.
G	Background—bulk transfers and other activities that are permitted on the network that should not impact the use of the network by other users and applications.

The IEEE defines seven priority classes, but Intellon only uses four. The IEEE gives the following recommendations on how to map from seven priority classes to four priorities.

IEEE 802.1D Recommended Mapping from User Priority to Four Traffic Classes – Derived from 801.2D (1998) – Section H.2.4

User Priority	Priority Class
A	3
B	3
C	2
D	2
E	1
F	1
G	0

Queuing Model of the Homeplug MAC Protocol

Homeplug defines a proprietary standard for implementing the packet prioritization. Using this standard, we can attempt to model the network in terms of priority queues. The actual standard is much more complicated than this, with many different options, but the following model is good enough to characterize the behavior of Intellon’s priority mechanism. The model is similar one discussed in class called the non-preemptive priority queue. Each priority class has its own queue on the network. Once a packet is on the network, it is allowed to finish. The model used here is slightly different from the one in class because each priority class is not first-come-first-served, but rather the packet to be served is chosen randomly from each class. Thus we have the following scenario:

In addition, we have a rule that packets cannot wait on the CA0 queue for more than 8ms. This rule is in place to insure that delay for the remaining packets waiting in the queue does not become so large that the packets are useless.

Testing

The tests we created are designed to measure the critical performance values of the network traffic as we increased the amount of real-time traffic on the network. They were run on Intellon’s eight node testing station.

Intellon Testing Station

Intellon has a testing station that consists of eight nodes on a simulated power line network, with each node being controlled by a controlling PC. The controlling PC is connected to each of the nodes via a dedicated serial connection as shjown in the following diagram.

Controlling PC

Each of the nodes is running software called PLCNetTest. This software sends packets from the node to another node, and records statistics for packets it received that were sent by other nodes. The controlling PC is running a program called Labview. This program communicates with PLCNetTest on each of the nodes, and tells them to send a series of packets to each other. It also controls any impairment that may be placed on the power line. After the nodes have finished sending and receiving they send back the results of the test. The controlling PC then compiles these results into a comprehensive spreadsheet filled with statistics.

Capabilities of the Labview GUI and PLCNetTest

Labview also has a user interface that allows the user to define the different test scenarios that it will run. One of our main tasks in this project was to define these test scenarios so that we could measure performance. Some of its capabilities are:

• For each test is can specify:
• Which each of the 8² - 8 = 56 network paths in the 8-node network will communicate
• Any line impairments such as white noise or a drill that is contained on the line (not used – we assume a clean line)
• For each node it can specify:
• The number of retries to attempt for non-acknowledged packets (either none, one or “normal”)
• Whether to allow backpressure. (This feature is used for LAN bridges and is not useful for this project)
• For each path it can specify:
• The number of packets that will be sent on each path
• The length of the packets that will be sent on each path. A fixed value can be chosen, or the packet length can vary linearly or randomly through the test
• The priority of the packets sent on this path
• The delay between the packets sent on this path. This can be used to throttle the data rate on simulated real-time data paths

Test Scenarios

Because of limited time available on Intellon’s testing station, we decided to limit our test to one scenario while changing three independent variables. The test scenario consists of the following (shown in Diagram of Logical Paths for Test Scenario):

· Eight nodes, each involved in two-way communication with only one other node, to create four full-duplex channels.

· Two channels were communicating using conventional data (i.e. simultaneous file transfers)

§ The normal retry setting was used to allow MAC level retries

§ The packet length was set to 1514 bytes

§ The inter-packet delay was set to zero

· Other two channels were communicating using real-time packets with a fixed rate of speed.

The independent variables were:

1. The fixed real-time communications rate for the real-time channels (16 different values were used)

2. The priority used by the real-time channels (either CA3 or CA0)

3. The number of MAC layer reties on the real-time channels (none or “normal”)

Diagram of Logical Paths for Test Scenario

Real-Time Communications Rate

We wanted to vary the percentage of the total available power line throughput used by the real-time channels from 0-100%. This would give us a full range test points, and would ensure that we cause the system to become overloaded, so that some of the tests would fail, thereby allowing us to find the maximum real-time throughput the network can handle. The first test we ran was to determine this maximum value. It was 8.1 Mbps. Since we have four real-time channels, this would give us a value of 2 Mbps second for each channel. We varied the data rate on the real-time channel from 0 to 2 Mbps in ¼ second increments, for a total of 17 values for the real-time communications rate.

So the minimum value tested was running 2-250 kbps real-time duplex channels. This is quite a bit of real-time traffic, corresponding to two simultaneous medium-quality videoconferences. The maximum value tested – 2-2Mbps real-time duplex channels correspond to more than two very high quality videoconference channels. Thus our tests are almost unrealistic in stressing the network.

For a throughput of 2Mbps, using the 1514 byte length packet, we must send the packet every 6ms. We decided to use the 6ms value for all of the real-time channels to be consistent, and to just vary the length of the packets from 1514 bytes to 95 bytes.

Real-Time Priority

We wanted to compare the network performance using a priority for real-time channels versus not using any priority for real-time channels. This would allow us to determine the effectiveness of Intellon’s priority mechanism as compared to using a standard network.

Retries

The Homeplug specification requires that packets (actually pack segments) be retransmitted if they are not received. This allows for a higher channel quality by reducing the packet error rate as seen by the transport layer. For real-time packets, however, too many reties might reduce the channel quality by increasing the delay of the packet in the network. We wanted to test this hypothesis by running all of our tests with and without retries.

Intellon’s implementation allows us to adjust the number of times a packet may be retransmitted before it is dropped. Normally, this value is set to 6, but we can change it to 0 or 1 on a node-by-node basis. We ran all of the tests using the “normal” number of retries and no retries.

Test Summary

Changing these three variables independently gives us a total of 68 tests to run, with each test using eight paths on eight nodes.

Expected Results

For our test scenarios, we are mainly looking for three results: Throughput, Maximum Delay, and Packet Error Rate (PER). For each test, there are two types of packets: real-time and conventional. We determine the real-time throughput a priori by how often we send real-time packets. The other five variables are dependent: Real-Time Delay, Real-Time PER, Data Throughput, Data Delay, and Data PER. We are not very concerned about Data Delay, but the other four graphs are very important in determining the quality of service provided by the network.

Real-Time Delay vs. Real-Time Throughput

With Priority Queuing

To find the expected value of the real-time delay, we need to use our queuing model for the network. We have up to four CA3 nodes and four CA0 nodes. Since there is no inter-packet delay we can always assume that when a packet arrives for a CA3 node to send, the line will be occupied with either a CA3 packet or a CA0 packet (the network is always running at capacity). The average time (in bytes) needed for a CA3 packet to get to the contention period (i.e. to get a chance to compete with other CA3 packets) is the residual time R, where

, where P is the packet length of a CA3 packet

Once the residual packet has been served, the packet must vie between the other packets of the same priority for a chance to go. If we assume that all of the CA3 packets arrive at random times, then the probability that another packet is also vying for transmission is l*W_Q for a deterministic system with random initial offsets. Thus the probability of being transmitted this time is . If this packet cannot transmit the first time, another CA3 queue must have transmitted, and it won’t be transmitting again for another 1/l period. If we assume that the waiting time of the other two queues are independent of the our waiting time (which isn’t necessarily true, but simplifies analysis), then the probability of being transmitted the second time is , and the third time is , and the fourth time is w₄ = 1. The average waiting time in the queue is then

And

W = W_Q + P

If we convert to ms and Mbps, and plot the real-time delay per simplex channel vs. real-time delay per simplex channel, we get the following graph:

Real-Time Delay (in ms) vs. Real-Time Throughput (theoretically)

Note that the delay is increasing with the increased throughput.

Without Priority Queuing

Without priority queuing, the queuing model used is just a random queuing model. We will simplify the analysis by assuming that all nodes will vie for contention when every packet is sent. This is actually not the case, because the real-time nodes will only vie for contention some of the time.

We can model this queuing system as eight independent queues all vying for one server who randomly selects one of the queues for a packet to serve. Assuming the selection is fair, the effective average service time, from the point of view of packets in the queue, (in bytes) is then eight times the average packet length of all queues. The average packet length of all queues is then , where P is the packet length for the real-time packets. Notice that this value is between 1514/2 and 1514. When we designed l and P, we designed 1514*l to be ¼ of the total throughput, so the four real-time simplex channels would consume the total throughput. If the total throughput is T, and L is 1514, then . But m, the effective average service rate is between T/8L and T/4L, so r = l/m is between 1 and 2. This is unstable. Thus this setup is inherently bad for real-time traffic.

Data Throughput vs. Real-Time Throughput

If we assume no collisions, and that the channel is always full, then we get that the data_throughput = max_channel_capacity – real_time_throughput. Thus the plot should look like:

Data Throughput vs. Real-Time Throughput

Real-Time and Data PER vs. Real-Time Throughput

Ideally, there should be no PER, but because there are collisions, we will definitely get some. The expected value here is highly complicated and its calculation is beyond the scope of this project. Suffice it to say that this value should be less than about 0.1% before the quality of service begins to suffer (for voice or video anyway). This is just a rough guideline and the actual value depends highly on the application and especially on any compression algorithm used.

Data Post-Processing

Since the *.csv files have all test data in one line, which were too long to be edited with most editors, we used Ultraedit to convert each column into rows. Then UNIX was used to merge files column by column. Each column was a full set of data of each test. At last, Excel was used to convert the plain text format data into Excel spread sheet.

The following models were used to analyze the data:

Data Extraction

X: RT- Throughput	*.CSV: SUM( TX Mbps for all RT paths )
Y1: Data-Throughput	*.CSV: SUM( TX Mbps for all Data paths )
Y2: RT-Avg. delay	*.CSV: AVG(CA3 Latency, for all RT paths )
Y3: RT-PER	*.CSV: SUM(Pkt Err RX, for each RT path) / SUM( Pckts Tx, for each RT path )

Real-Time ® Nodes 5 – 8 and paths 5 ® 6, 6 ® 5, 7 ® 8 & 8 ® 7

Data ® Nodes 1 – 4 and paths 1 ® 2, 2 ® 1, 3 ® 4 & 4 ® 3

Results

The following graphs show the results obtained for each of the four categories of data: Priority with Retries, No Priority with Retries, Priority with No Retries and Priority with Retries.

In some of our data points were duplicated and others were lost in both of the No Priority plots. Fortunately, those are the less interesting of the two.

Real-Time Delay vs. Real-Time Throughput

We plotted this graph on a log scale so that we can see the great advantage to using the priority mechanism, while still showing the general increasing tend in the delay as the throughput increases. Note that the Priority with No Retries chart has a smaller delay than the Priority with Retries, which is expected, but they both are less than 10 ms.

Obviously the priority mechanism is required if one is to use real-time traffic at the same time as conventional traffic on this network. The priority graphs compare well with the theoretical results (especially the No Retries one), except they are a little larger in magnitude.

Data Throughput vs. Real-Time Throughput

The values here are lower than expected (without any real-time traffic, the total throughput is 8Mpbs). The trend for data throughput to decreased is apparent for the Priority data, but it is not apparent for the No Priority data. More research needs to be done to determine why the data throughput does not sum to around 8Mbps.

Real-Time PER vs. Real-Time Throughput

This graph shows a packet error rate much higher than expected. Clearly the Priority with Retries is the only method that comes close to meeting the general criterion. The real-time PER is only acceptable for less than 1 Mbps of real-time data. However, this is a lot of bandwidth for real-time data.

We need to gather more data points in this region for the Priority with Retries method to get a more accurate error threshold.

Data PER vs. Real-Time Throughput

This graph shows the data PER. This value should be low to prevent retransmissions from occurring on the transport layer. Here we see that the PER is highest for the two Priority methods, which could have been anticipated. These values when using Priority methods become rather high at the higher real-time throughput rates. But these high rates aren’t practical because the real-time PER is much too high at those rates.

Conclusions

The data that we gathered was somewhat limited, but with it we were able to show acceptable performance of the Intellon network, with real-time throughputs of under 1Mbps, using Intellon’s priority mechanism with retries turned on. We also showed that the priority mechanism is necessary for acceptable performance of the network at high real-time rates.

The network performance was troublesome for two metrics however: The data throughput metric was not as large as our expectations, even for comparatively low real-time data rates. Also, the packet error rates were rather large for the data packets.

References

[1] Dimitri Bertsekas, Robert Gallager, Data Networks, 2^nd Edition, Prentice-Hall, Inc. 1992.

[2] Homeplug^TM Alliance, Homeplug Specification 1.0, 2001.

[3] IEEE 802.1D Standards Committee, IEEE 802.1D Standard 1998 Edition, 1998.

[4] Telecommunications Research Associates, Understanding Voice over IP^TM, 1999.