Real-Time Industrial Ethernet: Mini Literature Survey

Poovappa Subbaiah (Dinu)

Introduction:
Fast real time data exchange between controllers and sensors/actuators are normally
performed in the Factory Communication networks. Vendor specific fieldbus solutions
dominate the Industrial network market. While these fieldbus solutions have a much
smaller market and therefore remain high cost, Ethernet solutions are extremely cost
efficient. This along with the higher data rates offered by Ethernet has prompted
numerous studies on its use in factory networks.

An end to end communication, integrating factory data with enterprise application is

seeing new trends in the automation world today. High speed, low cost and dominance
in the office networks is making Ethernet the prime communication protocol for
automation networking.

While Ethernet offers numerous advantages over fieldbus, some problems associated to
its non deterministic behavior prevent its deployment on a wide scale. Standards such
as IEEE 802.1p and Fieldbus HSE have been introduced to address this issue. Although
it is impossible to determine the arrival time of queued messages in Ethernet networks, a
switched architecture helps in eliminating the non determinism due to CSMA/CD by
separating collision domains. A high overhead is also a feature of the Ethernet when
small factory data packets need to be communicated. While several such factors have
been the deterrent for the use of Ethernet for low level real-time factory data
communication, a plethora of studies and information suggest different approaches to
achieve real time traffic on Ethernet networks.

This document is a survey of literature on Ethernet for real-time communications in

Industries. The list of research papers that are studied in this survey are given in the
reference section.

1. On the use of Ethernet at low level of factory

communication systems [5]
Introduction: The paper discusses the possibility of using Ethernet for low level factory
communications. The study proposes UDP based protocols that compare with Fieldbus
protocols represented by Profibus DP and WorldFIP. The Master-Slave protocol and the
Producer Consumer protocols are analyzed with Ethernet and the corresponding
fieldbus implementations. The theoretical analysis results for fieldbus are derived from
other works on the subject, and the comparison with the Ethernet is presented in the
paper.

Description: The author emphasizes the changes brought into the Ethernet standards
that enable Ethernet to satisfy real-time requirements. The main obstacle for real-time
use of Ethernet is the non determinism caused by the possible collisions between

Poovappa (Dinu) Subbaiah Real-Time Industrial Ethernet: Mini Literature Survey 1/9
frames deriving from multiple simultaneous accesses of stations to networks. The
author writes “This [collisions] problem is often overrated because suitable network
configurations eliminated collisions”. The “suitable network configurations” include a
switching architecture that eliminates collisions, prioritizations standard 802.1p, etc. The
absence of “a standard and a well settled protocol, which allows for the implementation
of typical functions foreseen at device level” is seen as a critical issue.

The paper deals with Ethernet’s role in the device level communications within the three
tier factory communication systems architecture. The device level communication
requirements pose unique challenges in terms of strict timelines. The main functions of a
fieldbus operating at device level are the cyclical data exchange and the handling of
asynchronous urgent traffic. This is realized by techniques such as Master-Slave and
Producer-Consumer. The same techniques are implemented on Ethernet, with the IP
protocol layer is interfaced to MAC through network adapter module. UDP is adopted
since the packet size for factory data is small and overhead needs to be minimized.

It is important to note that the paper deals

with the device level communications as
identified in the following model of factory
communication systems. The figure shows
the three tiers of factory communication
systems. Ethernet is already being used for
the the plant and cell level communications.

The figure is taken from Ref.[5].

The essence of the Master-Slave protocol Ethernet implementation foresees the

presence of just one station which can arbitrarily access the network, while others can
only respond to specific interrogations. In other words, the Master sends a request frame
to a slave to poll and gets the answer with the response frame. The Producer-Consumer
on the other hand requires each station to be supplied with a list of the variables it can
produce. The producers of the variable, upon receiving a request, send a packet with the
data, which can be picked up by the consumers.

The cycle time for Ethernet Master-Slave, neglecting the acyclic activity is expressed as:

TMSc = N
i=1 Tslv (i) + 2NTif
Where N is number of slaves, Tif is inter packet time and Tslv is the time employed to poll the i’th slave

For the Producer-Consumer, based on a few assumptions the cycle time is given as:

TPCc = Tout + Tin + Tift where Tout is the time necessary to transmit all the output variables, Tin is the time
necessary to read all input variables and Tift is the total inter packet time.

The performances of the two real time protocols analyzed for Ethernet is compared with
those of two fieldbuses Profibus DP and WorldFIP. The Profibus DP performance is
analyzed in Ref.[6] and the results are substituted here for comparison. Similarly, for the
Producer-Consumer, the performance is evaluated based on comparison with the
analysis in Ref.[7].

Poovappa (Dinu) Subbaiah Real-Time Industrial Ethernet: Mini Literature Survey 2/9
Conclusions: The Master-Slave protocol, based on 100BASE-T Ethernet is seen to
perform significantly better than Profibus DP. The comparison is done based on the
cycle time achieved. Similar comparison between Producer-Consumer on 100BASE-T
Ethernet is seen to perform better than WorldFIP. However, the data coding efficiency is
somewhat lower with Ethernet.

Although the theoretical comparison shows that Ethernet performs better, the
assumption of no collisions upon network access by Ethernet devices needs to be
satisfied. Also, the implementation of the protocol in the factory data acquisition and
distribution devices may not be trivial.

2. The Road to an End-to-End Deterministic Ethernet [2]

Introduction: The authors of this paper see the need for introducing a prioritization
mechanism into the protocol stacks and give arguments for applying these mechanisms
within automation networks for deterministic Ethernet behavior. The argument that the
latency behavior within the end nodes causes severe jitter is presented and studied in
the paper. The paper also presents the results of experiments using priority queuing,
showing very promising results.

Description: The problems of fieldbus such as cost and bandwidth, is seen as a

motivation for Ethernet’s candidature for automation networks. However, Ethernet
comes with a different set of problems in relation to factory data communications.
Ethernet’s democratic nature where all nodes have an equal chance of accessing the
network, unpredictability and a fairly large overhead pose a problem in its
implementation in factory communication networks.

While most of these problems are being studied and solved, the issue of end to end
predictability requires the end nodes to also be precise. While there are studies
confirming that most of the end to end latency is a result of delay at the end nodes, the
authors believe that a study of QoS for an embedded environment and an evaluation of
performance are necessary and have not been done before.

Switching is seen as a way to separate collision domains but the queues associated with
the switches introduce non deterministic delays. In the case of broadcasts, the Ethernet
switch has to route this data to every drop link in the system resulting in packets of same
priority landing in the same queue. Also, within each node there is usually a single
hardware queue associated with the Ethernet controller which also contributes to delay
in certain circumstances.

The IEEE 802.1p alleviates the switch queue problem. A simple protocol associated with
the VLAN identifier in the TCI (Tag Control Info) allows for the removal of all
unnecessary drop link traffic in an automation network based on “publish and subscribe”.
Network data transfer time only forms a small part of the delay and to improve the
latency performance the concept of priority must be extended to the protocol layers in
both the sending and receiving end. This is accomplished, in the paper, by matching
process priority to packet priority and running the associated task when the packet

Poovappa (Dinu) Subbaiah Real-Time Industrial Ethernet: Mini Literature Survey 3/9
arrives. Some issues that could emerge with this implementation are also discussed in
the paper.

The figure (taken from the paper)

shows an example of a stack
implementation that would cure
most of the queue problems of the
standard implementation. The
solution consists of introducing a
thin layer between the Ethernet
controller driver and the protocol
multiplexer.

Overall, an implementation
wherein multiple priorities based processes or reentrant code (that can immediately
execute the receiving of the higher priority packets) is suggested for the receiving end. A
multiple priority based transmission queues seem to address the issue of buffer delays
for the sent packets. Finally, a measurement setup with an application sending a time-
stamped packet to the slave node and receiving it back was implemented and used for
measurement. Tests were conducted based on various criteria.

Conclusion: The paper addresses the issue of achieving application-to-application

predictable transfer time on Ethernet. The introduction of IEEE 802.1p standard has
allowed latency to be calculable in switched Ethernet infrastructure. Introducing a
protocol stack QoS mechanism and testing the end to end performance based on
several criteria shows very positive results in terms of predictability and control of an end
to end communication. Taking the priority criteria for transmission and reception of
packets is shown to reduce the latency for real time traffic and also introduce
determinism.

3. Genetic Algorithms for Industrial Ethernet Network Design[4]

Introduction: Ethernet is intrinsically non deterministic, while industrial communication
networks must provide bound end to end delay. The use of optimization and design
techniques for network control can result in better QoS. A genetic algorithm, for
optimization and network design is proposed and investigated in this paper. The
proposal includes finding the best distribution of industrial devices on a switched
Ethernet architecture.

Description: The design of efficient topologies and the distribution of devices on

different switches are done through graph partitioning techniques. While the graph
partitioning is a NP-Complete problem, several heuristics exist that simplify the problem.
This paper is a proposal and evaluation of such a heuristic method, the Genetic
Algorithms.

While a bad management of network cabling can create bottlenecks, adhering to some
optimization techniques, fault tolerance and real time scheduling enables the Ethernet to
be used in safety critical applications with hard real time constraints. The graph
partitioning techniques are used to find a good solution that groups the devices on the
switch according to their co-operation, and minimizes the inter switches exchanges.

Poovappa (Dinu) Subbaiah Real-Time Industrial Ethernet: Mini Literature Survey 4/9
The weighted graph partitioning problem is stated as “Given a graph with vertex and
edge weights, partition the vertices into disjoint subsets such that sum of the vertex
weights of each subset is close to the average sum, and the total cost of the cut edges
(edges that connect vertices in distinct sub domains) is minimized”. In other words, if we
know the amount of data flow between devices, the optimized network topology would
keep the devices that communicate with each other under one distinct switch such that
the data bandwidth under each switch is about the same and the data between switches
is minimized. The optimization objective is to minimize cross traffic flows between
federative switch and maximize traffic on the second level switches.
The figures taken from the paper illustrates the objectives clearly;

The genetic algorithm is a Meta heuristic inspired by the genetic processes of biological
organisms. A typical genetic algorithm can be briefly illustrated as follows.
1. Create initial population.
2. Choose parent1 and parent2 from population
3. compute offspring = crossover(parent1, parent2); mutation(offspring);
4. if ok replace(population, offspring)
The metrics used to determine the quality of the solution are the sum of all edges which
connect a vertex in one partition to a vertex in another partition and the balance between
groups (grouping efficacy).

Conclusion: To demonstrate the algorithm, a traffic matrix consisting of device ids’ and
their communication with other devices is taken. A ‘1’ shows that a particular device
communicates with the other device while the rest have a null. The java implementation
of the algorithm is shown to generate a better or equal solution than the RSB algorithm.
The evaluation is done by using the survey of [8] and [9] and the results demonstrate the
importance of having an algorithm to design the network topology. The objective is to be
able to modify topology by using different kinds of switches until the required QoS is
achieved.

4. Fuzzy Traffic Smoothing: an Approach for Real-Time

Communication over Ethernet Networks[3]
Introduction: Soft real time applications in industrial data networks do not require
determinism. This is the premise with which the paper proposes an approach of
delivering a statistical bound packet delivery. A solution for realizing a real time statistical
channel on a shared Ethernet, called Traffic Smoothing was introduced in [10]. The idea
is to provide statistical guarantees on timely delivery of Ethernet packets by keeping the

Poovappa (Dinu) Subbaiah Real-Time Industrial Ethernet: Mini Literature Survey 5/9
total arrival rate of new packets generated by nodes below a given network-wide input.
This approach is extended to provide dynamic traffic smoothing in several studies and in
this paper, these dynamic techniques are improved.

Description: In this paper, improvements to the dynamic traffic smoothing techniques

are introduced in two respects. The overall throughput and the number of collisions
observed over an interval are used as input parameters for the smoothing. This results in
a more complete indication of the actual network workload. Also a fuzzy controller
dynamically gauges the smoothing action according to the actual workload.

“Traffic smoothing” deals with the timely delivery of packets in statistical terms. In other
words the probability P that a packet is lost during its transmission or misses its deadline
is less than a certain loss tolerance. Architecturally the traffic smoother is inserted
between the TCP/IP and the Ethernet layers. The smoothing occurs based on the credit
bucket depth and refresh period parameters. Every refresh period credits not exceeding
the credit bucket depth are replenished and the packet is sent to the Ethernet layer only
if credits are available. The real time traffic is unaffected by the scheme and the
parameters can be tweaked to produce desirable results.

In dynamic traffic smoothing these criteria is distributed among stations that really need
to transmit. A fuzzy controller is also introduced which, according to the total throughput
and the number of collisions locally detected, applies rules to choose the most
appropriate station input limit on a case by case basis. Three variables, throughput,
collisions and refresh period form nine combinations with three possible values of high,
medium and low. The fuzzy controller interpolates according to fuzzy arithmetic to
generate the output.

A Linux test bed connected with 10BASE-T Ethernet and a collision diameter of 10
meter was used on 11 workstations. One of these was a monitoring workstation, whose
task was to configure the system parameters, activate and deactivate Real-Time/Non-
Real-Time messages etc. The test results were compared with the HIMD (Harmonic
Increase Multiplicative decrease on collision, traffic smoothing technique) approach test
results.
Conclusion: While several tests were run, the most interesting were the ones where the
roundtrip time was measured in the scenario with both RT and NRT traffic. The fuzzy
approach shows a much lesser delay of the RT traffic even when there are bursts of
NRT traffic, when compared to the HIMD approach. This improvement in performance is
also proved by the comparison of the total network throughput, where the throughput vs
workload graph is more regular and higher throughput is achieved.

5. Deterministic Real-Time Communication with Switched

Ethernet[1]

Introduction: Switch technology divides collision domains into simple point to point
connections ensuring collisions no longer occur. This paper evaluates the real time
characteristics by looking at distributions and upper bounds on transaction times. Both
simulation methods and analytical method with network calculus is used to evaluate the
results. The conditions under which the network can be called deterministic are
evaluated.

Poovappa (Dinu) Subbaiah Real-Time Industrial Ethernet: Mini Literature Survey 6/9
Description: A line topology with a device
assigned to every switch, creating a “fieldbus
compliant” cabling concept is considered for the
analysis. Several types of messages are taken into
account such as MT1, cyclic send and receive of
process data, MT2, Alarms etc., MT3, Monitoring
and control, MT4, acyclic transmission like file
transfer. Since the data usually flows to a sink, the
sink link is considered for bottleneck analysis.

Network Calculus is widely applicable for the assessment of the real time performance of
communication networks. While traditional queuing theory deals with probability
distributions, network calculus involves bounding constraints on packet arrival and
service. These constraints allow bounds on packet delays to be derived from which real
time network behavior can be quantified. It is shown that the end to end delay of the
highest order packets is bounded by DH = DHout + (b + rDHout) drsp + Ndrsp + (N + 1)v
Where v is the maximum service time of lower priority packet, N the number of stations,
drsp the service time of the response and DHout the delay of the highest priority packet.

With the Real-Time evaluation, for the line topology, it is shown that an upper
deterministic limit can be kept at 19ms for all message types for a load of p = 0.8, where
p is the ratio of the arrival rate to the link capacity. While the corresponding higher
extreme values for the line and the star topologies are found to be near equivalent, the
median varies significantly. All these results are for the FCFS scheduling. For the PQ
scheduling with the two topologies under similar conditions, the extreme values are seen
to be much lower. The preferential treatment results in a very high upper limit for the
MT4 but this is not important as the ones of higher priority outperform the same ones in
FCFS.

Conclusion: With the upper bounds in transaction time seen in the simulations, we
know that these upper bounds can occur. However, higher values than these upper
bounds could occur if the conditions are changed. Taking an analytical approach to the
same conditions as for the simulation shows that the values are 2 to 40 times higher.
The values in the analytical approach are for the worst case source behavior, while with
the simulation it was random bad cases. The question if switched Ethernet is real time
or not still remains, but as shown in this paper, for a typical application with 50 devices
maximum transaction times on the MAC level within milliseconds could be guaranteed
for real time services. It was also shown that using traffic class with user priorities leads
to a good isolation of time critical data.

Summary:
The papers selected for the survey cover various aspects of real time behavior in
Ethernet for Factory data communications. While all the papers seem to acknowledge
that the priority standard IEEE 802.1p and the switched architecture address this issue,
it is obviously not enough. Approaches based on Network design, algorithms, protocols
and traffic management are studied extensively in the search for a deterministic Ethernet
industrial data communication networks. Each of these approaches is briefly looked into
in this survey.

Each study presents a different approach to the problem. The paper “The Road to an
End-to-End Deterministic Ethernet” Ref.[2] looks into the queuing delays and jitter in

Poovappa (Dinu) Subbaiah Real-Time Industrial Ethernet: Mini Literature Survey 7/9
transmission and reception to address determinism. This proves to be useful but not
complete since among the assumptions is that the network is designed optimally. This is
also looked into in the Ref.[3] where a solution for realizing a real time statistical channel
on a shared Ethernet, called Traffic Smoothing was introduced and evaluated. The
premise is that soft real time applications will only require statistical guarantees; however
the results of the evaluation are encouraging.

The Ref.[5] takes the opinion that the collision problem with Ethernet is overrated and
presents a “fieldbus like” implementation with Ethernet. Comparison is done analytically
and the Ethernet is shown to perform better. However, the implementation seems to be
non trivial and the assumption of no collisions must be realized. The study in Ref.[1]
evaluates the real time characteristics by looking at distributions and upper bounds on
transaction times. Both simulation and analytical methods are demonstrated and the
worst case with the analytical method is suggested as the upper bound, and therefore
deterministic.

The architecture design and optimization techniques for network control can result in
better QoS. This NP complete problem is approached through the genetic algorithm that
is shown to result in a highly optimized design in Ref.[4]. The objective is to be able to
modify topology by using different kinds of switches until the required QoS is achieved.

Overall, some very interesting approaches to determinism are presented in the papers
surveyed. Although determinism, in strictly analytical terms is not achieved all the papers
show that Ethernet has come far on the road to a predictable end to end industrial
automation system.

Poovappa (Dinu) Subbaiah Real-Time Industrial Ethernet: Mini Literature Survey 8/9
Reference:

[1] Jurgen Jasperneite, Peter Neumann, Michael Theis and Kym Watson. “Deterministic
th
Real-Time Communication with Switched Ethernet”. 4 IEEE International Workshop
on Factory Communication Systems, Vasteras, Sweden, August 28-30, 2002.

[2] Tor Skeie, Svein Johannessen and Oyvind Holmeide. “The Road to an End-to-End
th
Deterministic Ethernet”. 4 IEEE International Workshop on Factory Communication
Systems, Vasteras, Sweden, August 28-30, 2002.

[3] A. Carpenzano, R. Caponetto, L. LoBello and O.Mirabella. “Fuzzy Traffic Smoothing:

th
an Approach for Real-Time Communication over Ethernet Networks”. 4 IEEE
International Workshop on Factory Communication Systems, Vasteras, Sweden,
August 28-30, 2002.

[4] Nicolas Krommenacker, Eric Rondeau and Thierry Divoux. “Genetic Algorithms for
th
Industrial Ethernet Network Design”. 4 IEEE International Workshop on Factory
Communication Systems, Vasteras, Sweden, August 28-30, 2002.

[5] S. Vitturi. “On the use of Ethernet at low level of factory communication systems”.
Computer Standards & Interfaces 23 (2001) 267 – 277.

[6] S. Vitturi. “The effects of acyclic traffic on Profibus DP networks, LADSEB-CNR

international report, October 2000.

[7] G. Cena, L. Durante, A. Valenzano. “Standard fieldbus networks for industrial applications”.
Computer Standard and Interfaces 17 (2) (1995) January.

[8] E.W.Kamen, P.Torab, K.Cooper, G.Custodi, “Design and analysis of packet switched
networks in control systems”, IEEE conference on decision and control, Phoenix, AZ,
December 1999.
th
[9] P.Torab, E.W.Kamen, “Load analysis of packet switched networks in control systems”, 25
annual conference of the IEEE industrial Electronics Society, IECON’99, USA, 1999, pp.
1222-1227.

[10] S. Kweon, K.G.Shin, Q. Zheng, “Statistical Real-Time Communication over Ethernet for
th
manufacturing automation systems”, Proc. of the 5 IEEE Real-Time Technology and
Application Symposium, Vancouver, Canada, June 1999.

Poovappa (Dinu) Subbaiah Real-Time Industrial Ethernet: Mini Literature Survey 9/9