How to Guarantee that Devices are Properly Time Synchronized?

Carlos Alberto Dutra Sérgio Luiz Zimath Jurandir Paz de Oliveira

[email protected] [email protected] [email protected]

Reason Tecnologia S.A.

Abstract The timing networks are being used in new ventures

not only because of technical requirements but also
The use of time synchronization is essential in because of the requirement of regulatory agencies.
different applications. Time synchronization accuracy
depends on the method being used, the structure of Therefore, it is important to know about different
the time network and the implementation of time time synchronization methods, considering the
servers and IEDs. This paper discusses these topics characteristics of each method and how to ensure that
and presents methods to help users to be able to all devices, time servers or time clients, are operating
check the quality of their devices. properly.

1. Introduction Regarding time synchronization, it is also important

to understand the concept of accuracy and precision
The use of time synchronism in substations and as it is possible to have an IED synchronized
power plants has been increasingly expanded and is precisely but not accurately, as is shown in this paper.
essential in applications such as synchrophasor Figure 1 shows the difference.
measurement and traveling wave-based fault location
because of the need of accurate data synchronization
of several units. However, time synchronization is
also related to protection systems monitoring, not
only for the digital fault recorders, but also for remote
terminal units and protective relays.

Normally, the internal clocks of different devices use

high drift oscillators and crystals. Such devices
require external references to control their internal
clocks to provide timestamp accuracy. Some devices Figure 1 – Difference between accuracy and precision
have to measure angle quantities with high precision,
In this paper, Section 2 presents some information
such as Phasor Measurement Units (PMU). To
related to the structure of the time synchronization
provide this required precision, the internal clock and
networks. Section 3 describes the characteristics of
the acquisition system need to be time synchronized
the different methods of time synchronization.
with good accuracy.
Methods to evaluate GPS clocks in laboratories and
The use of time synchronization also provides a in substations are discussed in Section 4. Section 5
precise timestamp of events in the power system, in presents a methodology to check IED time
order to compare the behavior of such events in synchronization, and finally, Section 6 presents the
different parts of the system. conclusions.

In the absence of time synchronization systems, the 2. Structure of the Time Synchronization
fault and disturbance analysis using oscillography Networks
records requires manual adjustment, increasing the
Time servers generally provide the time reference to
time and the effort for the analysis.
several devices in a system. These time servers are
typically installed in racks and are connected to IEDs
creating a time synchronization network. Considering and time can be obtained. The precision of the GPS
that different IEDs support different methods of time system follows a stochastic model and normally GPS
synchronization, it is important to evaluate which receivers provide a precision of ±100 ns for 1 sigma
method is more appropriate for the whole system. (~70%). Figure 2 shows the curve of precision
distribution.
The connection between IEDs and time servers
depends on the cable installation services and
requires design and implementation by skilled
professionals.

Before establishing the structure of the time

synchronization network, it is important to consider
some different aspects that impact on the costs of the
system:

 Project: requires skilled professional to design Figure 2 – Curve of precision distribution

and document the network structure considering
Based on this precision the GPS clocks generate
the characteristics of each device to be installed.
different types of time signals.
The technological level of each device can
impact on the network topology and can bring There are other systems similar to the GPS controlled
project difficulties. by the U.S. such as the European Galileo and the
 Implementation: requires specific professionals Chinese Beidou. However they are not popular
to build the network structure, to purchase among non-military, and do not provide the same
materials such as cables and conduits, and to precision and global coverage.
manage civil works.
 Expansion: requires specific professionals to The methods of time synchronization commonly used
analyze the network structure and the devices are the pulsed electrical signals such as PPS (pulse
that are already installed and those that would be per second) and IRIG-B, specific serial datagrams
installed, as well as to adapt the project and the such as NMEA, and sophisticated protocols such as
documentation. Additional costs related to NTP and PTP (IEEE 1588), which allow time
further implementations must be considered. synchronism using Ethernet networks. The evolution
 Maintenance: the whole system must be of these methods occurred with the technological
maintained in service and some maintenance may evolution of the devices. However, with the fast
be necessary. paced changes in technology, there are also questions
from users about the best method of time
Normally time synchronization infrastructure costs synchronization and what the benefits and limitations
are higher than the time servers costs. of each one are.

3. Methods to Provide Time Synchronization It is also important to note that although an IED is
capable to be synchronized by IRIG-B, NTP, or other
Technically there are several methods to provide time protocol, it does not mean that it will necessarily have
synchronization, each one related to a certain its internal clock very accurate. It is very common to
topology. Therefore, all of them are basically find implementations in IEDs, such as protective
associated with the use of GPS-based clocks. relays, that give high priority to protection functions
and do not achieve a good precision.
The GPS (Global Positioning System) is based on a
satellite constellation orbiting the Earth. These 3.1 Pulsed Electrical Signals
satellites transmit signals that are decoded by the
GPS receivers. With the information received by at The use of pulsed signals is the simplest method for
least four satellites, the longitude, latitude, altitude time synchronization. The time server generates
pulses synchronized to the second rollover (PPS) or group in charge of creating the standard for the
other pre-defined frequency. The equipment to be Range Commanders Council (RCC) of the U.S. army.
synchronized adjusts its internal clock with this The first edition was launched in 1960 and the most
signal. recent version, IRIG Standard 200, was published in
2004.
The precision obtained with the pulsed signals is
higher, especially when used for frequency control. IRIG-B signal is a 100-bit frame, containing
However, the main disadvantage is that the information related to the date, time and optionally,
timestamp information is not transmitted and clock control functions (CF) and straight binary seconds of
changes are not considered. day (SBS). This signal is typically transmitted in a
PWM format by a DC level shift signal (no carrier
Time synchronization networks demands cabling signal) or by an amplitude-modulated signal (sine
installation and maintenance, as well as skilled wave carrier). Figure 3 shows these signals.
people.

The status of the time servers cannot be monitored.

This method is rarely used in power systems,
although the PPS signal was considered to provide
time synchronization to Merging Units in the edition
1 of IEC 61850.

3.2 Serial Datagrams

Serial datagrams are data frames transmitted to serial

ports (usually RS232 or RS485) and provide the
Figure 3 – PWM format for the IRIG-B signal
timestamp and sometimes time servers status. Such
information is typically transmitted once a second, The use of IRIG-B time protocol can provide the
either in the beginning of the second (first character precision in the order of nanoseconds, but it requires
starts in the second rollover) or immediately before a lot of care in the project to meet this precision.
the second rollover (last character ends in the second Additionally, the architecture of the time
rollover). synchronization network must consider the way that
the IEDs are connected to the time server, the use of
Despite timestamps information being transmitted,
time repeaters and transceivers, the level of electrical
serial datagrams precision is low due to serial
insulation, etc. In addition to the cost of design, the
communication limitation. Sometimes PPS signal is
cost of implement a dedicated network for time
used in conjunction with serial datagrams to increase
synchronization should also be considered even if a
the precision.
data communication network is in use.
The time synchronization network requires an end-to-
IRIG-B time protocol is widely used by electric
end connection. The expansion and maintenance are
utilities to provide time synchronization of power
not simple and requires skilled professionals.
system devices such as protection relays and Digital
Time server status may be monitored if the serial Fault Recorders (DFR), although it is being gradually
changed by methods of time synchronization based
datagrams support status information. Time
synchronization by using serial datagrams is common on Ethernet networks.
for meters and remote terminal units.
3.4 Network Time Protocol (NTP) and Simple
Network Time Protocol (SNTP)
3.3 IRIG-B

IRIG-B is the widely used time format developed by The NTP (Network Time Protocol), was developed
the Inter-Range Instrumentation Group (IRIG), the by professor David L. Mills of Delaware University
in 1980, and has been used to provide time  When the network traffic level is low, especially
synchronization of devices connected in a Ethernet without GOOSE messages.
network. The NTP protocol has the following  When the NTP/SNTP client that is running in the
characteristics: IED is able to adjust the internal clock with good
precision.
 Since time information is transmitted by using
the Ethernet network, the protocol considers the Time synchronization by using the Ethernet network
measurement of time to send and receive the provides a stronger reduction in the costs because the
messages, allowing the delay compensation cost of design and implementation of a specific time
caused by cables, switches, signal adapters and network is not considered.
other devices.
 Uses an algorithm that adjusts the time clock of 3.5 Precision Time Protocol (PTP)
the IED monotonically, i.e., without “jumps” in
When analyzing the characteristics of the NTP
time to avoid discontinuities. Thus, it does not
protocol and the IRIG-B time protocol, it is possible
simply copy the timestamp received in the NTP
to realize that the NTP has the advantage of using the
message to the internal clock, but it adjusts the
data communication network, however, the precision
frequency controller to decrease the timestamp
is lower. On the other hand, the IRIG-B signal
error. The disadvantage is the delay to converge.
provides good precision but needs a dedicated
 Uses the time clock drift estimation to
network for time synchronization.
compensate time deviation and provide time
stability in the absence of the time source. The PTP protocol was developed to provide time
synchronization with the same precision as that of the
It is important to note that most of IEDs with the
IRIG-B signals by using the Ethernet network.
capability of time synchronization with Ethernet
protocols do not use the NTP protocol, but instead The use of the PTP protocol achieves higher
they use the SNTP, where the S means SIMPLE, i.e., precision over Ethernet communication because of
a simplified implementation of the method that does the support in hardware. The time information in the
not consider some algorithms and does not keep the frame is established only when the message is being
internal clock stable over long periods of time. The transmitted and being received in the Ethernet port.
SNTP protocol messages are equal to the NTP In this approach, the time processing delays
protocol. (especially the nondeterministic) in GPS clocks,
switches and IEDs are compensated.
The first edition of IEC 61850 considered the SNTP
for the IED time synchronization, however, with this Figure 4 shows the nondeterministic delays in a
protocol it is not possible to achieve the required communication between a time server and a client
precision when synchronizing Merging Units. without considering any kind of compensation (NTP
and PTP without hardware support).
The use of NTP/SNTP protocols for time
synchronization of IEDs can lead to a precision better The delays ∆t1, ∆t2, ∆t3, ∆t4 and ∆t5 vary because of
than 1 ms. Real world implementation in substations the protocol processing times, of queues in lower
with many switches usually achieve accuracy layers of the network communication process, and
between 2 and 10 ms. Laboratory tests show that because of the delays in the switches considering that
higher level of precision can be achieved under the the GOOSE messages are prioritized.
following conditions:
Considering the use of the PTP protocol with
 When using a good quality GPS clock. There are hardware support in all devices of the network, the
GPS clocks that support NTP messages but the errors are minimized because the timestamp is
timestamp in the message does not have good assigned in the lowest layer of the communication
precision. process. Thus, all errors related to the processing time
are irrelevant. Additionally, it is mandatory to use synchronization network in conjunction with the data
switches capable to compensate the delays related to communication network.
prioritization of GOOSE messages. Figure 5 shows
the behavior of timestamps when using devices with 4. Evaluation of GPS clocks
hardware support to compensate the delays.
The choice of which GPS clock will be used depends
on the technology and the architecture of the time
synchronization network, the methods of
synchronization, the auxiliary equipment, and the
IEDs to be synchronized.

It is important that users be able to evaluate if the

time reference equipment are providing suitable
signals to the devices that need to be synchronized.
And, how to guarantee that these time servers are
Figure 4 – Nondeterministic delays in a
operating correctly?
communication between a time server and a client.
Basically, all GPS clocks have an indicator that
signalizes if the internal clock is “locked” with the
primary timing source. "Locked" means that the
equipment was able to recognize the GPS satellites,
decode the received signals and finally adjust its
internal clock to provide time signals within the
specified precision. If the equipment does not
indicate this condition, there is the first evidence that
there is some failure in the time system.
Figure 5 – Delay compensation using hardware
support for PTP protocol It is important to note that the time system does not
depend sole on the GPS satellites. Both the antenna
PTP protocol is commonly used in
and the cable that connects the antenna to the
telecommunication systems. In power systems,
equipment have to be adequate. It is common practice
considering technological aspects, by using the
to install cables with characteristics of impedance and
concepts of IEC 61850 in new ventures, the natural
attenuation improper for the required length, as well
tendency is the use of PTP protocol as the alternative
as the antenna is installed in places with low visibility
to the dedicated time synchronization network. In this
of the sky.
scenario, to provide the time precision required for
the system, it is imperative that the network structure Once the GPS clock has indicated "locked", it is
be properly designed. possible to consider that probably the precision of the
equipment met the specification.
Because most of the devices on the market do not
support the PTP protocol, an alternative is the use of For the evaluation of GPS clocks two specific
small devices especially designed as PTP clients, situations will be considered: laboratory testing and
which locally provide time synchronization protocols field monitoring.
such as IRIG-B signal in outputs proper to be
connected directly to the IED. Therefore, if the 4.1 Laboratory Testing
structure of the communication network is
appropriated to the use of the PTP protocol, it is In this section the GPS clocks are analyzed based on
possible to install IEDs not compliant with this time the comparison of signals produced by the equipment
protocol and take advantage of having the time under test and a reference equipment. In the test, the
behavior of the GPS clock when generating
frequency signals and network protocols such as NTP sends request messages and the server responds
and PTP are evaluated in a controlled environment. indicating the timestamp of the moment when the
request message was received and when the response
4.1.1 Frequency Signals message was sent.
Initially the type of frequency signal to be monitored To evaluate the behavior of the NTP servers, a typical
is established. Typically the PPS (Pulse Per Second) NTP client is configured to be synchronized by
signal is used. different NTP servers simultaneously.
Considering the equipment under test and the Two GPS clocks from different manufacturers were
reference in service for a previous period of time, the configured as time servers for a computer with an
pulsed outputs of both pieces of equipment are NTP client. The reference GPS clock was considered
simultaneously monitored by using an oscilloscope in as the preferable server so that the computer's internal
trace mode and triggered by the reference equipment. clock is directly synchronized to this source. The
After several hours, the time differences between time differences between the reference server and the
pulsed signals generated by both units can be equipment under test are evidenced by the differences
graphically displayed and the statistical errors for reported by the NTP client. The servers and the client
these signals can be evaluated. were directly connected to a switch creating an
isolated network to minimize the impacts of delays
Figure 6 shows the comparison of the PPS signal
caused by network traffic. No GOOSE messages
generated by two GPS clocks from different
were transmitted during the test.
manufacturers.
After some hours of monitoring, based on the
statistical information provided by the NTP client, the
time differences were evaluated and the results can be
seen in Figure 7.

Figure 6 – Differences of PPS signals between the

equipment under test and a reference

Statistically, it is expected that in 70 % of the pulses,

Figure 7 – Time differences between the reference
the difference of the PPS signals must be lower than
NTP server and the equipment under test
200 ns (considering that the precision of each piece
of equipment is ±100 ns for 1 sigma). Based on the At the beginning of the test, the errors are naturally
test results, the differences of the PPS signals are in higher because the NTP client is not time
accordance with the statistical errors expected for the synchronized. After a period of time, the NTP client
test. stabilized and became synchronized to the reference
server. After that, the time differences evaluated were
4.1.2 NTP Protocol
lower than 50 us.
The NTP protocol is based on a client-server
Based on the results, the NTP server of the GPS
architecture where the equipment to be synchronized
clock under test had a good performance.
4.1.3 PTP Protocol Considering the results of the test, it is noted that the
precision of the PTP internal clock of the equipment
Usually, GPS clocks compliant with PTP protocol under test is in the order of nanoseconds.
(IEEE 1588) are very precise clocks. The higher
precision is provided by the hardware support to the PTP has a very important advantage in comparison to
protocol. A simple way to evaluate the precision of NTP and IRIG-B, namely the built in redundancy
the timestamp provided by the PTP protocol is to use support. To have a time server redundancy simply
the event input commonly available in GPS clocks install a second time server. The IEDs will
like that. The events generated in this input are synchronize to the most accurate server available in
reported with the precision of the PTP internal clock. the network.
In the test, events produced with high precision
timestamps allow the comparison between the 4.2 Field Monitoring
timestamps reported by the event input and the real
Monitoring the equipment in the field is essential to
timestamp of the event, and thus, evaluate the
ensure that the whole synchronization system is
precision of the GPS clock.
operating properly. This monitoring can be done in
For the test, the PTP client was configured to the most basic form, by using a simple contact failure
generate PPS signals from a TTL output. This signal indicator associated with the SCADA, or by using
was connected to the event input of the reference sophisticated protocols for on-line monitoring.
GPS clock and the events related to the PPS signals
In general, each manufacturer offers their own
were reported in the event log for a period of 5
software for the equipment configuration and
minutes.
monitoring. By using the monitoring software it is
Figure 8 shows the connection diagram. possible to check the information provided by each
device such as the synchronization status, number of
satellites in use, etc.

Depending on the number of GPS clocks to be

monitored in a whole utility, accessing each one
individually becomes a hard task. The use of some
network protocols such as SNMP (Simple Network
Figure 8 – PTP client testing Management Protocol) provides the automatic
monitoring of all GPS clocks in the utility. Based on
The difference between the timestamps of the events
this protocol, it is possible to periodically request
and the related PPS during the test is shown in Figure
information from some equipment. By using special
9.
algorithms, it is possible to automatically evaluate its
120 status, and thus, warn operators of a failure and
100
consequently, have a specific problem solved more
quickly.
80

60
Monitoring makes it possible to check the behavior of
the equipment and the installation characteristics.
40
Monitoring the number of visible satellites by each
20 piece of equipment allows to evaluate the position of
0
the GPS antenna and, with the level of signals from
satellites gives an indication of the quality of the
Figure 9 – Difference (in nanoseconds) between the cable that connects the antenna to the GPS clock.
timestamp of the PPS provided by the PTP client and Furthermore, it is possible to generate statistical data
the timestamp reported by the event input related to the equipment such as the number of
operating hours, network communication access, etc.
The tendency for the future is that monitoring of GPS using a standard switch and, being a lower traffic
clocks will be accomplished using the approach of level environment.
IEC 61850. The second edition of the standard
provides the settings for the logical node related to 5.1 Test Results
time synchronization, leading to the use of
All IEDs were synchronized by the reference GPS
monitoring methods like any other IED compliant
clock. For all IEDs under test, the results with IRIG-
with IEC 61850.
B and SNTP were equivalent. Since the events were
For the optimal operation of a GPS clock, it is related to the PPS (one pulse per second) signal, the
necessary that at least four GPS satellites be visible. timestamp of the event reported by the IED was
Some equipment have special features such as compared to the second rollover and shown in
oscillators with low drift or drift compensation milliseconds.
algorithms that keep the internal clock stable for a
A total of 60 pulses were used for each IED. The
period of time even without the minimum number of
results in Figures 11, 12 and 13, show that IED A has
visible satellites.
poor accuracy and precision. IED B has expected
5. Methodology for Evaluation of the Quality of accuracy and precision and IED C has an accuracy
IEDs time synchronization error of 3 milliseconds but a good precision because
all pulses were with the same error.
In this section a methodology for performance
evaluation of IEDs when synchronized in different 20
methods is presented. To perform this test, different 15
events with known timestamps are produced and the 10
IED event log is analyzed. The comparison between 5
the timestamp reported in the event log and the real 0
timestamp of the event produces the estimate -3 -2 -1 0 1 2 3 4 5 6 7 8
synchronization error.
Figure 11 – Test results for the IED A
The best way to produce events with known
timestamps is by using a GPS clock. Usually these
devices have an open collector output that can be 100
configured to generate signals in frequency. In the
50
test, the PPS (Pulse Per Second) is generated by a
reference GPS clock. The open collector output 0
drives the digital input of the equipment under test. -3 -2 -1 0 1 2 3 4 5 6 7 8
The performance test considered three IEDs from
different manufacturers. Figure 10 shows the
connection diagram for the test. Figure 12 – Test results for the IED B

100

50

0
-3 -2 -1 0 1 2 3 4 5 6 7 8

Figure 10 – Connection diagram Figure 13 – Test results for the IED C

For the SNTP performance test, all IEDs under test

were connected to a dedicated Ethernet network by
The events were applied to all the IEDs [3] “IRIG Serial Code Formats”, IRIG Standard 200-
simultaneously. The IEDs were configured by 04
qualified personnel. All the IEDs were maintained in
service for at least two hours before testing. [4] “IEEE Standard for a Precision Clock
Synchronization Protocol for Networked
6. Conclusions Measurement and Control Systems”, IEEE Std 1588-
2008
The most common methods of time synchronization,
as well as its main features and general applications, [5] “Sincronização Temporal de Equipamentos
are shown in this present work. Usando Rede Ethernet – Análise do Estado da Arte”,
XXI SNTPEE, Florianópolis, Brazil, 2011
Based on the evaluation of different time signals and
protocols in a laboratory environment, it was shown [6] “Abordagem Prática para Redes de Sincronismo
that it is possible to guarantee that GPS clocks are Temporal Baseadas no Sistema GPS”, VIII STPC,
providing precise time information, as well as the Rio de Janeiro, Brazil, 2005
means to analyze the behavior of time servers when
they are in service. Biography

Management of GPS clocks in a single substation, or Carlos Alberto Dutra received his degree in Control
in a whole utility, can be accomplished by using and Automation Engineering from the Federal
SNMP services. Thus, the status of GPS clocks can University of Santa Catarina, Brazil, in 1999. He has
be monitored by a central station, reporting any been working in Reason Tecnologia since 2003. In
malfunctioning of the equipment as well as problems the Reason Research and Development Department,
in the installation such as weak satellite coverage. he worked in the development of different devices
such as the traveling wave-based fault locator, digital
A methodology that makes is possible to evaluate the fault recorders, and GPS clocks. He is currently the
quality of time synchronization of IEDs was Development Manager, and is in charge of the
presented. This methodology was based on the management of the development and the lifecycle of
comparison of time differences between timestamps Reason products.
of events produced by a reference GPS clock and
timestamps of these events reported by the IED. Sergio Luiz Zimath majored in Control and
Different IEDs synchronized by IRIG-B signals and Automation Engineering in 1997 at the Federal
by SNTP protocol were analyzed by using this University of Santa Catarina. He has been working in
methodology. Reason Tecnologia since 1995. He was involved in
the development of several products, including
The performance tests showed that the timestamps RT1000 and RT2000 GPS clocks and the digital fault
reported by some IEDs may not be precise. The recorder RPIV. He is currently the Director of
methodology described in this paper enable the users Innovation and is responsible for the development of
to check the IEDs performance by themselves, new products and research projects.
leading them to know the limitations of each one.
Jurandir Paz de Oliveira received his Ms.C. (1997)
7. References and Electrical Engineering (1988) degrees from
Federal University of Santa Catarina (UFSC), Brazil.
[1] “An architecture for Describing Sinple Network He has been related with Reason Tecnologia since
Management Protocol (SNMP) Management 1995, as partner and Technical Director.
Frameworks”, RFC 3411, 2002

[2] “Simple Network Time Protocol (SNTP) Version

4”, RFC4330,