Vacuum Controls and Interlocks: P. Strubin
Vacuum Controls and Interlocks: P. Strubin
P. Strubin
CERN, Geneva, Switzerland
Abstract
The vacuum control system is, in most cases, a subset of the general control
system of an accelerator. As such, it shares the architecture and
communication infrastructure of the main control system. Considered as a
‘slow process’ to control in the frame of accelerators, the vacuum control
system can be built using commercial industrial controllers (PLCs). A data
driven approach allows for changes in configuration without changing the
software code but at the expense of a solid database. Modelling the
equipment allows for easy adaptation of a variety of control units with the
same functionality but different physical interfaces. It also allows for a
uniform display of the available data and status values. Interlocks are
required to protect the vacuum equipment itself against abnormal conditions,
but also to protect other systems, like RF, which need a good vacuum to
operate. They are an integral part of any vacuum control system.
1 Introduction
The vacuum control system is, in most cases, a subset of the general control system of an accelerator.
As such, it shares the architecture and the communication infrastructure of the main control system,
but often requires specific solutions at the process level to accommodate a large variety of equipment
(pumps, gauges, valves, interlocks, bakeout controllers, etc.). In an environment like CERN, running
many accelerators (Fig. 1) over many years, an additional challenge is to follow the evolution of the
control techniques while minimizing the modifications to the hardware.
369
P. S TRUBIN
The format of presentation of the acquired data varies, depending on the final end-user and must
allow for easy operation by machine operators as well as detailed diagnostic and fault hunting for the
vacuum specialist.
Interlock strategy and implementation is also an important part of the vacuum control system to
ensure the safe operation of a vacuum system.
Finally, unlike the control systems for most accelerator sub-systems, the vacuum control system
must be up and running at all times, including shutdown periods.
What is illustrated here is that the state of this particular equipment depends on different
stimuli: an external action like a command (here referred to as ‘actuation’), the status of
subcomponents or faults (here referred to as ‘exception’), but also external conditions and time. The
latter is the case of the transition from the ‘Running Up’ state to the ‘Running’ state, illustrated by a
turbo-molecular pump reaching its nominal speed.
On the measurement front, the main datum to be acquired is obviously pressure and, for more
demanding applications, the composition of the residual gas. In many cases, the pressure is evaluated
by the measurement of another signal (ion current, temperature, etc.), via a control unit. In this case
also, the vacuum user must receive a meaningful end value, whereas the controls specialist must be
able to read all signals. Table 1 shows an example of the required variables.
370
VACUUM CONTROLS AND INTERLOCKS
3 Architecture
3.1 Definitions
The most common architecture in modern control systems is the so-called ‘three tiers’ architecture.
The main feature of this architecture is a layered approach, allowing for a more generic approach for
common services, like the communication layers or the application program interface. Figure 3 shows
the way CERN implements this architecture for cryogenics controls [2], and Fig. 4 shows an example
taken from DESY [3].
The first example emphasizes the hardware layers, whereas the second one is more software
oriented. It is not always evident to map the various software layers to physical ones. In general,
however, it is more important to have clearly defined software layers and interfaces. The actual
implementation of these layers matters less and, in any case, should not be the concern of someone
developing vacuum controls (or any other process control, like cryogenics) who will concentrate on
the lowest layer, close to the equipment.
371
P. S TRUBIN
3.2 Standards
The driving force in modern control systems is the use of industrial standards (e.g. TCP/IP for
communication protocols or Web-based servers for graphical interfaces) and industrial equipment
(e.g., programmable logic controllers or PLCs for process control and data acquisition). The advantage
is of course less expense, multi-vendor solutions as compared to custom-made development. The
maintainability can also be easily outsourced. These solutions may not, however, always cover the
needs, in particular for fast controls (e.g. beam diagnostic and beam steering), but they are suitable for
vacuum controls. On the negative side, it is important to realize that using widely used and
documented communication standards makes the control system more vulnerable to hackers. It already
happened at CERN that the PLC programs controlling the cryogenic plant used to test the super-
conducting magnets for the LHC were blocked by an outside attack. Similarly, it was found that
malicious persons were remotely using the computing resources of a digital oscilloscope (essentially
an industrial PC). In both cases, the equipment was connected to the main CERN backbone Ethernet
network, using TCP/IP communication protocols. It therefore becomes very important to protect the
networks. Using proprietary protocols like Profibus™ at the equipment level also helps.
3.3 Models
Another common feature of modern control systems is to develop an ‘object-oriented’ or ‘model-
based’ approach, where the actual hardware is hidden behind a virtual representation which describes
the data provided or requested by the equipment as well as the dynamic behaviour of the equipment,
independently from the hardware. There exist many different implementations of the same concept,
often motivated by the necessity to stay compatible with previous implementations of the control
system and allow for a smooth transition. The layered approach allows for this and also takes into
account that the lifetime of the lower layer (typically at least 10 to 15 years) by far exceeds the
lifetime of the upper layer (often less than 5 years).
372
VACUUM CONTROLS AND INTERLOCKS
Figure 6 shows the way the lower layer, or equipment layer, has been implemented. The S7
PLC family from Siemens has been chosen in this case, together with Profibus™ and the Siemens
proprietary communication protocol MPI. Not only is there a large diversity of equipment to connect,
but some equipment with identical functionality, like Pirani or cold cathode gauges, comes with
different hardware configurations. This is required in the LHC to minimize costs by using active
gauges (sensitive to radiation) where the radiation level is low and passive gauges with a remote
controller elsewhere. Slave PLCs are used to connect to the equipment which require a local control
process (like a pumping station).
373
P. S TRUBIN
Other equipment, in general self-protected, is connected to the master PLC using deported
input–output modules. Finally, some equipment controllers come with a compatible interface and can
be connected to the master using Profibus™.
One specificity of large modern accelerators is the use of mobile pumping and diagnostic means
to reduce the cost. This requires that the control system be dynamically configured when mobile
equipment is connected to it.
This approach is independent of the control system. Analysis of the control models reveals the
common functionality and capabilities between vacuum devices. This leads to the definition of a class
hierarchy in which the common features are placed at the top. Specialization of these generic classes
provides the interface to more specific vacuum equipment, like gauges. Common functionality, like
pressure measurement, can be added as another abstract class, from which ‘gauges’ would inherit.
Finally, further specialization will lead to the classes representing the real equipment.
374
VACUUM CONTROLS AND INTERLOCKS
The above description covers ‘classes’ of equipment. The PLC controller will manage a number
of instances of every class. Hence, for each instance of equipment, a set of data blocks has to be
defined and created in the memory of the PLC. Therefore additional information is required to define
the physical location of the data blocks for every single instance of a piece of equipment. This is also
done via the ‘DEV_DB’ data block. Figure 9 shows the structure and mapping of the attributes the
‘DEV_DB’ data blocks for some typical devices.
375
P. S TRUBIN
6.1 Functionality
The previous section describes one way to implement the lower or equipment layer of the ‘three tiers’
approach. At this point, the representation of the data is still quite close to the physical equipment,
calling for an additional layer to make a homogeneous interface to the applications (graphical
interface, logging, alarm services, etc.). At the same time, as the processing power of the PLC
controllers is limited and best used to control the process, it makes sense to have a single access point
376
VACUUM CONTROLS AND INTERLOCKS
which is materialized by one or more servers, sometimes called a SCADA server (Supervisory Control
And Data Acquisition). The standard SCADA functionality can be summarized as follows:
– data acquisition;
– data logging and archiving;
– alarm handling;
– access control mechanism;
– human–machine interface including many standard features e.g., alarm display or trending.
As such, the SCADA server can replace the two upper layers of the ‘three tiers’ architecture. In
general, however, a SCADA server also includes a flexible Application Program Interface (API)
allowing access to all its features from an external application that would typically be on the upper
layer of the system.
A specific SCADA product has been chosen by CERN, PVSS from the Austrian company ETM
[6]. The system, illustrated in Fig. 10 is built around an ‘Event Manager’ which dispatches the data to
and from the possible users. Towards the lower part of the ‘three tiers’ architecture, PVSS connects to
the hardware (typically PLCs) via a set of drivers. Towards the upper layer, PVSS provides an
Application Programming Interface (API) and a possibility for processing data in the background (this
is where the control over a distributed process would happen). PVSS also provides the tools to build
so-called ‘User Interface Managers’, overlapping in this case with the ‘Presentation Layer’ of the
‘three tiers’ architecture. Finally, PVSS maintains a database which contains all data from all
connected devices, be they physical or virtual (e.g., a software process).
As for the PLCs, the data to and from the equipment is collected into a data structure, in this
case labelled ‘data points’ (roughly the equivalent of the data blocks in the PLC). The description of
these data points is done via a ‘data point type’ structure, which is similar to a ‘class’ in object-
oriented terminology. A data point contains the information related to a particular instance of an
equipment defined by a data point type and a ‘data point element’ represent one particular value in a
data point.
7 Databases
An important aspect of a modern control system is to avoid hard-coding device-specific features in the
control programs as much as possible. This goal has been achieved by using the hardware and
software structure described earlier. The data blocks describing the equipment in the PLCs are
prepared in a central ORACLE based database. The same database also stores the description of the
PVSS data points. Figure 11 shows the global database scheme used. The ‘survey database’ provides
377
P. S TRUBIN
static information about the layout (typically the geographical locations of the equipment), the ‘LHC
equipment database’ describes how the vacuum equipment is distributed (e.g., to which vacuum sector
it belongs). The native export facilities of the database management system are used to move the data
from one database to the other, via intermediate text files.
In an environment like CERN, it is desirable that a single database structure be used for all
accelerators. This is achieved by having a single set of tables describing all families and types of
equipment to control and a set of tables for every accelerator which is closely linked to the layout
(includes, for instance, geographical information taken from the surveyors database). In this way it is
possible to use the database not only to configure the controls equipment, but also to configure the
way data is displayed on the user interface.
378
VACUUM CONTROLS AND INTERLOCKS
Grouped actions are very often required in a vacuum system. For example, switch on all pumps
in one vacuum sector or close all sector valves. These are normally implemented as a menu item of the
synoptic view (for sector wide commands) or as a menu item of the global schematic view.
379
P. S TRUBIN
Fig. 15: Histogram of the pressure during the warm-up of the LHC helium distribution line
380
VACUUM CONTROLS AND INTERLOCKS
A histogram can also help in identifying beam-related effects (losses, electrically induced noise,
etc.). Figure 16 represents the values returned by an ion-gauge in LEIR while an ion beam was
injected into the accelerator. The equipment used is very new and still quite some debugging is
required to understand the reason for these very unstable readings.
8.5 Alarms
A typical vacuum system includes much controlled equipment, and hence may generate a considerable
number of warnings and alarms. These range from pressure limits being reached to faulty power
supplies or inappropriate valve states. The SCADA server is in charge of generating the alarms,
usually based on a set of rules defined in a database. The SCADA server itself can display and log
alarms, but they are most often also sent to a central alarm server which collates the alarms from all
systems and presents them in an uniform way to the operators in the control room.
Reduction algorithms are used to minimize the number of messages an operator must cope with.
They group similar alarms by vacuum sectors or sometime larger machine areas, like an octant.
Figure 17 shows an example of an alarm screen, where some alarms are indeed reduced. It is possible
though for the operator to display the details of the alarm sources.
Very often, the origin of the fault comes from an external event, like a power failure or a
communication network problem. In this case, the reduction algorithm should make a global analysis
on all systems and hierarchize the display of alarms. For instance, it does not make much sense to
display a list of faulty power supplies if there is no power on the grid in the racks where the supplies
are installed. So far, at least at CERN, the central alarm server does not allow for such overall
analysis.
381
P. S TRUBIN
9 Special cases
382
VACUUM CONTROLS AND INTERLOCKS
Regulation and protection can be significantly improved using a PLC-based system. The
program can automatically detect the time constants of the heated equipment. Measuring the power to
the heating elements, allows for detection of faulty (or wrongly connected) thermocouples. Figure 19
shows how power supervision can prevent overheating a vacuum component.
Table 2 shows a set of possible scenarios for dealing with various faults, showing that the
available processing power allows one to protect against or to recover from a number of failures using
information from other equipment.
Using a PLC-based controller is also economically more attractive. A single PLC can control as
many as 24 channels (the limitation is more the number of electrical connections to make, than the
processing power) with a single interface to the control network.
383
P. S TRUBIN
Short power cut Y (1) Y/N Stop the bakeout Either stop or continue the
(<5mn) bakeout (setting) & generate
an alarm
Long power cut Y (2) Y (2) Stop the bakeout Smoothly top the bakeout
(>5mn) (with a decrease temp.
slope) & generate an alarm
Too high temp. Y (1) Y (1) Nothing Stop the channel, continue
measurement or smoothly stoop the other
channels (with a decrease
temp. slope) & generate an
alarm
Too low temp. Y (1) Y (1) Nothing (100% of the Stop the channel, continue
measurement power will be delivered) or smoothly stoop the other
channels (with a decrease
temp. slope) & generate an
alarm
High pressure Y (3) Y (2) Stop the bakeout (need Smoothly stop the bakeout
hardware interlock) (with a decrease temp.
slope) & generate an alarm
Leak (RGA pick Y (3) Y (1) Stop the bakeout (need Smoothly stop the bakeout
40 leak detection) hardware interlock) (with a decrease temp.
slope) & generate an alarm
10 Web-server-based applications
An alternative to central user interfaces (at the presentation layer) is to use embedded Web servers.
This technique has been tested at ESRF for residual gas analysers and PLCs. Embedded Web servers
are essentially interface modules that can access the equipment on one side and accept html commands
from Ethernet on the other side. Web pages generated by the embedded server can be called from any
browser.
Commercial Web servers come with graphical tools which are fine to develop simple control
panels and display simple graphics, but their limits are quickly reached when approaching any
scientific display. Also, the Web server is not designed to store any important amount of data,
meaning that it does not replace the traditional control path. The main reason to use them is to have
384
VACUUM CONTROLS AND INTERLOCKS
access to local processes even when the main control system is not available (typically during some
shutdown periods). An alternative reason is to allow for quick prototyping of new user interfaces.
Figure 20 shows a Web page produced to control a residual gas analyser. The Web server
module is part of the control unit, itself located in the synchrotron ring and not accessible during
operation. Therefore, the Web interface is exclusively used during shutdown periods, for diagnostic
while the residual gas analyser is connected to the main control system during operation.
A second example is shown in Fig. 21 with the use of an embedded Web server in a PLC
dedicated to the control of valves, shutters, absorbers, and vacuum interlocks in beam lines. Using a
Web server on beam lines appears really useful since the set-up changes very frequently, making
changes and tests much quicker than from the central control system. In this case, the synoptic is
displayed, a point and click approach allows one to open the specific control panel of each piece of
equipment.
Other applications have been successfully tested with this approach at ESRF, like upgrades of
bakeout controllers or even the controls for the recently installed NEG coating facility.
385
P. S TRUBIN
11 Interlocks
386
VACUUM CONTROLS AND INTERLOCKS
387
P. S TRUBIN
References
[1] R. Gavaggio, M. Steffensen and P. M. Strubin, Using models to allow for object oriented
programming of the vacuum control systems, EPAC94, London, UK.
[2] P. Gayet and C. H. Sicard, Deployment and integration of industrial controls: the case of LHC
cryogenics controls, Proceedings of ICALEPCS2003, Gyeongju, Korea.
[3] K. Rehlich, Status of TTF VUV-FEL control system, PCaPAC 2005, Hayama, Japan.
[4] R. Gavaggio and I. Laugier (CERN, Geneva, Switzerland), L. Kopylov, S. Merker and
M. Mikheev (IHEP, Protvino, Russia), Development of the vacuum control system for the LHC.
[5] P.M. Strubin and N. Trofimov, Control and operational models for vacuum equipment, PAC97,
Vancouver, Canada, Eds. M. Coryn et al. (IEEE, Piscataway, NJ, 1998), pp. 2440–42.
[6] http://itcobe.web.cern.ch/itcobe/Services/Pvss/
[7] U. Hahn, M. Hesse, T. Kracht, E. W. Weiner, K. Porges and U. Miße, The residual gas analyzer
system for the HASYLAB beamline vacuum system, Nucl. Instrum. Methods Phys. Res. A
467/468 (2001), 805–808.
388