Reengineering an Industrial HMI: Approach,

Objectives, and Challenges

Bernhard Dorninger∗ , Michael Moser∗ , and Albin Kern†
∗ Software Competence Center Hagenberg, Austria
{firstname.lastname}@scch.at
† ENGEL Austria GmbH, Austria

[email protected]

Abstract—Human Machine Interfaces (HMI) play a pivotal A further interesting aspect concerns the flexibility of the
role in operating industrial machines. Depending on the extension HMI in the context of product assembly. Particularly in series
of a manufacturers domain and the range of its machines as well production, HMIs are not a one-off effort, but are usually
as the possible options and variants, the ensuing HMI component
repository may become substantially large, resulting in significant assembled from a number of reusable components. Having
maintenance requirements and subsequent cost. A combination a high number of machine types with a lot of options and
of cost pressure and other factors, such as significant change variants demands a certain degree of flexibility. Maintenance
of requirements, may then call for a substantial reengineering. for the sake of ad-hoc flexibilizing then often thwarts the
A viable alternative to manually reengineering the whole HMI existing architecture - frequently out of incomprehension of
framework might be the use of (semi)-automated reengineering
techniques for suitable parts. We describe such a model based the originally intended design. Parnas called this ”Ignorant
reengineering procedure relying on static analysis of the existing Surgery” , having an even more negative impact when follow-
source code for suited aspects of a large HMI framework. We will ing a minimal effort strategy in maintenance [2]. Another in-
sketch our overall approach including the objectives and highlight teresting aspect is the HMI part of industrial machine software
some important challenges of transforming HMI component having become more and more an essential - because visible
information extracted from source code into a representation
developed for the completely redesigned HMI infrastructure in - asset in recent years. With the rapidly evolving technologies
the light of an existing product assembly and configuration and design principles of HMIs - at least compared to the
process at a large machinery manufacturer. evolution of Programmable Logic Control (PLC) development
standards like IEC-61131, it has become more likely to subject
I. I NTRODUCTION the HMI to reengineering than the PLC software. This imposes
Reengineering of large software systems is a complicated a somewhat strong requirement in such a way that it shall
and usually labor-intensive task, with the reasons for reengi- become more amenable to recurring reengineering or evolution
neering a productive software system being manifold. In [1], (”Design for success” [1]).
Parnas clusters reasons for software becoming obsolete and The remainder of the paper is organized as follows: In
sketches possible strategies to deal with the inevitable process Section II, we provide an introduction to the industrial context
of software aging. These reasons may include end-of-life of of our project and its most important objectives and require-
key technologies, new requirements difficult or impossible ments, plus its connection with the current HMI assembly and
to fulfill with operational technologies, or just exuberant configuration process. In Section III, we outline our proposed
maintenance cost. reengineering approach in the context of a proposed adaption
Once the decision to reengineer a software system has of the assembly process. In Section IV, we highlight some
been made the extent of reengineering has to be determined. important challenges and how we plan to address or how we
A complete redesign from scratch is not always possible or have addressed them Section V briefly cites related work and
(economically) reasonable, but on the other hand quite often Section VI provides a short conclusion to this paper.
inevitable due to structural degrading of a software system,
often referred to as design erosion[2], which is quite frequently II. C ONTEXT, M OTIVATION , AND O BJECTIVES
caused by maintenance [3]. Another interesting question arises ENGEL is a large machine manufacturer, who offers a
deciding upon manual or (semi)automated reengineering of broad portfolio of machines tailorable to specific customer
specific parts of the software, e.g. by applying model based needs, where each machine can be equipped and configured
techniques. However, many parts of a system may be so with numerous different options. Regarding the engineering
complex a cost efficient model based reengineering is not process, the company covers all relevant fields and disciplines
feasible [4], [5] . from mechanical construction up to development of control
All of this applies to HMIs, too. Additional typical reasons system and HMI software. The productive version of the
to initiate a major HMI reengineering include the introduction latter is still based on a proprietary Java UI library, which
of new and contemporary interaction concepts and/or provid- is long past its end-of-life date and thus no more maintained.
ing a fresh and modern look and feel (L&F). Despite this, the current HMI framework is still maintained

SANER 2018, Campobasso, Italy

978-1-5386-4969-5/18/$31.00
c 2018 IEEE 547
Industry Track
and evolving. To address the growing concerns especially with
maintenance, it has been decided to completely reengineer the
HMI software based on JavaFX [6]. This decision has also
been backed by the fact of evolution in the connectivity to the
control system, as OPC-UA [7] is set to play a key role here
in the future.
In short, the key motivational factors for reengineering the
HMI are:
• End-of-life of base HMI library - including unclear
licensing issues
• Inflexible reflection of product variants and options in the
HMI
• Antiquated L&F of old HMI, outdated interaction con-
cepts Fig. 1. Sample of a HMI screen
• Tight coupling of HMI to PLC system impeding main-
tainability and testability
i.e. which stripes a screen contains. Screen assembly and
A. Characteristics of the Legacy HMI configuration is already automation-supported by the current
Apart from infrastructural components, the HMI consists product assembly and configuration process. The result is
of a high number of screens, with each screen covering a a screen model, which is processed by the machine HMI
certain aspect of a machine’s components or production related application at runtime. However, as mentioned in Section II-A,
tasks. Screens are usually made up of de-facto standardized UI the current process does not fully extend this approach to the
components in terms of L&F and layout behavior, but can also level of stripes - as the stripes themselves are hardcoded.
be a highly specific intra-HMI application. The former are of Stripe options and variants may be present in three types:
interest in our reengineering, whilst the latter are considered OT-1 Hardcoded options in stripes: Typically, stripe code
to be too complex to be processed automatically. adds additional widget rows shielded by conditional ex-
Model based screens consist of so called ”stripes”, which pressions.
in turn consist of one or more so called ”base widgets” (e.g. OT-2 Subclass variants: the stripe code defining the content
an component made up of a descriptive label, a number input exists in the form of an abstract class with subclasses
field and a label for the technical unit) - typically arranged in injecting a hardcoded PLC variable name prefix or some
rows. The building blocks for these base widgets can be more other parameters at construction time.
simple base widgets or can be standard widgets provided by OT-3 Project Clones: a whole stripe project is copied and
the underlying UI library. Base Widgets display process data slightly altered at various spots.
of a machine (PLC variables), e.g. a certain state, selection or
actual value and sometimes also allow alterations of reference The first form of variability is attended to at runtime -
values or set-points. upon starting the HMI. During HMI initialization, the stripe
Where the composition of screens from stripes is already code checks the presence of PLC variables or its value to
achieved via a model, stripes and base widgets are manually decide upon the widgets to be created and displayed. The other
coded. In addition, stripe code may contain hardcoded options, two forms are covered at engineering time by the assembly
typically conditional code blocks creating additional rows in and configuration process, which has to select the correct
the stripe. Conditions typically check the presence or even the stripe code and resource artifacts, depending on the machine
value of one or more PLC variables on the machine. configuration.
Figure 1 shows a simplified example of a screen consisting
C. Reengineering Objectives
of two stripes. The first strip consists of a single line using a
”double input field ” base widget. The second stripe consists of An overall objective of the new HMI framework is allowing
more lines using different types of base widgets. The callout the definition of the stripe contents and options with models.
shows one of the base widgets, which features a description Thus, the screen meta model of the existing HMI framework
label, an input field, a label for the physical unit and an has to be extended to not only allow the specification screens
indicative mark, e.g. for signaling the value exceeding a limit. and the contained stripes, but also to enable the specification
of a stripes content in a straightforward manner. Building new
B. The HMI in the Product Assembly Process stripes and assembling them to screens - manually as well
As mentioned, XYZ builds machines allowing different op- as in the context of the established automated assembly and
tions and variants from a broad range of de-facto standardized configuration process - shall be supported in a comprehensible
core products. This variability also is reflected in the HMI. As way.
of now, the set of screens a machine HMI provides, is decided On the other hand, reuse of stripes from the legacy HMI
at engineering time. This also includes the content of a screen, repository shall be supported, too. Consequently, we identify

548
five main objectives, which dominate our efforts of stripe fragment references is determined by evaluating the conditions
legacy code: from the OI. The references are used to fetch the respective
1) Extract the HMI containment tree of each stripe to a im- fragments from the stripe fragment repository in step (4). In
plementation technology agnostic meta model (Abstract the final step (5), these fragments are combined and the result
User Interface - AUI) is merged into the machine screen model. These steps have to
2) Preserve stripe component attributes, especially the bound be repeated for each stripe in a screen.
PLC variables B. Reengineering of Legacy Stripes
3) Preserve conditional variants and options
In the light of this intended process (enhancement), we
4) Provide repeatable generation of target artifacts, partic-
take the following 4 step approach depicted in figure 3 to re-
ularly screen model fragments for stripe content and
engineering, which relies on static analysis of the stripe source
binding information for PLC variables.
code
5) Resulting representation of legacy stripe reengineering
must fit the existing automated HMI assembly and con-
figuration process
III. A PPROACH
Before elaborating on our approach of re-engineering the
stripe code, we will sketch our approach of enhancing the
existing HMI assembly and configuration process.
Fig. 3. Approach
A. Enhancing the HMI Assembly Process
1) Candidate Analysis: The first step has been a predomi-
nantly search-based identification of the relevant base widgets
used by the stripes and the relevant methods of these widgets
being called during initialization. In addition we examined the
stripe source code regarding structural peculiarities which may
originate in e.g. the programmer’s coding habits to get a rough
idea of the language features the analyzer software needs to
support.
2) Parsing Stripe Source: For parsing and generating the
AST of a stripe, we employed a standard Java Parser provided
by the Eclipse IDE (JDT). The choice for this parser was
natural due to Eclipse being the established development tool
Fig. 2. HMI assembly process (simplified) for HMI components at ENGEL.
3) AST-Based Analysis: The AST ensuing from parsing
Figure 2 shows the intended process for simplification provides the basis for the analysis step, which accomplishes
we focus on the aspects of assembling a stripe and not the the extraction of the required information to the AUI. The
full HMI. The cornerstones of this process enhancements is AUI acts as an intermediate entity for decoupling analysis and
the decomposition of stripes to fragments plus extracting the the following processing step. Information extraction works
information concerning their combination (Option Info, OI) by partial interpretation of the method source code along the
with respect to machine type and available PLC variables. We creation call path of the analyzed stripe.
identify five steps for a single stripe model to be created. 4) Processing Analysis Models: After the analysis the stripe
1) Query Option Info AUI instances are subjected to three processing tasks, which
2) Find relevant PLC variable definitions shall produce the information sketched in III-A.
3) Determine required fragments a) Stripe AUI Decomposition: The stripe AUI instances
4) Gather stripe fragments from Repository gained from the analysis are decomposed into fragments
5) Combine Fragments and merge result model with each none-empty condition branch in a stripe be-
Stripe fragments as well as OI reside in a database. Assem- coming a distinct fragment in context of that stripe. This
bling a single stripe starts by querying the OI for this specific includes PLC relevant information such as a widgets PLC
stripe providing the machine type and the screen the stripe will variable attributes as well as image references and text
be part of (1). The next step (2) looks up the PLC variables keys, which are evaluated to the concrete images/texts
needed for evaluating the conditions specified by the OI. A at runtime. The resulting fragments are stored into the
machine PLC project produces files containing the PLC system fragment repository.
variables (SV) of each machine component. During this step, b) Condition to Fragment Mapping: In legacy stripe code,
the SV files relevant for a stripe are searched for the presence the addition of a widget to a stripe may be coded
of the necessary variables. In step (3), the set of relevant stripe within the block of a conditional branch. In the AUI

549
 from the analysis, this fact is memorized by mapping presence of a PLC variable). In contrast, stripes may contain
the conditions to the partial sub-tree of a container. In code computing their content from PLC values (e.g. the num-
this step, this mapping information is externalized into a ber of input fields depending on a number of installed machine
separate index record in the repository, becoming the key components) at runtime. Static options may be externalized as
part of the OI in the stripe fragment repository. described in III-A, while the latter must be retained in the
c) L&F Attribute Management: An additional task deals with target model. Analysis needs to be able to delimit these two
the HMI widget attributes not directly relevant for the forms.
content of a stripe, but rather its appearance (e.g. color 3) Correctness of the Analysis: The analyzer processing the
and font attributes). JavaFX follows a different paradigm legacy stripe code of course has to be tested. Considering the
with L&F attributes than legacy Java UI frameworks sheer amount of stripe components, it is neither possible nor
like AWT or Swing. While the latter only offer the rational to provide a test case for each of them. We try to deal
direct setting of attributes via API calls, JavaFX strongly with this issue by providing specifically coded pseudostripes,
encourages such attributes to be set via one or more each representing at least one concrete problem pattern (e.g.
centralized stylesheets (a reduced variant of W3C CSS a pseudostripe for testing a correct unrolling of for loops).
[8]). Thus, at the moment this task serves merely to
collect used colors and fonts as a means of verification B. Transformation and Target Model
for the HMI designers (e.g. for detecting special color 1) Eliminating Code Clones: Code clones are described
definitions). to have the potential of causing significant trouble in soft-
ware systems, especially in maintenance. The current stripe
IV. C HALLENGES repository and the HMI assembly process employ altered
The development of the new HMI framework as well as the code duplicates for variability, resembling type-2 and type-
reengineering of the legacy screens and stripes is still work in 3 code clones as described by Koschke [10]. OT-2 options
progress. There are quite a number of challenges concerning from Section II-B are a consequence of the tight coupling
all aspects of the reengineering, which have to be addressed - of stripe code with the underlying PLC API, as there is no
with some of them not solved yet. Due to space reasons we flexible way to attach PLC variables to a stripe or its contained
cannot describe all of them; instead we will pick some of the widgets. However, this option type can be dealt with by our
more important. reengineering approach, as all necessary information resides
in the stripe code. We have to make sure to gather the correct
A. Parser / Analyzer PLC variable names for each stripe subclass.
1) Evolving Codebase: The stripe legacy codebase, which In contrast, processing project clones as described in Section
is the subject of reengineering is still the basis for productive II-B, OT-3 lacks the necessary information concerning the
HMI releases. This implies ongoing maintenance and even stripe project selection criteria, which is buried in the product
the development of additional stripes, which of course may configuration system. In addition, alterations in cloned projects
contain options and variants. While having a manageable are not confined to the source code, but may also affect other
number of newly introduced components is not a challenge artifacts such as resources. This is an issue, which has yet to
per se, the issue here is the changing coding style. With the be discussed.
evolution of the Java language, certain new features are used 2) Target Model Sustainability vs Information Gap: A very
by some of the programmers and must thus be addressed by interesting question is whether the target screen model is
the stripe analyzer. For instance, at the start of the project, none capable of representing the information gained from the stripe
of the stripe components used Java Generics, but their use in analysis. The new screen meta model is being developed
stripe development is increasing, so the AST analyzer has to primarily following the requirements of the new-style HMI
be adapted accordingly. Other troubles are brought by making applications, currently proposing a very simple row-wise ar-
heavy use of collections or other Java API classes, which rangement of base widgets. This poses no problem as long as
require considerable effort to the necessary interpretation of a legacy stripe conforms to this scheme. But as it turned out,
the legacy code AST. This is an issue as described in [9], there is a significant amount of stripes not adhering to this
albeit in a different context. practice, instead introducing additional UI containers with ad-
In this case it is important to weigh up cost and benefit of hoc coded component arrangement (e.g. proprietary layouts)
automation support. For instance, during the candidate analysis resulting in a stripe AUI exhibiting a deep content structure not
it was detected that the stripe codebase contains more than over matching the intended flat structure in the new screen model.
400 instances of for loops, while there was only one instance Thus, our first attempt to neglect layout information proved
of a do-while loop. It is out of question the analyzer ought to be too simplistic, as these cases imply deep structure and
to support interpretation of for loops, but supporting do-while layout information needs to be considered or even retained.
does not seem reasonable. One challenge here is to distinguish between irrelevant -
2) Delimiting Static From Runtime Options: Static options which may be omitted - and significant structural and layout
are options hardcoded in stripe code, whose evaluation is information. Significant information has either to be dealt with
feasible with information present at engineering time (e.g. the by extending the newly proposed screen meta model in such

550
a way that it allows more complex content specifications or well as handling implementation specific event processing.
by representing ”problematic” parts of a legacy stripe with a Wolff and Forbrig [14] propose an approach allowing altering
specific base widget. However, we have to observe carefully reverse engineered HMIs within an UI editor before generating
not to allow the collection of base widgets to grow too the target. A significant work is from Ramon et.al [15]
much creating the same problems which are manifest in the highlighting the relevance of structural and layout information
legacy stripe repository. The work of Sinha and Karim [11] and how to retrieve it from source code.
on compacting layout trees might provide some inspiration
VI. C ONCLUSION
here. Furthermore -as mentioned in Section IV-A2 - runtime
options need to be preserved in a stripe fragment and cannot In this paper, we have described our ongoing work of
be externalized to the assembly process OI. We have to make reengineering a considerably large source code collection of
sure that the target screen model is capable of expressing such HMI components (so-called stripes) of a machine visualization
dynamic constructs. framework. The stripe reengineering is part of a much more
On the other hand, we must stay put for an easily compre- extensive project aiming to provide a completely new base for
hensible screen model. machine visualization applications. The currently hardcoded
3) Target Fragmentation: Considering the proposed HMI stripes shall be replaced by model based definitions, which
assembly process, it is obvious that a stripe usually will not are part of a machines screen and desktop model. We have
be present as a whole except if it has no options or variants. described a possible adaption of the current HMI assembly
It is represented by a number of stripe model fragments, process and have provided an overview of our reengineering
which will be combined depending on the machine type and approach. Furthermore we described and discussed some of
its configuration. This fact implies a few consequences and the important challenges we have come across and are still
reveals some pros and cons. dealing with.
Expectedly, the stripe repository will become extensive, but ACKNOWLEDGMENT
this would also be the case if not relying on a fragmenting ap- The research reported in this paper has been supported by the Austrian
proach. However, fragmented components might be considered Ministry for Transport, Innovation and Technology, the Federal Ministry for
a bit of a downside. A good tool support will be the key here Digital and Economic Affairs, and the Province of Upper Austria within the
frame of the COMET center SCCH.
to allow the simulation of machine configurations resulting in
the generation of the appropriate stripe definitions. As there is R EFERENCES
no stripe code anymore, testing and verification is transferred [1] D. L. Parnas, “Software Aging,” in ICSE ’94: Proceedings of the 16th
to the assembly and configuration process: Does a stripe international conference on Software engineering. Los Alamitos, CA,
USA: IEEE Computer Society Press, 1994, pp. 279–287.
fulfill all expectations given a specific engineering context (i.e. [2] J. van Gurp and J. Bosch, “Design erosion: problems and causes,”
concrete machine setting)? Are all expected widgets present Journal of Systems and Software, vol. 61, no. 2, pp. 105 – 119, 2002.
with the correct type? Referring the correct data points (PLC [3] K. Bennett, “Legacy systems: coping with success,” IEEE Software,
vol. 12, no. 1, pp. 19–23, Jan 1995.
variables)? Regarding the correct look of a stripe, there will be [4] I. Sommerville et al., Software engineering. Addison-wesley, 2007,
a generic renderer, who will interpret the given screen models ch. 28.
and builds the HMI at machine runtime. The same renderer [5] H. M. Sneed, “Planning the reengineering of legacy systems,” IEEE
software, vol. 12, no. 1, pp. 24–34, 1995.
can be used to preview resulting stripes or full screens during [6] (2014) Javafx overview. Oracle. Last visited: 20180110. [Online].
engineering and lay the base for (visual) testing. Available: https://docs.oracle.com/javase/8/javafx/get-started-tutorial/
We expect a substantial benefit arising with this approach jfx-overview.htm
[7] U. Enste and W. Mahnke, “OPC Unified Architecture,” at-
by reducing the number of conditions to check at runtime Automatisierungstechnik, vol. 59, no. 7, pp. 397–404, 2011.
considerably. A concrete screen model deployed on a specific [8] B. Bos and H. W. Lie, “Cascading style sheets (CSS1) level
machine will induce little or no static condition checking at all, 1 specification,” W3C, W3C Recommendation, Apr. 2008,
http://www.w3.org/TR/2008/REC-CSS1-20080411/.
as the model is built from the stripe fragments at engineering [9] H. M. Sneed, “The drawbacks of model-driven software evolution,”
time for that specific machine. As each runtime check for PLC IEEE CSMR, vol. 7, 2007.
variable existence or even its value consumes a fair amount [10] R. Koschke, Identifying and Removing Software Clones. Berlin,
Heidelberg: Springer Berlin Heidelberg, 2008, pp. 15–36.
of time, we hope for considerably lower HMI startup times. [11] N. Sinha and R. Karim, “Compiling Mockups to Flexible UIs,” in
Another advantage is that the approach of decoupling stripe Proceedings of the 2013 9th Joint Meeting on Foundations of Software
content and PLC variable references unlocks the potential of Engineering, ser. ESEC/FSE 2013. New York, NY, USA: ACM, 2013,
pp. 312–322.
getting rid of code clones, as each stripe fragment will exist [12] E. Merlo, P.-Y. Gagné, J.-F. Girard, K. Kontogiannis, L. Hendren,
only once in the repository. P. Panangaden, and R. De Mori, “Reengineering User Interfaces,” IEEE
Software, vol. 12, no. 1, pp. 64–73, 1995.
V. R ELATED W ORK [13] S. Staiger, “Reverse engineering of graphical user interfaces using static
analyses,” in Reverse Engineering, 2007. WCRE 2007. 14th Working
There are certainly yet a lot of efforts in reengineering or Conference on, Oct 2007, pp. 189–198.
migrating a HMI. An early work employing reverse engineer- [14] A. Wolff and P. Forbig, “Model based reengineering of user interfaces,”
in Model Driven Development of Advanced User Interfaces, 2007.
ing and model based reengineering is described by Merlo [15] Ó. S. Ramón, J. S. Cuadrado, and J. G. Molina, “Model-driven reverse
et.al [12]. Staiger [13] describes the reengineering of a QT engineering of legacy graphical user interfaces,” Automated Software
application, amplifying the recovery of content structure as Engineering, vol. 21, no. 2, pp. 147–186, 2014.

551