0% found this document useful (0 votes)
17 views4 pages

Delta Models As A Measurement For Improving The Quality of IEC 61499-Based Control Software

Uploaded by

Yanshi Dong
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
17 views4 pages

Delta Models As A Measurement For Improving The Quality of IEC 61499-Based Control Software

Uploaded by

Yanshi Dong
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Delta Models as a Measurement for Improving the

Quality of IEC 61499-based Control Software


2023 IEEE 28th International Conference on Emerging Technologies and Factory Automation (ETFA) | 979-8-3503-3991-8/23/$31.00 ©2023 IEEE | DOI: 10.1109/ETFA54631.2023.10275452

Prankur Agarwal∗ , Shubham Sharma∗ , Hafiyyan Sayyid Fadhlillah∗ ,


Rick Rabiser∗ , Alois Zoitl∗ , Senior Member, IEEE
∗ CDL
VaSiCS, LIT CPS Lab, Johannes Kepler University Linz
Linz, Austria
{prankur.agarwal, shubham.sharma, hafiyyan.fadhlillah, rick.rabiser, alois.zoitl}@jku.at

Abstract—Industrial-scale control software is designed as Most modern software systems are variability-intensive sys-
variability-intensive, i.e., highly configurable and adaptable soft- tems [5], where systems are designed to be highly configurable
ware, to support diverse hardware capabilities and to fulfill and adaptable to address diverse requirements. Specifically in
diverse customer requirements. Various control software archi-
tectures have been proposed for developing highly configurable the production automation industry, the market demands rapid
and adaptable control software. However, each architecture has adaptation to changing requirements or different hardware
its strengths and weaknesses in configurability and adaptability. technologies [6]. As a result, control software engineers are
Measuring the suitability of a given architecture to deal with driven to design and implement control software that can be
variability can further guide control software engineers in imple- easily adjusted to the changing requirements. Deciding the
menting highly configurable and adaptable control software. In
this paper, we propose measurement approaches for IEC 61499- control software architecture is crucial, as software architec-
based control software that indicate how well a particular ture is essential to facilitate variability management [7].
architecture can manage variability. Control software engineers Existing works have proposed architectural patterns or
are first encouraged to describe all the functionalities in their guidelines to implement IEC 61499-based control software.
system. They can also describe which elements must be added For instance, Sonnleithner et al. [8] have discussed the advan-
or removed when implementing a particular functionality. They
then store these descriptions using a variability mechanism called tages and disadvantages of using hierarchical and distributed
delta models. Next, we can measure the size of delta models, design patterns for IEC 61499. Patil et al. [9] have proposed
the cohesion and coupling of the control software modules, and design patterns for IEC 61499 inspired by existing software
the degree of mapping complexity between features and delta engineering patterns. Mayrhofer et al. [10] have also discussed
models. We argue that by using our proposed measurements, several architectural patterns for developing Cyber-Physical
we can indicate whether the control software’s architecture is
suitable for variability-intensive software. Production System (CPPS) control software. However, the
Index Terms—Delta Modeling, Software Measurements, Soft- software architecture that implements a variability-intensive
ware Adaptability system can introduce “variability debt”, increasing mainte-
nance and evolution efforts [11]. Furthermore, there are no
I. I NTRODUCTION clear satisfaction criteria for architecture quality requirements
IEC 61499 [1] is an emerging standard that defines a leading to high variability in implementing the software [12].
Domain-Specific Modeling Language (DSML) for develop- Having measurements to quantify how well the software
ing industrial-control software [2]. This standard provides architecture can deal with variability can support the decision
a platform-independent application model that can later be to select a proper control software architecture.
linked to a hardware configuration. IEC 61499 mainly utilizes In this paper, we propose an approach for IEC 61499-
Function Blocks (FBs) to implement the control functionality. based control software to measure how well the chosen control
An FB represents an encapsulated algorithm to execute a software architecture can manage variability. Our measurement
specific functionality. It also defines the interface to exchange approach is based on the delta modeling approach [13] to
data and/or events with other FBs. Additionally, the user can describe the changes required when adjusting one software
also define a subapplication to encapsulate some intercon- variant to another. We propose first to describe the function-
nected FBs into several sub-modules/sub-network hierarchy. ality existing in the control software using feature models.
The subapplication also provides an interface to exchange Then, we can use delta modeling provided in Variability for
data and/or events. As a result, the application model is built 4diac (V4rdiac) [14] to describe software changes needed for
by creating a network of interconnected FBs representing adjusting the FB network when implementing each feature. We
the overall system’s functionality. Control software engineers measure the quality of the control software architecture by
can develop variants of FBs to represent various system calculating the size of delta models, the degree of cohesion
functionalities and later store them in the type library. The and coupling between modules, and the degree of mapping
IEC 61499 standard has entered an early adoption phase for complexity between delta models & features. Based on these
developing industrial-scale control software [3] despite some measures, we can indicate how well the architecture facilitates
early challenges [4]. variability.

Authorized licensed use limited to: JILIN UNIVERSITY. Downloaded on December 16,2024 at 12:08:11 UTC from IEEE Xplore. Restrictions apply.
II. BACKGROUND & R ELATED W ORK such as the size of deltas, degree of cohesion and coupling
between the modules, or degree of mapping complexity be-
Existing works have proposed various measurements and tween features and delta models.
metrics for measuring control software quality. Wilch et
al. [15] have proposed metrics for measuring IEC 61131-3- III. D ELTA - BASED M EASUREMENTS
based control software quality by adopting Halstead Metrics We propose three measurements for analyzing how well
and McCabe’s Cyclomatic Complexity metric. In other work, IEC 61499-based control software architecture can deal with
Mayrhofer et al. [10] have proposed the BASE framework to variability. The first measurement is the size of delta models
assess control software architecture to deal with runtime vari- created with new functionality as the software evolves. The
ability. Sonnleithner et al. [16] have also defined measurements second measurement measures the degree of cohesion and
for Basic FB in IEC 61499 control software. However, all coupling between the interconnected FBs representing process
these proposed metrics and measurements do not concern the functionality. The third measurement is the degree of mapping
capability of control software to manage software variability, complexity between delta models and features.
which comes with the evolution of the design. To illustrate our measurements, we apply delta modeling
Various software architectures and design patterns have to the control software for controlling different valve variants
been published in the last three decades [17]. For instance, in metallurgical production plants. We implement this control
the SOLID principle has been proposed as a guideline for software using two different architectures: (1) Parametrized-
developing modular software, albeit it is mainly used for based Architecture (PBA) and (2) Module Replacement Archi-
developing object-oriented software [18]. tecture (MRA) (cf., Fig. 1). PBA controls software variability
Existing works have also shown the correlation between by using parameters to select or deselect variable components.
software architecture and adaptability [19]. To quantify soft- On the other hand, MRA replaces the variable module based
ware adaptability, Allen et al. [20] have proposed a mea- on the selected functionality. Currently, both architectures
surement based on Coupling and Cohesion between software support two functionalities, i.e., Servo and Proportional valve.
modules. Strong cohesion and weak coupling based on the Then, we simulate how this implementation can support a new
guidelines from the SOLID principle indicate a good software functionality: (1) Solenoid valve, (2) Monitoring Sensor for
architecture. In another work, Nakamura et al. [21] defined a valve, and (3) Simulation. We describe the changes needed to
metric that takes the two extreme endpoints of a significant implement these functionalities and then create delta models
change in the architecture as reference points and modifies for both architectures.
the other transitions based on these endpoints, and uses it
to compare and classify the transition patterns. In this paper, A. Measurement 1: Size of Delta Models
we use the assumption from architectural differences and the We measure the size of a delta model D by calculating the
SOLID principle to develop our proposed measurements. number of delta operations DOp stored inside of it:
The original purpose of delta modeling [13] is to describe
software changes for transforming one software implemen- Size(D) = ΣDOp (1)
tation into another. Each delta model contains one or more
delta operation(s) to describe which model element/line of As previously mentioned, each delta model contains one
code must be added/removed/modified in a particular software or more delta operations to describe the necessary changes
module. Each delta model must also represent changes ac- for implementing a specific feature. Additionally, each delta
cording to the variation points (e.g., features or configuration operation implies an effort to implement or remove a specific
options). We use application conditions in delta modeling feature in the control software. Thus, having more delta
to explicitly define the relation between variation points and operations in a delta model indicates that more effort is needed
delta models (using when-clause). Optionally, we can also de- to implement a feature.
fine a dependency between delta models (using after-clause).
Additionally, these application conditions are stored in a
configuration file called delta configurations file. In this paper,
we propose an approach for measuring the capability of control
software to deal with architectural variability based on the
delta modeling concept. Our previous work described a delta
modeling approach for IEC 61499-based control software [22].
Sharma et al. [23] proposed creating functional modules (a) Parametrized-based Architecture.
by comparing control software between two process variants.
Differences form delta models, which can be mapped to a
variability model. Functional modules were identified based on
variation points and created as control software modules. They
can be mapped to delta models for implementing variation (b) Module Replacement Architecture
points. The authors did not consider quality measurements Fig. 1. Control Software Architecture Comparison.

Authorized licensed use limited to: JILIN UNIVERSITY. Downloaded on December 16,2024 at 12:08:11 UTC from IEEE Xplore. Restrictions apply.
We use delta models in Listing 1 to implement a solenoid software that complies to these criteria will have the least
valve using PBA. On the other hand, we use delta models in possible DCC value.
Listing 2 to implement a solenoid valve using MRA. Listing 1 Listing 3 consists of the application conditions for both PBA
has a total of eleven delta operations in comparison to six and MRA. In this example, we monitor the valve operation
delta operations in Listing 2. Based on this simulation, we using different type of sensors – digital & analog. Furthermore,
can conclude that PBA requires higher maintenance effort than these sensor variants monitor the Solenoid valve activity. We
MRA since it has 45% more delta operations. The size of the have respective delta models to implement each sensor variant
delta model in Listing 1 is expected to grow at a similar rate as with the Solenoid valve. Each application condition defines the
software evolves further and increases the maintenance effort. relation between features and delta models as well as the order
d e l t a PBA_Solenoid ; on which delta model need to be executed first.
u s e s App ;
{ p r o d u c t l i n e MonitorPL ;
<Add> FB name= S o l e n o i d M o d u l e t y p e = S o l e n o i d ; ...
<Modify > S u b a p p l i c a t i o n name= C o n f i g P a r a m { d e l t a P B A _ H W _ A n a l o g S e n s o r _ S o l e n o i d A c t u a t o r a f t e r PBA_SensorAnalog && PBA_Solenoid
<Add> I n t e r f a c e E l e m e n t name= CNF_Solenoid ; when S e n s o r A n a l o g && A c t u a t o r S o l e n o i d ;
} d e l t a P B A _ H W _ D i g i t a l S e n s o r _ S o l e n o i d A c t u a t o r a f t e r P B A _ S e n s o r D i g i t a l && PBA_Solenoid
<Modify > S u b a p p l i c a t i o n name= D a t a S w i t c h { when S e n s o r D i g i t a l && A c t u a t o r S o l e n o i d ;
<Add> I n t e r f a c e E l e m e n t name= P I _ S o l e n o i d ; ...
} d e l t a MRA_Solenoid a f t e r MRA_HardwareControlCore when A c t u a t o r S o l e n o i d ;
<Modify > V a r i a b l e name= C o n f i g P a r a m . ValveType v a l u e = ‘ S o l e n o i d ‘ ; d e l t a M R A _ S e n s o r D i g i t a l a f t e r MRA_HardwareControlCore when S e n s o r D i g i t a l ;
<Add> C o n n e c t i o n s o u r c e = C o n f i g P a r a m . CNF_Solenoid d e s t = S o l e n o i d M o d u l e . REQ ; d e l t a MRA_SensorAnalog a f t e r MRA_HardwareControlCore when S e n s o r A n a l o g ;
<Add> C o n n e c t i o n s o u r c e =HMIBlock . CNF d e s t = S o l e n o i d M o d u l e . REQ ;
<Add> C o n n e c t i o n s o u r c e =HMIBlock . U s e r I n p u t d e s t = S o l e n o i d M o d u l e . P I 1 ; Listing 3. Delta Configuration Example for Calculating Cohesion & Coupling.
<Add> C o n n e c t i o n s o u r c e = S o l e n o i d M o d u l e . PO1 d e s t = D a t a S w i t c h . P I _ S o l e n o i d ;
<Add> C o n n e c t i o n s o u r c e = S o l e n o i d M o d u l e . CNF d e s t = D a t a S w i t c h . REQ ;
} In the above example, each application condition in PBA
Listing 1. Delta Model - Solenoid Valve in Parametrized-based Architecture. uses two after-clauses, whereas only one in MRA. There are
two application conditions for PBA consisting of two delta
d e l t a MRA_Solenoid ;
u s e s App ; models in the after-clause, whereas there are three application
{
<Remove> FB name= ServoModule ; conditions with only one after-clause for each delta in the
<Add> FB name= S o l e n o i d M o d u l e t y p e = S o l e n o i d ;
<Add> C o n n e c t i o n s o u r c e =HMIBlock . CNF d e s t = S o l e n o i d M o d u l e . REQ ; MRA. Using Equation 2, we calculate the DCC value for PBA
<Add> C o n n e c t i o n s o u r c e =HMIBlock . U s e r I n p u t d e s t = S o l e n o i d M o d u l e . P I 1 ;
<Add> C o n n e c t i o n s o u r c e =HMIBlock . S o l e n o i d M o d u l e . CNF d e s t = N e x t P r o c e s s . REQ ; as 2+2
2 = 2, while for MRA, the value is
1+1+1
3 = 1. Based
<Add> C o n n e c t i o n s o u r c e = S o l e n o i d M o d u l e . PO1 d e s t = N e x t P r o c e s s . P I 1 ;
} on this result, we conclude that MRA has higher cohesion and
Listing 2. Delta Model - Solenoid Valve in Module Replacement Architecture. lower coupling than PBA.
C. Measurement 3: Degree of Mapping Complexity between
B. Measurement 2: Cohesion and Coupling between the FBs Delta Models and Features
Cohesion and coupling in IEC 61499-based control software Our third measurement calculates the degree of mapping
refer to how the FBs are interconnected to represent func- complexity between the delta models and their related features.
tionality. Cohesion is the degree of which elements within a We used the following equation to calculate the degree of
module can represent individual functionality. On the other mapping complexity MC:
hand, coupling is a degree of interdependence between mod-
ules in the software. We can measure the degree of cohesion ΣRF
MC =
(3)
and coupling, DCC of delta models as follows: ΣAC
RF represents the number of related features inside an
ΣDDep
DCC = (2) application condition AC. The ideal condition for this mapping
ΣAC is when it follows the Single Responsibility Principle (SRP)
DDep represents the number of delta models used in an correctly, i.e., one feature is mapped to one or more delta
after-clause of an application condition AC. The control soft- models (1-to-n mapping). Thus, highly configurable software
ware architecture will have high cohesion if FBs representing will have a minimum number of n-to-m mappings between
a functionality are stored in the same network hierarchy. features and delta models. A control software architecture that
Additionally, the control software architecture will have low complies with this criteria will produce a low MC value.
coupling if changes on one network hierarchy do not affect p r o d u c t l i n e ValvePL ;
the other. ...
d e l t a PBA_Solenoid when S o l e n o i d ;
Both of these phenomena are also reflected in delta models. d e l t a P B A _ S o l e S i m u l a t i o n B l o c k when S o l e n o i d && M a n u a l S i m u l a t i o n ;
d e l t a P B A _ P r o p S i m u l a t i o n B l o c k when P r o p && M a n u a l S i m u l a t i o n ;
In IEC 61499, one delta model can only be used to change one d e l t a P B A _ S e r v o S i m u l a t i o n B l o c k when S e r v o && M a n u a l S i m u l a t i o n ;
...
network hierarchy. If each module in control software is highly d e l t a MRA_Solenoid when S o l e n o i d ;
d e l t a MRA_SimulationBlock when M a n u a l S i m u l a t i o n ;
cohesive, then a minimum number of network hierarchies will
Listing 4. Delta Configuration Example for Implementing Simulation Block.
be changed when implementing a particular feature. Addi-
tionally, if modules in control software have low coupling, Listing 4 shows the application conditions for both PBA and
then changes to network hierarchies are interdependent. Thus, MRA in the product line ValvePL. In this example, we can
a control software architecture with low coupling and high observe that adding a ManualSimulation functionality
cohesion will produce a minimum set of highly interdependent in PBA is more complex as compared to MRA. Different
delta models when implementing a particular feature. Control adjustments are required to implement ManualSimulation

Authorized licensed use limited to: JILIN UNIVERSITY. Downloaded on December 16,2024 at 12:08:11 UTC from IEEE Xplore. Restrictions apply.
in PBA depending on the type of valve currently being used. [7] M. Galster and P. Avgeriou, “Handling variability in software architec-
In comparison, we only need one adjustment to implement ture: Problems and implications,” in 2011 9th Working IEEE/IFIP Conf.
on Software Architecture, 2011, pp. 171–180.
ManualSimulation in MRA. [8] L. Sonnleithner, B. Wiesmayr, V. Ashiwal, and A. Zoitl, “IEC 61499
This complexity is also reflected in the MC value. PBA has distributed design patterns,” in 26th IEEE Int’l Conf. on Emerging
four application conditions where three of these conditions Technologies and Factory Automation, ETFA 2021, Vasteras, Sweden,
September 7-10, 2021. IEEE, 2021, pp. 1–8.
specify two features in the when-clause. On the other hand, [9] S. Patil, D. Drozdov, and V. Vyatkin, “Adapting software design patterns
MRA has two application conditions and all of them only to develop reusable iec 61499 function block applications,” in 2018 IEEE
specify one feature in the when-clause. Using Equation 3, the 16th Int. Conf. on Industrial Informatics (INDIN), 2018, pp. 725–732.
[10] M. Mayrhofer, C. Mayr-Dorn, A. Zoitl, O. Guiza, G. Weichhart, and
MC value for PBA is 2+2+2+1 4 = 1.75, while for MRA, the A. Egyed, “Assessing adaptability of software architectures for cyber
value is 1+12 = 1. Based on this result, we can conclude physical production systems,” in Software Architecture - 13th European
thatMRA has a lower mapping complexity between delta Conf., ECSA 2019, Paris, France, Sept. 9-13, 2019, Proc., ser. Lecture
Notes in Computer Science, vol. 11681. Springer, 2019, pp. 143–158.
models and features when compared to PBA. [11] D. Wolfart, W. K. G. Assunção, and J. Martinez, “Variability debt:
The three Delta-based measurements indicate that MRA Characterization, causes and consequences,” in SBQS ’21: XX Brazilian
scores better than PBA for managing control software vari- Symposium on Software Quality, Virtual Event, Brazil, November 8 - 11,
2021. ACM, 2021, p. 17.
ability. Our initial results also comply with studies on software [12] A. Alebrahim and M. Heisel, “Supporting quality-driven design deci-
modules in traditional software engineering [18], [20]. sions by modeling variability,” in Proc. of the 8th Int’l ACM SIGSOFT
Conference on Quality of Software Architectures, ser. QoSA ’12. New
IV. C ONCLUSION & R ESEARCH AGENDA York, NY, USA: Association for Computing Machinery, 2012, p. 43–48.
[13] I. Schaefer, “Variability modelling for model-driven development of
In this paper, we provided a theoretical background on the software product lines.” in Proc. of the 4th Int’l Workshop on Variability
Modelling of Software-Intensive Systems. ICB-Research Report 37,
subject matter and observed a gap in the existing approaches Universität Duisburg-Essen 2010, 2010, pp. 85–92.
for measuring the capability of control software regarding [14] H. S. Fadhlillah, K. Feichtinger, K. Meixner, L. Sonnleithner, R. Ra-
design variability. To address this gap, we proposed three mea- biser, and A. Zoitl, “Towards multidisciplinary delta-oriented variability
management in cyber-physical production systems,” in Proc. of the 16th
surements for analyzing IEC 61499-based control software’s Int’l Working Conference on Variability Modelling of Software-Intensive
ability to handle architectural variability using delta modeling. Systems, ser. VaMoS ’22. New York, NY, USA: Association for
As our next step, we will investigate how our proposal can Computing Machinery, 2022.
[15] J. Wilch, J. Fischer, E.-M. Neumann, S. Diehm, M. Schwarz, E. Lah,
be implemented for more extensive and complex IEC 61499- M. Wander, and B. Vogel-Heuser, “Introduction and evaluation of com-
based control software from our industrial partners to measure plexity metrics for network-based, graphical iec 61131-3 programming
the quality of their systems and report how well their archi- languages,” in IECON 2019 - 45th Annual Conf. of the IEEE Industrial
Electronics Society, vol. 1, 2019, pp. 417–423.
tecture can deal with variability. We will also validate the [16] L. Sonnleithner and A. Zoitl, “A software measure for iec 61499
proposed measurement approaches using existing validation basic function blocks,” in 2020 25th IEEE Int’l Conf. on Emerging
mechanisms [24], [25]. Technologies and Factory Automation (ETFA), vol. 1, 2020, pp. 997–
1000.
[17] M. Fowler, D. Rice, M. Foemmel, E. Hieatt, R. Mee, and R. Stafford,
ACKNOWLEDGMENT Patterns of Enterprise Application Architecture. Addison-Wesley
The financial support by the Austrian Federal Ministry for Professional, 2002.
[18] V. Madasu, T. Venna, and T. Eltaeib, “Solid principles in software
Digital and Economic Affairs, the National Foundation for architecture and introduction to resm concept in oop,” Journal of
Research, Technology and Development, and the Christian Engineering Science and Technology, vol. 2, pp. 3159–40, 03 2015.
Doppler Research Association is gratefully acknowledged. [19] E. Kouroshfar, M. Mirakhorli, H. Bagheri, L. Xiao, S. Malek, and
Y. Cai, “A study on the role of software architecture in the evolution and
We explicitly want to thank our industry partner for their quality of software,” in 2015 IEEE/ACM 12th Working Conf. on Mining
continuous support. Software Repositories, 2015, pp. 246–257.
[20] E. Allen, T. Khoshgoftaar, and Y. Chen, “Measuring coupling and
R EFERENCES cohesion of software modules: an information-theory approach,” in Proc.
7th Int’l Software Metrics Symposium, 2001, pp. 124–134.
[1] Int’l Electrotechnical Commission (IEC), TC65/WG6, “IEC 61499-1, [21] T. Nakamura and V. R. Basili, “Metrics of software architecture changes
Function Blocks - part 1: Architecture: Edition 2.0,” Geneva, 2012. based on structural distance,” in 11th IEEE Int’l Software Metrics
[2] A. Zoitl and R. Lewis, Modelling control systems using IEC 61499. 2nd Symposium (METRICS’05), 2005, pp. 24–24.
Edition. Institute of Electrical Engineers, 2014. [22] H. S. Fadhlillah, S. Sharma, A. M. G. Fernández, R. Rabiser, and
[3] G. Lyu and R. W. Brennan, “Towards iec 61499-based distributed intel- A. Zoitl, “Delta modeling in iec 61499: Expressing control software
ligent automation: A literature review,” IEEE Transactions on Industrial variability in cyber-physical production systems,” in 28th IEEE Int’l
Informatics, vol. 17, no. 4, pp. 2295–2306, 2021. Conf. on Emerging Technologies and Factory Automation, ETFA 2023,
[4] A. Zoitl, T. Strasser, K. Hall, R. Staron, C. Sünder, and B. Favre-Bulle, Sinaia, Romania, September 12-15, 2023. IEEE, 2023.
“The past, present, and future of iec 61499,” in Holonic and Multi- [23] S. Sharma, H. S. Fadhlillah, A. M. Gutiérrez Fernández, R. Rabiser,
Agent Systems for Manufacturing. Berlin, Heidelberg: Springer Berlin and A. Zoitl, “Modularization technique to support software variability
Heidelberg, 2007, pp. 1–14. in cyber-physical production systems,” in Proc. of the 17th Int’l Work-
[5] M. Galster, D. Weyns, D. Tofan, B. Michalik, and P. Avgeriou, “Variabil- ing Conf. on Variability Modelling of Software-Intensive Systems, ser.
ity in software systems - A systematic literature review,” IEEE Trans. VaMoS ’23. New York, NY, USA: ACM, 2023, p. 71–76.
Software Eng., vol. 40, no. 3, pp. 282–306, 2014. [24] B. Kitchenham, S. Pfleeger, and N. Fenton, “Towards a framework
[6] B. Vogel-Heuser, S. Feldmann, J. Folmer, J. Ladiges, A. Fay, S. Lity, for software measurement validation,” IEEE Transactions on Software
M. Tichy, M. Kowal, I. Schaefer, C. Haubeck, W. Lamersdorf, T. Kehrer, Engineering, vol. 21, no. 12, pp. 929–944, 1995.
S. Getir, M. Ulbrich, V. Klebanov, and B. Beckert, “Selected challenges [25] L. Briand, S. Morasca, and V. Basili, “Property-based software engineer-
of software evolution for automated production systems,” in 13th IEEE ing measurement,” IEEE Transactions on Software Engineering, vol. 22,
Int’l Conf. on Industrial Informatics. IEEE, 2015, pp. 314–321. no. 1, pp. 68–86, 1996.

Authorized licensed use limited to: JILIN UNIVERSITY. Downloaded on December 16,2024 at 12:08:11 UTC from IEEE Xplore. Restrictions apply.

You might also like