Software Metrics for Real Time System

Aasim Khan, MSc.

Computer Science & Engineering Department
University of Engineering & Technology
Lahore, Pakistan

Abstract
Real Time Systems are gaining more importance day by day in our daily life. But unfortunately
not much work has been done to measure the performance, efficiency and quality of a real time
system. The paper presents a software metrics to measure the performance of a real time
system. To define this performance metrics we use a parameter called “percentage of deadlines
misses”. Depending on how many times deadlines are missed we calculate the performance of
the system. In the end of paper we propose to extend the performance metrics for dynamic
properties of the system.

Keywords: Software Metrics, Real Time System

Introduction which the brakes must be released to prevent

the wheel from locking. Real-time
In computer science, real-time computations can be said to have failed if they
computing (RTC) is the study of hardware and are not completed before their deadline,
software systems which are subject to a "real- where their deadline is relative to an event. A
time constraint"—i.e., operational deadlines real-time deadline must be met, regardless of
from event to system response. By contrast, a system load.
non-real-time system is one for which there is
no deadline, even if fast response or high Real Time systems can be classified as
performance is desired or even preferred. The
needs of real-time software are often 1- Hard Real-time systems
addressed in the context of real-time
operating systems, and synchronous
A system is said to be real-time if the
programming languages, which provide
total correctness of an operation depends not
frameworks on which to build real-time
only upon its logical correctness, but also upon
application software. [1]
the time in which it is performed. The classical
conception is that in a hard or immediate real-
A real time system may be one where
time system, the completion of an operation
its application can be considered (within
after its deadline is considered useless -
context) to be mission critical. The anti-lock
brakes on a car are a simple example of a real- ultimately, this may lead to a critical failure of
time computing system — the real-time the complete system.
constraint in this system is the short time in
Hard real-time systems are typically found commercial airliners. These can operate to a
interacting at a low level with physical latency of seconds. Live audio-video systems
hardware, in embedded systems. For example, are also usually soft real-time; violation of
a car engine control system is a hard real-time constraints results in degraded quality, but the
system because a delayed signal may cause system can continue to operate.
engine failure or damage. Other examples of
hard real-time embedded systems include Software metrics lies on the ancient
medical systems such as heart pacemakers discipline of measurement mainly developed
and industrial process controllers. by scientists (physicians). On this basis, some
take up measurement principles in order to
Hard real-time systems are used when it is measure software activities. [2]
imperative that an event is reacted to within a
strict deadline. Usually such strong guarantees software metrics is a collective term used to
are required of systems for which not reacting describe the very wide range of activities
in a certain window of time would cause great concerned with measurement in software
loss in some manner, especially physically engineering. These activities range from
damaging the surroundings or threatening producing numbers that characterize
human lives (although the strict definition is properties of software code (these are the
simply that missing the deadline constitutes classic software ’metrics’) through to models
failure of the system). Systems that always that help predict software resource
have hard real-time constraints (due to the requirement and software quality. The subject
potentially severe outcome of missing a also includes the quantitative aspects of
deadline) include nuclear power stations and quality control and assurance - and this covers
car airbags. In the context of multitasking activities like recording and monitoring defects
systems the scheduling policy is normally during development and testing.
priority driven (pre-emptive schedulers). Other An other definition of software metrics is due
scheduling algorithms include Earliest to Paul Goodman: The continuous application
Deadline First, which, ignoring the overhead of of measurement-based techniques to the
context switching, is sufficient for system loads software development process and its
of less than 100% products to supply meaningful and timely
management information, together with the
2- Soft Real-time systems use of those techniques to improve that
process and its products".
A soft real-time system on the other
hand will tolerate such lateness, and may In this Paper we first describe some type of
respond with decreased service quality (e.g., real time systems. Then we have discussed a
dropping frames while displaying a video). performance metrics to base on “Percentage
of deadline misses” parameter. After that we
Soft real-time systems are typically those used applied the metrics on sample real time
where there is some issue of concurrent application. In the end for future work we
access and the need to keep a number of have recommended to extend the
connected systems up to date with changing performance for dynamic characteristics of the
situations. Example: the software that real time system.
maintains and updates the flight plans for
 testing, model-checking, compositional
Literature Survey analysis, correctness-by-construction
and abstract interpretation. [3]
1. RTSS 2008 The 29th IEEE Real-Time
Systems Symposium November 30 -
December 3, 2008 Barcelona, Spain 2. Designing Real-Time Applications with
(Design and Verification of Embedded the COMET/UML Method: Most
Real-Time Systems): Designing object-oriented analysis and design
embedded real-time systems is methods only address the design of
becoming an increasingly important sequential systems or omit the
and difficult task due to the important design issues that need to
widespread applications, and be addressed when designing real-time
increasing complexity of such systems, and distributed applications [Bacon97,
and the stringent constraints on Douglas99, Selic94]. It is essential to
reliability, performance, energy blend object-oriented concepts with
consumption, cost, and time-to- the concepts of concurrent processing
market. The objective of this track is to [MageeKramer99] in order to
promote research on design and successfully design these applications.
analysis, and verification of embedded This paper describes some of the key
real-time systems. It intends to cover aspects of the COMET method for
the whole spectrum from theoretical designing real-time and distributed
results to concrete applications with an applications, which integrates object-
emphasis on practical and scalable oriented and concurrent processing
techniques and tools providing the concepts and uses the UML notation
designers with automated support for [Booch98, Rumbaugh99]. [4]
obtaining high-quality software and
hardware systems. A particular goal is
to provide a forum for interaction 3. Real-time Control Systems: A Tutorial:
between different research The literature about real-time systems
communities, such as scheduling, presents digital control or computer
hardware/software co-design, and controlled systems as one of its most
formal techniques. Topics of interest important practical application field.
include (but are not limited to) the
However, it is very difficult to find in
following: Modeling, evaluation and
these textbooks real-time control
optimization of non-functional aspects
aspects. It seems to be more natural
such as timing, memory usage,
that these applications should be
communication bandwidth, and energy treated as part of digital control
consumption. Design space courses. In spite of that, control system
exploration, performance analysis, and literature rarely includes extensively
mapping of abstract designs onto the real-time subject and it does
target platforms such as time-triggered normally not pay attention to real-time
architectures. Model-based validation aspects beyond algorithms and choice
techniques ranging from simulation, of sampling times. The aim of this
 paper is to highlight important issues event to the start of its processing by an
about real-time systems that should be application task, and in absolute terms is equal
taken into account at the moment to to the interrupt task response time defined in
implement digital control. [5] Fig. 1.

Proposed Solution I will concentrate on in this paper that how to

evaluate behavior of the application processes
This paper addresses the problem of (tasks) in terms of its performance. To do this,
evaluating the performance of real-time we have to look at another parameter, which
software architecture. When building a real- can be termed timeliness, that is, the property
time application especially when designing its characterized by the worst-case time that is
software architecture the developers need to needed for processing of an already perceived
have some way of assessing its performance event by an application task.
before deployment. Therefore it is essential to Under the assumption that a real-time
find good metrics and measures the application demonstrates satisfiable
performance of the system. performance if it meets its timing constraints
(deadlines), we should consider the following
To deal with performance of real-time two metrics for evaluation of timeliness:
software we need to realize first, how such
software responds to events that it has to • the number of times the deadlines are
handle. In general, an event, after passing missed (percentage of missed deadlines),
through hardware, is first handled by a real-
time kernel (part of an operating system layer) • the overall accumulated time by which the
and only then by an application task. A deadlines are missed.
respective sequence of handling events,
following the layered approach has been To measure these criteria’s let us define a
outlined in Figure. 1. metrics called “Software Sensitivity Metrics”.
This metrics can be defined as “Software
sensitivity is a measure of the magnitude of a
software response to changes in the values of
deadlines (when they are increased or
decreased).” This means that when software is
sensitive, a small change of a deadline length
causes relatively larger changes in the number
of deadlines missed. Practically, software
sensitivity considered as metric shows
whether performance degradation occurs
Figure 1: Systematic view of event handling sharply or gracefully.

All latencies and response times illustrated in

Figure 1 contribute to the basic real-time
system parameter we can call responsiveness.
It can be measured as the worst-case time that
elapses from the occurrence of a particular
Results & Discussions performance of the whole system degrades
from two times for light-load modules (such as
To validate the metrics, I applied this metrics a Collision Detection module, Fig. 2) to two
on already available data, which is available in orders of magnitude for heavy-load modules,
[6]. The results reported in [6] validated its in terms of the overall time the deadlines are
concept confirming essential facts, such as missed.
better performance of real-time CORBA than
CORBA’s non-real-time versions. At the same Conclusion & Future
time, based on validation experiments, several Recommendations
other observations were made, such as:
how much one implementation is The objective of this work was to introduce a
better or worse than another or new metric to characterize performance of
At what particular values of deadlines real-time software architectures. Using an
performance of the application begins experiment we built on the concept of
to degrade. changing deadlines to see impact on the
The metrics was applied to a large scale number of times (percentage) the deadlines
applications implemented in CORBA: an air are missed, and the overall time the deadlines
traffic control system simulator (ATCS) [6]. are missed, I presented a measures called
software sensitivity, that characterizes the
magnitude of software response to changes in
deadlines.

I have restricted the performance metrics to

software sensitivity. Although it is used for
assessing the timing behavior of software but
it is essentially a steady-state parameter,
because it does not depend on dynamic
properties. One would, however, be tempted
Figure 2: Overall time the deadlines are to characterize software behavior using true
missed (ms) in 100 experiments, for Collision dynamic terms. E.g. one may ask a question:
Detection module. How linear is the curve in its descending
region and would another type of
In the air traffic control problem, it was found approximation describe it better? Trying
to be effective for detecting unusual behavior exponential approximation, under the
of software in real time, when applied to the assumption that the descending part of the
evaluation of performance. For example, curve is not a straight line but an exponential
experimental results show quantitatively how curve, we can characterize it by a new notion
much degradation in performance of a certain called software dynamic i.e. capturing the
component in ATCS (module performing dynamic response to changes in software
conflict detection and resolution) can be behavior under varying load. So for future I
expected, if the number of aircraft directly would like to recommend enhancing it for
involved in computations increases. If the Software dynamics which goes one step
number of aircraft is increased five times, the further and involves a concept of a first-order
dynamic system by associating dynamic
properties with software. Thus, software can
be described by notions from the theory of
dynamical systems and its characteristics can
be included in computational models involving
Laplace transforms and time constants.
References
[1] http://en.wikipedia.org/wiki/Real-
time_computing

[2] Software Metrics An Overview Simon Alexandre

CETIC asbl - University of Namur Software Quality
Lab Belgium July 2002

[3] hhttp://www.rtss.org/

[4] Designing Real-Time Applications with the

COMET/UML Method by Hassan Gomma
Department of Information and Software
Engineering George Mason University USA.

[5] Real Time Control System: A Tutorial by A

Gambier Automation Laboratory University of
Mannhein, Germany.

[6] Guo D., J. van Katwijk, J. Zalewski, A New

Benchmark for Distributed Real-Time Systems:
Some Experimental Results, Proc. 27th
IFAC/IFIP/IEEE Workshop on Real-Time
Programming, Łagów, Poland, May 14-17, 2003, pp.
141-146