International Journal of Advanced Engineering Research and Science (IJAERS) Vol-3, Issue-6, June- 2016

ISSN: 2349-6495

Handling of Priority Inversion Problem in RT-

Linux using Priority Ceiling Protocol
Silambarasan D1, Ramanatha Venkatesan M2
1
Assitant Professor, Department of EEE, AURC- CBE University, Coimbatore, Tamil Nadu
2
Department of EEE, AURC- CBE University, Coimbatore, Tamil Nadu

Abstract— Real time system which runs multiple task embedded products. So, Embedded Linux could get in
concurrently or pseudo concurrently shares the resources Embedded Products easily. Embedded Linux has
will face priority inversion phenomenon. This priority powerful community available in internet which helps to
inversion phenomenon will reduce the Real Time System have good technical support. Embedded Linux has
predictability which in turn leads to un-predictable error. various real time capabilities which make it suitable for
Continuous Priority Inversion phenomenon will lead the the Embedded Systems. Moreover, the Embedded Linux
Real Time System to collapse. This paper analyses the is available in free of cost in most scenarios. Linux has
cause and effect of the priority inversion phenomenon. proven its existence in huge servers and in small handheld
Further analyze the various solution and implementation devices. Above advantages are applicable for RT-Linux.
in RT-Linux. This paper improves the already existing Linux could be made as hard real time Operating System
priority inheritance protocol and priority ceiling in two ways. One is only to modify the Linux Kernel, the
protocol. The proposed algorithm will prevent the Real other is to add an abstract hardware layer, that is to say, to
Time System from deadlock related to priority inversion add a real-time kernel to have the real-time performance.
prevention protocol. Experimental results and analysis in First approach modifies the Linux and adopting POSIX
theoretical way indicate that the methods to solve the 1.b standard. In order to reduce the time that Linux masks
priority inversion problem in RT-Linux are effective and the interrupts, pre-emption points are inserted in Linux
concise, provides reasonable technical details for the safe Kernel Code. This method improves the real-time
running of complex real-time application in RT-Linux. performance of the Linux kernel. But, this method could
Keywords— Priority Inversion, Priority Reversal, not meet the hard real-time performance. The second
Priority Ceiling, Real Time Systems, RT-Linux. method solute the problem and reaches the hard real-time
performance [3].
RT-Linux comes under the category of the Embedded
I. INTRODUCTION Linux. RT-Linux has relatively good real-time
Real Time System is a system where a timely response by performance compared to its counterparts like RTAI,
the computer to external stimuli is vital [1]. The etc.,. RT-Linux is a hard real-time RTOS microkernel that
correctness of the response not only depends on logical runs the entire Linux operating system as a fully pre-
value of the result but also the time at which response is emptive process. The hard real-time property makes it
given. The operating System used in such system should possible to control robots, data acquisition systems,
adhere with above definition and should strive to achieve manufacturing plants, and other time-sensitive
this. Real Time Operating System has one of the instruments and machines from RT-Linux applications.
important components in Real Time Systems to meet Real The key RT-Linux design objective was to add hard real-
Time system’s demands. Nowadays, various commercial time capabilities to a commodity operating system to
Real Time Operating System are available off the shelf. facilitate the development of complex control programs
But to reduce the cost of the Real Time Operating with both capabilities. For example, one might want to
System, achieve the reliability of the Real Time Operating develop a real-time motor controller that used a
System and have high configurability of Real Time commodity database and exported a web operator
Operating System, Linux has been considered for having interface. Instead of attempting to build a single operating
it in Real Time Systems. Embedded Linux has emerged system that could support real-time and non-real-time
as one such Operating System. Embedded Linux is an capabilities, RT-Linux was designed to share a computing
Open Source which paves the way to customize and fine device between a real-time and non-real-time operating
tune the Operating System based on the embedded system so that (1) the real-time operating system could
system. Embedded Linux supports wide range of never be blocked from execution by the non-real-time
hardware which eases the effort of porting it into new operating system and (2) components running in the two

www.ijaers.com Page | 144

International Journal of Advanced Engineering Research and Science (IJAERS) Vol-3, Issue-6, June- 2016
ISSN: 2349-6495
different environments could easily share data. As the moment a hardware interrupt is know by the processor
name implies RT-Linux was originally designed to use and the moment an interrupt handler starts to execute its
Linux as the non-real-time system but it eventually first instruction is under 15 microseconds on RT-Linux
evolved so that the RT-Core real-time kernel could run running on a generic x86. These times are hardware
with either Linux or BSD UNIX. limited, and as hardware improves RT-Linux will also
Thus RT-Linux has become an interesting Software improve. Standard Linux has very good average
component which could be used for Real Time performance and can even provide millisecond level
Computing in a Real Time System. But Real time scheduling precision for tasks using the POSIX soft real-
Computing has many more challenges to meet like higher time capabilities. However, Standard Linux is not
predictability, higher reliability, Should handle the widely designed to provide precision in the range of sub-
varying computational loads [1]. millisecond and reliable timing guarantees. RT-Linux was
Some of the challenges could be better handled by Real based on a lightweight virtual machine where the Linux
Time Operating System. They could be grouped in below runs as guest Operating System and it was given a
headings Task assignment and scheduling, virtualized interrupt controller and virtualized timer, and
Communication protocols, Failure Management and all other hardware access was direct. For the real-time
recoveries [1]. In this paper one such issue related to "host", the Linux kernel is a thread. Interrupts needed for
Communication Protocols in RT-Linux is discussed. deterministic processing are taken care by the real-time
The key RT-Linux design objective was to add hard real- core, while other interrupts are sent to Linux kernel,
time capabilities to a commodity operating system to which runs at a lower priority when compared to real-time
facilitate the development of complex control programs threads. Linux kernel device drivers handle almost all
with both capabilities. For example, one might want to operations related to I/O. First-In-First-Out pipes (FIFOs)
develop a real-time motor controller that used a or shared memory can be used to share data between the
commodity database and exported a web operator General purpose Linux operating system and real-time
interface. Instead of attempting to build a single operating core RT-Linux.
system that could support real-time and non-real-time RT-Linux built a virtual software layer. When Linux
capabilities, RT-Linux was designed to share a computing disables or enables interrupts, one variable of the virtual
device between a real-time and non-real-time operating software layer is set. When one interrupt happens, RT-
system so that (1) the real-time operating system could Linux decides if the interrupt should be handled by Linux
never be blocked from execution by the non-real-time or RT-Linux according to the value of the variable. In
operating system and (2) components running in the two respect of memory allocation of the real-time task, RT-
different environments could easily share data. As the Linux allocates the memory for all real-time tasks as
name implies RTLinux was originally designed to use Linux kernel modules, so that all real-time tasks have
Linux as the non-real-time system but it eventually same address space as the Linux kernel. It can lessen the
evolved so that the RTCore real-time kernel could run difficulty of RT-Linux development to do like this. But
with either Linux or BSD UNIX. Real Time task programmer has to design carefully real-
Thus RT-Linux has become an interesting Software time programs in case of any crash in Real Time task the
component which could be used for Real Time whole system would be affected. In respect of task
Computing in a Real Time System. But Real time scheduling, the scheduler of RT-Linux bases on the
Computing has many more challenges to meet like higher priority scheduling. But the priority scheduling is not
predictability, higher reliability, Should handle the widely suitable for all real-time application, So RT-Linux has
varying computational loads[1]. modularized the scheduler, therefore the user can use the
Some of the challenges could be better handled by Real schedulers based on various policies and algorithms. In
Time Operating System. They could be categorized as respect of real-time clock, in order to realize the precise
Task assignment and scheduling, Communication real-time trimming and avoid the task release jitter, RT-
protocols, Failure Management and recoveries[1]. In this Linux adopts internal hardware timer chips as timer
paper one such issue related to Communication Protocols interrupt generator. In respect of inter process
in RT-Linux is discussed. communication, RT-Linux provides the mechanism of
II. RT-LINUX ARCHITECTURE semaphore message queue and especially for the needs of
RT-Linux provides the capability of running special real- communication between real-time processes and none
time tasks and interrupt handlers on the same machine as real-time processes. RT-Linux provides FIFO, and shared
standard Linux. These tasks and handlers are executed memory for inter task communication.
when they are needed to be executed no matter what
Linux is executing. The worst case time between the

www.ijaers.com Page | 145

International Journal of Advanced Engineering Research and Science (IJAERS) Vol-3, Issue-6, June- 2016
ISSN: 2349-6495
priority which leads to un-bounded priority inversion
problem.

Fig.1: Architecture of RT-Linux

RT Linux kernel is module based kernel, the scheduler is Fig.2: Priority Inversion Phenomenon.
itself a loadable kernel module. So, various scheduling Priority inversion phenomenon is an important reason of
policies like Earliest Deadline First(EDF), RateMonotic unpredictable errors in real-time system. Serious
(RM) could be used. In RT-Linux the Linux runs as low continuous priority inversion phenomenon will lead to
priority process. Advantages of RT-Linux are small foot real-time system to collapse. The priority inversion is a
size (approx. 150 KB), higher degree of predictability, most common problem that will affect the real-time
response in terms milli-second and sophisticated service performance of a real – time kernel. Priority inversion
from GPOS Linux. affects heavily on the system predictability. So the real-
time system may enter into unpredictable mode.
III. PRIORITY INVERSION PHENOMENON
Priority inversion is phenomenon that occurs when a IV. EXISTING SOLUTION
higher priority task waits for a lower priority task to There are many methods to solve the priority inversion
release a resource it needs but that is held by lower problem and each method has its own advantages as well
priority task and meanwhile the intermediate priority as disadvantage. A Solution will suite only for the
tasks pre-empt the lower priority task from CPU. So, high specific application environment.
priority task would be blocked. Now the priority of the 4.1. Locking the Scheduler
task effectively gets inversed with respect to the medium Locking the scheduler will suspend the scheduler.
priority task. Priority Inversion will occur in Multi- Whenever a task enters into any of the critical section, the
tasking system when resource is shared across the tasks. scheduler will be locked. Once the task comes out the
Priority inversion would happen when high priority task critical region the scheduler would be released. This
T(1) and low priority task(T3) share critical resource and method stops temporarily the scheduler till the task is in
T3 first gets the resource, while T1 is ready and ask for the critical section.
access the critical resource, it is blocked for waiting T3 to If task (T1) wants to access critical section, It will lock
release the resource, at this time, middle priority task T2 the scheduler, enters the critical section. Now even a high
which does not need the resource is ready, T2 seizes the priority task (T2) is ready in run queue, Task(T1) could
processor to made T1 continue block until T2 ends, and not be pre-empted due to fact the scheduler is locked.
then T3 gets the processor again to complete the use of Once the task(T1) has finished its update on critical
shared resource, finally T1 pre-empts T3 and running. section. It will release the scheduler. Now, the scheduler
In above scenario , the priority of T1 comes down to level will schedule the high priority task (T2).
of T3’s priority, So the high priority task T1 cannot meet
its deadline first, if many middle priority has emerged,
high priority task would be blocked for undetermined
duration which is called as Continuous Priority Inversion
Phenomenon. Serious continuous priority inversion
phenomenon will end up with collapse of the whole real-
time system. One of the most famous victims of Priority
Inversion problem is the “mars path finder”. Although
priority inversion phenomenon was found early in 1970
of the last century, there is no effective and simple
solution yet. In some other scenario, there would be
another task (T4) which would tries to acquire the
common resource which may lead to cyclic change of Fig.3: Solution to priority inversion – Locking scheduler.

www.ijaers.com Page | 146

International Journal of Advanced Engineering Research and Science (IJAERS) Vol-3, Issue-6, June- 2016
ISSN: 2349-6495
This approach is simple and easy to implement by priority task would inherit the highest prioirty among all
programmer. System behaviour Analysis could be done priority tasks. Prioirty inheritance mehod makes
easily. Consider, there are three task in the system namely developers do not need to know anything about the task’s
task(TH) with highest priority, task(TM) with medium requirements of resurces which reduce the burden of the
priority and task(TL) with lower priority. A resource programmers. This improves the easier predictiability and
(RA) is shared between task(TM) and task(TL). Task TL development of huge real-time embedded system. But this
is ready and running enters the critical section. Now the method can’t prevent deadlock.
scheduler is locked and the high priority task(TH) is
entering into run queue. But, since the scheduler is in
locked state, It could not have CPU. So, the task(TH) is
blocked due to a resource that is nowhere related to TH.
So, this approach will result in high degree un-
predictability in the system if the number of task is more.
Higher priority task may often miss the deadline. If the
critical section access time is high which in turn increase
the time for which scheduler is in locked state. So, this
method is not suitable for system with more number of Fig.4: Solution to priority inversion – Priority inheritance
task or system with larger critical section.
4.2. Priority Remapping V. PROPOSED SOLUTION
This priority remapping method improves the priority The Proposed solution is another algorithm that is new to
inheritance protocol. The method is expanding the RT-Linux. But this algorithm has wide usage in some
priority from 64 to 128 without changing the external commercial Real Time Operating Systems. Proposed
interface. For users, there are still only 64 priorities. But solution contains more than one algorithm which discuss
in internal task creation function, priority is multiplied by advantage and disadvantage in each use case level.
2 to achieve the priority remapping effect, so internal 5.1. Highest Locker Protocol
priority is extended to (0, 2 ..., 126) even priority, the rest The basic idea of priority ceiling protocol shows in Figure
odd priority are left for changing when priority inversion 5. Programmer sets a ceiling priority for each shared
phenomenon arises. For example, priority of task that resource, Highest priority value of the task which is going
accesses critical resource is 80 while priority of task that to use this resource. The ceiling is the highest task
asks for the resource is 30, then priority of the task which priority for requests the resource. High priority task T1
accesses the critical resource will promote to 31 to and low priority task T3 need the same resource (the
prevent priority inversion phenomenon black part in Figure 4). T3 gets ready first and access the
4.3. Priority Exchange resource. At t1 time point, T1 gets ready and tries to
Priority exchange method is also deriving from the access the resource which has already held by T3. So T3’s
improvement of priority inheritance protocol. The basic priority promotes to the ceiling at t2 time point. After T3
idea is that exchange tasks’ priorities when high priority finishes the use of resource at t3 time point, T3’s priority
task is blocked and priorities will be changed back after resume to the original low priority; T1 gets the resource
critical region is finished. This method makes sure every and begin to run. After T1 ends, T2 begins to run at t4
task has unique priority in the system and solves the time point. After T2 ends, T3 continues to run at t5 time
priority inversion problem. The drawback is that point. This method extends priority inheritance protocol.
priority’s exchange requires additional costs. Priority ceiling protocol prevents deadlock and reduces
4.4. Priority Inheritance the block time. The drawback is that developers need to
The idea of the priority inheritance protocol is, When a do static analysis in advance and assign a highest priority
high priority task is blocked by a low priority task for for each shared resource. This make the programs become
getting the resource, then the low priority task will inhert more complicated and developers must know all tasks’
temporarily the priority of the higher priority task. When priorities and all idle resources’ priorities. And each
the resource is released then low priority task’s priority resource size a ceiling priority also decreases the number
will be assigned the same value as earlier. of task created by application system. So priority ceiling
If task (TH) is blocked by low priority task Tl, TL will protocol is not suitable for complex applications.
inherit TH’s priority to avoid middle priority task TM
seizing the processor. After TL withdraw its critical
region, its priority resume to the original low priority. If
there are many high priority task being blowe, low

www.ijaers.com Page | 147

International Journal of Advanced Engineering Research and Science (IJAERS) Vol-3, Issue-6, June- 2016
ISSN: 2349-6495
priority of a task does not change upon acquiring a
resource merely the value of a system variable CSC
changes. The priority of a task changes by the inheritance
clause of PCP only when one or more tasks wait for a
resource it is holding. Tasks requesting a resource block
almost under identical situations under PCP and HLP.
The only difference with PCP is that a task Ti can also be
blocked from entering a critical section, if there exists any
resource currently held by some other task whose priority
ceiling is greater or equal to that of T2. A little thought
would show that this arrangement prevents the
unnecessary inheritance blockings caused due to the
priority of a task acquiring a resource being raised to very
Fig.5: Solution to priority inversion – Highest Locker
high values (ceiling priority) at the instant it acquires a
Protocol.
resource. In PCP, instead of actually raising the priority
5.2. Prioirity Ceiling Protocol
of the task acquiring a resource, merely the value of a
Priority Ceiling Protocol (PCP) extends the ideas of PIP
system variable (CSC) is raised to the ceiling value. By
and HLP to solve the problems of unbounded priority
comparing the value of CSC against the priority of a task
inversion, chain blocking, and deadlocks, while at the
requesting a resource, the possibility of deadlocks is
same time minimizing inheritance-related inversions.
avoided. If no comparison with CSC would have been
Resource sharing among tasks under PCP is regulated
made (as in PIP), a higher priority task may later lock
using two rules for handling resource requests: resource
some resource required by this task leading to a potential
grant and resource release. We elaborate these two rules
deadlock situation where each task holds a part of the
in the following:
resources required by the other task.
Resource grant rule:
Resource grant rule consists of two clauses. These two
VI. IMPLEMENTATION DETAILS
clauses are applied when a task requests to lock a
By analyzing the RT-Linux source code, priority reverse
resource.
problem may not happen due to the priority inheritance
5.2.1. Resource request clause:
protocol that is enabled in the RT linux kernel patch. But
(a) If a task Ti is holding a resource whose ceiling
the Priority Inheritance Protocol is not much suitable for
priority equals CSC, then the task is granted access to the
the Real Time Application. As it only avoids the Un
resource.
Bounded Priority inversion problem. The problem arises
(b) Otherwise, Ti will not be granted CRS, unless its
from the fact that Priority Inheritance Protocol has its own
priority is greater than CSC (i.e. pri(Ti) > CSC). In both
limitation like chain blocking, deadlocks. Moreover chain
(a) and (b) above, if Ti is granted access to the resource
blocking will result in un-predictable behavior in the
CRS, and if CSC < C eil(C Ri), then CSC is set to C eil(C
system and many task may miss their respective deadline.
2. Inheritance clause: When a task is prevented from
But Priority Inheritance is used due to its simplicity. To
locking a resource by failing to meet the resource grant
solve this problem of priority reverse or priority inverse
clause, it blocks and the task holding the resource inherits
problem alogn with chain blocking problem and deadlock
the priority of the blocked task if the priority of the task
problem, priority ceiling or variant of priority ceiling
holding the resource is less than that of the blocked task.
protocols have been adopted. The algorithm and
5.2.2 Resource Release Rule:
implementation details are discussed below.
If a task releases a critical resource it was holding and if
• A global variable ‘System_Ceiling_Priority’ is
the ceiling priority of this resource equals CSC, then CSC
declared and initialized to zero.
is made equal to the maximum of the ceiling value of all
• If a task Ti is holding a resource whose ceiling
other resources in use; else CSC remains unchanged. The
priority equals to the value in
task releasing the resource either gets back its original
System_Ceiling_Priority, then the task is granted
priority or the highest priority of all tasks waiting for any
access to the resource.
resources which it might still be holding, whichever is
• (b) Otherwise, Ti will not be granted the resource,
higher.
unless its priority is greater than
PCP is very similar to HLP except that in PCP a task
System_Ceiling_Priority (i.e. pri(Ti) >
when granted a resource does not immediately acquire the
System_Ceiling_Priority). In both (a) and (b) above,
ceiling priority of the resource. In fact, under PCP the
if Ti is granted access to the resource , and if

www.ijaers.com Page | 148

International Journal of Advanced Engineering Research and Science (IJAERS) Vol-3, Issue-6, June- 2016
ISSN: 2349-6495
System_Ceiling_Priority < C eil(C Ri), then critical region is relatively long, higher priority tasks
System_Ceiling_Priority is set to Ceil(Resource) would miss their deadline often. Priority exchange
• Inheritance clause: When a task is prevented from method is relative complex in design, need to modify RT-
locking a resource by failing to meet the resource Linux kernel, but it is convenient to develop complex
grant clause, it blocks and the task holding the real-time application for application programmer. The
resource inherits the priority of the blocked task if drawback is that lack of deadlock prevention, chain
the priority of the task holding the resource is less blocking and the exchange of priority will pay some
than that of the blocked task. unnecessary overhead to Real Time System. In cases of
exchange priority frequently, the method will increase
VII. EXPERIMENTAL RESULTS AND burden of the system and effect the predictability of
DISCUSSION system. Priority Inheritance protocol introduce
Create task T1, T2, T3, T4, T5, T6 with priority value inheritance related inversion which is not suitable for
higher for lower task number. And T2, T5 shares the some Real Time System requirements due to lack of dead
resource R1 and R2. T1 to T2 is grouped as high priority lock prevention, chain blocking, Inherited Inversion
task. T3, T4 is grouped as medium priority task. T5, T6 problem. Modified Priority Ceiling Protocol takes the
are grouped as low priority task. advantages of priority ceiling protocol (Higher Locker
Figure 6 shows the running result that didn’t modify the protocol) and the priority Inheritance protocol. By this
kernel, T2 seizes the processor make T1 blocked long method, system is free from unbounded priority
time and give raise to priority inversion phenomenon. inversion, Chain Lock, Dead Lock. And, this minimizes
the effect of Inheritance related inversion. The discussed
method improves effectively the removal of priority
inversion problem with relatively less overhead. The
Inheritance related inversion could be further reduced.
But this algorithm only minimize the impact but not
removing the same. By having this algorithm, RT-Linux
could be used in many time critical complex systems.

Fig.6: Priority Inversion Problem. REFERENCES

After use modified priority ceiling method, priority [1] Ianlin Zhu, Keou Liu, Yu Tu, Yi Yuan, Xialoliang
inversion phenomenon is confined to one level, and there Gao, “RT-Linux priority reversal and priority
is not chain blocking and deadlock possibility. The inheritance mechanisms”, 2013 Fifth Conference on
experiment running result is shown in Figure 7. Measuring Technology and Mechatrnics
Automation.
[2] XU Liang, XU Zhongwei. “Task scheduling
optimization in in μC/OSII system”[J] computer
project. 2007,33(19):57-59
[3] C.M. Krishna, Kag G. Shin “Real-time Systems”
Tenth reprint 2013.
[4] Lee J H, Kim H N. “Implementing Priority
Inheritance Semaphore on uC/OS Real-time
Kernel[C]” Proc.of IEEE Workshop on Software
Technologies for Future Embedded Systems.2003-
Fig7: Modified Priority Ceiling Protocol.
05.
[5] WANG Jigang, GU Guochang, XIE Shibo, LI Yi.
VIII. CONCLUSION
“Priority inheritance Protocol and an improved
This paper discuss several methods to solve the priority
algorithm” Computer engineering,2007,33(8):41-44.
inversion problem in RT-Linux such as disabling
[6] ZHAO Qi, SUO Xiaoran. “Research of priority
interrupt, priority inheritance protocol, priority remapping
inversion in real-time system”[J] Computer
method, priority exchange and modified priority ceiling
Application Research .2008,25(6):1728-1730
protocol. The simplest method is that to lock the
[7] ZHOU Xuchuan. “A method to solve priority
scheduler preventing the system to context switch. It is
inversion problem in μC/OS-II system”[J]
very simple and effective to limit priority inversion
Application of embedded operating system
problem when critical region is very short. But if the
2007,23(5-2) :58-64.

www.ijaers.com Page | 149