1282 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO.

6, NOVEMBEWDECEMBER 1992

The Chimera I1 Real-Time Operating System for

Advanced Sensor-Based Control Applications
David B. Stewart, Donald E. Schmitz, and Pradeep K. Khosla, Senior Member

Abstract-The Chimera I1 Real-Time Operating System, which critical factor. Examples of processes at this level include
has been developed for advanced sensor-based control appli- generating accounting or performance logs of the real-time
cations. It has been designed as a local operating system, to system, simulating a task, and programming new tasks for the
be used in conjunction with a global operating system, is de-
scribed. It executes on one or more single board computers in a system to take on. In order to develop sensor-based control
VMEbus-based system. Advanced sensor-based control systems applications, a multitasking, multiprocessing, and flexible real-
are both statically and dynamically reconfigurable. As a result, time operating system (RTOS) is needed.
they require many special features, which are currently not An RTOS can be subdivided into several parts, including
found in commercial real-time operating systems. Several design the real-time kernel, the multiprocessor support, the file sys-
issues for such systems are presented as well as the features we
have developed and implemented as part of Chimera 11. These tem, and the programming environment, The real-time kernel
features include a real-time kernel with dynamic scheduling, provides local task management, scheduling, timing primi-
global error handling, user signals, and two, levels of device tives, memory management, local communication, interrupt
drivers; an enhanced collection of interprocessor communication handling, error handling, and an interface to hardware devices.
mechanisms, including global shared memory, spin-locks, remote
semaphores, priority message passing, global state variable tables,
The multiprocessor support includes interprocessor communi-
multiprocessor servo task control, and host workstation integra- cation and synchronization, remote interrupts, access to special
tion; and several support utilities, including UNIX C and math purpose processors, and distributed task management. The
libraries, a matrix library, a command interpreter library, and file system provides access to secondary storage, such as
a configuration file library. Chimera I1 is currently being used disks and tapes, and to local-area-networks. The programming
with a variety of systems, including the CMU Direct Drive Arm
11, the CMU Reconfigurable Modular Manipulator System, the environment provides the tools for building applications; it
'hikabot System for Rapid Assembly, and the Self-Mobile Space includes the editor, compiler, loader, debugger, windowing
Manipulator. environment, graphic interface, and command interpreter (also
called a sheI1). The level of support provided for each part of
I. INTRODUCTION the operating system (OS) varies greatly among RTOS.
In this paper, we present the Chimera I1 RTOS, which has
A DVANCED sensor-based control applications, such as
robotics, process control, and intelligent manufacturing
systems have several different hierarchical levels of control,
been designed especially to support advanced sensor-based
control applications. Chimera I1 is designed as a local OS
within a global/local OS framework, as shown in Fig. 1. In
which typically fall into three broad categories: servo lev-
such a framework, the global OS provides the programming
els, supervisory levels, and planning levels. The servo levels
environment and file system, while the local OS provides the
involve reading data from sensors, analyzing the data, and
real-time kernel, multiprocessor support, and an interface to
controlling electromechanical devices, such as robots and
the global OS. For many applications the global OS may be
machines. The timing of these levels is critical, and often
non-real-time, such as UNIX or Mach. However, the use of a
involves periodic processes ranging from 1 Hz to 1000 Hi.
real-time global OS such as Alpha OS [7] and RT-Mach [30]
The supervisory levels are higher level actions, such as spec-
can add real-time predictability to file accesses, networking,
ifying a task, issuing commands like turn on motor 3 or
and graphical user interfaces.
move to position B, and selecting different modes of control
Many commercial RTOS, including iRMX I1 [5], OS-9
based on data received from sensors at the servo level. Time
[13], and pSOS+ [24]. do not use the global/local OS frame-
at these levels is a factor, but not as critical as for the
work, and hence they provide their own custom programming
servo levels. In the planning levels time is usually not a
environment and file system. The environments, including
Manuscript received January 26, 1991; revised February 21, 1992. This the editors, compilers, file system, and graphics facilities
work was supported in part by U S . Army AMCOM and DARPA under
contract DAAA-2189-C-0001, in part by NASA under contract NAGS-1091, are generally inferior to their counterparts in
by the Department of Electrical and Computer Engineering, and in part by The os. In addition, since much development effort for these
Robotics Institute at Carnegie Mellon University, and in Part by the Natural RToS goes into the programming environment, they have
Sciences and Engineering Research Council of Canada (NSERC) through a
Graduate Scholarship. inferior real-time kernels as compared to other RTOS. Some
D. B. Stewart and P. K. Khosla are with the ECE Department, Carnegie commercial RTOS, such as VRTX [20] and VxWorks [32],
Mellon University, 5000 Forbes Ave., Pittsburgh, PA 15213. do use the global/local OS framework. However, as compared
D. E. Schmitz is with Transarc Corporation, 707 Grant St., Pittsburgh, PA
15219. to Chimera 11, they provide very little multiprocessor support,
IEEE Log Number 9201396. and their communications interface to the global OS is limited
0018-9472/92$03.00 0 1992 IEEE
STEWART et al.: THE CHIMERA I1 REAL-TIME OPERATING SYSTEMS 1283

Local Area Network software configurations may have to be changed to support

a different class of applications. Thus advanced sensor-based
control systems must also be statically reconfigurable.
Several design issues, discussed in detail below, were con-
workstation workstation
with global sidered in developing Chimera 11, an RTOS for advanced
operating operating sensor-based control systems that are both statically and dy-
system system namically reconfigurable.
(e.g. Unix)

~~ ~~~~

multiprocessor
dedicated
communication
links 4 multiprocessor
A. Programming Environment and File System
The programming environment is required to quickly de-
velop, debug, and maintain code. The basic requirements for
the environment are an editor, a high-level language compiler,
a linker, and a loader. The file system is required to store
open-architecture open-architecture
hardware hardware code and data on secondary storage, and to electronically
with local real-time with local real-time transfer information to other systems. In order to provide all of
operating system operating system
(e.g. Chimera II) (e.g. Chimera II) the advantages of the full-featured programming environments
and file systems of today’s UNIX workstations, we adopted
Fig. 1. Framework of global/local operating system separation. the global/local OS framework, and developed Chimera I1
as a local OS. Chimera I1 then requires a global OS to
operate as the host, which makes all of the global OS’s
to networking protocols, thus making the communication slow programming environment and file system features available
and inflexible. The commercial RTOS only provide basic to Chimera 11. Given such a framework, we have developed a
kernel features, such as static priority scheduling and very powerful interface between the host workstation and real-time
limited exception handling capabilities, and multiprocessor environment, as described in Section 1II-B.
support is minimal or nonexistent. Previous research efforts in
developing an RTOS for sensor-based control systems include B. ooen Architecture Real-Eme Hardware
Condor [MI, the Spring Kernel [25], Sage [21], Chimera [23],
A typical hardware configuration for advanced sensor-based
and Harmony [3]. They have generally only concentrated on
control applications may consist of multiple general pur-
selected features for the real-time kernel, or were designed
pose processors, possibly on multiple buses. The system
for a specific target application. Chimera I1 differs from these
may contain special processing units, such as floating point
systems in that it not only provides the basic necessities of
accelerators, digital signal processors, and image processing
an RTOS, but also provides the advanced features required
systems. The system also includes interfaces to sensory de-
for implementation of advanced sensor-based control systems,
vices, such as force sensors, cameras, tactile sensors, and
which may be both dynamically and statically reconfigurable.
range finders to gather data about the environment, and to
control devices such as actuators, switches, and amplifiers to
11. DESIGNISSUES control electromechanical equipment. We have implemented
Advanced sensor-based control systems should by dynam- Chimera I1 around the VMEbus [16], since it is a popular
ically reconfigurable, even for the simplest of applications. bus standard. However, much of the design of Chimera I1 is
Consider the example of a robotic manipulator that is required independent of the target hardware architecture, and hence can
to move an object whose location is known. This task can be be used with other architectures.
broken up into three separate phases: pick up object; move Fig. 2 shows a typical hardware configuration. General pur-
to new position; put down object. When the manipulator is pose processing is provided by commercially available single
picking up the object, it must use force control for contacting board computers, which we call real-time processing units
the object, and gripper control to properly pick up the object. (RTPUs). We have chosen to support the Motorola MC680x0
To move the object a different controller is required for the family of general purpose processors because of their popular-
free motion phase. Possibly vision processing is also required ity and their one-to-one mapping of hardware signals with the
to track the object’s target location. To put down the object, VMEbus signals [15]. The design of Chimera I1 allows other
force control and gripper control is again needed. The set of processor-based RTPUs, such as the Intel 80x86 and SPARC
modules executing and the sensors required during each phase families, to also be used; however, we currently have not
is different. Some of the modules and sensors are shared by the ported any software to those platforms. Any VMEbus-based
different phases, while others must be dynamically changed. 1/0 device or special purpose processor can be incorporated
As applications become more complex, the number of possible into Chimera 11, by using the two-level device driver support
configurations also increases. offered by our RTOS.
Advanced sensor-based control systems should be imple-
mented on open-architecture hardware, so that new sensors and C. Real-Time Kernel
additional processing may be incrementally added to increase The real-time kernel must include all the basic features
the system’s intelligence. In addition, the hardware and set of found in any RTOS. These include task management, mem-
1284 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBEWDECEMBER 1992

Ethernet

Planning

mainframe
User Interface

I Host VME bus

Software Development and Simulation (non-real-tune)

Real-Tlme Execution Gateway

# I
Primary VME bus

I I I I Supervb0ry
Levels

- Real-Time
Processing
Unit #1
Real-Tme
Processing
Un8#2
Spedal
High-Speed
Processor
Real-lime
Processing
UnitX3
(May consist 01
more than one
VME bus)

I I I
Aiiernate High Speed Local Bus
Cnmmunication Link
(e.g. Fiber Optics) Gateway
Secondary VME bus

I I I I I ] Servo Levels

- Red-Time
Processing
Unit#
usion
System
Seriai and
Parallel
ID ports
Real-Time
Processing
unn#t
(may consist of
more than one
VME bus)

ory management, local shared memory, local semaphores, these mechanisms become bound to a single processor. The
timer access, and hardware independence. Task management same code cannot execute on a different processor, without
includes actions such as creating, destroying, blocking, waking modification of VMEbus addresses, or in the case of message
up, setting priorities, and scheduling of concurrent tasks. passing, modifying the names of the source and destination
Memory management is the allocation and deallocation of of the messages. In a reconfigurable system, modules must
physical memory. We do not consider virtual memory, because be designed such that they are independent of the target
we are not aware of any method to do paging in real-time. RTPU; hence these methods are not satisfactory. Second, these
Local shared memory allows tasks on the same RTPU to share mechanisms do not provide all the tools desirable for quickly
memory. Local semaphores provide basic synchronization for developing multiprocessor applications. In Section 111-B. we
tasks on the same RTPU. Timer access allows the execution describe the enhanced IPC and synchronization mechanisms
of tasks to be controlled by time. Hardware independence is developed in Chimera 11, including global shared memory,
a virtual machine layer., which allows user software to use spin-locks, remote semaphore, prioritized message passing,
the hardware without having to program hardware specific global state variables, and multiprocessor task control.
code. The level of hardware independence provided by an
OS varies. We have developed an expanded real-time kernel
E. Predictability
suitable for reconfigurable systems. In addition to the basic
kernel functions, our kernel also provides dynamic scheduling, The primary concern in a real-time system is not that
two levels of device drivers, built-in control of timers, and it is fast, but rather that it is predictable. A fast system
global error handling. Details of Chimera I1 kernel are given with unpredictable behavior can cause serious damage. In
in Section 111-A. theory, the rate monomonic algorithm [lo] is used to en-
sure predictable execution. However, this static scheduling
algorithm is not suitable for dynamically reconfigurable sys-
D.Interprocessor Communication and Synchronization tems and does not provide the CPU utilization achievable
Most RTOS give very few mechanisms for interprocessor with dynamic scheduling algorithms [ll]. We have developed
communication (IPC) and synchronization. The mechanisms the maximum-urgency-first algorithm to provide a predictable
available generally include support for an atomic read-modify- dynamic scheduling algorithm [26]. This algorithm has been
write instruction for limiting access to critical sections, and implemented as the default scheduler for Chimera 11. Support
some form of message passing for higher-level communica- for the rate monotonic algorithm, as is provided in most RTOS,
tion. The VMEbus by default offers shared memory; how- is also available with the default scheduler.
ever, most RTOS do not make any provisions for allocating Most real-time scheduling theory concentrates on ensuring
that memory, or for automatically performing the address that tasks always meet their deadlines. However, nothing is
calculations required to access different parts of memory said about what happens to tasks that fail to meet deadlines. In
on the VMEbus. As a result, programs that use any of addition, even though a task can meet all deadlines in theory,
STEWART er al.: THE CHIMERA I1 REAL-TIME OPERATING SYSTEMS 1285

abnormalities in the implementation may cause the task to The intercommunication between the modules is handled
miss its deadline in practice. In Chimera I1 we have addressed automatically using a high-performance global state variable
this issue by designing a deadline failure handing mechanism, table mechanism. The servo task control mechanism and global
which allows an exception handler to be automatically called state variable table are described in Section 111-B.Details on
when a task fails to meet its deadline. Possible error handling developing a library of reusable modules can be found in [27].
includes aborting the task and preparing it to restart the next
period; sending a message to some other part of the system G. Libraries
to handler the error; performing emergency handling, such as The usefulness of an operating system does not only lie
a graceful shutdown of the system or sounding an alarm; in the features given, but also on the supporting libraries,
maintaining statistics on failure frequency to aid in tuning which save on application development time. Like most other
the system; or in the case of iterative algorithms, returning RTOS, Chimera I1 provides the standard UNIX C and math
the current approximate value regardless of precision. Details libraries. It also provides a concurrent standard I/O library
of the Chimera I1 scheduler and deadline failure handling are (stdio), which is suitable for using the stdio facilities in a
included in Section 111-A. multiprocessor environment. In addition, several other libraries
Deadline failures account for only one of the many types of are provided, which are generally not found in other OS. These
errors that may occur in a system. Deadline failures are unique include a matrix math library, a command line interpreter
in that the errors are a function of time. Other errors that may library, and a configuration file support library. These libraries
occur in a system include hardware failures, software bugs, provide utilities that are often required by advanced sensor-
invalid data, invalid states, and processor exceptions. These based control applications. More details on these libraries are
errors may occur at any time, and are often detected within the in Section 111-C.
user’s code through the use of consistency if-then statements.
The most typical method of dealing with these errors is to
have the routine detecting the error to either handle it itself, 111. SOFTWAREARCHITECTURE
or to return an error value, such as -1. Unfortunately, in a The Chimera I1 software is divided into two distinct parts:
hierarchical software architecture, this method has two major 1) Code that runs on the RTPUs, and 2) Code that runs on
problems. First, the exception handling code is embedded the host workstation.
within the application code, making the code inflexible and Fig. 3 shows the data flow diagram for code that runs on
difficult to maintain. Second, if an error occurs at the lowest the RTPUs. A copy of the Chimera I1 kernel executes on each
level of the architecture and it is only to be handled by a RTPU. User tasks execute concurrently, communicating with
higher level module, then the error must be continually passed each other though local shared memory and local semaphores.
up through the intermediate software levels. This method is Tasks also have direct access to local I/O devices, and can
both cumbersome to code, and is also very inefficient, in that communicate with other processors using any of the IPC
the time to enter the exception handling code requires the mechanisms available. Each RTPU has a server task that con-
additional overhead of propagating the error through multiple stantly monitors the express mail. The express mail is a high-
hierarchical levels. In Chimera I1 we have developed a global performance message passing mechanism between kernels on
error handling mechanism, which allows exception handling each RTPU, which in turn allows tasks on different RTPUs
code and main program code to be coded independently. to communicate with each other transparently. The server
Details are given in Section 111-A. handles express mail messages, translates symbolic names into
pointers, and performs any necessary calculations to translate
F. Modular and Reusable Software addresses within various address spaces on the VMEbus.
In order to save on development time and make software The kernel provides a real-time multitasking environment for
more maintainable, it is generally accepted that applications executing the tasks concurrently.
should be designed in a modular fashion, and the software The host workstation is also an integral part of the
modules should be reusable. Chimera I1 extends this ideology Chimera I1 environment. Fig. 4 shows the data-flow diagram
to practice, by providing the tools that make it possible to for code that executes on the host. Three categories of
quickly develop modular and reusable software. First, all processes execute on the host: the server process, console
communication mechanisms are processor transparent, and processes, and user processes. All processes communicate
hence modules can be developed without knowledge of the via local shared memory and semaphore facilities available
final target hardware architecture. Second, the two-level device in the host’s global OS. The host’s server provides similar
drivers supported by Chimera I1 provide a standard interface functionality as the server on the RTPUs. In addition, it can
not only to I/O devices, as offered by other RTOS, but also to access the host file system directly, and it includes special
sensors and actuators. Third, the servo task control mechanism primitives to support the console processes. The console
forces the programmer to develop code as reconfigurable, process provides the user interface that serves to download
hence resulting in reusable modules. Each module can then and execute programs on the RTPUs. The console process also
execute on any RTPU at any frequency, and all modules provides the stdin, stdout, and stderr files for tasks executing
can be controlled by a single task. The modules form a on the RTPUs. A single console process can control all RTPUs
library of modules, any of which can be used in later systems within the system. However, if multiple RTPUs are using
without the need for recompiling any parts of the code. stdin, only one of them can have it active at once. Other
1286 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBERIDECEMBER 1992

VME bus

Ethemet
v Network File System

express mail
Multiprocessor Operations

extended file system

host procedure calls
VME and mailbox interrupts
special purpose processors
global shared memoty
spin locks
remote semaphores
message passing
global state variable table
multiprocessor servo task wntrol
I I a User Process 1

+ + +
shared memory User Process N
A I I semaphores

local shared memory

C libraries

I I / . w I I

local stadc
Express Mail
Server
server
(optional)

XEp
real time kernel
local state

+ Express Mail
RTPU Downloadand Control
Host Workstation

Global Shared Memory

Remote Semaphores
task management memory management Host Prcadure Calls Message Passing
static and dynamic scheduling system tables 1 I
deadline failure handlers local Semaphores
interrupt handlers user signals
exception handlers U0 device drivers
global error handlino sensorlactuator interface VME bus
~~

Real-Time Processing Unit (RTPU)

Fig. 4. Software architecture of host workstation.
Fig. 3. Software architecture of a RTF'U.
Our basis for measuring performance is the amount of CPU
tasks reading from stdin block, and send a Waiting for TTY time during normal execution that must be dedicated to the
Input message to the user's terminal, similar to the way UNIX operating system functions, such as scheduling, task switching,
handles background processes. If multiple RTPUs required and communication overhead. The major design decisions that
stdin simultaneously, then multiple instances of the console we made in developing the Chimera I1 kernel are described
process can be created, each in a separate window on the host below.
workstation. I ) Tasks: A task in Chimera I1 is also known as a thread
User processes are just like any other UNIX process running or lightweight process in other operating systems. A user
on the host workstation, except that an additional Chimera I1 program that is downloaded into an RTPU consists of a single
library is linked in, allowing the process to use the Chimera I1 executable file. The kernel is supplied as a C library and is
IPC package. By using this library, the host workstation linked into the executable image. When a program is down-
appears as an RTPU to the other RTPUs, thus making commu- loaded to an RTPU and executed, some kernel initialization
nication between the host workstation and RTPUs transparent. is performed, following which the user's main() routine is
The host processes can thus read from and write into the mem- spawned as a task. Any other task can then be started from
ory of any RTPU, send or receive messages, or synchronize main().
using remote semaphores. This powerful feature allows users 2) Intertask Security: Typically, all of the tasks running on
to create their own custom user interfaces, which possibly a given RTPU (or set of RTPUs) are designed to cooperate.
include graphics or windowing facilities offered by the host We have sacrificed inter-task protection, allowing one task to
workstation. access the address space of any other task. This eliminates a
large amount of overhead incurred in performing system calls
A . Real-Time Kernel or their equivalents. In general, inter-task security is desirable
One of the important goals of the kernel is to provide the for two reasons: 1) to prevent multiple users or independent
required functionality at the highest performance possible. In processes from infringing on other processes in the system;
our system, this is achieved by sacrificing traditional operating and 2) to contain software errors within a faulting module
system features, such as virtual memory and intertask security. for better control of error handling. Although Chimera I1 is
STEWART et al.: THE CHIMERA I1 REAL-TIME OPERATING SYSTEMS 1287

multitasking and multiprocessor, it is designed to be single to indefinitely lock a task into the CPU for atomic or critical
user. Two totally separate control systems should each have emergency handling code.
their own installation of Chimera 11. This is necessary if 7) Real-Time Task Scheduler: Chimera I1 supports both
predictable execution of each system must be maintained. static and dynamic preemptive scheduling of real-time tasks.
Therefore the first reason for wanting inter-task security is The default scheduler supports the rate monotonic (RM)
not applicable to Chimera 11. As for containing errors within static priority scheduling algorithm [lo], [111, the earliest-
a software module, inter-task security prevents corrupting deadline-first (EDF) dynamic scheduling algorithm [111, [33],
memory of other modules, and causes the faulting module and the maximum-urgency-first (MUF) mixed (static and dy-
to abort. These types of errors typically occur due to software namic) scheduling algorithm [26]. In addition, the scheduler
bugs. The global error handling mechanism has some facilities is designed as a replaceable module, allowing user-defined
for automatically detecting bugs, such as memory corruption schedulers to easily override the default scheduler, just by
errors or bad arguments to system calls, and hence provides linking the new scheduler with the application.
an alternate method for handling software errors. The wide range of support for scheduling algorithms with
3) Task Management: Chimera I1 provides the kernel task the default scheduler allows Chimera I1 to be used in many
management features typical to all RTOS, including creating, applications, without being restricted by the default scheduler,
suspending, restarting, preempting and scheduling. In addition, as is the case with the commercial RTOS, which restricts the
Chimera I1 provides a different approach to handling task programmer to using a static priority scheduling algorithm. For
timing. Whereas other RTOS require the user to program example, RM is usually the algorithm of choice when develop-
timers, the Chimera I1 kernel performs all timer program- ing a single-configuration system. EDF and MLF are used in
ming automatically. A pause(restart-time) routine is provided, dynamically changing systems when transient overloads of the
which tells the kernel to pause the task until the specified system are not possible, or in static systems when maximum
restart time. The kernel schedules the tasks using virtual CPU utilization is required. MUF is an algorithm we have
timers, all based on a single hardware timer. Using this method designed especially for dynamically reconfigurable systems,
the number of tasks in an application requiring the use of where critical tasks can be guaranteed to execute, even during
timers is not limited by the number of timers available on the transient overloads of the system [26].
RTPU. Therefore the user does not have to manually perform 8) Deadline Failure Handling: One novel and powerful fea-
any timer multiplexing, as is necessary with most other RTOS ture in Chimera I1 is its deadline failure handling mechanism.
when there are insufficient hardware timers. Whenever a task fails to meet its deadline, an optional failure
4) Local Intertask Communication: There is no parent/child handler is called on behalf of the failing task. The failure
relationship among tasks. Any task can communicate or syn- handler can be programmed to execute either at the same of
chronize with any other task either through local shared different priority than the failing task. Such functionality is
memory, local semaphores, or user signals. Within a sin- essential in predictable systems. Any of the actions specified
gle executable file, all global variables are automatically in Section 11-E. and other user-defined actions can be imple-
shared. Local semaphores provide high-speed synchronization mented using the deadline failure handling available with our
between tasks, and can be used either to provide mutual MUF scheduler.
exclusion during critical sections (binary semaphores), or to Estimating the execution time of tasks is often difficult.
provide synchronization among tasks (general or counting For example, most commercially-available hardware is geared
semaphores). User signals provide an alternate method of towards increasing average performance via the use of caches
synchronization, allowing the receiving task to be interrupted and pipelines. Such hardware is often used to implement
when the signal arrives, instead of explicitly checking if the real-time systems. As a result, the execution time cannot
signal has arrived as done with the local semaphores. Any of necessarily be predicted accurately. Underestimating worst-
the IPC mechanisms described in Section 111-B.can also be case execution times can create serious problems, as a task in
used locally. the critical set may fail. The use of deadline failure handlers
5) Memory Management: The total address space used by all is thus recommended for all tasks in a system. The Chimera I1
tasks on one system is limited by the physical memory avail- default scheduler provides this ability.
able on the RTPU. Chimera I1 does not provide any virtual 9) Error and Exception Handling: Three types of nontiming
memory, as the memory management and swapping overhead errors may occur in an advanced sensor-based control system:
not only decreases the performance of a system drastically, hardware errors, state errors, and software errors [2], [4].A
but it also causes the system to become unpredictable, thus hardware error is either directly generated by the hardware,
violating one of the important rules of real-time systems. such as a processor exception (e.g., bus error), or be detected
Chimera I1 provides its own version of the maZloc() family by software, as is usually the case with I/O devices. Some-
of routines to allocate physical memory. times these errors are only transient, or can be corrected by
6) Interrupt Handling: The Chimera I1 kernel provides the software; other times human intervention is required to reset
interfacing routines to easily define and install C-language the hardware. State errors are software generated, generally
interrupt handlers for VMEbus IRQ interrupts, local LRQ inter- after failing some form of if-then comparison. These errors are
rupts, and a user-definable number of mailbox interrupts, even more abstract than hardware errors, and indicate that an invalid
if the hardware only supports one or two mailbox interrupts. software state has been detected. For example, a state error
Utilities are also given to enable and disable interrupts, and occurs if a robot pick-up command fails, because the object
1288 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBEWECEMBER 1992

was not there or the grasp operation failed. The hardware itself such as read() and printf() return error codes when an error is
does not generate an error; but the state holding object after detected. However, because they are almost always successful,
the pick-up operation is not correct. Software errors are due programmers tend to not check the return values, in the interest
to software design oversights, limitations, or bugs. In general of increased performance, decreased program complexity, and
they should be fixed promptly. However, if they occur while faster development time. The global error handling automat-
the system is executing, appropriate error handling is required ically detects and handles any errors in these routines, thus
to ensure predictable execution. eliminating the need for the programmer to check the return
In a predictable system, every error must be handled in a values. As a result, the code becomes more robust and behavior
known fashion. In the best case, an error handler is called is more predictable, as even unexpected or rarely occurring
that corrects the error. An intermediate solution is to operate errors are handled.
the system with degraded performance; this is often necessary State errors are usually detected by software. The following
with a fully autonomous system. In the worst case, the system code segment is a typical example of detecting a state error:
must be shutdown. To prevent damage to the system and the
environment, system shutdown must be graceful, such that
i f (count > 10 && !found){
moving actuators slowly come to a halt, and all power is turned errnum = ENOTFOUND;
off after the actuator comes to a halt. return(-1); / * error, o b j e c t not found * /
One of the most powerful features in Chimera I1 is the
global error handling mechanism. It allows user programs
1.
to be developed without ever having to explicitly check the If using the global error handling mechanism, the following
return value of an error. Instead, whenever an error is detected, code segment would instead be written as follows:
the error handling mechanism is invoked, by generating an if(count > 10 && !found)
error signal. By default, a detailed error message is printed,
errInvoke(moduleptr, ENOTFOUND); .
and the task at fault is aborted. The mechanism allows error
messages to be very specific, thus aiding in tracing down The routine errInvoke() generates an error signal, which is
errors in the system. When the debug mode is enabled, the handled either by the default error handler or by a user-
source code filename and line number are also printed. The defined error handler. The moduleptr argument is obtained
programmer may override the default handler, on a per-error- during initialization of the particular module containing this
code, per-module, or per-scope basis. After handling the error, code.
the default action of aborting the task can also be overridden, The basic design philosophy of the global error handling
either by continuing at the line after the error, or by returning mechanism in Chimera I1 is similar to that found in some
to a previously marked place in the program. Every task has languages, such as Ada, A M L K [17], and Exceptional C[2].
its own set of error handlers. However, by making it part of the OS it has several major
The design of the global error handling allows the defini- advantages such as programming language independence, au-
tion of programs and error handlers to be kept separate. In tomatic detection of system call and kernel errors, and the
traditional UNIX C code, error handling is built-in to the code ability to automatically detect such things as bad arguments
by using if statements to check return values of routines, as to system calls or memory corruption, which are generally a
shown in the following code segment: result of software errors.
i f ( ( f d = open(”f ilename”. 0-RDONLY)) The global error handling mechanism coexists with standard
== -1){ UNIX error handling. If it is not enabled, then any routines
that generate errors, such as system calls, return an appropriate
f p r i n t f ( s t d e r r , ” E r r o r opening f i l e % s / n ” error value (e.g. -1 or NULL) and set the global variable
filename); errno, as is consistent with UNIX. Processor exceptions, such
exit(); as bus error and divide-by-zero, also generate error signals.
However, when the global error handling is disabled, C-lang-
1. uage processor exception handlers can be installed to override
When the global error handling mechanism is enabled, the the default action of aborting the task.
open() routine generates an error signal instead of returning 10) I t 0 Devices, Sensors, and Actuators: The Chimera I1
-1. The default error handling of printing a detailed error kernel adopts a two-level approach to developing and using
message and aborting the task then occurs, unless a user- device drivers to isolate the user from the details of special
defined handler was previously installed to handle errors hardware. The first level drivers provide a hardware indepen-
occurring in the open() routine. As a result, the main program dent interface to I/O devices, such as serial ports, parallel ports,
becomes much simpler, in that error checking after each analog-to-digital converters, digital-to-analog converters, and
system call is not needed. The above code segment then frame grabbers. It supports the open, close, read, write, ioctl,
reduces to the following: and mmap drivers, as is customary in most UNIX-based OS.
These drivers are usually much simpler to write than their
f d = open(”f ilename”, 0-RONDLY); .
UNIX counterparts because intertask security is not a concern.
It has also been our experience that most software bugs go The second level device drivers are not required, but are highly
unnoticed because return values are not checked. Routines recommended. They provide a standard hardware independent
STEWART et al.: THE CHIMERA I1 REAL-TIME OPERATING SYSTEMS 1289

interface to the sensors and actuators that are connected to on the same or different RTPUs. The message passing uses
the system via the 1/0 ports. This modular interface allows the express mail to initialize a message queue. Variable-length
applications to be developed without prior knowledge of the messages are then sent by placing them into the queue. The
details of the sensors and actuators, and allows the sensors and length of the queue is user-definable. Messages are received
actuators to be developed independently, without knowledge by retrieving them from the queue, either on a first-in-first-
of the target application. out, last-in-first-out,or highest-priority-first basis. The sending
and receiving of messages bypasses the express mail, thus
B. Multiprocessor Support eliminating unnecessary run-time overhead.
The Chimera I1 kernel executing on one RTPU is designed 6) Global State Variable Table Mechanism: A state variable
to communicate with kernels on other RTPUs through express table mechanism has been developed to allow control tasks
mail, in order to provide the base for a multiprocessor OS. on multiple RTPUs to share state information, and to update
Many other interprocessor communication and synchroniza- the state information quickly and correctly. This mechanism
tion mechanisms are built into Chimera I1 as a layer above automatically creates one global table, and a local copy of the
the express mail, for simplifying the development of complex table for each task that requires access to the state variables.
applications, These IPC mechanisms are described as follows. Tasks always make use of the local copy, with periodic
I ) Express Mail: The express mail mechanism is a high- updates (typically at the beginning and end of each periodic
speed communication protocol we have developed especially cycle) from the global table. This mechanism allows the exact
for backplane communication. It is the lowest level of commu- calculation of the required VMEbus bandwidth and transfer
nication used by the system level software. It has three basic time required when integrating multiple real-time control tasks
functions: 1) provide a layer of processor transparency, so that ~71.
all other IPC mechanisms remain processor independent; 2) 7) Multiprocessor Servo Task Control: In conjunction with
handle the I/O between the host workstation and RTPUs; and the global state variable table mechanism, this mechanism is
3) send system signals and mailbox interrupts between RTPUs. designed to support the automatic integration and control of
In order to keep IPC overhead to a minimum, most other IPC reconfigurable servo task control modules [27]. One task in
mechanisms only use the express mail during initialization. the system may take control over some or all of the RTPUs
After a communication link is made, all subsequent commu- in the system. The task can then spawn new tasks on any
nication bypasses this layer, so that unnecessary overhead is RTPU, and control the execution of the other tasks through
removed. User’s cannot directly send and receive express mail simple “on,” “off,” “control,” and “status” mechanisms. The
messages; rather, the other IPC mechanisms must be used. integration of multiprocessor applications becomes fairly sim-
2) Global Shared Memory: The VMEbus has several differ- ple. Reconfiguring an application to use different controllers
ent address spaces, and each processor addresses the spaces can be performed dynamically. This mechanism also makes the
differently. By using the automatic address resolution features updates of the local copies of state variable tables (described
of the express mail, memory can be dynamically allocated above) automatic, thus removing that responsibility from the
from any RTPU on any other RTPU or memory card, and it programmer.
can be shared by tasks on all RTPUs. All the address resolution 8) Extended File System: The real-time system makes use of
and address offsets are performed during initialization of the the file systems on the host workstation, instead of requiring
shared memory segment, so that transfers between local and a separate disk file system in the real-time environment. This
shared memory have no operating system overhead. allows the real-time tasks to read and write the same files
3) Spin-Locks: Spin-locks make use of atomic test-and-set as any process on the host workstation. A standard UNIX
(TAS) instructions in order to provide mutual exclusion for system call interface, using the open(), close{), read{), write{),
shared data [14]. In general they require the least amount of ioctl{), and lseek{) routines, are used to access the extended
overhead of any synchronized IPC mechanism. However, they file system. The concurrent standard I/O library can also be
use polling to obtain the lock, which may waste significant used for buffered I/O. All remote operations are completely
CPU time. The polling time and a bounded time for retrying transparent to the user.
before issuing a time-out error can be set by the user. 9) Host Procedure Calls: Tasks running in the real-time
4) Remote Semaphores: Most real-time operating systems environment can trigger execution of routines on the host
only provide local semaphores; Chimera I1 also provides re- workstation by using host procedure calls. In such cases,
mote semaphores that allow tasks on multiple processors to execution of the routines is performed on the host, but it
use semaphores. Like the spin-locks, the remote semaphores appears to the user that it is executing in the real-time
use TAS instructions to obtain a lock; however, unlike the environment. Therefore, an interactive program running on an
spin-locks, a task will block instead of polling for the lock. RTPU can call an editor (such as vi or emacs), and allowing
When the lock is released, the blocked task is automatically the user to edit information. Once the user exits, execution
be woken up. Both binary and counting remote semaphores of the task on the RTPU then continues. The host procedure
are supported. The remote semaphores can be used either for calls provided include changing current directory, getting an
mutual exclusion when accessing a shared memory segment environment variable, and executing any stand-alone program,
or for synchronizing tasks on different RTPUs. such as Is, vi, and grep.
5) Priority Message Passing: The Chimera I1 priority mes- 10) Host Workstation Integration: The host workstation is
sage passing allows typed messages to be sent between tasks fully integrated into the real-time environment; it appears to
1290 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBEWDECEMBER 1992

all the RTPUs as just another RTPU. As a result, processes a standardized, yet general format for configuration files.
on the host workstation can make use of the global shared When using this library, most error checking is automatically
memory, remote semaphores, or prioritized message passing to performed by the library routines, removing that need from
communicate rapidly with tasks in the real-time environment. the programmer.
For example, a host process may be graphically displaying Details of any of the features described in this section can
data on the workstation. The data is being collected in real- be found in the Chimera I1 program documentation [28].
time, and being stored in a shared memory segment on one
of the RTPUs. The host process can then periodically read
that location, and update the graphics display. To bypass IV. IMPLEMENTATION
the lack of timing on the host workstation, the task in the Chimera I1 has been used with several different systems,
real-time environment can use the binary remote semaphores both at Carnegie Mellon University (CMU) and elsewhere,
to periodically signal the host process, using the more ac- including at the Jet Propulsion Laboratory, California Institute
curate real-time timers, instead of the less accurate UNIX of Technology; the Air Force Institute of Technology; Wright
timers. State University; and the University of Alberta. At CMU, it is
11) Special Purpose Processors: Special purpose proces- being used with the CMU Direct Drive Arm I1 (DDArm 11)
sors are generally added to real-time systems in order to [8], the CMU Reconfigurable Modular Manipulator Systems
increase performance for specialized computations. Examples (RMMS) [22], the Self-Mobile Space Manipulator [ 11, the
of special purpose processors are floating point accelerators, Troikabot System for Rapid Assembly [9], [19], and the
image processors, digital signal processors, LISP machines, Amada Steel Sheet Bending System. Each of these systems
and transputers. In order to simplify the integration and use is very different in nature, but can be controlled using the
of the processors, we have designed a hardware independent same base operating system, which is proof of the flexibility
interface that allows a task to download, copy data, and call of Chimera 11.
subroutines on the processors. The processors are treated as As an example, a block diagram of the system components
slaves in the system that do not execute a kernel. However, of the CMU DDArm I1 is shown in Figure 5. The system
nonpreemptive scheduling, such as first-come-first-serve or has a total of 12 processors in the real-time environment:
shortest-job-first can be used to schedule multiple subroutine three general purpose MC680XO processors, a 20 Mflop peak
calls to these processors. Mercury floating point accelerator, six TMS320 digital signal
processors, and an Imaging Technology vision system with
two Weitek processors. The Chimera I1 kernel is running on
C. Libraries
two of those processors: the Ironics MC68020 and Ironics
Chimera I1 provides several libraries that ease the program- MC68030 processors. The other processors are all treated as
ming of large applications: special purpose processors. Note that the Heurikon MC68030
1) UNIX C and Math Libraries: These libraries provide is executing the OS-9 real-time kernel. The only reason for
compatibility with UNIX-based OS and other RTOS. The C not using Chimera I1 on this processor is that the commercial
library includes routines for manipulating strings, block copy, libraries for the vision system were only available as OS-9
fill, and move operations, random number generating, sorting object code. With such a setup, we show that Chimera I1 is
and searching routines, time-of-day utilities, and memory al- capable of working in conjunction with other RTOS, if the
localion routines. The math library includes the transcendental need arises.
and logarithmic functions, as well as other useful floating point The system has a total of four buses: the host VMEbus,
functions. the control subsystem VMEbus, the vision subsystemVMEbus,
2) Concurrent Standard Inputloutput Library (stdio): This and a Multibus for the robot controllers. Bus adaptors are
library provides a multiprocessor version of the UNIX stdio used to isolate the bus traffic of each subsystem. Chimera I1
library, which allows multiple tasks on multiple processors to automatically handles the address translations required when
share files, including the standard input, standard output, and crossing adaptors.
standard error of the remote console. Several sensors are connected to the system, including a
3) Matrix Math Library: This library provides optimized 6-degree-of-freedom trackball, a tactile sensor, a forcehorque
floating point routines for manipulating vectors and matrices. sensor, a radiation sensor, and a camera mounted on the end-
Functions available include matrix addition and multiplication, effector. By using Chimera 11, sensors can easily be added
dot and cross products, determinants, Gaussian elimination, or removed, and cooperating software quickly written. As a
and matrix copy and subcopy. result, application programmers can concentrate on the higher
4) Command Interpreter Library: This library provides a level details of sensor integration, as opposed to the low-level
set of routines for quickly building custom command line and hardware details.
interfaces. The custom command interpreter automatically has
built-in searching for closest match, shell execution, help files,
and script file support. A. Performance
5) Configuration File Library: In a reconfigurable system, The success of a real-time operating system in developing
configuration files are used to define the current configura- robotic applications is based both on the ease of use and on
tion. This library provides the utilities for quickly reading its performance. Writing code to run under the Chimera I1
 __

STEWART et al.: THE CHIMERA I1 REAL-TIME OPERATING SYSTEMS 1291

O(7)1”..6 pq
Ethernet

workstation

Software Development and

User Interlace (non-real-time)

W E to VME Real-The Execution

adaptor

-r
Control
Subsystem 6-dol trackball

....
tactile sensor

forcehorque sensoi

radiation sensor

controller 1 controller 6

totfrom tohom
robot joint 1 robot joint 6

Fig. 5. Block diagram of system components of the CMV Direct Drive ARM 11.

environment is similar to writing a C language program The scheduling time in this case is 28 ps, resulting in a
to run under UNIX. Several critical operating system fea- peak CPU utilization of 97% with a one millisecond time
tures were timed to provide a general idea of the perfor- quantum. At the other extreme, a full context switch is
mance of Chimera 11. The timings shown in Table I were needed, which executes in 66 ys, for a total of 94 ps, thus
performed on Ironics model IV3230 RTPUs [6], each with providing minimum CPU utilization of 91 percent for CPU-
a 25 MHz M68030 CPU and a 25 MHz M68882 floating bound jobs. The scheduling time includes all checks for missed
point coprocessor (FPU). All times were measured using deadlines, updating dynamic priorities, and selecting a new
a 25 MHz VMETRO VBT-321 Advanced VMEbus Tracer task if the highest priority task changes. A full context switch
~311. involves suspending the current task by saving its entire
A task can be created in 220 ps, and destroyed in 99 ps. context, including the FPU registers, and resuming the new
In general, these operations are done during initialization and task by restoring its entire context. Note that saving and
termination, and not during time crucial code. As a result, restoring the FPU registers accounts for over half the context
performance of these routines is sometimes sacrificed, in switch time alone. Although it is possible to improve the
favor of performing any computations beforehand, so that the context switch time for tasks that do not use the FPU, it
performance of run-time operations, such as context switch- was decided that most control tasks use at least some floating
ing, can be improved. It takes 20 ps to update the static point operations. Instead of adding overhead to keep track
priority of a task, which includes rearranging any queues of when floating point operations were used, the full FPU
sorted on priority. The time to update the dynamic priority context is always saved and restored at each context switch.
of a task is included in the rescheduled time, described A reschedule and context switch arising as a result of the
next. running task blocking due to resource contention does not
One of the important performance criterion of a multitasking have to recalculate any dynamic priorities; it takes 82 ps.
kernel is the reschedule and context switch time. Upon the Worst case CPU utilization can be increased to over 99% by
expiration of a time quantum, a timer interrupt is generated, increasing the time quantum to 10 ms, which can be done
forcing a time-driven reschedule. If the currently running task for all but demanding servo level tasks that must run at
has the minimum laxity or is highest priority, it remains frequencies typically above 100 Hz. A task can also be locked
running, and the timer interrupt results in no task swapping. into the CPU, thus disabling the preemption. In this case only
1292 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBERDECEMBER 1992

TABLE I
CHIMERA 11 PERFORMANCE OF SELECTED FEATURES

Task-Management
Create a task with default arguments 220 ps
Destroy a task 99 ps
Modify static priority of a task 20 p s
Timer-driven reschedule, no task swapping 28 p s
Context switch time, including CPU and FPU save and restore 66 p s
Timer-drive reschedule, with context switch 94 p s
Resource contention reschedule, with context switch 82 ps
Timer interrupt, CPU locked and preemption disable, no rescheduding 6 PS
Local Semaphores
P( ) operation, no blocking 7 c”s
P( ) operation, blocking 89 p s
\-( ) operation, no waking up 7 PS
l‘( ) operation, waking up task 34 p s
Remote Semaphores
semP( ) operation, no blocking
semP( ) operation, blocking, with resource contention reschedule

semV( ) operation, no waking up

semV( ) operation, waking up task on local RTPU
semV( ) operation, waking up task on remote RTPU

Multiprocessor Priority Message Passing

Place a 24-byte message into queue, queue in local memory
Place a 24-byte message into queue, queue in remote memory
Retrieve a 24-byte message from queue, queue in local memory
Retrieve a 24-byte message from queue, queue in remote memory
Global Shared Memory
Overhead for reading from shared memory
Overhead for writing into shared memory

t Time includes 11 p s overhead for TAS workaround.

6 ps are used at each timer interrupt to update the system that is physically stored in local memory, or 79 ps to store
clock. the message into a queue in remote memory. These times
The local P ( ) (wait) and V ( ) (signal) semaphores each include the C subroutine call overhead. It takes 64 ps for a
execute in 7 ps when there is no blocking or waking-up occur- task to retrieve that message from a queue in local mem-
ring. A blocking task adds the time of the resource contention ory, or 81 ps for a queue in remote memory. Thus an in-
context switch before the next task begins executing. The V ( ) terprocessor message can be sent and received in 160 ps.
operation takes an additional 27 ps to wake up a blocked These times all include the 11 ps overhead for the TAS
task. workaround.
Inevitably, remote semaphores take longer than local sema- There is no software overhead involved in accessing global
phores. A non-blocking semP( ) remote semaphore operation shared memory. The speed of shared memory transfers
takes 27 ps. If the s e m P ( ) operation results in blocking across the VMEbus is limited only by the hardware.
the running task, then the time of the resource contention As a result, global shared memory is the fastest means
context switch is also needed before the next task can ex- of communication among tasks within a multiprocessor
ecute. The semV( ) remote semaphore wakeup operation system.
takes 26 ps, if no tasks are to be woken up. Waking up Note that the times above are subject to small variations
a blocked task that is on the same RTPU takes an ad- as Chimera TI continues to be developed. Nevertheless, the
ditional 38 ps, while waking up an off-board task takes timing measurements provide a fairly good representation
another 36 ps to send a mailbox interrupt to the proper of the performance of Chimera 11. The high performance
RTPU for a total of 100 ps. There is a documented hardware of Chimera I1 allows it to maintain over 90 percent CPU
bug in the implementation of the TAS instruction on the utilization for control tasks running up to 1000 Hz, thus
Sun workstation, preventing it from being ltomic over the allowing it to be used with the most computational and time
VMEbus. A TAS workaround is built-in to all Chimera I1 IPC demanding servo tasks.
mechanisms. Thus the remote semaphore operations include
an 11 ps overhead, which can be saved if a host workstation
without the bug is used, and hence the TAS workaround not B. Comparison of Chimera 11 with other RTOS
needed. In this section, we briefly describe how the features and
For the multiprocessor priority message passing, it takes performance of Chimera I1 compare with other RTOS. We
62 ps for a task to place a 24-byte message into a queue are only interested in local RTOS for control applications,
STEWART et al.: THE CHIMERA 11 REALTIME OPERATING SYSTEMS 1293

which are capable of supporting tasks up to 1000 Hz. Thus, especially for advanced sensor-based control applications, and
global RTOS such as ARTS [29], Real-Time MACH [30], implemented in Chimera 11:
and Alpha [7] are not considered. Although they provide Combined static and dynamic scheduling: the de-
various powerful real-time features, their designs do not fault Chimera I1 scheduler is capable of supporting
allow them to be used effectively with tasks executing up to the rate monotonic, earliest-deadline-first, maximum-
1000 Hz because of significant amounts of system overhead. urgency-first, and user-defined real-time scheduling
These operating systems, however, are excellent candidates for algorithms.
providing real-time facilities on the host workstation, allowing Two-level device driver support: a first level of device
the host workstation and the RTPUs executing Chimera I1 to drivers provides the standard UNIX-like interface to
communicate with each other in real-time. I/O devices. The second level provides a standardized,
It has been very difficult to obtain performance times hardware independent layer to sensors and actuators.
of the same features with each RTOS because we did Special Purpose Processor Interface: A standardized in-
not have their required hardware and software available. terface to special purpose processors makes it easy to
In addition, the timings published for one RTOS are integrate and use such processors for enhancing real-time
not necessarily comparable to those in a different RTOS, performance.
because of differences in functionality for similar features. Global error handling mechanism: Programmers no
What is needed is a standard, independent benchmarking longer have to worry about checking the return values of
of each RTOS for proper comparison. Such an effort has any routine that may generate an error. All errors in the
begun [12] but has not yet been completed. However, system, including those which can detect software bugs,
based on the performance benchmarks available in the produce an error signal, which by default prints an error
literature referenced earlier, the following observations can be message and aborts a task, but can be overridden with
made. any user-defined error handler.
The context switch time for Chimera I1 is 66 ps on a Express mail: A new communication protocol based on
25 MHz M68030 CPU, of which at least half of the time the shared memory available on open-architecture stan-
is for saving and restoring all the M68882 FPU context dard buses allows high-speed system communication be-
and registers. VxWorks claims a 35 ps context switch tween RTPUs.
time, but that does 8ot include the time to save FPU Global shared memory: The automatic address resolution
registers. OS-9 advertises 55 ps, which possibly includes and transparent no-software-overhead access of shared
the FPU context save. As for research operating systems, memory allows RTPU-independent modules to use global
SAGE takes 170 ps, and no timings are available for and shared memory efficiently.
Harmony and Condor. Remote “stmaphores: both binary and counting sema-
Interprocessor priority messages in Chimera I1 take 79 ps phores can be used by tasks on different RTPUs, instead
to send a 24-byte message, and another 81 ps to retrieve of being restricted to a single RTPU as in other RTOS.
that messages, even if the message is being sent to the Global state variable table mechanism: This mechanism
host workstation. SAGE takes as much as 920 ps for has been developed to allow control tasks on multiple
a 4-byte RTPU-to-RTPU message. It takes as much as RTPUs to share state information, and to update the state
15 ms when the htkt is involved, using the UDP protocol. information quickly and correctly. The mechanism allows
CGndor takes at least 200 p, for RTPU-to-RTPU, and as the maximum bus bandwidth and execution time of the
much as 34 ms when the host workstation is involved system to be computed a priori.
in the transfer. No times are available for interprocessor Multiprocessor servo task control mechanism: This mech-
messages using any of the commercial RTOS, although anism is designed to support the automatic integration and
those systems that do support multiprocessing use TCP/IP control of dynamically reconfigurable servo task control
for messages, which puts the timing of these messdges modules in a multiprocessor environment.
on the order of ms. Host workstation integration: By linking a Chimera I1
VxWorks estimates 10 ps for a non-blocking local library with processes on the host workstation, those
semaphore operation on a 25 MHz M68020 processor. processes can communicate with tasks running on the
VRTX takes 24 ps on a 25 MHZ MC68030. Chimera I1 RTPUs, using the global shared memory, remote sema-
performs the same operation in 7 ps on a 25 MHz phores, and priority message passing. The host work-
M68030 processor. Some of the other RTOS do not even station appears to the RTPUs as just another RTPU.
support local semaphores, and only Chimera I1 supports Host procedure calls: Facilities that are not available
remote semaphores. in the real-time environment, but are available on the
These observations show that Chimera I1 performs as well, host workstation, can be accessed transparently by tasks
if not better, than other RTOS. However, the strength of running on RTPUs by using the host procedure calls.
Chimera I1 lies in the multitude of features it has that are In addition to the above features, Chimera I1 also contains
highly desirable for developing advanced sensor-based control several other features useful for advanced sensor-based control
applications, but are not available with the other RTOS. systems, which are similar to features found in other RTOS:
Following is an annotated list of the features we have designed Priority message passing: Many RTOS offer this fea-
1294 IEEE TRANSACTIONS ON SYSTEMS. MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBEWDECEMBER 1992

ture, but the Chimera I1 implementation, based on the Intel Corporation, Santa Clara, CA 96051, Extended iRMX 11.3, 1988.
express mail, allows variable-length user messages to be Ironics Incorporated, 1V3230 VMEbus Single Board Computer and Mul-
tiprocessing Engine User’s Manual, Technical Support Group, 798
transferred across processors in 160 ps. Cascadilla St., Ithaca, New York 14850.
Extended file system: All RTOS provide some form of E.D. Jensen and J.D. Northcutt, “Alpha: A nonproprietary OS for
file system. In Chimera 11, the extended file system uses large, complex, distributed real-time systems,” in Proc. IEEE Work-
shop on Experimental Distributed Systems, Huntsville, AL, Oct. 1990,
the express mail to provide high-speed access to the file pp. 35-41.
system of the host workstation. T. Kanade, P.K. Khosla, and N. Tanaka, “Real-time control of the
CMU Direct Drive Arm 11 using customized inverse dynamics,” in
C language interface to interrupt handlers: the program- Proc. 23rd IEEE Con) Decision Contr., Las Vegas, NV, Dec. 1984,
ming of interrupt handlers is difficult in most RTOS, pp. 1345-1352.
because assembly language must be used. Chimera I1 P.K. Khosla, R. S. Mattikalli, B. Nelson, and Y. Xu, “CMU Rapid
Assembly System,” in Kdeo Proc. 1992 IEEE Int. Con) Robotics
provides an interface allowing C-language handlers to be Automat., Nice, France, May 1992.
installed without the need for any interfacing assembly J. Lehoczky, L. Sha, and Y. Ding, “The rate monotonic scheduling
code. algorithm: exact characterization and average case behavior,” in Proc.
10th IEEE Real-Time Syst. Symp., Santa Monica, CA, Dec. 1989,
User signals: a task on an RTPU may interrupt another pp. 166-171.
task on the same RTPU by sending a user signal. The C. L. Liu, and J. W. Layland, “Scheduling algorithms for multiprogram-
handling of signals is completely user-definable. ming in a hard real time environment,”J. CM, vol. 20, no. 1, pp. 44-61,
January 1973.
Spin-locks: the Chimera I1 implementation of spin-locks S. Malone, “Benchmarking kernels as a means of evaluating real-
allows the polling time and a bounded retry time to be time operating systems,’’ Master’s thesis, Dept. Elect. Comput. Eng.,
specified by the user. Carnegie Mellon Univ., Pittsburgh, PA, Dec. 1991.
Microwave, The OS-9 Catalog, (Des Moines, IA), 1989.
”

Chimera I1 also provides a matrix library, command L. D. Molesky, C. Shen, and G. Zlokapa, “Predictable synchronization
interpreter library, a configuration file reading library, mechanisms for multiprocessor real-time systems,” The J . Real-Time
Syst., vol. 2, no. 3, Sept. 1990.
and a concurrent version of the standard 1/0 library, in Motorola, Inc., MC68030 enhanced 32-bit microprocessor user’s man-
addition to the standard UNIX C library. These libraries ual, third ed. Englewood Cliffs, NJ: Prentice Hall, 1990.
Motorola Microsystems, The VMEbus Specification, Rev. C.l, 1985.
reduce the development time required for applications by L. R. Nackman, et al., “AMLIS: a programming language for design
providing already-debugged utility routines for often used and manufacturing,” in Proc. 1986 Fall Joint Comput. Con), 1986,
functions. pp. 145-159.
S. Narasimhan, D. Siegel, and J. Hollerbach, “Condor: A revised
architecture for controlling the Utah-MIT hand,” in Proc. IEEE Con5
Robotics Automat.,. Philadelphia, PA, April 1988, pp. 446-449.
V. CONCLUSION N. P. Papanikolopoulos, B. Nelson, and P. K. Khosla, “Monocular 3-D
visual tracking of a moving target by an eye-in-hand robotic sys-
When implementing a real-time advanced sensor-based con- tem,” in Proc. IEEE Con) Decision Contr. {CDC),Tucson, AZ, Dec.
trol systems, too much time is typically spent with low-level 1992.
J. F. Ready, “VRTX: A real-time operating system for embedded micro-
details to get the hardware to work, as opposed to higher
processor applications,” IEEE Micro, vol. 6, pp. 8-17, Aug. 1986.
level applications that allow the system to do something L. Salkind, “The Sage operating system,” in Proc. I989 IEEE Int. Con)
useful. Chimera I1 is a real-time operating system that adds Robotics Automat., Phoenix, AZ, May 1989, pp. 860-865.
D. E. Schmitz, P. K. Khosla, and T. Kanade, “The CMU reconfigurable
a layer of transparency between the user and the hardware modular manipulator system,” in Proc. Int. Symp. Exposition on Robots
by providing a high-performance real-time kernel and a va- (designated 19th ISIR), Sydney, Australia, Nov. 1988, pp. 473-488.
riety of IPC features. The hardware platform required to D. E. Schmitz, P. K. Khosla, R. Hoffman, and T. Kanade, “CHIMERA:
A real-time programming environment for manipulator control,” in
run Chimera I1 consists of commercially available hardware, Proc. 1989 IEEE Int. Con) Robotics Automat., Phoenix, AZ, May 1989,
and allows custom hardware to easily be integrated; and pp. 846-852.
the design allows it to be used with almost any type of Software Components Group, Inc., pSOS+I68K User’s Manual, Version
1.0, (San Jose, CA 95110) 1989.
VMEbus-based processors and devices. It allows radically J. A. Stankovic and K. Ramamritham, “The design of the Spring kernel,”
differing hardware to be programmed using a common sys- in Proc. Real-Time Systems Symp., Dec. 1987, pp. 146-157.
D. B. Stewart and P. K. Khosla, “Real-time scheduling of sensor-based
tem, thus providing a first and necessary step. towards the control systems,” in Proc Eighth IEEE Workshop Real-Time Operating
standardization of reconfigurable systems that results in a Syst. Software, Atlanta, GA, May 1991, pp. 144-150.
reduction of development time and cost. Chimera I1 is already D. B. Stewart, R. A. Volpe, and P. K. Khosla, “Integration of real-time
software modules for reconfigurable sensor-based control systems,” in
being used with several different systems, both at CMU and Proc. 1992 IEEEIRU Int. Conf Intelligent Robots Syst. {IROS ’92),
elsewhere. Rayleigh, NC, July 1992, pp. 325-332
D.B. Stewart, D.E. Schmitz, and P.K. Khosla, Chimera II Real-
Time ProgrammingEnvironment, Program Documentation, Version 1.11,
Dept. EE, Carnegie Mellon Univ., Pittsburgh, PA, April 1992.
REFERENCES H. Tokuda and C. Mercer, “ARTS: A distributed real-time kerne1,”ACM
Operating Syst. Rev., vol. 23, no. 4, July 1989.
[ l ] H. B. Brown, M. B. Friedman, and T. Kanade, “Development of a 5-DOF H. Tokuda, T. Nakajima, and P. Rao, “Real-time Mach: Towards a
walking robot for space station application: overview,” in Proc. I990 predictable real-time system,” in Proc. USENIX Mach Workshop, Oct.
IEEE Int. Con$ Syst. Eng., Pittsburgh, PA, Aug. 1990, pp. 194-197. 1990.
[2] I. J. Cox and N. H. Gehani, “Exception handling in robotics,” Computer, VMETRO Inc., YET-322 Advanced VMEbus Tracer User’s Manual,
vol. 22, no. 3, pp. 43-49, Mar. 1989. 2500 Wilcrest, Suite 530, Houston, TX.
[3] W.M. Gentleman, S.A. MacKay, D.A. Stewart, and M. Wein, “ A n Wind River Systems, VxWorks Reference Manual, Release 4.0.2,
introduction to the Harmony real-time operating system,” IEEE Comput. Alameda, CA, 1990.
Soc. Tech. Comm. Newsletter, pp. 3-6, Summer 1988. W. Zhao, K. Ramamritham, and J.A. Stankovic, “Scheduling tasks with
[4] J. B. Goodenough, “Exception handling: issues and a proposed notation,” resource requirements in hard real-time systems,” IEEE Trans. Software
Comm. ACM, vol. 18, no. 12, pp. 683-696, Dec. 1975. Eng., vol. SE-13, no. 5, pp. 564-577, May 1987.
STEWART et al.: THE CHIMERA I1 REAL-TIME OPERATING SYSTEMS 1295

David B. Stewart (S’87-M’88-S’88) was born in Pradeep K. Khosla (S’83-M’83-SM’91) received

Montreal, PQ, Canada on September 30, 1965. He both the M.S. and Ph.D. degrees from Carnegie
received the B.Eng. degree in computer engineering Mellon University, Pittsburgh, PA.
with great distinction (summa cum laude) from He is currently an Associate Professor in the
Concordia University, Montreal, PQ, Canada in Department of Electrical and Computer Engineering
1988 and the M.S. degree in electrical and com- at Carnegie Mellon University. He is also a member
puter engineering from Carnegie Mellon University, of the The Robotics Institute and Director of the
Pittsburgh, PA, in 1989, where he is currently a Advanced Manipulators Laboratory. Prior to joining
Ph.D. student in the Department of Electrical and Carnegie Mellon, he worked with Tata Consulting
Computer Engineering. Engineers and Siemens in the area of real-time
In 1991 he was a Visiting Researcher for four control. Professor Khosla’s research interests are in
months at the Jet Propulsion Laboratory, NASA, Pasadena. In 1990, he the area of real-time sensor-based manipulation, architectures for real-time
attended the summer session of the International Space University, held in control, integrated design-assembly systems, Robotic Applications, in Space,
Toronto, ON, Canada. He is a recipient of a Natural Sciences and Engineering Field, and Manufacturing environments. He is involved in Electrical and
Research Council of Canada (NSERC) 1967 Science and Engineering Scholar- Computer Engineering and Robotics education both at the graduate and the
ship. He has also been awarded the Canadian Foundation for the International undergraduate level. He was a member of the committee that formulated
Space University Scholarship, also sponsored by NSERC (1990), the Chait a curriculum for the Ph.D. program in Robotics at Carnegie Mellon. He
Medal and Computer Engineering Medal from Concordia University (1988), was also a member of the Wipe the Slate Clean Committee that created a
the Prize of Excellence from the Professional Order of Engineer’s of Quebec, new four-year undergraduate ECE degree curriculum at CMU. He is the
Canada (1988), twice the NSERC Undergraduate Research Award (1986, Chairman of the Education Committee of the IEEE Robotics and Automation
1987), the Myer F. Pollock Scholarship for academic excellence (1986), the Society. He was the Program Vice-chairman for the 1989 IEEE International
Concordia University Entrance Scholarship (1984- 1987), the Independent Or- Conference on Systems Engineering and the General chairman for the 1990
der of Forester’s University Entrance Scholarship (1984- 1988), the National IEEE International Conference on Systems Engineering. He has also served on
Honor Society Scholarship (1982), and the Josten’s Foundation Scholarship the Program Committees of several international conferences. He is currently
(1982- 1983). His research interests include reconfigurable sensor-based sys- the Associate Director of the Robotics and Expert Systems Division of the
tems, robotics, real-time systems theory and implementation, and automatic Instrument Society of America, a member of the AdCom of the IEEE Systems,
systems integration. Man and Cybernetics Society, a Technical Editor of the IEEE TRANSACTIONS
Mr. Stewart is a student member of ACM. ON ROBOTICS AND AUTOMATION, and PACE chairman of the IEEE Robotics
and Automation Society.
Dr. Khosla is a recipient of the Inlaks Fellowship, United Kingdom. He
received the Carnegie Institute of Technology LADD award for excellence
in research in 1989. His research has resulted in more than 80 journal
Donald E. Schmitz holds the B.S. degree in me- articles, conference papers and book contributions. He is a member of the
chanical engineering from Carnegie Mellon Univer- Technical Advisory Board of the Next Generation Controller Project (WPAFB,
sity, Pittsburgh, PA. He is now a software systems US Air Force and Martin Marietta) and the Sample Acquisition Analysis
designer at Transarc Corporation, Pittsburgh, PA. and Preservation Project (Jet Propulsion Laboratory, NASA). He has been
He is currently working on the implementation of an invited participant to the Department of Commerce workshops on the
threading software, and developing testing strategies Intelligent Manufacturing Systems program and USA-Japan R&D consortia
for large highly concurrent and distributed trans- and collaboration. He is also a consultant to several industries in the USA.
action processing systems. Prior to accepting the
position at Transarc, he was a senior engineer at the
Robotics Institute, Carnegie Mellon University. His
work included the design and system integration of
experimental robot manipulators, including major efforts in the CMU Direct
Drive Arm II and Reconjigurable Module Manipulator System projects. During
the years 1987 to 1990, this work also concentrated on the design of real-
time operating systems for application to robot control. Research interests
include the implementation of real-time task scheduling and synchronization,
and quality assurance of large software systems.