An Overview of The Spring System
An Overview of The Spring System
Nucleus B Nucleus A
FIGURE 5. Domains, doors, and door tables
For example, say that user X has a file object foo, which
has a restricted access control list specifying that only X is
allowed to read the file. However X would like to print the
file on a printserver P. P is not on the ACL for foo, so it memory objects
would not normally have access to foo’s data. However, X
can obtain an object reference that will act as a software FIGURE 8. User’s view of address spaces
capability, encapsulating the read access that X is allowed An address space is a linear range of addresses with regions
to foo. X can then pass that object reference on to the mapped to memory objects. Each region is mapped to a (part
printserver P and P will be able to read the file. of) a memory object. Each page within a mapped region may
be mapped with read/write/execute permissions and may be
locked in memory.
machine than the pager object server which provides the
contents of the memory object. Pager 1 Pager 2
pager pager pager pager
object object object object
8.2 Cache and Pager Objects
In order to allow data to be coherently cached by more
than one VMM, there needs to be a two-way connection
between the VMM and an external pager (e.g., a file cache cache cache cache
server). The VMM needs a connection to the external object object object object
pager to allow the VMM to obtain and write out data, and
the external pager needs a connection to the VMM to VMM 1 VMM 2
allow the provider to perform coherency actions (e.g., to
invalidate data cached by the VMM). We represent this FIGURE 9. Pager-cache object example
two-way connection as two objects. A VMM and an external pager have a two-way pager-cache
The VMM obtains data by invoking a pager object imple- object connection. In this example, Pager 1 is the pager for
mented by an external pager, and an external pager per- two distinct memory objects cached by VMM 1, so there
are two pager-cache object connections, one for each mem-
forms coherency actions by invoking a cache object
ory object. Pager 2 is the pager for a single memory object
implemented by a VMM. cached at both VMM 1 and VMM 2, so there is a pager-
When a VMM is asked to map a memory object into an cache object connection between Pager 2 and each of the
address space, the VMM must be able to obtain a pager VMMs.
object to allow it to manipulate the object’s data. Associ-
bility of the external pager implementing the memory
ated with this pager object must be a cache object that the
object. The coherency protocol is not specified by the
external pager can use for coherency.
architecture—external pagers are free to implement what-
A VMM wants to ensure that two equivalent memory ever coherency protocol they wish. The cache and pager
objects (e.g., two memory objects that refer to the same object interfaces provide basic building blocks for con-
file on disk), when mapped, will share the data cached by structing the coherency protocol. Our current external
the VMM. To do this, the VMM invokes a bind operation pager implementations use a single-writer/multiple-reader
on the memory object. The bind operation returns a cach- per-block coherency protocol [12, 13].
e_rights object, which is always implemented by the
VMM itself. If two equivalent memory objects are
mapped, then the same cache_rights object will be 9 File System
returned. The VMM uses the returned object to find a
pager-cache object connection to use, and to find any The file system architecture defines file objects that are
pages cached for the memory object. implemented by file servers. The file object interface
inherits from the memory object and io interfaces. There-
When a memory object receives a bind operation from a
fore, file objects may be memory mapped (because they
VMM, it must determine if there is already a pager-cache
are also memory objects), and they can also be accessed
object connection for the memory object at the given
using the read/write operations of the io interface.
VMM. If there is no connection, the external pager imple-
menting the memory object contacts the VMM, and the Spring includes file systems giving access to files on local
VMM and the external pager exchange pager, cache, and disks as well as over the network. Each file system uses
cache_rights objects. Once the connection is set up, the the Spring security and naming architectures to provide
memory object returns the appropriate cache_rights object access control and directory services.
to the VMM. A Spring file system typically consists of several layered
Typically, there are many pager-cache object channels file servers [5]. The pager-cache object paradigm is used
between a given pager and a VMM (see Figure 9 for an by file systems as a general layering mechanism between
example). the different file servers and virtual memory managers.
Among other things, this has enabled us to provide per-
machine caching of data and attributes to decrease the
8.3 Maintaining Data Coherency number of network accesses for remote files.
The task of maintaining data coherency between different
VMMs that are caching a memory object is the responsi-
9.1 File Server Implementations In contrast, Spring provides a uniform name service [17].
The Spring Storage File System (SFS) is implemented In principle, any object can be bound to any name. This
using two layers as shown in Figure 10. applies whether the object is local to a process, local to a
machine, or resident elsewhere on the network, whether it
is transient or persistent; whether it is a standard system
SFS object, a process environment object, or a user specific
Coherency layer All files are object. Name services and name spaces do not need to be
exported by
coherency segregated by object type. Different name spaces can be
Disk drive Disk layer layer composed to create new name spaces.
By using a common name service, we avoid burdening cli-
ents with the requirement to use different names or differ-
FIGURE 10. Spring SFS ent name services depending on what objects are being
accessed. Similarly, we avoid burdening all object imple-
The base disk layer implements an on-disk Unix compati- mentations with constructing name spaces—the name ser-
ble file system. It does not, however, implement a coher- vice provides critical support to integrate new kinds of
ency algorithm. Instead, an instance of the coherency file objects and new implementations of existing objects into
server is stacked on the disk layer, and all files are Spring. Object implementations maintain control over the
exported to clients via the coherency layer. representation and storage of their objects, who is allowed
The coherency layer implements a per-block multiple- access to them, and other crucial details. Although Spring
reader/single-writer coherency protocol. Among other has a common name service and naming interface, the
things, the implementation keeps track of the state of each architecture allows different name servers with different
file block (read-only vs. read-write) and of each cache implementation properties to be used as part of the name
object that holds the block at any point in time. Coherency service.
actions are triggered depending on the state and the cur- The name service allows an object to be associated with a
rent request using a single-writer/multiple-reader per- name in a context, an object that contains a set of name–
block coherency algorithm. The coherency layer also to–object associations, or name bindings, and which is
caches file attributes. used by clients to perform all naming operations. An
The Caching File System (CFS) is an attribute-caching file object may be bound to several different names in possibly
system. Its main function is to interpose itself between several different contexts at the same time. Indeed, an
remote files and local clients when they are passed to the object need not be bound to a name at all.
local machine so as to increase the efficiency of many By binding contexts in other contexts we can create a
operations. Once interposed on, all calls to remote files naming graph (informally called a name space), a directed
end up being diverted to the local CFS. graph with nodes and labeled edges, where the nodes with
An interesting aspect of CFS is the manner in which it outgoing edges are contexts.
dynamically interposes on individual remote DFS files. A Unlike naming in traditional systems, Spring contexts and
caching subcontract is used to contact the local CFS in the name spaces are first class objects: they can be accessed
process of unmarshalling file objects. When CFS is asked and manipulated directly. For example, two applications
to interpose on a file, it becomes a cache manager for the can exchange and share a private name space. Tradition-
remote file by invoking the bind operation on the file as ally, such applications would have had to build their own
described in Section 8.2. naming facility, or incorporate the private name space into
a larger system–wide name space, and access it indirectly
via the root or working context.
10 Spring Naming Since Spring objects are not persistent by default, naming
An operating system has various kinds of objects that need is used to provide persistence [16]. It is expected that
to be given names, such as users, files, printers, machines, applications generally will (re)acquire objects from the
services, etc. Most operating systems provide several name service. If the part of the name space in which the
name services, each tailored for a specific kind of object. object is found is persistent, then the object will have been
Such type specific name services are usually built into the made persistent also.
subsystem implementing those objects. For example, file A Spring name server managing a persistent part of a
systems typically implement their own naming service for name space converts objects to and from their persistent
naming files (directories).
form (much like the UNIX file system, which converts exported via Spring objects implemented by a UNIX pro-
open files to and from their persistent form). However, cess server (in a separate domain). See Figure 4.
since naming is a generic service for an open–ended col-
The UNIX process server implements functions that are
lection of object types, a context cannot be expected to
not part of the Spring base system and which cannot reside
know how to make each object type persistent. Spring
in libue.so due to security reasons.
object managers have ultimate control of the (hidden)
states of their objects. Therefore we provide a general
interface between object managers and the name service 11.1 Libue
that allows persistence to be integrated into the name ser-
When a program is execed, libue.so is dynamically linked
vice while allowing the implementation to control how its
with the application image in place of libc, thus enabling
(hidden) objects’ states are mapped to and from a persis-
the application to run unchanged.
tent representation.
The libue.so library encapsulates some of the functionality
Because the name service is the most common mechanism
that normally resides in a monolithic UNIX kernel. In par-
for acquiring objects, it is a natural place for access control
ticular, it delivers signals forwarded by the UNIX process
and authentication. Since the name service must provide
server, and keeps track of the association between UNIX
these functions to protect the name space, it is reasonable
file descriptor numbers (fd’s) and Spring file objects.
to use the same mechanism to protect named objects. The
naming architecture allows object managers to determine For each UNIX system call, we implemented a library
how much to trust a particular name server, and an object stub. In general, there are three kinds of calls:
manager is permitted to forego the convenience and 1. Calls that simply take as an argument an fd, parse any
implement its own access control and authentication if it passed flags, and invoke a Spring service (e.g., read,
wishes. Similarly, name servers can choose to trust or not write, and mmap). Most file system and virtual mem-
to trust other name servers. ory operations fall in this category.
The Spring name service does not prescribe particular 2. Calls that eventually call a UNIX-specific service in
naming policies; different policies can be built on the top. the UNIX process server. Examples include pipe and
Our current policy is to provide a combination of system- kill.
supplied shared name spaces, per-user name spaces, and
3. Calls that change the local state without calling any
per-domain name spaces that can be customized by attach- other domain. Dup, parts of fcntl, and many signal han-
ing name spaces from different parts of the distributed dling calls fall into this category.
environment.
By default, at start-up each domain is passed from its par-
ent a private domain name space, which incorporates the 11.2 UNIX Process Server
user and system name spaces. A domain can acquire other The UNIX process server maintains the parent-child rela-
name spaces and contexts if it desires. tionship among processes, keeps track of process and
group ids, provides sockets and pipes, and forwards sig-
nals.
11 UNIX Emulation The UNIX process server is involved in forking and exec-
Spring can run Solaris binaries using the UNIX emulation ing of new processes. It is also involved in forwarding (but
subsystem [6]. It is implemented entirely by user-level not delivering signals). Since it keeps track of process and
code, employs no actual UNIX code, and provides binary group ids, it enforces UNIX security semantics when ser-
compatibility for a large set of Solaris programs. The sub- vicing requests from client processes.
system uses services already provided by the underlying
Spring system and only implements UNIX-specific fea-
tures that have no counterpart in Spring (e.g., signals). No 12 Conclusions
modifications to the base Spring system were necessary to
The Spring project chose to build a different operating sys-
implement Solaris emulation.
tem, one based on the notions of strong interfaces, open-
The implementation consists of two components: a shared ness and extensibility and designed to be distributed and
library (libue.so) that is dynamically linked with each suited to multiprocessors. Using object-oriented ideas and
Solaris binary, and a set of UNIX-specific services strong interfaces has been a natural fit, with a number of
benefits:
• A standardized basis for open, distributed object sys- Proc. Winter 1993 USENIX Conference, pp. 469-479, Janu-
tems via the Interface Definition Language and a sim- ary 1993.
ple client model for objects [7] Yousef A. Khalidi and Michael N. Nelson, “The Spring Vir-
• Easy distributed services and applications tual Memory System,” Sun Microsystems Laboratories
Technical Report SMLI-93-9, March 1993.
• Readily extensible system facilities, such as file sys-
[8] Yousef A. Khalidi and Michael N. Nelson, “A Flexible Exter-
tems and name services nal Paging Interface,” Proc. 2nd Workshop on Microkernels
• Unity of architecture together with a wide range of and Other Kernel Architectures, September 1993.
implementation opportunities as in virtual memory [9] Michael N. Nelson and Graham Hamilton, “High Perfor-
management, naming, subcontract, and serverless mance Dynamic Linking Through Caching,” Proc. 1993
objects Summer USENIX Conference, pp. 253-266, June 1993.
• Highly efficient inter-address space object invocation [10] Michael N. Nelson, Graham Hamilton, and Yousef A. Kha-
in support of a microkernel-based architecture. lidi, “Caching in an Object-Oriented System,” Proc. 3rd
International Workshop on Object Orientation in Operating
Finally, designing in security mechanisms from the start Systems (I-WOOOS III), pp. 95-106, December 1993.
has provided a system that can support a wide range of [11] Michael N. Nelson and Yousef A. Khalidi, “Generic Support
secure mechanisms in a networked environment, from the for Caching and Disconnected Operation,” Proc. 4th Work-
most relaxed to the most secure. shop on Workstation Operating Systems (WWOS-IV), pp.
61-65, October 1993.
[12] Michael N. Nelson, Yousef A. Khalidi, and Peter W.
13 References Madany, “Experience Building a File System on a Highly
Modular Operating System,” Proc. 4th Symposium on Expe-
[1] Graham Hamilton and Panos Kougiouris, “The Spring riences with Distributed and Multiprocessor Systems
Nucleus: A Microkernel for Objects,” Proc. 1993 Summer (SEDMS IV), September 1993.
USENIX Conference, pp. 147-160, June 1993. [13] Michael N. Nelson, Yousef A. Khalidi, and Peter W.
[2] Graham Hamilton, Michael L. Powell, and James G. Mitch- Madany, “The Spring File System,” Sun Microsystems Lab-
ell, “Subcontract: A Flexible Base for Distributed Program- oratories Technical Report SMLI-93-10, March 1993.
ming,” Proc. 14th ACM Symposium on Operating Systems [14] Michael N. Nelson and Sanjay R. Radia, “A Uniform Name
Principles, pp. 69-79, December 1993. Service for Spring’s Unix Environment,” Proc. Winter 1994
[3] Graham Hamilton and Sanjay Radia, “Using Interface Inher- USENIX Conference, Jan. 1994.
itance to Address Problems in System Software Evolution,” [15] Object Management Group, “Common Object Request Bro-
Proc. ACM Workshop on Interface Definition Languages, ker Architecture and Specification,” OMG Document
January 1994. 91.12.1, December 1991.
[4] Peter B. Kessler, “A Client-Side Stub Interpreter,” Proc. [16] Sanjay Radia, Peter Madany, and Michael L. Powell, “Per-
ACM Workshop on Interface Definition Languages, January sistence in the Spring System,” Proc. 3rd International Work-
1994. shop on Object Orientation in Operating Systems (I-
[5] Yousef A. Khalidi and Michael N. Nelson, “Extensible File WOOOS III), pp. 12-23, December 1993.
Systems in Spring,” Proc. 14th ACM Symposium on Operat- [17] Sanjay R. Radia, Michael N. Nelson, and Michael L. Pow-
ing Systems Principles, pp. 1-14, December 1993. ell, “The Spring Name Service,” Sun Microsystems Labora-
[6] Yousef A. Khalidi and Michael N. Nelson, “An Implementa- tories Technical Report SMLI-93-16, October 1993.
tion of UNIX on an Object-oriented Operating System,”