Development of an Object-Oriented DBMS

David Maier
Jacob Stein
Servio Logic Development Corp.
and Oregon Graduate Center

Allen Otis
Alan Purdy
Servio Logic Development Corp.
15025 S.W. Koll Parkway, l a
Beaverton, Oregon 97006
(503) 644-4242

Abstract
We dascdbe the results of developing the GemStone object- traditional data management system will result in a system that
oriented database sawer, which supports a model of objects offers further reductions in application development efforts. The
similar to that of Smalltalk-80. We begin with a summary of the extensible data-typing facility of the system will facilitate storing
goals and requirements for the system: an extensible data model information not suited to normalized relations. In addition, we
that captures behavioral semantics, no artificial bounds on the believe that an object-oriented language can be complete enough
number or size of database objects, database amenities (con- to handle database design, database access, and applications.
currency, transactions, recovery, associative access, authodza- Object-like models have long been popular in CAD [CFHL, EM,
lion) and an interactive development environment. Object- Ka82, Ka83, LP, MNP, SMF], and seem well suited to support
odented languages, Smaiitalk in particular, answer some of these programming environments [PL], knowledge bases [DK], and
requirements. We discuss satisfying the remaining requirements office information systems [Ah-,-, Zd84]. Other groups are in the
in an object odentad context, and report briefly on the status of process of implementing object model database systems [DKL,
the development efforts. This paper is directed at an audience Ni, ZW].
familiar with object-oriented languages and their implementation,
2. Goals and Requirements
but perhaps unacquainted with the difficulties and techniques of
database system development. It updates the odginal report on 2.1. A n Extensible Data Model
the project ~M], and expands upon a more recent article [MDP].
The system must have a data model that supports the defini-
lion of new data types, rather than constraining programmers to
1. Introduction use a fixed set of predefined types. New types should also be
The GemStone database system is the result of a develop- indistinguishable from system-supplied types for the purposes of
ment project started three years ago at Servio. Our goal was to application programming: operations that apply to new types
merge object-oriented language concepts with those of database should be syntactically similar to the built-in operations on prede-
systems. GemStone provides an object-oriented database fined types. The distinction between data types and data struc-
language called OPAL, which is used for data definition, data tures is important in achieving this goal of extensibility.
manipulation and general computation. Data structures are made up of atomic values (integers,
Conventional record-oriented database systems, such as strings, etc.), plus constructors (record, relation, set, tree). A
commercial relational systems, often reduce application develop- data type is really a collection of operators, the protocol, for
ment time and improve data sharing among applications. How- operating on a particular structure. The underlying structure
ever, these DBMSs are subject to the limitations of a finite set of need not be the same as the appearance provided by the proto-
data types and the need to normalize data lea, Si]. In contrast, col. In conventional database systems, the types correspond to
object-oriented languages offer flexible abstract data-typing fecill- the structures. There is usually a fixed set of operations on
lies, and the ability to encapsulate data and operations via the atomic values, such as arithmetic and comparisen operations.
message metaphor. Smalltalk-80 is an example of a completely Each constructor has a fixed set of operations; for example, a
implemented object-oriented system [GR, Kr]. record constructor has "set field" and "get field", and a relation
Our premise is that a combination of object-oriented constructor has "add record", "delete record" and "select record".
language capabilities with the storage management functions of a It Is not possible to add new operations that appear syntactically
similar to the built-in operations. Thus the set of data types that
are directly supported is the same as the set of data structures
since nested application of the constructors is not supported.
Permisdon to copy withom fee all or ira1 of this material is iFanted provided Our goal is to model the behavior, not just the structure, of
that the copies are not made or distributed for direct commercial ndvant~e, entities in the real wodd [Mo]. Further, we must be able to pack-
the ACM copy~t noticeand the title of the publicationand its dateappear, age behavior with structure to create new data types. To get
and notice is given thai copying is by permi,~on of the Association for reasonable performance from such a system, the collection of
Computing Machinery. To copy otherwise, or to republish, requires a fee and/ constructors must be rich enough that most data types have fairly
or qx~f~: permimon. direct implementations. In particular, we should be able to cap-
O 1996 ACM 0-89791-204-7/86/09000472 75¢ ture many-to-many relationships, collections, and sequences

472 OOPSLA ' ~ Procesdin~s S e m m ~ 1986

directly. For an easily usable system, we should be able to nest object, not as multiple tuples spread amongst several relations
the structuring operations to arbitrary levels, and use previously [HL, JSW, LoP, PKLM]. Properties of entities need not be simple
defined data types as building blocks for other types. GemStone data values, but can be other entities of arbitrary complexity.
must have system management functions for monitoring system The address component of an employee object need not be just
performance, performing backups, recovering from failures, a text string. In GemStone it can be a structured object, itself
adding and removing users, and altering user privileges. having components for street number, street and city, and its own
defined behavior. (See Figure 1.) GemStone directly supports
2.2. Database Amenities set-valued entities, without the encoding required in the relational
GemStone is first a database system, so it must provide model. Furthermore, sets can have arbitrary objects as ele-
shared access to persistent data in a multi-user environment It ments, and need not be homogenous. We provide the physical
should suooort stable storage of data objects on disk, while data independence of relational databases without the limitations
providing location transparency to the application programmer on on modeling power,
the movement of objects between main memory and secondary
storage. GemStone must provide for ownership of data objects,
and requests by the owner to authorize sharing with other users.
Each database session should appear to have complete control
of a consistent version of the database, even while users are run.
ning concurrently, and should be provided with a transaction
Employee I
mechanism to commit or abort a set of changes atomically.
Users should be able to request replication of critical data to Pers~Name ]
guard against localized media failure.
GemStone should support auxiliary storage structures that first
provide alternative access paths to data, and should give users
some control over physical grouping of objects, to improve effi-
ciency of specific access patterns. Bounds on the number and
size of data objects should be determined only by the amount of last
secondary storage, not main-memory limitations or artificial res- I I
trictions on data definition. Thus, fields in a record should be
variable-length, with no fixed upper bound. Collections of
objects, such as arrays and sets, should not have a bound on the
number of elements. Similarly, the total number of objects in a
I ~a.,nte~, I
database system should not be arbitrarily limited. Finally, the I 11223333 J
system should handle both small and large objects with reason-
able efficiency [SSB]. 5treetAddress I
2.3. Programming Environment
We feel that GemStone should provide at least the following
Address
st.Number ISma,I,°t
1055
Ii
tools and features for application development:

1. An interactive interface for defining new database objects, street

writing OPAL routines, and executing ad hoc queries in
OPAL.
2. A windowing package upon which end-user interfaces can be city
built.
3. A procedural interface to conventional languages, such as C I I
and Pascal.

3. Advantages of an Object-Oriented Model Salary lSmalllnteger I

During the research stages of the GemStone project, we 45558 I
developed a mostly declarative query language that was deficient
in procedural capabilities. Given the problems with providing pro-
cedural extensions and educating the marketplace to a com-
pletely new language, we decided to use an existing object- 3.1.1. Object Identity
oriented language, Smalitalk-80 [GR], as the basis for product GemStone supports object identity [Ms]. A data object
development. We have made extensions to Smalltalk in the retains its identity through arbitrary changes in its own state.
areas of associative access support for queries, basic storage Entities with information in common can be modeled as two
structures, typing and support for a multi-usar environment. In objects with a shared subobject containing the common informa-
the following subsections, we cover the advantages of an object- tion. Such shadng reduces "update anomalies" that exist in the
odented approach as regards modeling and application develop- relational data model. In the relational model, the procerties of
ment. an entity must be sufficient to distinguish it from all other entities.
For one entity to refer to another, there must be some fields that
3.1. Modeling Power uniquely and immutably identify the other entity. (Some exten-
GemStone supports modeling of complex objects and rela- sions to the relational model incorporate forms of identity [Co,
tionships directly and organizes classes of data items into an Za].) Uniqueness and immutability are ideals seldom present in
inheritance hierarchy. A single entity is modeled as a single the real world. We may choose to refer to departments by name,

September1986 OOPSLA '86 Proceedings 473

but what happens if a department's name changes? (We note ment schedule based on scheme definition, application writing,
that all of our objects are assumed independent. Knowing that database pol~ulation and debugging is reasonable. Not so with a
one object is owned by, or depends on, another could be useful CAD task being modeled for the first time, or a database to sup-
for storage management [Gr, Ni, We].) port an expert system. The application area has not been
modeled before, and there will be many iterations of the data-
3.1.2. Modeling Behavior base schema before the application is mature [ACO, AO, MP].
GemStone supports simulation of the behavior of real-world Being able to start writing database routines without completely
entities. Data manipulation commands in conventional systems specifying the structure and behavior of every class of entities
are oriented towards machine representations: "modify field," can be of great advantage. Later, when the model has stabil-
"insert tuple," "get next within parent." For an office management ized, typing can be associated with fields for integrity or effi-
system, several applications might reserve a room. In a conven- ciency.
tional database system, each application would contain DML By not associating types with variables, unanticipated cases
(data manipulation language) statements to test for room availa- (a-company car might be assigned to a department as well as an
blility, insert or change a record to indicate the reservation, and employee) can be more easily handled. A routine (method)
perhaps create another record with a reminder to the reserver. makes assumptions about the protocol of its arguments, not their
Changes to the structure of the database may require locating internal structure. Such routines are robust in the face of new
and modifying every application that makes use of the database. classes. If every object responds to the prt.,~:,S~z':Lz;9 mes-
In GemStone, a z'esez'.,'eRoo., message can be defined that sage to retum a string representing itself, then we can write an
takes a date and a time as parameters, and performs all the OPAL method for Set. that prints a string representation of all its
necessary checks and updates to the database to reserve a elements, regardless of their classes.
room.
The OPAL method that implements a message can execute 3.2. A Unified Language
any number of database queries and updates, with many advan- OPAL is much more powerful than standard data manipula-
tages. Applications are more concise" sending one message tion languages. It is computationally complete, with assignment
takes the place of many database operations. The code is more and flow of control constructs. Almost all the computation
reliable, as every application that reserves a room uses exactly required in an application can be written within OPAL. This abil-
the same procedure---the method associated with the r e s e r v e - ity helps avoid the problem of impedance mismatch, where infor-
Room message. The scope of changes required by alterations to mation must pass between two languages that are semantically
the structure of a type is limited to the methods for the type. and structurally different, such as a declarative data sublanguage
Further, messages can protect the integrity of the database, by and an imperative general-purpose language. GemStone
consistency checks in their methods. If all applications that enter stresses uniformity of access to all system objects and functions,
an item in the room reservation list are reauired to use the using the same mechanisms as for regular data objects.
z'esex'veRoom message, double-booking of rooms can be fores-
balled. 4. Turning Smalltalk into a DBMS
Smalltalk is a single-user, memory-based, single-processor
3.1.3. Classes system. It does not meet the requirements of a database sys-
The class structure of GemStone speeds application tem. While Smalltalk provides a powerful user interface and
development in several ways. GemStone comes with a large many tools for application development, it is oriented to a single
complement of classes implementing frequently used data types. user workstation. To meet the requirements of a database sys-
Class definitions are the analogue to schemes in other'database tem the following enhancements have been added.
systems, but classes also package operations with the structure,
to encapsulate behavior. Thus the message definition facilities
4.1. Support of a Multi-User, Disk-Based Environment
along with class mechanism meet the requirement of an extensi-
ble data model. GemStone includes a hierarchy of classes. Being disk-based does not mean simply paging main
Whereas a class helps organize data, the class hierarchy helps memory to disk as it overflows. The database must be intelligent
organize the classes. The subclassing mechanism allows a about staging objects between disk and memory. It should try to
database scheme to capture similarities among various classes group objects accessed together onto the same disk pages, try to
of entities that are not totally identical in structure or behavior. anticipate which objects in main memory are likely to be used
Subclassing also provides a means to handle special cases again soon, and organize its query processing to minimize disk
without cluttering up the definition of the normal case (MBW]. traffic.
Since GemStone data is shared by multiple users, the sys-
3.1.4. Associating Types with Objects tem must provide concurrent access. Each user should see a
Unlike most programming languages that support abstract cohsistent version of the database, even with other users running
data types, Smalltalk associates types with values, not the slots simultaneously. Since a user may make changes that are not
holding the values. Typing objects rather than names has liabili- committed permanently to the database, GemStone must support
ties for query processing, which we consider in the next section. some notion of multiple workspaces, in which proposed changes
We consider some advantages here. to the database can later be discarded or committed. A related
We hope to enable database designers to model apphcation requirement is management of multiple name spaces. Smalltalk
domains they previously may have shied away from because of assumes a single user per image, and so provides a single glo-
complexity or lack of regular structure. However, modeling an bal name space. Several partially related or unrelated applica-
enterprise for the first time is a much different undertaking than tions can be under development on a single database at one
building a database application for an area that has already been time. It is unreasonable to expect either that users share a sin-
modeled, but has not yet been computerized. The basic model- gle global name space, or, at the other extreme, that user name
ing for financial record keeping was done thousands of years spaces are disjoint.
ago. The structure of the information involved is such that it Current Smalltalk implementations use a single processor for
readily fits into standard record-based data models. A develop- both display processing and object management. We expect

474 OOPSLA '86 Proceedings September1986

 and ~" total words of object memory [GR]. More recent imple-
GemStone to support multiple, interactive applications. Hence, it
does not seem wise to use the same processor for secondary mentations raise these limits, but still use the same techniques to
storage management as for display processing at the end-user represent and manage objects [KK]. Large disk-based objects
interface. We felt that the storage-management and user- require new storage techniques. Some objects will be too large
interface functions in GemStone must be decoupled to run as to fit in main memory, and must be paged in.
separate processes on separate processors. While virtual-memory implementations page large objects, we
felt we must get away from linear representations of long objects.
4.2. Data Integrity Requiring objects that span disk pages to be laid out contigu-
Various kinds of failures (program, processor, media) and ously in secondary storage (or even virtual memory) will lead to
violations (consistency, access, typing) can compromise the vali- unacceptable fragmentation or expensive compaction passes. In
dity and integrity of a database. A database system must be Smalltalk, to "grow" an object, such as an array, a new, larger
able to cope with failure by restoring the database to a consistent object is created and the contents of the smaller object are
state while minimizing the amount of computation lost. It must copied into it. We want the time required to update or extend an
also prevent violations from occurring. object to be proportional to the size of the update or extension,
By program failure we mean that an application program may not to the size of the object being updated. We also felt that
fail to complete, say, because of a run-time error. If the program SmaUtalk's repertoire of basic storage representations was inade-
fails after some updates to the database have been made, the quate for supporting large unordered collections. Having to map
database can be left in an unexpected state. Database systems such a collection into an ordered underlying representation
provide for multiple updates to be performed atomically (in an imposes artificial restrictions. Thus, GemStone needs a basic
all-or-nothing manner) through the use of transactions. A tran- storage type for unordered collections.
saction is used to mark a section of processing so that all its Finally, searching a long collection by a sequential scan will
changes are made permanent (the transaction commits), or none give unacceptable performance with a disk-based object.
are made permanent (the transaction aborts). Searching for elements should be at most logarithmic in the size
By processor failure, we mean that the processor handling of the collection, rather than linear. Thus, GemStone should sup-
GemStone storage management fails. For such failures, the port associative access on elements of large collections: It should
database must be kept intact. Recovering from program and pro- supply storage representations and auxiliary structures to support
cessor failure imply that master copies of objects on secondary locating an element by its internal state. This requirement rein-
storage must be updated carefully. Additionally, a good database forces the need for typing on collections and instance variables.
system should be robust enough to tolerate additional failures To index a collection E of employees on the value of the
during the recovery period. =ala.z'y instance variable, the system need assurances that
every element in E has a s a ] . a r y entry. Furthermore, if that
By media failure we mean that damage or flaws in the index is to support range queries on szLla.z-y, the systems'
secondary storage devices may cause committed data to be lost.
needs a declaration that all s a l a . r y values will be comparable
No strategy can provide complete protection against media
according to some total order.
failure. We wanted GemStone to provide for both periodic
backup and dynamic replication of sensitive information, By Along with storage-level support for associative access,
dynamic replication, we mean keeping multiple, on-line copies of OPAL must have language constructs that allow associative
a database, all of which are updated on every transaction. access.
Turning to violations, database consistency can be violated if 4.4. Physical Storage Management
transactions from multiple users interleave their updates. Gem- GemStone must provide features for managing the physical
Stone must support serializability of transactions: the net effect of placement of objects on disk. Smalltalk is a memory-resident
concurrent transactions on the database must be equivalent to system, and so there is not much need to say where an object
some serial execution of those transactions. The integrity of a goes. The database administrator, or a savvy application pro-
database can also be violated if a user accesses data that he or grammer, should be able to hint to GemStone that certain objects
she should not be permitted to see. In Smalltalk, all objects are are often used together, and so should be clustered on the disk.
available to the user. Gemstone must assign unique ownership The administrator should be able to take objects off line, say for
to every object, and give the owner of an object to power to grant archiving, and bring them back on line later. Finally, as objects
access to others. are never explicitly deleted, the system will be responsible for
Integrity constraints, such as keys and referential integrity, reclaiming the space used by unreferenced objects. (An alterna-
are assertions that a priori exclude certain states of the database. tive is to assume that a permanent object is never deleted, and
It is always a judgment call whether the database system should that objects not referenced in the current state of the database
check constraints after each transaction, or whether the applica- should be shifted to archival storage.)
tion programmer should be responsible for preserving con-
sistency in each transaction. The former course is more reliable, 4.5, Access From Other Systems
but almost always more expensive. At a minimum, the database While OPAL goes much further than conventional database
should support constraints that require subparts of an entity or languages in providing a single language for database application
collections to belong to a certain class. We note that referential programming, we wanted to concentrate our initial efforts on
integrity comes "for free" in GemStone. One object refers storage management issues, rather than user interfaces. Thus,
directly to another object, not to a name for that object. The GemStone provides for access to its facilities from other pro-
reference cannot be created if the other object does not exist. gramming languages. We want to support an application
Hence, there are no dangling references. development environment for OPAL along the lines of the
Smalltalk programming environment [GR], but we recognize that
4.3. Large Object Space the application development environment may not be the same
Gemstone must store both large numbers of objects and as the environment in which the finished application runs. How-
objects that are large in size. The first Smalltalk-80 implementa- ever, we are committed to providing procedural interfaces to C
tions had a limit of z,' objects, z,, instance variables in any object, and Pascal.

Sep(ember 1986 OOPSLA '86 Proceedings 475

5. Our Approach Gem sits atop Stone, and elaborates Stone's storage model
This section addresses how we provided the enhancements into the full GemStone model. Gem also adds the capabilities of
needed to Smalltalk to make it a database system. We start with compiling OPAL methods into bytecodes and executing that
an overview of the architecture of GemStone. code, user authentication, and session control. (OPAL bytecodes
are similar, but not identical, to the bytecodes used in Smalltaik.)
5.1. GemStone Architecture Part of the Gem layer is the virtual image: the collection of OPAL
Figure 2 shows the major pieces of the GemStone system. classes, methods and objects that are supplied with every Gem.
Stone and Gem correspond roughly to the object memory and Stone system.
the virtual machine of the standard Smalltalk implementation Figure 3 shows the class hierarchy in the current GemStone
[GR]. Stone provides secondary storage management, con- virtual image. Comparing it to the Smalltalk hierarchy, we have
currency control, authorization, transactions, recovery, and sup- removed classes for file access, communication, screen manipu-
port for associative access. Stone also manages workspaces for lation and the programming environment. The file classes are
active sessions. Stone uses unique surrogates, called object- unnecessary, as we have persistent storage for all GemStone
oriented pointers (OOPs) to refer to objects. Stone uses an objects. Computation for screen manipulation needs to happen
object table to map an OOP to a physical location. This level of near the end user, and needs fast byte,code execution. Gem-
indirection means that objects can easily be moved in memory. Stone is optimized toward maintaining large numbers of per-
While the object table can potentially have 2*' entries, we expect sistent objects, rather than fast bytecode execution. The pro-
that the portion for objects currently in use by various sessions is gramming environment classes are replaced by a browser appli-
small enough to fit in main memory. Stone is built upon the cation that runs on top of GemStone, which we describe in a
underlying VMS file system. The data model that Stone provides later subsection. We have added classes and methods to make
is simpler than the full GemStone model, and provides only the data management functions of transaction control, account-
operators for structural update and access. An object may be ing, ownership, authorization, replication, user profiles and index
stored separately from its subobjects, but the oops for the values creation controllable from within OPAL.
of an object's instance variables are grouped together. Others
have considered decomposed representations of objects [Ch-,-,
CK]. FmtmE 3

FIGURE 2 Object
tBM-K Association
SymbolAssociation
WINDOWS Behavior
Class
OPB Metaclass
Oth~ Boolean
AppflaUon Collect=on
SequenceableCollection
AGDIT AGlaTF Array
InvariantArray
I I Repository
String
InvariantString
Symbol
I LAN Bag
VAX Set
Dictionary
I NlffWOIU[ SOFT'A/~E J SymbolD=ctionary
I . I LanguageDtctionary
SymbolSet

i J °"'
UserProldeSet
G1M
PROCSSS • • • PROCESSi CompiledMethod
Magnitude
Character
DateTtme
Number
STONS Float
Pll0CZSS Integer
Smalllnteger
MethodContext
I Block
SelectionBlock
¥lLq FILB ilO Segment
Stream
PositionableStream
ReadStream
DATA WriteStream
System
UndefinedObject
UserProfile

476 OOPSLA '86 Proceedings September1986

 The Agent interface is a set of routines to facilitate communi- tion is allowed to commit. For a commit, the shadow object table
cation from other programs in other languages running on pro- of the session is treated as if it were transparent on the entries
cessors (possibly) remote from Gem. The Agent Interface that have not been modified, and is overlaid on the most recent
currently supports calls from C and Pascal programs running on version of the shared table. Thus, only entries in the shared
an IBM-PC for session and transaction control, sending mes- table for objects that have been copied by the session are
sages to GemStone objects, executing a sequence of OPAL changed. In this way, the changes made by the committing ses-
statements, compiling OPAL methods, and error explanation. In sion are merged with those of other transactions that committed
addition, the Agent provides "structural access" calls, which per- after the committing session began. If the current transaction
form the following functions: conflicts with a previously committed transaction (or is aborted by
the application), the changes in its shadow table are discarded,
1. determining an object's size, class, and implementation; after using the table to reclaim pages used for new copies of
objects.
2. inspecting a class-defining object;
This optimistic scheme ensures that read-only transactions
3. fetching bytes or pointers from an object; never conflict with other transactions. Such a transaction gets a
4. storing bytes or pointers in an object; consistent copy of the database state, does its reading, and has
5. creating objects. no changes to make to the shared table on commit. Only tran-
sactions that write can conflict with each other. This scheme
Information passes between the Agent and Gem in the form never deadlocks, as a session experiences no contention with
of bytes and GemStone object pointers. Certain objects have other sessions before a commit point. However, it is possible
predefined object pointers, such as instances of Boolea.n, that an application that writes a large portion of the database
Cb.a.r=.c~er and S m a . l l l " n t , e g e r . Instances of Flo&¢, and may fail to commit any transactions for an arbitrarily long time.
S t ; r i - 9 are passed as byte sequences. Instances of other While shadowing has had some bad press, it seems a natural
classes must be decomposed into Instances of the classes man- approach to us, given that we have an object table. It avoids
tioned, in order to pass their internal structure between the Agent some extra reads, makes commit and abort simple, and is a
and Gem. However, the identity of any object can be passed excellent candidate for writs-once memory, since active pages
between the Agent and Gem, regardless of its complexity. The are never changed in place.
idea is to do the computation and manipulation of objects in
Gem, and only pass data used for display through the Agent to
SH/~[O SHADOW
the interface routines.
Gem, Stone and the Agent interface are structured as
separate processes. Our current mapping of processes to pro-
cessors has Gem and Stone running on on a VAX under VMS.
The Agent interface supports communication with Gem from an
IBM PC. While a GemStone system has a single Stone process,
it maintains a separate Gem process for each active user, and
the Agent interface handles communication on a per.application
basis.
5.2. Multiple Users
Stone supports multiple concurrent users by providing each FIGUI~ 4
user session with a workspace that contains a shadow copy of
the object table derived from the most recent committed object 5.3. Efficiency Considerations
table, called the shared table. Whenever a session modifies an One problem with recording all the objects a session reads or
object, a new copy of that object is created, and placed on a writes is that the list can grow quite long, and GemStone will
page that is inaccessible to other sessions. "l'he shadow copy of spend a lot of time adding entries to such lists. One optimization
the object table is updated to have the object's OOP map into is that certain classes of objects, such as Sma.llZnl:,eger,
the new page. Cha.z'a.ct.er, and ~ l e a . n are known to contain only instances
Conceptually, the shadow object table for a workspace is a that cannot be udpated. Thus, such objects need not be
complete copy of the version of the shared table when the ses- recorded for concurrency control. Even excluding these objects,
sion starts. Actually, we do not make a copy all at once. Object single objeCts are just too fine a granularity for concurrency con-
tables are represented as S-trees, indexed on OOPs. For a sha- trol. Thus, we introduced the notion of segments, which are logi-
dow object table, we need only copy the top node of the commit- cal groupings of objects that are the unit of concurrenr.Y control in
ted object table. As the objects are changed by a session, the GemStone, much like the segments of the ADAPLEX LDM
shadow object table adds new nodes that are copies of its [CFLR]. A segment may contain any number of objects. Gem-
shared object table with the proper changes. Figure 4 shows the Stone keeps a list of just the segments read or wdttan by a ses-
state of a shadow object table after the alteration of a single sion, rather than all objects. Also, at a physical level, pages
object. Multiple paths have been copied since several objects respect segment boundaries. Thus the practice of copying all
may have been on the same page as the altered object. objects in ~ page when one is changed causes no additional con-
We chose an optimistic concurrency control scheme: one in flicts. Segments are visible from within OPAL through the class
which access conflicts are checked at commit time, rather than Segment,. Users can control placement of objects in segments,
prevented from occuring through locking. For each transaction, tO group objects to try to avoid conflict. If an application has a
Stone keeps track of which objects the transaction has read or group of private objects, all those objects and no others can be
written. At commit time, Stone checks for read-write and writs- placed together in one segment. At the system level, system
write conflicts with transactions that have committed since the objects that are shared by many users, but are almost never
time the transaction began. If there are no conflicts, the transac- updated (such as the class describing object for a system class)

September1986 OOPSLA '86 Proceedings 477

can be placed on a single segment, so that accesses to them may be taken off line, which means all its objects become inac-
never causes a conflict. cessible. • Repoe~.t.ory is an OPAL class providing internal
representatives of repositories. A Repo,,~.t,o r y instance can
5.4. Name Spaces respond to a message repZ:Lc=t,e, which means two copies of
Multiple name spaces are managed by Gem. The virtual the repository will be maintained (at increased cost in time and
image has a c l ~ UeerProf:l.le that is used to represent pro- space), usually on separate extamal devices. The copies know
parties of each' user that are of interest to the system, such as about each other, and if the medium for one fails, the other is still
user id, password, native language and local time zone. A available.
U s e r P z ' o t i l e object also COntains a list of dictionaries that are
used to resolve symbols when compiling OPAL code for that 5.6. Authorization
user. When an identifier is encountered in OPAL code, and that Segments are also the unit of ownership and authorization.
identifier is neither an instance variable nor a class variable, the Every user has at least one segment, and when he or she
dictionaries are searched in order to find an object corresponding creates new objects, they go in an owned segment. A user may
to that identifier. There may be any number of dictionaries for a grant read or write permission (write implies read) on a segment
user, to accommoalate various degrees of shadng. For example, to other users or groups of users. Such grants must always
a programmer's first dictionary may contain objects and classes come from the original owner. Read or write permission on a
for his or her portion of a project, the second may be for objects segment implies the same permission on all objects assigned to
shared with other programmers working on the same project, and the segment..User Identification is handled by Gem, using
the third COuld COntainsystem objects. Note that symbol resolu- userZd and p=e,:word from a U s e r P r o f i l e .
tion can be performed at runtime, thus providing for dynamic There are some subtleties of read and write permission in an
symbol resolution during method execution. object model. First, having the identity of an object (its OOP) is
not the same as reading the object. Second. having permission
5.5. Transactions and Recovery
on an object does not imply have permission on all its subob-
Most of our approach to transactions has been COvered in a iects. So, for example, an Employee object, along with the
previous section. To reiterate, every session gets a shadow copy objects that are values for instance variables empName, seNo
of the shared object table when it begins, and installs its shadow and a.ddz-ese COuld reside in one segment. By putting a
copy as the shared copy when it successfully commits a transac- S,,Ze.z'yHlst,o r y object in another segment, authorization can
tion. Further, a session always writes changed and new objects be granted to just a portion of an employee's personnel informa-
into pages that are not accessible to any other transaction before tion, Third, name spaces are the first line of defense against
commit time. Thus, aborting a transaction means throwinc away unauthorized access. If a user cannot find an object, he or she
its shadow object table, and committing means replacing the cannot read the object.
shared table with a shadow copy. The only issue that needs
more elaboration is atomicity--that the changes of a transaction 5.7. Large Object Space
are made, seemingly, all at once. As object tables are trees, In designing GemStone, we have tried to always set limits on
atomicity is not hard to pro,,ide. When a shadow table is to object numbers and sizes so that physical storage limits will be
replace the shared table, and the shadow table differs from the encountered first. A GemStone system can support t', objects
table it is about to replace, the new table can replace the old by (e', COunting instances of b-"ma,l l I n t , eger) and an object can
simply overwriting the root of the shared object table with the root have up to ==, instance variables. Segments have no upper
of the new object table. (Actually, there is a "root of the data- bound on the number of objects they can contain, other than the
base" above the root of the object table that gets overwritten. number of objects in the system.
The database root references some other information besides the
When an object is larger than a page, the object is broken
object table, such as a list of active transactions.) Rewriting the
into pieces and organized as a tree spanning several pages. To
root is the only place where any part of the shared copy of the
handle large unordered objects (instances of ~ and its subc-
database is overwritten.
lasses), we have added a new basic storage structure called a
Recovery from processor failure does not require a great non-sequenceable collection (NSC). This structure supports
amount of additional mechanism over what we have for con- adding, removing and testing for membership, along with iteration
currency control. Our unit of recovery is a transaction. Changes over all the elements. However, NSCs have anonymous
made by committed transactions are kept, changes not yet com- instance variables, which means their component objects may
mitted are Inst. Since the shared version of the database is not be referenced by name or index. Large NSCs are also
never overwritten, we need almost nothing in the way of logs to stored as trees, but ordered by OOP. In the next section, we
bring the database to a consistent state, since it never leaves show how content-based retrieval is supported for an NSC
one. The only tricky part is a processor crash while writing a object.
new database root. To handle that eventuality, we keep two A large object can be accessed and updated without bringing
copies of the root, which reside at a known place. To restore a all the pages of an object into a workspace. The tree structure
consistent state of the database after the crash, we simply check for large objects makes it possible to update pieces of them
those two pages. If they are different, we copy one that can be without rewriting the whole object, much as for the object table.
determined to be uncorrupted over the other. The real work on Since pages of a large object need not be contiguous in secon-
recovery is garbage collection: removing detritus of the transac- dary storage, such objects can grow and shrink with no need to
tions that had not committed before the crash. recopy the entire object.
To guard against media failure, we have introduced a struc-
ture called a repository. A repository is the unit of replication, 5.8. Associative Access and Typing
and also the unit of storage that can be taken off line. Most of We briefly cover some of the language and typing issues
what we said before about the database actually pertains to repo- relating to associative access, along with index structures and
sitories. Segments partition repositories, and all the objects in a their maintenance. The fundamental language issue is being
segment are stored in the segmenrs repository. A repoaiton/ able to detect opportunities for using auxiliary storage structures.

478 OOPSLA '86 Proceedings September1986

In a computationally complete language such as OPAL, it is nei- lections (like having several relations on the same scheme [Ha]).
ther necessary or desireable to consider using auxiliary structures Indexing on the class means that applications that do not use the
for every database manipulation. Conceivably, we could analyze index still bear the overhead for instances they use being in the
all OPAL methods to detect places where alternative access index. Further, a classwide index presents authorization prob-
paths might be used. We felt that approach was too complex, lems. No one user may have read access to set all the objects
and instead decided that the programmer must flag opportunities in the class, so no one is able to request the index be created.
to use auxiliary structures. Also, indexing a collection allows the possibility that instances of
OPAL supports the use of indices to speed the evaluation of subclasses be included in a collection that is indexed. Indexing
expressions of the form on a class basis makes it easier to trace changes to the state of
an object that could cause the object to be positioned differently
within an index. We decided that minimizing cost for programs
select: =Block
not using an index was the top priority, so we index on collec-
tions, but other systems have chosen class indexing [ZW]. Addi-
The block has one variable and retums a Boolean. The result tionally, if indexing by class, intersecting the result of a lookup
of the expression is the subset of elements of a3ag for wllich with a collection may be time consuming with respect to the size
a B l o c k returns t r u e , and resembles the relational selection of the collection. Note that a class can be implemented to keep
operator in the Cypress data model [Ca]. This statement is a collection of all instances if desired, and that collection can be
evaluable in OPAL without indices, but at the cost of examining indexed.
every element in aSa9. Since a block can contain arbitrary
Indices are created and abandoned by sending messages to
OPAL expressions, indices are not useful in evaluating every
a Bag or S e t object, giving the path to index. For example, if
expression in a blocck. Hence, for use with indices, we added
empSet is a set of Employee objects, we can request an index
path syntax to the OPAL language. For any variable, we can on eatpNa.,.e, l = e t . There are two kinds of indices: identity and
append to it a path composed of a sequence of pieces called
equality. An identity index supports searching a collection on the
links, which specify some subpart of an object. For example,
identity of some subobject of one of the elements, without refer-
anEmp.empName.last might access the last name of an
ence to the subobject's state. An equality index supports lookup
Emplo:y'ee object. A question arises why sequences of unary
on the basis of the value or internal state of objects, and range
messages do not suffice to the same thing, such as a n E m p searches on values.
name last The reason is that we want to support associative
access at runtime without performing message sends, and so the The path syntax for associative selections and the kind of
support can come from the Stone level. index desired dictate what typing information is required to sup-
pod indexing. Referring back to the discussion in Section 4.3, to
A selection block for associative access can contain an con-
have an identity index on a collection using a particular path, we
junction of path comparisons, where a path comparison an
must know that the path expression is defined (leads some-
expression of the form <path expression> <comparator>
where) for every object in the collection. For an equality index,
<literal> or <:path expression> <comparator> <path expres- we must additionally know that the values of the paths for every
sion>: element of the collection are comparable with respect to equality
and the other comparisons supported by equality indices. OPAL
a41Emp.empName.last = ' S a n d e r s ' provides typing for names and anonymous instance variables.
smEmp, salary • smEmp, dept.manager, salary For any named instance variable, the value of that variable can
be constrained to be a kind of a given class. A value is a kind of
Index use is requested by using set braces in place of brackets a class if it is an instance of that class or of some subclass
around the block in a s e l e c t : message thereof. For example, we can declare that the empN=u~e
instance variable of class Employee must have kind
PersonName, which means the value can be a Pez'sord~lazae
empSet s e l e c t : instance, or an instance of some subclass, say T l t l e d P e r s o n -
{ : anEmp I a.nEmp, empName, lastName = ' S a n d e r s ' ). Name. Subclasses of Bag and S e t can be restricted in the kind
of elements their instances may contain.
rather than Both named and anonymous instance variable typing are
inherited through the class hierarchy. Additionally, typing can be
further restricted in a class's subclasses. If in Employee class
empSet select : instance variable empName iS constrained to PersonName.
[ : anF_.mp I a.r.F.mp,empName, lastName = ' S a n d e r s ' ] then in a subclass of Employee empName can be constrained
to TttlecI.Per sonNLme.
The two expressions give the same result, but the first one In order to create an identity index into empSet on
requests OPAL to use an index if available, while the second will smpnams, l a s t . variable empNa...e must be constrained to a
always be evaluated by iterating through empSet. If no class in which a variable l a s t is defined. However, it would not
appropriate index exists, then the first expression might still be be necessary for l a s t to be be constrained within that class, for
evaluated without the use of message sends if, as discussed the comparisons supported by identity indices need not know the
below, the path is appropriately typed. Otherwise, the first structure of employee's last names. In an equality index,
expression evaluates using the same method as the second. We employee's last names would need to be constrained to a class
found it a great help in testing associative access processing to whose instances are totally ordered, in order to provide the com-
have a brute-force way to evaluate selection queries, as a kind of parisons supported by equality indices. The constraint on the
a "semantic benchmark" for checking index-based evaluation. last link of a path upon which an equality index is built is res-
Another central decision in designing associative access was tricted to B o o l e a n , Character, D~teT:i.me. F l o a t .
what to index, "classes or collections? Many applications may String, Smalllnteger or subclasses thereof. For
use instances of the same class, and store them in different col- Boolean, Cha.ra.cter and S m a l l I n t e g e r , there is no dis-

Sel~mber 1986 OOPSLA ~6 Proceedings 479

tinction between equality and identity indices. 8. Future Development Plans
In addition to supporting indexed associative access, typing As with all computer-based systems, performance efficiency
of names can be used as an integrity constraint on named vari- is a perennial concern. We have two approaches to addressing
ables. In particular, it can be used to assure that the value of a GemStone's performance efficiency: Improving the execution
named instance variable will understand a given protocol in all model, and irriproving database functions. Research of Smalltalk
objects that are a kind of a given class. virtual machines has demonstrated several techniques for
Indices are implemented as a sequence of index com. improving their efficiency [Kr]. Database systems spend most of
ponente, one ~" each link in the path upon which the index is their time in searching and sorting tasks. These functions can be
improved by better algorithms and better buffer management.
built. Each component is implemented as a B-trse. An index on
empN~,,e, ls.st, into empSe¢, would have two components, one We also plan to add several features to GemStone. We plan
from names to employees in empSe¢, and one from last names to add multiple repositories to allow users to dismount a portion
to names of employees. Common prefixes of indexed paths of the database and either transport it to another site or preserve
share the index components that correspond to the common pre- it Offline. Support for distributed databases will allow multiple
fix. For example, an index on empNa.me,f:Lz'e¢, would share the sites to share a collection of geographically distributed data. As
Index component from names to employees with the index on our user population increases, we expect users to need
empName.la.e¢,. Additionally, identity indices are implicitly incceased performance efficiency in certain new classes. To sup-
created on the path prefixes of an indexed path. Creating an port this need, we expect to add selected new classes to the set
index on empName, la~t; implicitly creates an Identity index on of predefined classes. GemStone currently supports only the
en,pNaae. IBM-PC workstation. We plan to offer interfaces to additional
Indexing is discussed in further detail elsewhere [MS]. We workstations such as Lisp and Smalltalk systems.
here mention that the maintenance of indices is a problem that is 7. Acknowledgements
related to that of maintaining referential integrity constraints [Da]
in relational databases, and that Stone manages concurrent The authors would like to thank the following people for their
access to indexing structures since indices are maintained in contributions to the GemStone project: Ken Almond, quality and
object space. change control; Robert Bretl, Stone object manager; John Bruno,
browser; Mauraan Drury, virtual image; Jack Falk, documenta-
5.9. Garbage Collection tion; Lynn Galllnat, virtual image; Larry Male, OPE editor; Daniel
Gem uses teachability information to remove temporary Moss, PC/VAX communications; Bruce Schuchardt, OPE imple-
objects before 8 transaction commits. Objects that have been mentation, bulk loader and dumper; Harold WiUams, Gem imple-
created dudng the current transection and can not be accessed mentation; Monty Williams, quality and support implementation;
transitively from the current state of some object present in the Mike Nastos, documentation and bug hunting; Rick Nelson, VAX
shared object table are temporary. Permanent objects, those system manager; D. Jason Penney, Stone implementation, pro-
that have previously been committed, are garbage collected off- cess scheduler; Mun Tuck Yap, Gem object manager, PC/VAX
line using a mark-sweep algorithm. We believe this preferable to communications.
reference counting in that reference counting would require
accessing an object every time a reference to it is added or
removed.
5.10. Access From Other Systems 8. Bibliography and Trademarks
GemStone does not provide direct access to a human inter-
face through OPAL. Applications manage the human interface [Ah+] Ahlsen, M., A. Bjornerstedt, S. Britts, C. Hutten, and L.
through C modules running on a PC. These modules have com- Sudedand, An architecture for object management in
ptete access to OPAL through the Gemstone C Interface (GCI), OIS, ACM TOOLS2:3, July 1983.
which is the interface to the agent. The role of the GCI in appli-
cation developroent is depicted in Figure 5. [ACO] Albano, A., L. Cardelli, and R. Orsini, Galileo: a
The OPAL Programming Environment (OPE) is a collection strongly-typed interactive conceptual language, ACM
of Microsoft Windows-compatibte applications for the creation of TODS 10:2, June 1985.
OPAL classes and methods. In addition, the OPE provides appli-
cations for the bulk loading and dumping of GemStone data- /AOI Albano, A., and R. Orsini, A prototyping approach to
bases. The development of a GemStone application consists of database applications development, Database
two parts: first creating Gemstone classes and methods through Engineering 7:4, December 1984.
the OPE and then developing a human interface on a PC that
accesses OPAL through the GCI. [Ca] Catell, R.G., Design and implementation of a
relationship-entity-datum model, Xerox CSL 83-4, May
C APlIUCATIGR~ 0PN,. C00E ~TONE 1983.
(m IIItl-I~) (mmEIwilha~E~ mvta) 0ATASAS£
[Ch .-,-] Chan, A., A. Danberg, S. Fox, W.-T. K. Un, A. Nod,
and D. Ries, Storage and access structures to support
a semantic data model, Proc. Conference on Very
Large Databases, September 1982.

[CFLR] Chan, A., S.A. Fox, W.-T. K. Un, and D. Rles, Design
sln¢twm ImStml of an ADA compatible local database manager (LDM),
FIGURE 5 TR CCA 81-09, Computer Corporation of America,
November 1981.

480 OOPSLA '86 Proceedings September 1986

[CFHL] Chu, K.C., J.P. Fishbum, P. Honeyman, and Y.E. Liem, [KK] Keehler, T., and G. Krasner, LOOM . large object-
Vdd - a VLSI design database, Engineering Design oriented memory for Smalltalk-80 systems, In [Kr].
Application Proceedings from SIGMOD Database
Week, May 1983. [Ka82] Katz, R.H., A database approach for managing VLSI
design data, Proc. IEEE 9th Design Automation Con-
[Col Codd, E.F., Extending the database relational model to feranceafl, 1982.
capture more meaning, ACM TODS 4:4, December
1979, [za~i Katz, R.H., Managing the chip design database, IEEE
Computer, 16:12, December 1983.
lCK] Copeland, G., and S.N. Koshafian, A decomposition
storage model, Proc. ACM/SIGMOD International [Kd Krasner, G., Smalltalk.80: Bits of History, Words of
Conference on the Management of Data, 1985. Advice, Addison-Wesley, 1983.

[CMI Copeland, G., and D. Maier, Making Smalltalk a data- [LP1 La Croix, M., and A. Pirotte. Data structures for CAD
base system, Proc. ACM/SIGMOD International object description, Proc. IEEE 18th Design Automation
Conference on the Management of Data, 1984. Conference, 1981.

[Da] Date, C.J., An Intoduction to Database Systems, [LOP] Lode, R., and W. Plouffe, Complex objects and their
Volume 2, Addison-Wesley, 1983. use in design transactions, Engineering Design Appli-
cation Proceedings from SIGMOD Database Week,
[DKL] Derret, N., W. Kent, and P. Lynbaek, Some aspects of May 1983.
operations in an object-oriented database, Database
Engineering 8:4, December 1985. [Ma] MacLennan, B.J., A view o1 object oriented program-
ming, Naval Postgraduate School NPS52-83-001,
[OK] Dolk, D.R., and B.R. Konsynski, Knowledge represente- February 1983.
lion for model management systems. IEEE Transac-
tions on Software Engineering, 10:6, November 1984. [MOP] Maier, D., A. Otis, and A. Purdy, Object-oriented data-
base development at Servio Logic, Database Engineer-
[Ea] Eastman, CM., System facilities for CAD databases, ing 18:4, December 1985.
Proc. IEEE 17th Design Automation Conference, June
1980. [MPI Meier, O., and D. Price, Data model requirements for
engineering applications, Proc. International Workshop
[EM] Emond, J.C., and G. Marechad, Experience in building on Expert Database Systems, 1984
ARCADE, a computer aided design system based on a
relational DBMS, Engineering Design Application [MS] Maier, D., and J. Stein, Indexing in an object-oriented
Proceedings from SIGMOD Database Week, May DBMS, manuscript in preparation.
1983.
[MNP] McLeod, D., K. Narayanaswamy, and K.V. Bapa Rao.
[GR] Goldberg, A., and D. Robson, Smalltalk-80: The An approach to infomation management for CAD;VLSI
Language and Its Implementation, Addison-Wesley, applications, Engineering Design Application Proceed-
1983. ings from SIGMOD Database Week, May 1983.

[Gd Gray, M., Databases for computer-aided design, In New [Mo] Morgenstern, M., Active Databases as a paradign for
Applications of Databases, G.Garadarin, E. Gelenbe enhanced computing environments, Proc. Conference
eds., Academic Press, 1984. on Very Large Oatabases,19]]3.
[Hal Haynie, M.N., The relational/network hybrid data model
for desiqn automation databases, P.roc. IEEE 18th [MAW] Mylopoulos, J., P.A. Bernstein, and H.K.T. Wong, A
Design Automation Conference, 1981. language facility for designing database-intensive appli-
cations, ACM TODS 5:2, June 1980.
[HJ] Hewitt, C., and P. de Jong, Open systems, In On Con-
ceptual Modelling: Perspectives from Artificial Intelli- [Ni] Nierstrasz, O.M., Hybrid: a unified objact-onented sys-
gence, Databases and Programming Languages, M.L. tem, Database Engineering 8:4, December 1985.
Brodie, J. Mylopoulos, J.W. Schmidt eds., Springer-
Verlag, 1984. [PKLM] Plouffe, W., Kim, W., R. Lorie, and D. Mcplabb, A data-
base system for engineenng design, Database
[HI.] Haskin, R.L., and R.A. Lorie, On extending the func- Engineering 7:2, June 1984.
tions of a relational database system, Proc.
ACM/SIGMOD International Conference on the pL] Powell, M.L., and M.A. Linton, Database support for
Management of Data, 1982. programming environments, Engineering Oesign Appli-
cation Proceedings from SIGMOD Database Week,
[JSW] Johnson, H.R., J.E. Schweitzer, and E.R. Warkentlre, A May 1983.
DBMS facility for handling structural engineering enti-
ties, Engineering Design Application Proceedings from [sq Siddle, T.W., Weaknesses of commercial data base
SIGMOD Database Week, May 1983. management systems in engineering applications, Proc.
IEEE 17th Design Automation Conference, June 1980.

Sel~mber 1986 OOPSLA '86 Proceedings 481

[SMF] Spooner, D.L.M.A. Milican, and D.B. Fatz, Modelling Trademarks
mechanical CAD data with data abstractions and
object-oriented techniques, Pron. 2nd International
Conference on Data Engineering. February 1986.
Smalltalk-80 is a trademark of Xerox Corporation
[SSB] Stempts, D., T. Sheard, and R. Bunker, Abstract data
UNIX is a trademark of AT&T Ball Laboratories
types in databses: specification manipulation and
access, Proc. 2nd International Conference on Data
VAX and VMS are trademarks of Digital Equipment Corp.
Engineering, February 1986. Microsoft and Windows are trademarks of Microsoft Corp.
IBM-PC is a trademark of IBM Corp.
ADA Is a trademark of the Department of Defense
pNe) Weiener, S.P., An abject-oriented protocol for managing
data, Database Engineering, 8:4, December 1985. GemStone is a trademark of Servio Logic DevelopmentCorp.

[Za] Zanioio. C., The database language GEM, Pro¢.

ACM/SIGMOD International Conference on the
Management of Data. May 1983.

[Z~,=] Zdonik, S.B., Object management systems concepts,

Proc. ACM SIGOA Conference on Office Information
Systems, 1984.

[ZdS5] Zdonik. S.B., Object management systems for design

environments, Database Engineering 8:4, December
1985.

[zwl Zdonik, S.B., and P. Wogner, Towards object.oriented

database environments, Brown Univeds~ TR, 1985.

482 OOPSI,A '86 Proceedings Sq:mbw t~6