Setuid Demystified∗

Hao Chen David Wagner Drew Dean

University of California at Berkeley SRI International
{hchen,daw}@cs.berkeley.edu [email protected]

Abstract system resources. In particular, user ID zero, reserved for

the superuser root, allows a process to access all system
resources.
Access control in Unix systems is mainly based on user
IDs, yet the system calls that modify user IDs (uid-setting In some applications, a user process needs extra privi-
system calls), such as setuid, are poorly designed, in- leges, such as permission to read the password file. By
sufficiently documented, and widely misunderstood and the principle of least privilege, the process should drop
misused. This has caused many security vulnerabilities its privileges as soon as possible to minimize risk to the
in application programs. We propose to make progress system should it be compromised and execute malicious
on the setuid mystery through two approaches. First, code. Unix systems offer a set of system calls, called the
we study kernel sources and compare the semantics of uid-setting system calls, for a process to raise and drop
the uid-setting system calls in three major Unix systems: privileges. Such a process is called a setuid process. Un-
Linux, Solaris, and FreeBSD. Second, we develop a for- fortunately, for historical reasons, the uid-setting system
mal model of user IDs as a Finite State Automaton (FSA) calls are poorly designed, insufficiently documented, and
and develop new techniques for automatic construction widely misunderstood. “Many years after the inception
of such models. We use the resulting FSA to uncover of setuid programs, how to write them is still not well un-
pitfalls in the Unix API of the uid-setting system calls, to derstood by the majority of people who write them” [1].
identify differences in the semantics of these calls among In short, the Unix setuid model is mysterious, and the
various Unix systems, to detect inconsistency in the han- resulting confusion has caused many security vulnerabil-
dling of user IDs within an OS kernel, and to check the ities.
proper usage of these calls in programs automatically.
Finally, we provide general guidelines on the proper us- We approach the setuid mystery as follows. First, we
age of the uid-setting system calls, and we propose a study the semantics of the uid-setting system calls by
high-level API that is more comprehensible, usable, and reading kernel sources. We compare and contrast the se-
portable than the usual Unix API. mantics among different Unix systems, which is useful
for authors of setuid programs. In doing so, we found
that manual inspection is tedious and error-prone. This
motivates our second contribution: we construct a for-
1 Introduction mal model to capture the behavior of the operating sys-
tem and use it to guide our analysis. We will describe a
new technique for building this formal model in an au-
Access control in Unix systems is mainly based on the tomated way. We have used the resulting formal model
user IDs associated with a process. In this model, each to more accurately understand the semantics of the uid-
process has a set of user IDs and group IDs which deter- setting system calls, to uncover pitfalls in the Unix API
mine which system resources, such as files and network of these calls, to identify differences in the semantics of
ports, the process can access1 . Certain privileged user these calls among various Unix systems, to detect incon-
IDs and groups IDs allow a process to access restricted sistency in the handling of user IDs within an OS kernel,
∗ This research was supported in part by DARPA Contract ECU01-
and to check for the proper usage of user IDs in programs
401U subcontract 27-000765 and NSF CAREER 0093337.
automatically.
1 In many Unix systems, a process has also a set of supplementary

group IDs, which are not closely related to the topic of this paper and Formal methods have gained a reputation as being im-
which will not be discussed.
practical to apply to large software systems, so it may access. In particular, user ID zero is reserved for the su-
be surprising that we found formal methods so useful in peruser root who can access all resources.
our effort. We will show how our formal model enables
many tasks that would otherwise be too error-prone or Each process has three user IDs: the real user ID (real
laborious to undertake. Our success comes from using uid, or ruid), the effective user ID (effective uid, or euid),
lightweight techniques to answer a well-defined question and the saved user ID (saved uid, or suid). The real uid
about the system; we are not attempting to prove that a identifies the owner of the process, the effective uid is
kernel is correct! Abstraction plays a major role in sim- used in most access control decisions, and the saved uid
plifying the system so that simple analysis techniques are stores a previous user ID so that it can be restored later.
sufficient. Similarly, a process has three group IDs: the real group
ID, the effective group ID, and the saved group ID. In
This paper is organized as the follows. Section 2 dis- most cases, the properties of the group IDs parallel the
cusses related work. Section 3 provides background on properties of their user ID counterparts. For simplicity,
the user ID model. Section 4 reviews the evolution of we will focus on the user IDs and will mention the group
the uid-setting system calls. Section 5 compares and IDs only when there is the potential for confusion or pit-
contrasts the semantics of the uid-setting system calls in falls. In Linux, each process has also an fsuid and an
three major Unix systems. Section 6 describes the formal fsgid which are used for access control to the filesystem.
user ID model and its applications. Section 7 analyzes The fsuid usually follows the value in the effective uid
two security vulnerabilities caused by misuse of the uid- unless explicitly set by the setfsuid system call. Simi-
setting system calls. Section 8 provides guidelines on the larly, the fsgid usually follows the value in the effective
proper usage of the uid-setting system calls and proposes gid unless explicitly set by the setfsgid system call. Since
a high-level API to the user ID model. the fsuid and fsgid are Linux specific, we will not discuss
them except when we point out an inconsistency in the
handling of them in the Linux kernel.

2 Related Work When a process is created by fork, it inherits the three

user IDs from its parent process. When a process exe-
cutes a new file by exec. . . , it keeps its three user IDs
Manual pages in Unix systems are the primary source unless the set-user-ID bit of the new file is set, in which
of information on the user ID model for most program- case the effective uid and saved uid are assigned the user
mers. See, for example, setuid(2) and setgid(2). But ID of the owner of the new file.
unfortunately, they are often incomplete or even wrong
(Section 6.4.1). Many books on Unix programming also Since access control is based on the effective user ID, a
describe the user ID model, such as Stevens’ [2], but of- process gains privilege by assigning a privileged user ID
ten they are specific to one Unix system or release, are to its effective uid, and drops privilege by removing the
outdated, or lack important details. privileged user ID from its effective uid. Privilege may
be dropped either temporarily or permanently.
Bishop discussed security vulnerabilities in setuid pro-
grams [3]. His focus is on potential vulnerabilities that
a process may be susceptible to once it gains privilege,
while our focus is on how to gain and drop privilege con-
fidently and securely. Unix systems have evolved and • To drop privilege temporarily, a process removes
diversified a great deal since Bishop’s work in 1987, and the privileged user ID from its effective uid but
a big problem today is how to port setuid programs se- stores it in its saved uid. Later, the process may
curely to various Unix systems. restore privilege by restoring the privileged user ID
in its effective uid.

3 User ID Model

This section provides background on the user ID model. • To drop privilege permanently, a process removes
Each user in a Unix system has a unique user ID. The the privileged user ID from all three user IDs.
user ID determines which system resources the user can Thereafter, the process can never restore privilege.
4 History • If the effective uid was zero, then the real uid and
effective uid could be set to any user ID.

Bell Laboratories filed a patent application on Den- • Otherwise, either the real uid or the effective uid
nis Ritchie’s invention of a bit to specify that a pro- could be set to value of the other one.
gram should execute with the permissions of its owner,
rather than invoker, in 1973. The patent was granted in Therefore, the setreuid system call enabled a process to
1979 [4]. Thus, setuid programs and related system calls swap the real uid and effective uid.
have existed through most of Unix history.
The POSIX standard [5] codified a new specification for
the setuid call. In an attempt to be POSIX compliant,
4.1 Early Unix 4.4 BSD replaced 4.2 BSD’s old setreuid model with
the POSIX/System V style saved uid model. It modified
setuid so that setuid set all three user IDs regardless of
In early Unix systems, a process had two user IDs: the whether the effective uid of a process was zero, therefore
real uid and the effective uid. Only one system call, se- allowing any process to permanently drop privileges.
tuid, modified them according to the following rule: if
the effective uid was zero, setuid set both the real uid
and effective uid; otherwise, setuid could only set the 4.4 Modern Unix
effective uid to the real uid [1]. This model had the prob-
lem that a process could not temporarily drop the root
privilege in its effective uid and restore it later. As Unix As System V and BSD influenced each other, both sys-
diverged into System V and BSD, each system solved the tems implemented setuid, seteuid, and setreuid, although
problem in a different way. with different semantics. None of these system calls,
however, allowed the direct manipulation of the saved
uid (although it could be modified indirectly through se-
4.2 System V tuid and setreuid). Therefore, some modern Unix sys-
tems introduced a new call, setresuid, to allow the modi-
fication of each of the three user IDs directly.
System V added a new user ID called the saved uid to
each process. Also added was a new system call, seteuid,
whose rules were:
5 Complexity of Uid-setting System Calls
• If the effective uid was zero, seteuid could set the
effective uid to any user ID. A process modifies its user IDs by the uid-setting sys-
tem calls: setuid, seteuid, setreuid, and in some systems,
• Otherwise, seteuid could set the effective uid to only
setresuid. Each of the system calls involves two steps.
the real uid or saved uid.
First, it checks if the process has permission to invoke
the system call. If so, it then modifies the user IDs of the
seteuid did not change the real uid or saved uid. Further- process according to certain rules.
more, System V modified setuid so that if the effective
uid was not zero, setuid functioned as seteuid (changing In this section, we compare and contrast the semantics
only the effective uid); otherwise, setuid set all three user of the uid-setting system calls among Linux 2.4.18 [8],
IDs. Solaris 8 [6], and FreeBSD 4.4 [7]. The behavior of the
uid-setting system calls was discovered by a combina-
tion of manual inspection of kernel source code and for-
4.3 BSD mal methods. We will defer discussion of the latter until
Section 6.

4.2 BSD kept the real uid and effective uid but changed
the system call from setuid to setreuid. Processes could The POSIX Specification To understand the seman-
then directly control both their user IDs, under the fol- tics of the uid-setting system calls, we begin with the
lowing rules: POSIX standard, which has influenced the design of the
system calls in many systems. In particular, the behavior If { POSIX SAVED IDS} is defined:
of setuid(newuid) is defined by the POSIX specification.
See Figure 1 for the relevant text. 1. If the process has appropriate privileges, the se-
tuid() function sets the real user ID, effective user
The POSIX standard refers repeatedly to the term ap- ID, and the [saved user ID] to newuid.
propriate privileges, which is defined in Section 2.3 of 2. If the process does not have appropriate privi-
POSIX 1003.1-1988 as: leges, but newuid is equal to the real user ID or
the [saved user ID], the setuid() function sets the
An implementation-defined means of associ- effective user ID to newuid; the real user ID and
ating privileges with a process with regard to [saved user ID] remain unchanged by this func-
the function calls and function call options de- tion call.
fined in this standard that need special privi-
leges. There may be zero or more such means. Otherwise:
1. If the process has appropriate privileges, the se-
Essentially, the term appropriate privilege serves as a tuid() function sets the real user ID and effective
wildcard that allows compliant operating systems to user ID to newuid.
use any policy whatsoever for deeming when a call
to setuid should be allowed. The conditional flag 2. If the process does not have appropriate privi-
{ POSIX SAVED IDS} parametrizes the specification, leges, but newuid is equal to the real user ID,
allowing POSIX-compatible operating systems to use ei- the setuid() function sets the effective user ID to
ther of two schemes (as described in Figure 1). We will newuid; the real user ID remains unchanged by
see how different interpretations of the term appropriate this function call.
privilege have led to considerable differences in the be- (POSIX 1003.1-1988, Section 4.2.2.2)
havior of the uid-setting system calls between operating
systems. Figure 1: An excerpt from the POSIX specification [5]
covering the behavior of the setuid system call.
5.1 Operating System-Specific Differences
newuid=geteuid(), in addition to when its effective uid is
zero. Also in contrast to Solaris, FreeBSD does not de-
Much of the confusion is caused by different interpreta- fine { POSIX SAVED IDS}, although every FreeBSD
tions of appropriate privileges among Unix systems. process does have a saved uid. Therefore, by calling se-
tuid(newuid), a process sets both its real uid and effective
uid to newuid if the system call is permitted, in agree-
Solaris In Solaris 8, a System V based system, a ment with POSIX. FreeBSD also sets the saved uid in all
process is considered to have appropriate privileges permitted setuid calls.
if its effective uid is zero (root). Also, Solaris de-
fines { POSIX SAVED IDS}. Consequently, calling se-
tuid(newuid) sets all three user IDs to newuid if the ef-
fective uid is zero, but otherwise sets only the effective Linux Linux introduces a capability2 model for finer-
uid to newuid (if the setuid call is permitted). grained control of privileges. Instead of a single level
of privilege determined by the effective uid (i.e., root or
non-root), there are a number of capability bits each of
FreeBSD FreeBSD 4.4 interprets appropriate privi- which is used to determine access control to certain re-
leges differently, as noted in Appendix B4.2.2 of POSIX: sources3 . One of them, the SETUID capability, carries
the POSIX appropriate privileges. To make the new ca-

The behavior of 4.2BSD and 4.3BSD that al- 2 Beware: the word “capability” is a bit of a misnomer. In this con-

lows setting the real ID to the effective ID is text, it refers to special privileges that a process can possess, and not
to the usual meaning in the security literature of an unforgeable refer-
viewed as a value-dependent special case of ence. Regrettably, the former usage comes from the POSIX standard
appropriate privilege. and seems to be in common use, and so we follow their convention in
this paper.
3 More accurately, a Linux process has three sets of capabilities, but
This means that a process is deemed to have ap- only the set of effective capabilities determine access control. All ref-
propriate privileges when it calls setuid(newuid) with erences to capabilities in this paper refer to the effective capabilities.
pability model compatible with the traditional user ID uid unchanged. However, when the current effective uid
model where appropriate privileges are carried by a zero is not zero, there is a slight difference in the permis-
effective uid, the Linux SETUID capability tracks the ef- sion required by seteuid among Unix systems. While
fective uid during all uid-setting system calls: Whenever Solaris and Linux allow the parameter neweuid to be
the effective uid becomes zero, the SETUID capability equal to any of the three user IDs, FreeBSD only allows
is set; whenever the effective uid becomes non-zero, the neweuid to be equal to either the real uid or saved uid;
SETUID capability is cleared. in FreeBSD, the effective uid is not used in the decision.
As a surprising result, seteuid(geteuid()), which a pro-
However, the SETUID capability can be modified out- grammer might intuitively expect to be always permitted,
side the uid-setting system calls. A process can clear can fail in FreeBSD, e.g., when ruid=100, euid=200, and
its SETUID capability, and a process with the SETP- suid=100.
CAP capability can remove the SETUID capability of
other processes (but note that in Linux 2.4.18, no process
has or can acquire the SETPCAP capability, a change
setreuid() The semantics of setreuid is confusing. It
that was made to close a security hole; see Section 7.1
modifies the real uid and effective uid, and in some
for details). Therefore, explicitly setting or clearing the
cases, the saved uid. The rule by which the saved uid
SETUID capability changes the properties of uid-setting
is modified is complicated. Furthermore, the permis-
systems calls.
sion required for setreuid differs among the three op-
erating systems. In Solaris and Linux, a process can
always swap the real uid and effective uid by calling
5.2 Comparison among Uid-setting System
setreuid(geteuid(), getuid()). In FreeBSD, however, se-
Calls
treuid(geteuid(), getuid()) sometimes fails, e.g., when
ruid=100, euid=200, and suid=100.
Next we compare and contrast the uid-setting system
calls and point out several unexpected properties and an
inconsistency in the handling of fsuid in the Linux ker- setuid() Although setuid is the only uid-setting sys-
nel. tem call standardized in POSIX 1003.1-1988, it is also
the most confusing one. First, the required permission
differs among Unix systems. Both Linux and Solaris
setresuid() setresuid has the clearest semantics among require the parameter newuid to be equal to either the
the four uid-setting system calls. The permission check real uid or saved uid if the effective uid is not zero. As
for setresuid() is intuitive and common to all OSs: for the a surprising result, setuid(geteuid()), which a program-
setresuid() system call to be allowed, either the euid of mer might reasonably expect to be always permitted, can
the process must be root, or each of the three parameters fail in some cases, e.g., when ruid=100, euid=200, and
must be equal to one of the three user IDs of the process. suid=100. On the other hand, setuid(geteuid()) always
As each of the real uid, effective uid, and saved uid is succeeds in FreeBSD. Second, the action of setuid dif-
set directly by setresuid, the programmer knows clearly fers not only among different operating systems but also
what to expect after the call. Moreover, the setresuid between privileged and unprivileged processes. In So-
call is guaranteed to have an all-or-nothing effect: if it laris and Linux, if the effective uid is zero, a successful
succeeds, all user IDs are changed, and if it fails, none setuid(newuid) call sets all three user IDs to newuid; oth-
are; it will not fail after having changed some but not all erwise, it sets only the effective user ID to newuid. On
of the user IDs. the other hand, in FreeBSD a successful setuid(newuid)
call sets all three user IDs to newuid regardless of the
Note that while FreeBSD and Linux offer setresuid, So- effective uid.
laris does not. However, Solaris does offer equivalent
functionality via the /proc filesystem. Any process can
examine its three user IDs, and a superuser process can setfsuid() In Linux, each process has also an fsuid in
set any of them, in line with the traditional System V addition to its real uid, effective uid, and saved uid. The
notion of appropriate privilege. fsuid is used for access control to the filesystem. It nor-
mally follows the effective uid unless when explicitly set
by the setfsuid system call. The Linux kernel tries to
seteuid() seteuid has also a clear semantics. It sets maintain the invariant that the fsuid is zero only if at least
the effective uid while leaving the real uid and saved one of the real uid, effective uid, or saved uid is zero, as
 ruid=euid=suid=0 by the egid in the setgid-like calls, but this is not how
fsuid=0 it actually works. This misconception caused a mistake
setresuid(x,x,-1) in the manual page of setgid in Redhat Linux 7.2 (Sec-
ruid=euid=fsuid=x tion 6.4.1).
suid=0
setfsuid(0) In many Unix systems, a process has also a set of supple-
ruid=euid=x
mentary group IDs which are modified by the setgroups
suid=fsuid=0 and initgroups calls. They are not closely related to the
setresuid(-1,-1,x)
topic of this paper and will not be discussed.
ruid=euid=suid=x
fsuid=0

Figure 2: The call sequence shows that the invariant that

6 Formal Models
the fsuid is zero only if at least one of the ruid, euid, or
suid is zero is violated in Linux. In the figure, x repre-
We initially began developing the summary in the previ-
sents a non-zero user ID.
ous section by manually reading operating system source
code. Although reading kernel sources is a natural
manifested in the comment in a source files. The ratio- method to study the semantics of the uid-setting sys-
nale is that once a process has dropped root privilege in tem calls, it has many serious limitations. First, it is
each of its real uid, effective uid, and saved uid, the pro- a laborious task, especially when various Unix systems
cess cannot have any leftover root privilege in the fsuid. implement the system calls differently. Second, since
Since the fsuid is Linux specific, this invariant allows a our findings are based on current kernel sources, they
cross-platform application that is not aware of the fsuid may become invalid should the implementation change
to securely drop all privileges. in the future. Third, we cannot prove that our findings
are correct and that we have not misunderstood kernel
Unfortunately, we discovered that this invariant may be sources. Finally, informal specifications are not well-
violated due to a bug in the kernel up to the latest version suited to programmatic use, such as automated verifi-
of Linux (2.4.18, as of this writing). The bug is that while cation of properties of the operating system or use in
every successful setuid and setreuid call sets the fsuid to static analysis of application programs to check proper
the effective uid, a successful setresuid call will fail to usage of the uid-setting system calls. These problems
do the same if the effective uid does not change during with manual source code analysis motivate the need for
the call 4 . This causes the call sequence in Figure 2 to more principled methods for building a formal model of
violate the invariant. The bug has been confirmed by the the uid-setting system calls.
Linux community. Section 6.4.3 will describe how we
discovered this bug using a formal model.
6.1 Building a Formal Model

setgid() and relatives There are also a set of calls

for manipulating group IDs, namely, setgid, setegid, se- Our model of the uid-setting system calls is based on fi-
tregid, and setresgid. They behave much like their se- nite state automata. The operating system maintains per-
tuid counterpart, with only one minor exception (the per- process state (e.g., the real, effective, and saved uids) to
mission check in setregid differs slightly from setreuid track privilege levels, and thus it is natural to view the
in Solaris). However, the appropriate privileges are al- operating system as implementing a finite state automa-
ways carried by the euid in both setuid-like and setgid- ton (FSA). A state of the FSA contains all relevant in-
like calls. Thus, an effective group ID of zero does not formation about the process, e.g., the three uids. Each
accord any special privileges to change groups. This is uid-setting system call leads to a number of possible tran-
a potential source of confusion: it is tempting to assume sitions; we label each transition with the system call that
incorrectly that since appropriate privileges are carried it comes from.
by the euid in the setuid-like calls, they will be carried
4 The
We construct the FSA in two steps: (1) determine its
seteuid(euid) call in Linux is implemented as setreuid(-1,
euid) or setresuid(-1, euid, -1), depending on the version of the C li-
states by reading kernel sources; (2) determine its tran-
brary. Hence, the seteuid system call might or might not set the fsuid sitions by simulation. In the first step, we determine the
reliably, depending on the C library version. states in the FSA by identifying kernel variables that af-
fect the behavior of the uid-setting system calls. For ex- G ET S TATE():
ample, if only the real uid, effective uid, and saved uid 1. Call getresuid(&r,&e,&s).
can affect the uid-setting system calls, then each state of 2. Return (r, e, s).
the FSA is of the form (r, e, s), representing the values
of the real, effective, and saved user IDs, respectively. S ET S TATE(r, e, s):
1. Call setresuid(r, e, s).
This is a natural approach. However, the problem one 2. Check for error.
immediately faces is that the resulting FSA is much too
large: in Linux, uids are 32-bit values, and so there are G ETA LL S TATES():
(232 )3 = 296 possible states. Obviously, manipulating 1. Pick n arbitrary uids u1 , . . . , un .
an FSA of such size is infeasible. Therefore, we need 2. Let U := {u1 , . . . , un }.
to somehow abstract away inessential details and reduce 3. Let S := {(r, e, s) : r, e, s ∈ U }.
the size of the FSA dramatically. 4. Let C := {setuid(x), setreuid(x, y),
setresuid(x, y, z), · · ·
Fortunately, we can note that there is a lot of symme- : x, y, z ∈ U ∪ {−1}}.
try present. If we have a non-root user ID, the behav- 5. Return (S, C).
ior of the operating system is essentially independent
of the actual value of this user ID, and depends only B UILD M ODEL():
on the fact that it is non-zero. For example, the states 1. Let (S, C) := G ETA LL S TATES().
(ruid, euid, suid) = (100, 100, 100) and (200, 200, 200) 2. Create an empty FSA with statespace S.
are isomorphic up to a substitution of the value 100 by 3. For each s ∈ S, do:
the value 200, since the OS will behave similarly in both 4. For each c ∈ C, do:
cases (e.g., setuid(0) will fail in both cases). In general, 5. Fork a child process, and within the child, do:
we consider two states equivalent when each can be mu- 6. Call S ET S TATE(s), and then invoke c.
tated into the other by a consistent substitution on non- 7. Finally, let s0 := G ET S TATE(),
root user IDs. By identifying equivalent states, we can pass s0 to the parent process, and exit.
c
shrink the size of the FSA dramatically. 8. Add the transition s → s0 to the FSA.
9. Return the newly-constructed FSA as the model.
Now that we know that there must exist some reason-
able FSA model, the next problem is how to compute
it. Here we use simulation: if we simulate the presence Figure 3: The model-extraction algorithm.
of a pseudo-application that tries every possible system
call and we observe the state transitions performed by practice, we extend this basic algorithm with several op-
the operating system in response to these system calls, timizations and extensions.
we can infer how the operating system will behave when
invoked by real applications. Once we identify equiva- One simple optimization is to use a depth-first search to
lent states, the statespace will be small enough that we explore only the reachable states. In our case, the state-
can exhaustively explore the entire statespace of the op- space is small enough that the improvement is probably
erating system. This idea is made concrete in Figure 3, unimportant, and we did not implement this optimiza-
where we give an algorithm to construct an FSA model tion. A more dangerous optimization would be to em-
using these techniques. ulate the behavior of the operating system from user-
level by cutting-and-pasting the source code of the setuid
Note that by using simulation to create a model of the system calls from the kernel into our simulation engine.
uid-setting system calls, we assume that while a process This would speed up model construction, but the perfor-
is executing such a call, the user IDs of the process can- mance improvement comes at a severe price: it is hard
not be modified outside the call. In other words, there is to be sure that our emulation of the OS is completely
no race on the user IDs between a uid-setting system call faithful. In any case, our unoptimized implementation
and other parts of the kernel. This requirement might not already takes only a few seconds to generate the model.
hold in multi-threaded programs if multiple threads share For these reasons, we do not apply this optimization in
the same user IDs. We leave this topic for future work. our implementation.

To ensure maximum confidence in the correctness of our

Implementation Our implementation follows Figure 3 results, we check in two different ways that the call to
closely. (Note that the simulator must run as root.) In setresuid in line 1 of S ET S TATE () succeeds. First, we
check the return value from the operating system. Sec- instance, a state like (100, 200, 100) will never appear in
ond, we call getresuid and check that all three user IDs such an application. Each state in this simple FSA has
have been set as desired (see Section 8.1.3). three bits, each representing whether the real uid, effec-
tive uid, or saved uid is root or not. All together there are
On Solaris, there are no getresuid and setresuid system eight states in the FSA. In Figure 4 we show graphically
calls. However, we can simulate them using the /proc the models one obtains in this way for the setuid call on
filesystem. We read the three user IDs of a process from Linux, Solaris, and FreeBSD. Note that the models on
its cred file, and we modify the user IDs by writing to Solaris and Linux are equivalent, but they differ from the
its ctl file (see proc(4) for details). model on FreeBSD. Figure 5 shows the models for the
seteuid, setreuid, and setresuid calls on Linux.
On Linux, we also model the SETUID capability bit by
adding a fourth dimension to the state tuple. Thus, states A variation of the previous models is shown in Figure 6
are of the form (r, e, s, b) where the bit b is true when- where the set of user ID values is {x, y} where x and y
ever the SETUID capability is enabled. This allows us to are distinct non-root user ID values. This model is ap-
accurately model the case where an application explic- propriate for applications that switch between two non-
itly clears or sets its SETUID capability bit; though we root user IDs (rather than between the root and a non-
are not aware of any real application that does this, if we root user ID). This model is appropriate for analyzing
ever do encounter such an application our model will still BSD games [9] run under the dungeon master. Foley’s
remain valid. work [10] offers a more serious use of this model.

On all operating systems, we extend our model further to We can easily extend the simple models to include more
deal with system calls that fail (i.e., when invoking call user ID values, which are appropriate for applications
c in line 6 of B UILD M ODEL ()). It is sometimes useful that use more than two user ID values. Figure 7 shows a
to be able to reason about whether a system call has suc- model where the set of user ID values is {0, x, y} where
ceeded or failed, and one way is to add a bit to the state x and y are distinct non-root user ID values. This is the
denoting whether the previous system call returned suc- fully general model of Unix user IDs.
cessfully or not.

Also, on all operating systems we extend our model to

6.3 Correctness
include group IDs. This adds three additional dimensions
to the state: real gid, effective gid, and saved gid5 . In
this way, we can model the semantics of the gid-setting
system calls. On Linux, we also add a bit to indicate Our model-extraction algorithm (Figure 3) is an instance
whether the SETGID capability is enabled or not. of a more general schema for inferring finite-state mod-
els, specialized by including application-dependent im-
plementations of the G ET S TATE(), S ET S TATE(), and
6.2 Examples of Formal Models G ETA LL S TATES() subroutines. We argue that our al-
gorithm is correct by arguing that the general version is
correct. This section may be safely skipped on first read-
In this section, we show a series of formal models of ing.
the uid-setting system calls created using the algorithm
in Figure 3. These models differ in their set of user ID We frame our theoretical discussion in terms of equiva-
values. In other words, they differ in the user ID val- lence relations. Let S denote the set of concrete states
ues picked in step 1 of G ETA LL S TATES() subroutine in (e.g., triples of 32-bit uids) and C the set of concrete sys-
Figure 3. c
tem calls. Write s t if the operating system will al-
ways transition from state s to t upon invocation of c.
We start with a simple model where the set of user ID We will need equivalence relations ≡S on S and ≡OS on
values is {0, x} where x is a non-root user ID. Although S ×C that are respected by the operating system: in other
simple, this model is accurate for many applications that c
words, if s t and s ≡S s0 , then there is some state t0
manipulate at most one non-root user ID at a time. For and some call c0 so that (s, c) ≡OS (s0 , c0 ), t ≡S t0 , and
5 We c0
don’t currently model supplemental groups, though this would s0 t0 . The intuition is that calling c from s is some-
be straightforward to correct. Note that this omission does not affect
the correctness of our model, as supplemental groups are only used in
how isomorphic to calling c0 from s0 . Also, we require
access control checks and never affect the behavior of the setgid-like that whenever (s, c) ≡OS (s0 , c0 ) holds, then s ≡S s0
calls. does, too.
 R=1,E=1,S=0 setuid(1) R=0,E=1,S=1 setuid(1)

setuid(0) setuid(0)

R=1,E=0,S=0 R=0,E=1,S=0 setuid(1) R=1,E=0,S=1 R=0,E=0,S=1

setuid(0) setuid(0) setuid(0) setuid(0)

setuid(1) R=0,E=0,S=0 setuid(0) setuid(1) setuid(1)

setuid(1)

R=1,E=1,S=1 setuid(0) setuid(1)

(a) An FSA describing setuid in Linux 2.4.18

R=1,E=1,S=0 setuid(1) R=0,E=1,S=1 setuid(1)

setuid(0) setuid(0)

R=1,E=0,S=0 R=0,E=1,S=0 setuid(1) R=1,E=0,S=1 R=0,E=0,S=1

setuid(0) setuid(0) setuid(0) setuid(0)

setuid(1) R=0,E=0,S=0 setuid(0) setuid(1) setuid(1)

setuid(1)

R=1,E=1,S=1 setuid(0) setuid(1)

(b) An FSA describing setuid in Solaris 8

R=0,E=1,S=1 R=1,E=0,S=0 R=1,E=0,S=1 R=0,E=0,S=1 R=0,E=1,S=0

setuid(0) setuid(0) setuid(0) setuid(0) setuid(0)

R=1,E=1,S=0 setuid(0) setuid(1) setuid(1) setuid(1) R=0,E=0,S=0 setuid(0) setuid(1) setuid(1)

setuid(1) setuid(1)

R=1,E=1,S=1 setuid(0) setuid(1)

(c) An FSA describing setuid in FreeBSD 4.4

Figure 4: Three finite state automata describing the setuid system call in Linux, Solaris, and FreeBSD, respectively.
Ellipses represent states of the FSA, where a notation like “R=1,E=0,S=1” indicates that euid = 0 and ruid = suid 6= 0.
Each transition is labelled with the system call it corresponds to. To avoid cluttering the diagram, we omit the error
states and (in Linux) the capability bits that otherwise would appear in our deduced model.
 R=1,E=0,S=1 seteuid(0) R=1,E=1,S=0 seteuid(1) R=0,E=1,S=1 seteuid(1) R=0,E=1,S=0 seteuid(1)

seteuid(1) seteuid(0) seteuid(1) seteuid(0) seteuid(1) seteuid(0) seteuid(1)

R=1,E=1,S=1 seteuid(0) seteuid(1) R=1,E=0,S=0 seteuid(0) R=0,E=0,S=1 seteuid(0) R=0,E=0,S=0 seteuid(0)

(a) An FSA describing seteuid in Linux

R=1,E=1,S=0 setreuid(0, 0) setreuid(0, 1) R=1,E=0,S=1 R=0,E=1,S=0 R=0,E=0,S=1

setreuid(1, 0) setreuid(1, 0) setreuid(1, 0) setreuid(1, 0)

R=1,E=0,S=0 setreuid(1, 0) setreuid(0, 0) setreuid(0, 0) setreuid(0, 0)

setreuid(0, 0) setreuid(1, 0) setreuid(0, 1) setreuid(0, 1) setreuid(0, 1)

setreuid(1, 1) setreuid(1, 1) setreuid(0, 1) setreuid(1, 0) R=0,E=0,S=0 setreuid(0, 0) setreuid(1, 1) setreuid(1, 1)

setreuid(1, 1) setreuid(0, 1) setreuid(0, 0)

R=0,E=1,S=1 setreuid(0, 1) setreuid(1, 1)

setreuid(1, 1)

R=1,E=1,S=1 setreuid(0, 0) setreuid(0, 1) setreuid(1, 0) setreuid(1, 1)

(b) An FSA describing setreuid in Linux

R=1,E=1,S=0 setresuid(1, 1, 0)

setresuid(1, 1, 0) setresuid(0, 0, 0)

R=0,E=0,S=0 setresuid(0, 0, 0) setresuid(1, 0, 1) setresuid(1, 1, 0)

setresuid(1, 0, 1) setresuid(0, 0, 0) setresuid(1, 0, 0) setresuid(1, 1, 0)

setresuid(1, 0, 0) setresuid(0, 0, 0) R=1,E=0,S=1 setresuid(1, 0, 1) setresuid(0, 1, 1) setresuid(1, 1, 0)

setresuid(0, 1, 0) setresuid(1, 1, 0) setresuid(0, 1, 1) setresuid(0, 0, 0) setresuid(1, 0, 1) setresuid(1, 0, 0)

setresuid(0, 0, 0) setresuid(0, 1, 0) setresuid(1, 0, 1) setresuid(0, 1, 1) R=1,E=0,S=0 setresuid(1, 0, 0) setresuid(0, 0, 1) setresuid(1, 1, 0)

setresuid(1, 1, 1) setresuid(0, 0, 1) setresuid(0, 0, 0) setresuid(0, 1, 0) setresuid(1, 0, 1) setresuid(0, 1, 1) setresuid(1, 0, 0)

setresuid(1, 1, 1) setresuid(0, 1, 0) setresuid(1, 0, 0) R=0,E=1,S=1 setresuid(0, 1, 1) setresuid(0, 0, 1) setresuid(1, 0, 1)

setresuid(0, 1, 0) setresuid(0, 1, 1) setresuid(0, 0, 1) setresuid(1, 0, 0) setresuid(1, 1, 1)

R=0,E=1,S=0 setresuid(0, 1, 0) setresuid(0, 0, 1) setresuid(0, 1, 1) setresuid(1, 1, 1)

setresuid(0, 1, 0) setresuid(0, 0, 1) setresuid(1, 1, 1)

setresuid(1, 1, 1) R=0,E=0,S=1 setresuid(0, 0, 1)

setresuid(1, 1, 1)

R=1,E=1,S=1 setresuid(0, 0, 0) setresuid(0, 0, 1) setresuid(0, 1, 0) setresuid(0, 1, 1) setresuid(1, 0, 0) setresuid(1, 0, 1) setresuid(1, 1, 0) setresuid(1, 1, 1)

(c) An FSA describing setresuid in Linux

Figure 5: Three finite state automata describing the seteuid, setreuid, setresuid system calls in Linux respectively.
Ellipses represent states of the FSA, where a notation like “R=1,E=0,S=1” indicates that euid = 0 and ruid = suid 6= 0.
Each transition is labelled with the system call it corresponds to.
 R=y,E=x,S=y setuid(x) R=y,E=y,S=x setuid(y) R=x,E=y,S=x setuid(y) R=x,E=y,S=y setuid(y)

setuid(y) setuid(x) setuid(y) setuid(x) setuid(x) setuid(y)

R=y,E=y,S=y setuid(x) setuid(y) R=y,E=x,S=x setuid(x) R=x,E=x,S=x setuid(x) setuid(y) R=x,E=x,S=y setuid(x)

Figure 6: A finite state automaton describing the setuid system call in Linux. This FSA considers only two distinct
non-root user ID values x and y. Ellipses represent states of the FSA, where a notation like “R=x,E=y,S=x” indicates
that euid = y and ruid = suid = x. Each transition is labelled with the system call it corresponds to.

R=0,E=y,S=x setuid(y) R=y,E=x,S=0 setuid(x) R=x,E=y,S=0 setuid(y) R=0,E=x,S=y setuid(x) R=y,E=y,S=x setuid(0) setuid(y) R=x,E=y,S=y setuid(0) setuid(y)

setuid(x) setuid(y) setuid(x) setuid(y) setuid(x) setuid(y) setuid(x) setuid(y)

setuid(0) R=0,E=x,S=x setuid(x) setuid(y) R=y,E=y,S=0 setuid(x) setuid(y) setuid(0) setuid(0) R=x,E=x,S=0 setuid(x) setuid(y) R=0,E=y,S=y setuid(x) setuid(y) setuid(0) R=y,E=x,S=x setuid(0) setuid(x) R=x,E=x,S=y setuid(0) setuid(x)

setuid(0) setuid(0) setuid(0) setuid(0)

R=y,E=0,S=y R=0,E=0,S=x R=y,E=0,S=0 R=0,E=y,S=0 setuid(x) setuid(y) R=0,E=x,S=0 setuid(x) setuid(y) R=x,E=0,S=y R=x,E=0,S=x R=y,E=0,S=x R=x,E=0,S=0 R=0,E=0,S=y

setuid(0) setuid(0) setuid(0) setuid(0) setuid(0) setuid(0) setuid(0) setuid(0) setuid(0) setuid(0)

setuid(y) setuid(y) R=y,E=x,S=y setuid(0) setuid(x) setuid(y) setuid(x) setuid(x) setuid(x) R=0,E=0,S=0 setuid(0) setuid(y) setuid(x) setuid(y) setuid(x) setuid(y) setuid(y) setuid(x) setuid(y) R=x,E=y,S=x setuid(0) setuid(y) setuid(x) setuid(x)

setuid(y) setuid(y) setuid(x) setuid(x)

R=y,E=y,S=y setuid(0) setuid(x) setuid(y) R=x,E=x,S=x setuid(0) setuid(x) setuid(y)

Figure 7: A finite state automaton describing the setuid system call in Linux. This FSA considers three user ID values:
the root user ID and two distinct non-root user ID values x and y. Ellipses represent states of the FSA, where a notation
like “R=0,E=x,S=y” indicates that ruid = 0, euid = x and suid = y. Each transition is labelled with the system call it
corresponds to.

A critical requirement is that the operating system must s (the implementation may freely choose one). Finally,
behave deterministically given the equivalence class of the G ETA LL S TATES() function must return a pair (S, C)
c c0 so that S contains at least one representative from each
the current state. More precisely, if s t and s0 u
0 0
where (s, c) ≡OS (s , c ), then we require t ≡S u. The equivalence class of ≡S and so that every equivalence
intuition is that the behavior of the operating system will class of ≡OS contains some element (s, c) with c ∈ C.
depend only on which equivalence class we are in, and
not on any other information about the state. For in- When these general requirements are satisfied, the
stance, the behavior of the operating system cannot de- B UILD M ODEL() algorithm from Figure 3 will correctly
pend on any global variables that don’t appear in the state infer a valid finite-state model for the underlying oper-
s; if it does, these global variables must be included into ating system. The proof is easy. We will write [x] for
the statespace S. As another example, a system call im- the equivalence class containing x, e.g., [s] = {t ∈
c
plementation that attempts to allocate memory and re- S : s ≡S t}. If s → t appears in the final FSA out-
turned an error code if this allocation fails will violate put by B UILD M ODEL(), then there must have been a
our requirement, because the success or failure of the step at which, for some s0 ∈ [s], t0 ∈ [t], and c0 with
memory allocation introduces non-determinism, which (s, c) ≡OS (s0 , c0 ), we executed c0 in state s0 at line 6
is prohibited. We can see that this requirement is non- and transitioned to state t0 . (This follows from the cor-
trivial, and it must be verified by manual inspection of rectness of S ET S TATE() and G ET S TATE().) The latter
c0 c
the source code before our algorithm in Figure 3 can be means that s0 t0 , from which it follows that s t00 for
safely applied; we will return to this issue later. 00
some t ∈ [t], since the OS respects ≡OS . Conversely,
c0
if s0 t0 for some s0 , c0 , t0 , then by the correctness of
Next, there are three requirements on the instantiation of
G ETA LL S TATES(), there will be some s and c satisfy-
the G ET S TATE(), S ET S TATE(), and G ETA LL S TATES()
ing (s, c) ≡OS (s0 , c0 ) so that we enter line 6 with s, c,
subroutines. First, the G ET S TATE() routine must return
and thanks to the deterministic nature of the operating
(a representative for) the equivalence class of the current c
system we will discover the transition s → t for some
state of the operating system. Note that it is natural to 0
t ≡S t . Thus, the FSA output by B UILD M ODEL() is
represent equivalence classes internally by singling out
exactly what it should be. Consequently, all that remains
a unique representative for each equivalence class and
is to check that these requirements are satisfied by our
using this value. Second, the S ET S TATE() procedure
instantiation of the schema.
with parameter s must somehow cause the operating sys-
tem to enter a state s0 in the same equivalence class as
We argue this next for the implementation shown in This completes our justification for the correctness of our
Figure 3. Let U denote the set of concrete uids (e.g., method for extracting a formal model to capture the be-
all 32-bit values), so that S = U × U × U. Say havior of the operating system.
that a map σ : U → U is a valid substitution if it
is bijective and fixes 0, i.e., σ(0) = 0. Each such
substitution can be extended to one on S by working 6.4 Applications
component-wise, i.e., σ(r, e, s) = (σ(r), σ(e), σ(s)),
and we can extend it to work on system calls by apply-
ing the substitution to the arguments of the system call, The resulting formal model has many applications. We
e.g., σ(setreuid(r, e)) = setreuid(σ(r), σ(e)). have already discussed in Section 5 the semantics of the
We define our equivalence relation ≡S on S as fol- setuid system calls and pointed out pitfalls; this relied
lows: two states s, s0 ∈ S are equivalent if there is heavily on the FSA formal model. Next, we will dis-
a valid substitution σ such that σ(s) = s0 . Similarly, cuss several additional applications: verifying documen-
(s, c) ≡OS (s0 , c0 ) holds if there is some valid substitu- tation and checking conformance with informal specifi-
tion σ so that σ(s) = s0 and σ(c) = c0 . cations; identifying cross-platform semantic differences
that might indicate potential portability issues; detecting
The correctness of G ET S TATE() and S ET S TATE() is im- inconsistency in the handling of user IDs within an OS
mediate. Also, so long as n ≥ 6, G ETA LL S TATES() kernel; and checking the proper usage of the uid-setting
is correct since the choice of uids u1 , . . . , un is imma- system calls in programs automatically.
terial: every pair (s, c) ∈ S × C is equivalent to some
pair (s0 , c0 ) ∈ S × C, since we can simply map the first
six non-zero uids in (s, c) to u1 , . . . , u6 respectively, and
6.4.1 Verifying Accuracy of Manual Pages
there can be at most six non-zero uids in (s, c). Actu-
ally, we can see that the algorithm in Figure 3 comes
from a finer partition than that given by ≡OS : for exam- Manual pages are the primary source of information for
ple, (u1 , u1 , u1 ) and (u2 , u2 , u2 ) are unnecessarily dis- Unix programmers, but unfortunately they are often in-
tinguished. This causes no harm to the correctness of the complete or wrong. FSAs are useful in verifying the ac-
result, and only unnecessarily increases the size of the curacy of manual pages of uid-setting system calls. For
resulting FSA. We gave the variant shown in Figure 3 each call, if its FSA is small and its description in man-
because it is simpler to present, but in practice our im- ual pages is simple, we check if each transition in the
plementation does use the coarser relation ≡S . FSA agrees with the description by hand. Otherwise, we
build another FSA based on the description and compare
All that remains to check is that the operating system re- this FSA to the original FSA built by simulation. Differ-
spects and behaves deterministically with respect to this ences between the two FSAs indicate discrepancies be-
equivalence class. We verify this by manual inspection of tween the behavior of the system call and its description
the kernel sources, which shows that in Linux, FreeBSD, in manual pages.
and Solaris the only operations that the uid-setting sys-
tem calls perform on user IDs are equality testing of two The following are a few examples of problematic docu-
user IDs, comparison to zero, copying one user ID to an- mentation that we have found using our formal model:
other, and setting a user ID to zero. Moreover, the oper-
ating system behavior does not depend on anything else,
with one exception: Linux depends on whether the SE- • The man page of setuid in Redhat Linux 7.2 fails to
TUID capability is enabled for the process, so on Linux mention the SETUID capability, which affects the
we add an extra bit to each state indicating whether this behavior of setuid.
capability is enabled. Thus, our verification task amounts
to checking that user IDs are treated as an abstract data • The man page of setreuid in FreeBSD 4.4 says:
type with only four operations (equality testing, compar-
Unprivileged users may change the real
ison to zero, and so on) and that the side effects and re-
user ID to the effective user ID and vice-
sults of the system call do not depend on anything outside
versa; only the super-user may make
the state S. In our experience, verifying that the operat-
other changes.
ing system satisfies these conditions is much easier than
fully understanding its behavior, as the former is an al- However, this is incorrect. Swapping the real uid
most purely mechanical process. and effective uid does not always succeed, such as
when ruid=100, euid=200, suid=100, contrary to
 what the man page suggests. The correct descrip- uid, or saved uid is zero. To verify this invariant, we ex-
tion is “Unprivileged users may change the real user tend the formal model of user IDs with the fsuid and au-
ID to the real uid or saved uid, and change the effec- tomatically create an FSA of the model on Linux. From
tive uid to the real uid, effective uid, or saved uid.” the FSA, we discovered that the invariant does not always
hold, because the state where fsuid = 0 and ruid 6= 0,
• The man page of setgid in Redhat Linux 7.2 says euid 6= 0, suid 6= 0 is reachable. For example, the call
sequence in Figure 2 will violate the invariant. The prob-
The setgid function checks the effective lem results from an inconsistency in the handling of the
gid of the caller and if it is the superuser, fsuid in the uid-setting system calls. While every suc-
all process related group ID’s are set to cessful setuid and setreuid call sets the fsuid to the ef-
gid. fective uid, a successful setresuid call will fail to do the
same if the effective uid does not change during the call.
In reality, the effective uid is checked instead of the
The problem has been confirmed by the Linux commu-
effective gid.
nity.

6.4.2 Identifying Implementation Differences

6.4.4 Checking Proper Usage of Uid-setting System
Calls
Since various Unix systems implement the uid-setting
system calls differently, it is difficult to identify their se-
mantic differences via reading kernel sources. We can The formal model is also useful in checking proper us-
solve this problem by creating an FSA of the user ID age of uid-setting system calls in programs. We model
model in each Unix system and contrasting the FSAs. a program as an FSA, called the program FSA, which
For example, Figure 4 shows clearly that the semantics represents each program point as a state and each state-
of setuid in Solaris is different from that in FreeBSD and ment as a transition. We call the FSA describing the user
Linux. ID model a model FSA. By composing the program FSA
with the model FSA, we get a composite FSA. Each state
The approach can be further formalized by taking the in the composite FSA is a pair (s, s0 ) of one state s from
symmetric difference of FSAs. In particular, if M, M 0 the model FSA (representing a unique combination of
are two FSAs for two Unix platforms with the same state- the values in the real uid, effective uid, and saved uid)
space, we can find portability issues as follows. Compute and one state s0 from the program FSA (representing a
the parallel composition M × M 0 , whose states are pairs program point). Thus, a reachable state (s, s0 ) in the
(s, s0 ) with s a state from M and s0 a state from M 0 . composite FSA indicates that the state s in the model
Then, mark as an accepting state of M × M 0 any pair FSA is reachable at the program point s0 . Figure 8(b)
(s, s0 ) where s 6= s0 . Now any execution trace that starts shows the program FSA of the program in Figure 8(a).
at a non-accepting state and eventually reaches an accept- Figure 8(c) shows the composite FSA obtained by com-
ing state indicates a sequence of system calls whose se- posing the model FSA in 4(a) with the program FSA in
mantics is not the same on both operating systems. This Figure 8(b).
indicates a potential portability issue, and all such differ-
ences can be computed via a simple reachability compu- This method is useful for checking proper usage of uid-
tation (e.g., depth-first search). setting system calls in programs, such as:

• Can a uid-setting system call fail? If any error state

6.4.3 Detecting Inconsistency within an OS Kernel
in the model FSA is reachable at some program
point, it shows that a uid-setting system call may
An OS kernel maintains many invariants which both the fail there.
kernel itself and many application programs depend on.
Violation of the invariants may cause vulnerabilities in • Can a program fail to drop privilege? If any state
both the OS and applications. Therefore, it is important that contains a privileged user ID in the model FSA
to detect any violation of the invariants. is reachable at a program point where the program
should be unprivileged, it shows that the program
The Linux kernel tries to maintain the invariant that the may have failed to drop privilege at an earlier pro-
fsuid is zero only if at least one of the real uid, effective gram point.
 Fourth, the formal model is useful in detecting inconsis-
tency in an OS kernel. Finally, the formal model is use-
// ruid=1, euid=0, suid=0
ful in checking proper usage of uid-setting system calls
1: printf(“drop priv”);
in programs automatically.
2: setuid(1);
3: execl(“/bin/sh”, “sh”,NULL);

(a) A program segment

7 Case Studies of Security Vulnerability

printf() setuid(1)
Line 1 Line 2 Line 3 Misuses of uid-setting system calls have caused many se-
curity vulnerabilities, which are good lessons in learning
(b) Program FSA of the program in Figure 8(a) the proper usage of the system calls. We will analyze two
such incidents in older versions of sendmail.

printf() setuid(1) Sendmail [11] is a commonly used Mail Transmission

Line 1 Line 2 Line 3
R=1,E=0,S=0 R=1,E=0,S=0 R=1,E=1,S=1 Agent(MTA). It runs in two modes: (1) as a daemon that
listens on port 25 (SMTP), and (2) via a Mail User Agent
(c) Composite FSA of the model FSA in Figure 4(a) and
to submit mail to the mail queue. In the first case, all
the program FSA in Figure 8(a) three user IDs of the sendmail process are typically zero,
as it is run by the superuser root in the boot process. In
the second case, however, sendmail is run by an ordinary
Figure 8: Composing a model FSA with a program FSA user. As the mail queue is not world writable, sendmail
requires privilege to access the mail queue.
• Which part of the program may run with privilege?
To answer this question, we first identify all states
that contain a privileged user ID in the model FSA. 7.1 Misuse of Setuid
Then, we identify all program points where any of
those states are reachable. The program may run
with privilege at these program points.
Next we describe a vulnerability that was caused by a
misuse of setuid [12]. Sendmail 8.10.1 installed the
A full discussion is out of the scope of this paper, and sendmail binary as a setuid-root executable. When it was
we refer the interested reader to a companion paper for executed by a non-root user, the real uid of the process
details [14]. was the non-root user while both the effective uid and
saved uid were zero. This gave sendmail permission to
write to the mail queue since its effective uid was zero.
6.5 Advantages To minimize risks in the event that an attacker takes over
sendmail and executes malicious code with root privi-
lege, sendmail permanently dropped root privilege be-
The formal model holds several advantages over trying fore doing potentially dangerous operations requested by
to understand the behavior of the kernel through man- an user. This was done by calling setuid(getuid()), which
ual code inspection. First, our formal model makes it sets all three user IDs to the non-root user.
easier to describe the properties of the uid-setting sys-
tem calls. While we still need to read kernel code to POSIX specifies that if a process has appropriate priv-
determine the kernel variables that affect the uid-setting ileges, setuid(newuid) sets all three user IDs to newuid;
system calls, the majority of the workload, determining otherwise, setuid(newuid) only sets the effective uid to
their actions, is done automatically by simulation. Sec- newuid (if newuid is equal to the real uid or saved uid).
ond, the formal model is reliable because it is created In Linux, appropriate privileges are carried by the SE-
from the same environment where application programs TUID capability. Furthermore, after any uid-setting sys-
run. The formal model has corrected several mistakes tem call, the Linux kernel sets or clears the SETUID
in the user ID model that we created manually. Third, capability bit, if necessary, to establish a simple post-
the formal model is useful in identifying semantic differ- condition: the SETUID capability should be set if and
ences of uid-setting system calls among Unix systems. only if the effective uid is zero.
 7.2 Interaction between User IDs and Group
A normal non-root user A malicious non-root IDs
executes sendmail user executes sendmail
ruid!=0, euid=suid=0 ruid!=0, euid=suid=0
SETUID-capability=1 SETUID-capability=0
Another vulnerability in Sendmail was caused by an in-
sendmail calls sendmail calls
setuid(getuid()) teraction between the user IDs and the group IDs [13].
setuid(getuid())
To further reduce the risk from a malicious user taking
ruid=euid=suid!=0 ruid=euid!=0, suid=0 over sendmail, as of version 8.12.0 Sendmail no longer
SETUID-capability=0 SETUID-capability=0
installed sendmail as a setuid-root program. To give
sendmail executes The malicious user sendmail permission to write to the mail queue, the mail
the rest of code takes over
sendmail and executes queue was configured to be writable by group smmsp,
setreuid(-1,0) and sendmail was installed as setgid-smmsp. Therefore,
when sendmail was executed by a non-root user, the real
ruid!=0, euid=suid=0
gid of the process was the primary group of the user, but
The malicious user
the effective gid and saved gid were smmsp.
executes code
with root privilege For the same reason that it permanently dropped root
privilege in previous versions, now sendmail perma-
(a) A normal execution (b) An execution of send-
nently dropped smmsp group privilege before executing
of sendmail by a non-root mail by an attacker potentially malicious directives from a user. Similar to
user the use of setuid(getuid()) to permanently drop root priv-
ilege, sendmail called setgid(getgid()) to permanently
Figure 9: A vulnerability in sendmail due to a misuse drop smmsp group privilege. However, since sendmail
of setuid. Note the failure: the programmer assumed no longer had appropriate privileges because its effective
that setuid(getuid()) would always succeed in dropping uid was not zero anymore, setgid(getgid()) only dropped
all privilege, but by disabling the SETUID capability, the the privileged group ID smmsp from the effective gid but
attacker is able to violate that expectation. left it in the saved gid. Consequently, any malicious user
who found some way to take over sendmail (e.g., by a
buffer overrun) could restore the smmsp group privilege
in the effective gid by calling setgid(-1, smmsp). This is
illustrated in Figure 10.

The vulnerability was caused by an interaction between

However, prior to version 2.2.16 of Linux, there was a the user IDs and group IDs since changing user IDs may
bug in the kernel that made it possible for a process to affect the property of setgid. To avoid the vulnerabil-
clear its SETUID capability bit even when its effective ity, we can replace setgid(newgid) with setresgid(newgid,
uid was zero. In this case, calling setuid(getuid()) only newgid, newgid) if available, or setregid(newgid, newgid)
modified the effective uid, and under these conditions, otherwise. The vulnerability also shows that if both user
sendmail would only drop root privilege from its effec- IDs and group IDs are to be modified, the modification
tive uid but not its saved uid. Consequently, any mali- should follow a specific order (Section 8.1.2).
cious local user who could take over sendmail (e.g., with
a buffer overrun attack) could restore root privilege in the
effective uid by calling setreuid(-1, 0). In other words, an
attacker could ensure sendmail’s attempt to drop all priv-
ileges would fail, thereby raising the risk of a root attack 8 Guidelines
on sendmail. Figure 9 illustrates the vulnerability.

The vulnerability was caused by the overloaded seman- We provide guidelines on the proper usage of the uid-
tics of setuid. Depending on whether a process has the setting system calls. First, we discuss general guidelines
SETUID capability, setuid sets one user ID or all three that apply to all setuid programs. Then, we focus on ap-
user IDs, but it returns a success code in both cases. The plications that use the uid-setting system calls in a spe-
vulnerability can be avoided by replacing setuid(newuid) cific way. We propose a high-level API for these appli-
with setresuid(newuid, newuid, newuid) if available, or cations to manage their privileges. The API is easier to
with setreuid(newuid, newuid) otherwise. understand and to use than the Unix API.
 to newuid regardless of the effective user ID. We envision
A user A user the following scenarios where setuid may be misused:
executes sendmail executes sendmail
ruid=euid=suid!=0 ruid=euid=suid!=0
rgid!=smmsp rgid!=smmsp • If a setuid-root program temporarily drops root
egid=sgid=smmsp egid=sgid=smmsp privilege with seteuid(getuid()) and later calls se-
sendmail calls sendmail calls
tuid(getuid()) with the intention of permanently
setgid(getgid()) setgid(getgid()) dropping all root privileges, the program does not
get the intended behavior on Linux or Solaris, be-
ruid=euid=suid!=0 ruid=euid=suid!=0
rgid=egid=sgid!=smmsp
cause the saved user ID remains root. (However,
rgid=egid!=smmsp
(wrong assumption) sgid=smmsp the program does receive the intended behavior on
FreeBSD.)
sendmail executes An attacker
the rest of code takes over sendmail
• Also on Linux or Solaris, in a setuid-root pro-
and executes
setregid(-1, smmsp) gram, calling setuid(getuid()) permanently drops
root privileges; however, in a setuid-non-root pro-
ruid=euid=suid!=0
rgid!=smmsp
gram (e.g., a program that is setuid-Alice where Al-
egid=sgid=smmsp ice is a non-root user), calling setuid(getuid()) will
not permanently drop Alice’s privileges, because
The attacker the saved user ID remains Alice. This is particu-
executes code with
smmsp group privilege
larly confusing, because the way setuid-root pro-
grams permanently drop privileges does not work
in setuid-non-root programs on Linux or Solaris.
(a) The programmer’s (b) Real execution of send-
mental model of an mail by a malicious user
expected execution trace
8.1.2 Obeying the Proper Order of System Calls

Figure 10: A vulnerability in sendmail due to interac-

tion between user IDs and group IDs. The failure occurs The POSIX-defined appropriate privileges affect the ac-
because the programmer has overlooked that she has al- tions of both system calls that set user IDs and that set
ready dropped root privilege and hence no longer has the group IDs. Since often appropriate privileges are car-
appropriate privileges to drop all group privileges in the ried by the effective uid, a program should drop group
setgid call. privileges before dropping user privileges permanently.
Otherwise, after permanently dropping user privileges,
the program may be unable to permanently drop group
8.1 General Guidelines privileges. For example, the program in Figure 11(a) is
able to permanently drop both user and group privileges
because it calls setgid before setuid. In contrast, since
the program in Figure 11(b) calls setuid before setgid, it
8.1.1 Selecting an Appropriate System Call
fails to drop group privileges permanently.

Since setresuid has a clear semantics and is able to set

each user ID individually, it should always be used if 8.1.3 Verifying Proper Execution of System Calls
available. Otherwise, to set only the effective uid, se-
teuid(new euid) should be used; to set all three user IDs,
setreuid(new uid, new uid) should be used. Since the semantics of the uid-setting system calls may
change, e.g., when the kernel changes or when an appli-
setuid should be avoided because its overloaded seman- cation is ported to a different Unix system, it is impera-
tics and inconsistent implementation in different Unix tive to verify successful execution of these system calls.
systems may cause confusion and security vulnerabilities
for the unwary programmer. As described in Section 5.2,
in Linux or Solaris, if the effective user ID is zero, se- Checking Return Codes The uid-setting system calls
tuid(newuid) sets all three user IDs to newuid; otherwise, return zero on success and non-zero on failure. A process
it sets only the effective user ID to newuid. On the other should check the return codes to verify the successful ex-
hand, in FreeBSD setuid(newuid) sets all three user IDs ecution of these calls. This is especially important when
 // drop privilege
ruid=100, euid=suid=0 ruid=100, euid=suid=0 setuid(getuid());
rgid=200, egid=sgid=0 rgid=200, egid=sgid=0

setgid(getgid()) setuid(getuid()) // verify the process cannot restore privilege

if (setreuid(-1, 0) == 0)
ruid=100, euid=suid=0 ruid=euid=suid=100 return ERROR;
rgid=egid=sgid=200 rgid=200, egid=sgid=0

setuid(getuid()) setgid(getgid())
Figure 12: An example of a program that verifies that
ruid=euid=suid=100 ruid=euid=suid=100 it has properly dropped root privileges. The verification
rgid=egid=sgid=200 rgid=egid=200, sgid=0
is achieved by checking that unpermitted uid-setting sys-
tem calls will fail. Note that a full implementation should
(a) A program correctly (b) A program fails to also check the return code from setuid and verify that all
drops both user and group drop group privileges per- three user IDs are as expected after the call to setuid.
privileges permanently by manently because it calls
calling setgid(getgid()) be- setuid(getuid()) before set-
fore setuid(getuid) gid(getgid()) examine its fsuid via the /proc filesystem since Linux
does not offer a getfsuid call.
Figure 11: Proper order of dropping user and group priv-
ileges. Figure (a), on the left, shows proper usage; figure
(b) shows what can go wrong if one gets the order back- Verifying Failures Once an attacker takes control of a
wards. process, the attacker may insert arbitrary code into the
process. Therefore, for further assurance on security,
the process should ensure that all unpermitted uid-setting
a process permanently drops privilege, since such an ac- system calls will fail. For example, after dropping privi-
tion usually precedes operations that, if executed with lege permanently, the process should verify that attempts
privilege, may compromise the system. to restore privilege will fail. This is shown in Figure 12.
Be aware that the Linux-specific setfsuid system call re-
turns the previous fsuid from before the call and does not 8.2 An Improved API for Privilege Manage-
return any error message to the caller on failure. This is ment
one motivation for our next guideline.

Although the general guidelines in Section 8.1 can help

Verifying User IDs However, checking return codes programmers to use the uid-setting system calls more se-
may be insufficient for uid-setting system calls. For ex- curely, programmers still have to grapple with the com-
ample, in Linux and Solaris, depending on the effective plex semantics of the uid-setting system calls and their
uid, setuid(newuid) may either (1) set all three user IDs differences among Unix systems. The complexity is
(if the effective uid is zero), or (2) set only the effective partly due to a mismatch between the low-level seman-
uid (if it is non-zero), but the system call returns the same tics of the system calls, which describes how to modify
success code in both cases. The return code does not in- the user IDs, and the high-level goals of the programmer,
dicate to the process which case has happened, and thus which represent a policy for when the application should
checking return codes is not enough to guarantee suc- run with privilege. We propose to resolve this tension by
cessful completion of the uid operation in some cases. introducing an API that is better matched to the needs of
Moreover, checking the return code is infeasible for the application programmers.
setfsuid call since it does not return any error message on
failure.
8.2.1 Proposed API
Therefore, after each uid-setting system call, a program
should verify that each of its user IDs are as expected. A
In many applications, privilege management can typi-
process may call getresuid to check all three user IDs if it
cally be broken down into the following tasks:
is available, as in Linux and FreeBSD, or use the /proc
filesystem on Solaris. Otherwise, the process may call
getuid and geteuid to check the real uid and effective uid, • Drop privilege temporarily, in a way that allows the
if none of these are available. In Linux, a process must privilege to be restored later.
 int drop_priv_temp(uid_t new_uid)
{
priv if (setresuid(-1, new_uid, geteuid()) < 0)
return ERROR_SYSCALL;
if (geteuid() != new_uid)
drop_priv_temp() restore_priv() return ERROR_SYSCALL;
drop_priv_perm() return 0;
unpriv_temp }

int drop_priv_perm(uid_t new_uid)

{
unpriv_perm uid_t ruid, euid, suid;
if (setresuid(new_uid, new_uid, new_uid) < 0)
return ERROR_SYSCALL;
Figure 13: An FSA showing the behavior of a process if (getresuid(&ruid, &euid, &suid) < 0)
return ERROR_SYSCALL;
when calling the functions of the new API. if (ruid != new_uid || euid != new_uid ||
suid != new_uid)
return ERROR_SYSCALL;
• Drop privilege permanently, so that it can never be return 0;
restored. }

int restore_priv()
• Restore privilege. {
int ruid, euid, suid;
if (getresuid(&ruid, &euid, &suid) < 0)
We propose a new API that offers the ability to perform return ERROR_SYSCALL;
each of these tasks directly and easily. The API contains if (setresuid(-1, suid, -1) < 0)
return ERROR_SYSCALL;
three functions: if (geteuid() != suid)
return ERROR_SYSCALL;
return 0;
• drop priv temp(new uid): Drop privilege temporar- }
ily. Move the privileged user ID from the effective
uid to the saved uid. Assign new uid to the effective
uid. Figure 14: A possible implementation of the high-level
API for systems with setresuid.
• drop priv perm(new uid): Drop privilege perma-
nently. Assign new uid to all the real uid, effective To use this implementation, an application must meet the
uid, and saved uid. following requirements:
• restore priv: Restore privilege. Copy the privileged
user ID from the saved uid to the effective uid. • When the process starts, its effective uid contains
the privileged user ID. This is true in most circum-
stances. When a process is run by a privileged user,
By raising the level of abstraction, we free programmers all three user IDs contain the privileged user ID. If
to think more about their desired security policy and less the process is run as a privileged user, i.e., its exe-
about the mechanism of implementing this policy. Fig- cutable is setuid’ed to the privileged user and is run
ure 13 illustrates the action of these functions pictorially by an unprivileged user, both the effective uid and
with a simple state diagram. saved uid of the process contain the privilege user
ID.

8.2.2 Implementation • If the privileged user ID is not zero, then the unpriv-
ileged user ID must be stored in the real uid when
the process starts. This requirement enables the pro-
We implement the new API as wrapper functions to the cess to replace the privileged user ID in the effective
uid-setting system calls. The implementation uses setre- uid with the unprivileged user ID in drop priv temp
suid if available since it has the clearest semantics and and drop priv perm. This is the case when a non-
it is able to set each of the user IDs independently, as root user executes an executable that is setuid’ed
shown in Figure 14. If setresuid or its equivalent is not to another non-root user. On the other hand, if the
available, the implementation uses seteuid and setreuid, privileged user ID is zero, then there is no such re-
as shown in Figure 15. quirement, since the process can set its user IDs to
uid_t priv_uid;
• It does the right thing even in cases where root is not
int drop_priv_temp(uid_t new_uid) involved, i.e., where the privileged user ID is not the
{ superuser.
int old_euid = geteuid();

// copy euid to suid

if (setreuid(getuid(), old_euid) < 0)
We can extend this basic implementation to include
return ERROR_SYSCALL; stronger safeguards against programming errors or OS
// set euid as new_uid inconsistency. To prevent a program from restoring a
if (seteuid(new_uid) < 0) wrong privilege, we can let the function restore priv take
return ERROR_SYSCALL;
if (geteuid() != new_uid)
a parameter and check that the parameter matches the
return ERROR_SYSCALL; privilege stored in the saved user ID (Figure 14) or in the
priv_uid = old_euid; variable priv uid (Figure 15). Another improvement is to
return 0; let the function drop priv perm verify that an attempt to
}
regain privilege will fail, as described in Section 8.1.3.
int drop_priv_perm(uid_t new_uid)
{
uid_t suid;
if (setreuid(new_uid, new_uid) < 0) 8.2.3 Evaluation
return ERROR_SYSCALL;
// OS specific way of reading suid
suid = read_suid_from_proc_filesystem(); To evaluate the high-level API, we replaced every uid-
if (getuid() != new_uid || setting system call in OpenSSH 2.5.2 with functions
geteuid() != new_uid ||
suid != new_uid) from the new API. OpenSSH contains fifteen uid-setting
return ERROR_SYSCALL; system calls in eight tasks. Of the eight tasks, four are
return 0; to drop privilege permanently, two are to drop privilege
}
temporarily, and two are to restore privilege. We are able
int restore_priv() to implement all these tasks with the new API.
{
if (seteuid(priv_uid) < 0) One known limitation of our API is that it does not ad-
return ERROR_SYSCALL;
if (geteuid() != priv_uid)
dress group privileges. We leave this for future work.
return ERROR_SYSCALL;
return 0;
}

9 Future Work
Figure 15: A possible implementation of the high-level
API for systems without setresuid.
We plan to study how the uid-setting system calls affect
other properties of a process, such as the ability to receive
arbitrary values. signals and to dump cores. We may also study how to
• The process does not make any uid-setting system extend the formal models for multi-threaded programs.
calls that change any of the three user IDs. Such a Topics to investigate include in-kernel races and how the
call may cause the process to enter a state not cov- user IDs are inherited during the creation of new threads
ered by the FSA in Figure 13, on which the high- in different Unix systems.
level API and the implementation are based.

The implementation has the following beneficial proper-

ties:
10 Conclusion

• It does not affect the real uid. We have studied the proper usage of the uid-setting sys-
tem calls by two approaches. First, we documented the
• It guarantees that all transitions in Figure 13 suc-
semantics of the uid-setting system calls in three major
ceed.
Unix systems (Linux, Solaris, and FreeBSD) and identi-
• It verifies that the user IDs are as expected after each fied their differences. We then showed how to formalize
uid-setting system call. this problem using formal methods, and we proposed a
new algorithm for constructing a formal model of the se-
mantics of the uid-setting system calls. Using the result-
ing formal model, we identified semantic differences of
the uid-setting system calls among Unix systems and dis-
covered inconsistency within an OS kernel. Finally, we
provided guidelines for proper usage of the uid-setting
system calls and proposed a high-level API for manag-
ing user IDs that is more comprehensible, usable, and
portable than the usual Unix API.

Acknowledgment

We thank Monica Chew, Solar Designer, Peter Gutmann,

Robert Johnson, Ben Liblit, Zhendong Su, Theodore
Ts’o, Wietse Venema, Michal Zalewski, and the anony-
mous reviewers for their valuable comments.

References
[1] Chris Torek and Casper H.S. Dik. Setuid mess. http:
//yarchive.net/comp/setuid_mess.html.
[2] Richard Stevens. Advanced Programming in the UNIX
Environment. Addison-Wesley Publishing Company,
1992.
[3] Matt Bishop. How to write a setuid program. ;login:,
12(1):5–11, 1987.
[4] Dennis M. Ritchie. Protection of data file contents.
United States Patent #4,135,240. Available from http:
//www.uspto.gov.
[5] IEEE Standard 1003.1-1998: IEEE standard portable op-
erating system interface for computer environments. In-
stitute of Electrical and Electronics Engineers, 1988.
[6] http://www.sun.com/software/solaris/.
[7] http://www.freebsd.org.
[8] http://www.kernel.org.
[9] dm(8). 4.4 BSD System Manager’s Manual.
[10] Simon N. Foley. Implementing chinese walls in unix.
Computers and Security Journal, 16(6):551–563, Decem-
ber 1997.
[11] http://www.sendmail.org/.
[12] Sendmail Inc. Sendmail workaround for linux capabilities
bug. http://www.sendmail.org/sendmail.
8.10.1.LINUX-SECURITY.txt.
[13] Michal Zalewski. Multiple local sendmail vulnerabili-
ties. http://razor.bindview.com/publish/
advisories/adv_sm812.html.
[14] Hao Chen, David Wagner, and Drew Dean. An infras-
tructure for examining security properties of software.
manuscript in preparation.