IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-7, NO.

4, JULY 1981 333

Major Issues in Software Engineering Project

Management
RICHARD H. THAYER, MEMBER, IEEE,ARTHUR B. PYSTER, MEMBER, IEEE,
AND ROGER C. WOOD, MEMBER, IEEE

Abstract-Software engineering project management (SEPM) has been [25], improvements and developments in management have
the focus of much recent attention because of the enormous penalties not kept pace with advances in the technology of software
incurred during software development and maintenance resulting from development. A large number of articles addressing such topics
poor management. To date there has been no comprehensive study
performed to determine the most significant problems of SEPM, their as better coding style ("structured programming"), testing,
relative importance, or the research directions necessary to solve them. formal verification, language design for more reliable coding,
We conducted a major survey of individuals from all areas of the com- diagnostic compilers, and so forth, have appeared in the liter-
puter field to determine the general consensus on SEPM problems. ature (e.g., in the IEEE TRANSACTIONS ON SOFTWARE ENGI-
Twenty hypothesized problems were submitted to several hundred indi- NEERING, Proceedings of the International Conferences on
viduals for their opinions. The 294 respondents validated most of these
propositions. None of the propositions was rejected by the respondents Software Engineering, Proceedings of the ACM Conferences on
as unimportant. A number of research directions were indicated by the the Principles of Programming Languages, . . . ). Although the
respondents which, if foliowed, the respondents believed would lead to technology of software engineering as a well-defined discipline
solutions for these problems. is relatively new, software engineers hav-e progressed to the point
Index Terms-Project management, software engineering, survey, uni- where many major issues relevant to the technology of software
versity curriculum. production have been identified and considerable progress in
addressing these issues has been made [16], [17]. Practical
I. INTRODUCTION working tools to support improved software production are
N EARLY every software engineering development project commonly available, and their design and generation have be-
is plagued with numerous problems leading to late deliv- come a recognized topic for university instruction [21.
ery, cost overruns, and sometimes, unsatisfied customers. Often Software engineering project management (SEPM) has not
these problems are technical, but just as often, the software enjoyed the same progress. While it might be argued that SEPM
engineering development problems are managerial. How often has been defined, it is far from a recognized discipline. Soft-
have we personally heard or read that a project was late (or ware developers who have demonstrated competence as devel-
overbudget, or reduced in scope, or terminated early, or did opers and programmers have been elevated to project managers
not satisfy the user, . . .) because: without the benefit of education or training. The major issues
* programmers did not tell the truth (or did not know the and problems of SEPM have not even been agreed on by the
truth) about the status of their programs, or computing community as a whole, and, consequently, priorities
* management unreasonably reduced the delivery time of for addressing them have not been widely established. Research
(or budget of, or withheld necessary resources from) the project, in SEPM has been scant. As Commander Cooper reported in
or the IEEE TRANSACTIONS ON SOFTWARE ENGINEERING Spe-
* top management did not allow sufficient time for front cial Issue on Project Management (July 1978):
end planning, or
* the true status of the project was never known, or "Although the need is apparent, there appears to be
* programer productivity was lower than planned, or precious little innovative activity in the area of software
management. Perhaps this is so because computer scien-
* the customer did not know what he wanted (or changed tists believe that management per se is not their business,
his requirements), or and the management professionals assume that it is the
* government standards for requirement specifications (or computer scientists responsibility."
procurement policies) were not suitable for software, or ..
Richard Merwin stated in the same issue:
Although the technological and managerial aspects of soft-
ware engineering were both recognized about the same time "Programming disciplines, such as top-down design,
Manuscript received May 22, 1979; revised February 5, 1980. Any use of standard high level languages, and program library
opinions expressed herein are those of the authors and do not necessarily support systems all contribute to the production of reli-
reflect the opinions of the U.S. Air Force. able software on time, within budgets.... What is still
R. H. Thayer was with the Sacramento Air Logistics Center, Air Force missing is the overall management fabric which allows
Logistics Command, McClellan Air Force Base, CA 95652. He is now the senior project manager to understand and lead major
with the Department of Computer Science, California State University,
Sacramento, CA 95819. data processing development efforts."
A. B. Pyster and R. C. Wood are with the Department of Computer
Science, University of California, Santa Barabara, CA 93106. Some data were collected about the extent to which univer-
0098-5589/81/0700-0333$00.75 C 1981 IEEE
334 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-7, NO. 4, JULY 1981

sities teach SEPM. It revealed that only a handful of promi- Deliverables

nent universities surveyed had any courses exclusively on * Software
SEPM. On the other hand, most of these universities offered * Documentation
at least one course on the technological issues of software Success Attributes
engineering. More details on this study are presented in Sec- * On time
tion VI. * Within resources
* Meets requirements
Before we can agree on the progress accomplished in SEPM, * Usable
we must establish the yardstick for measuring that progress. * Reliable
The first step must be to identify the major issues and problems * Maintainable
which face project managers. This paper will attempt such an Fig. 1. Software development delivery and success model.
identification based on a survey of 294 individuals. In addition,
it will try to indicate the relative importance of these issues next step was to attempt to validate that these 20 "problems"
and problems. are truly problems for project managers in the field. Two
reasonable validation methods in this context are to:
II. THE MAJOR ISSUES
The major or key issues of an activity, technology, or task * perform a detailed case study of a number of projects,
are those portions of it which are: observing the problems encountered by project managers,
* survey various individuals for their opinions on the nature
* critical to the success or excellence of the larger activity, of managerial problems in software development projects.
technology, or task, and
* have an existing or potential weakness which may signifi- In fact, both methods were tried. The results of the case
cantly detract from the success or excellence of the activity, study will be reported in a forthcoming paper. The second
technology, or task, or in a worst case, cause failure. approach, surveying, is the method reported here for validating
the problems which is reported here.
The major issues of SEPM must be identified in order that: The survey, conducted between Autumn 1977 and Spring
* real problems can be separated from pseudoproblems en- 1979, obtained information from industrial, governmental, and
abling management attention to be properly focused, university leaders on the major problems of SEPM. The cate-
* the university/education system can properly apply its gories of participants desired and eventually obtained is shown
resources in the training of software managers as well as in Fig. 2. Experienced and knowledgeable data processing
engineers, managers and other personnel from the data processing area
* a greater level of abstraction can be identified to provide a were sent a copy ofthe 20 issues and asked to state their opinion
broader basis for understanding the management of a software as to whether they felt each of the hypothesized problems was
development project. "critical," "important," "not important," "not a problem," or,
lastly, "incorrectly stated." In addition, the respondents were
The first step taken to define the major issues of SEPM was asked whether they viewed a problem as being primarily mana-
to review the literature for software engineering problems since gerial, technical, both, or neither. The respondents were also
1974 (plus several early classical documents) concentrating on asked to complete two more questions about each problem.
secondary sources in order to take advantage of generalizations They were asked whether the solution to the problem was ob-
already made. By using the software engineering delivery and tainable through improvements in management, technology,
success model shown in Fig. 1, we hypothesized which of these both, or neither. Finally, they were asked to state in English
problems can most affect the success of software delivery. prose, how they would (or did) solve this problem. The last
These, we believed, were the major issues. question helped ensure that the participants thought through
Each software engineering issue was then evaluated from the their answers with more than a casual effort. A sample of one
viewpoint of the project manager, leaning towards a manager survey question is contained in Fig. 3.
who does not have a great deal of managerial experience and is The participants were obtained from published names and
seeking a means to ensure delivery of a successful project. These addresses available to the general computing community and
issues were then reworded as problems as seen by the project by distributing survey copies to people attending computer-
manager and compared to the classic management model of oriented conferences. The survey was mailed to highly visible
planning, organizing, staffing, directing, and controlling to individuals in government and private computer science, partic-
ensure that every area was covered. By far the dominant two ularly members so the IEEE Computer Society, ACM, and
activities are planning and controlling, which together account AFIPS. In addition, the survey was distributed at a number of
for 80 percent of the issues, with planning alone involving ten conference sessions in which the emphasis was on project
issues. The resulting 20 major issues of SEPM are shown in management. In addition, as word of the survey spread, we
Table I, along with their respective source references and a received dozens of requests for survey copies as well as recom-
short title for later reference. mendations for other candidate participants.
To further increase the pool of potential respondents, we
III. VALIDATING THE ISSUES examined several widely read computer journals for project
Based upon the criteria explained in the last section, the 20 management articles and software engineering articles that
problem statements shown in Table I were assembled. The could be related to project management, and mailed a survey
THAYER et al.: SOFTWARE ENGINEERING PROJECT MANAGEMENT 335

TABLE I
TWENTY HYPOTHESIZED PROBLEMS IN SOFTWARE ENGINEERING PROJECT
MANAGEMENT

Planning Problems Problem 12 (Organization A ccountability): The accountability

structure in many software engineering projects is poor, leaving some
Problem 1 (Plan Requirement): Requirement specifications are question as to who is responsible for various project functions [15],
frequently incomplete, ambiguous, inconsistent, and/or unmeasure- [29], [5], [13].
able [31], [301, [321, [3], [15], [12], [5], [7], [10], [25], [13], Staffing Problems
[9], [11].
Problem 2 (Plan Success): Success criteria for a software devel- Problem 13 (Staff Project Manager): Procedures and techniques for
opment are frequently inappropriate which results in poor "quality" the selection of project managers are poor [ 15], [121, [5], [10], [ 13].
delivered software, i.e., not maintainable, unreliable, difficult to use,
relatively undocumented, etc. [15], [32], [3], [22], [5], [101, [14], Directing Problems
[281, [91, [11].
Problem 3 (Plan Project): Planning for software engineering projects Problem 14 (Direct Techniques): Decision rules for use in selecting
isgenerally poor [6], [15], [22], [24], [12], [5], [10], [28], [13]. the correct management techniques for software engineering project
management are not available [ 10].
Problem 4 (Plan Cost): The ability to estimate accurately the re-
sources required to accomplish a software development is poor [1], Controlling Problems
[32], [81, [15], [29], [5], [10], [281, [11], [181.
Problem S (Plan Schedule): The ability to estimate accurately the Project 15 (Control Visibility): Procedures, techniques, strategies,
delivery time on a software development is poor [1], [30], [32], [8], and aids that will provide visibility of progress (not just resources used)
[15], [34], [5], [10], [28], [13], [18]. to the project manager are not available [19], [31], [81, [151, [29],
Problem 6 (Plan Design): Decision rules for use in selecting the cor-
[5], [12].
rect software design techniques, equipment, and aids to be used in Problem 16 (Control Reliability): Measurements or indexes of reli-
designing software in a software engineering project are not available ability that can be used as an element of software design are not available
[19], [15], [121, [5], [10], [28], [11]. and there is no way to predict software failure, Le., there is no practical
way to show the delivered software meets a given reliability criteria [3],
Problem 7 (Plan Test): Decision rules for use in selecting the correct [211, [22], [12], [10], [20], [28], [9].
procedures, strategies, and tools to be used in testing software developed
in a software engineering project [3], [ 121, [ 10], [ 281, [35 ]. Problem 17 (Control Maintainability): Measurements or indexes
of maintainability that can be used as an element of software design
Problem 8 (Plan Maintainable): Procedures, techniques, and strate- are not available, ie., there is no practical way to show that a given
gies for designing maintainable software are not available [121, [71, program is more maintainable than another [28], [9].
[281, [111. Problem 18 (Control Goodness): Measurements or indexes of
Problem 9 (Plan Warranty): Methods to guarantee or warrant that "goodness" of code that can be used as an element of software design
the delivered software will "work" for the user are not available [30]. are not available; i.e., there is no practical way to show that one program
is better than another [ 3 2], [ 26], [ 33 ].
Problem 10 (Plan Control): Procedures, methods, and techniques
for designing a project control system that will enable project managers Problem 19 (Control Programmers): Standards and techniques for
to successfully control their project are not readily available [ 19], [ 15], measuring the quality of performance and the-quantity of production
[5], [28], [91, [11]. expected from programmers and data processing analysts are not avail-
able [1], [31], [32], [15], [29], [34], [28], [111].
Organizing Problems
Problem 20 (Control Tracing): Techniques and aids that will pro-
Problem 11 (Organization Type): Decision rules for selecting the vide an acceptable means of tracing a software development from re-
proper organizational structure, e.g., project, matrix, function, are not quirements to completed code are not generally available [3], [28],
available. [5].*

problems in SEPM. It was not our intent to determine why

copy to the articles' authors. Finally, the survey was distri-
buted to a group of professional programmers and, to one the participants answered as they did, nor to analyze the rela-
tionships between the various propositions or their parts. We
graduate class in computer science at the University of Califor-
feel that because the problem statements are short, and sub-
nia at Santa Barbara. These last two distributions account for
ject to different interpretations, the reader should not place
the majority of the project individuals (as opposed to managers)
and students who completed the survey. too much significance on the specific percentages tabulated
for individual problems. Therefore, to determine whether one
As a twist on the original surveying method, we also sent a
of the hypothesized problems was truly a problem or not, a
modified questionaire to most major universities in the country
which offer computer science degrees to determine to what 30/40/30 rule was adopted. If fewer than 30 percent of the
degree SEPM was being taught. Twenty-seven responses were respondents felt that a proposed problem was at least "impor-
received, revealing that very little on SEPM is currently incor-
tant," then the hypothesis would be rejected. On the other
porated into the regular computer science programs at either
hand, if at least 70 percent felt that the issue was either critical
the undergraduate or graduate levels. Section VI contains the
or important, then the hypothesis was accepted and that issue
survey details. categorized as a major problem. Finally, if at least 30 percent
but less than 70 percent of the respondents felt that the prob-
IV. THE SURVEY RESULTS lem was important (the middle 40 percent), then the hypothesis
Our primary goal was to uncover whatever consensus exists could not be conclusively accepted or rejected. No more re-
among those involved in software development on the major fined ranking of the problems was done for this study. The
336 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-7, NO. 4, JULY 1981

* Technical leaders in computer science who are/have either: TABLE II

* a position of influence in their company, ATTRIBUTES OF PARTICIPANTS
* highly visible authors and/or speakers on computer science or GROUP ATrRIBUTE PERCENTAGE
project management OF RESPONDENTS
* Project managers
Job or position (only one applies)
* R&D personnel
line manager 10
* Educators (university, education institute,...) project manager 15
individual developer 21
* Other personnel interested in project management
senior staff, policy 4
supervisor, software 12
Fig. 2. Desired participants. corporate staff, software 6
supervisor, non-software
anonymous
1. PROBLEM-Requirement speciflcations are frequently incomplete, oniversity teaching 28
ambiguous, inconsistent, and/or unmeasureable. consultant 3
a. This problem is: critical ( ) . . . important ( ).. . not important
)... no problem ( ) . .. incorrectly stated ( ). General (more than one applies)

b. This is a problem in: management( )... technology( ) . .. both R./D oriented 23

educators 26
( ) . . . neither ( ) ... not a problem ( ). faculty CS-Dept 20
c. This problem can be solved through improvements in: manage- student 7
ment ( ) . technology ( ).. . both ( )... neither ( ) ... not program rer/soituare analyst 21
engineer/functional anal yst 9
a problem ( ).
quality assurance/technical 2
d. How would (did) you solve this problem? director
manager/supervisor 45
government employee 24
PhD 17
Fig. 3. Sample survey question. consultant 10
establishes general software a
development policy
choice of the 30/40/30 figures is somewhat arbitrary, but does, national author/speaker 24
affiliated with CS professional 36
we believe, reflect the notion of "consensus" fairly well. We organization
have provided the percentage figures in the tables presented in affiliated with aerospace
professional organization
14

this section so that if the reader is unhappy with our classifica- American/Canadian influence 96
tion scheme, the information to construct a new scheme is European influence 6
available. Far East influence

The 30/40/30 rule can also be applied to parts B and C of the Employer or Firn (only one applies)
questionaire. Parts B snd C, which refer to the problem type manufacturer of computer hardware 15
and solution type respectively, were evaluated to determine manufacturer of other than
computer hardware
12

whether the problems and solutions are managerial, technical, software house 3
both, or neither. If managerial has a weighted average of at engineering services and technical a
support organization
least 70 percent, then the problem (solution) type is consid- g over nment 23
ered to be managerial; if fewer than 30 percent felt that the university/RZ&D laboratory 31
proposition is a managerial problem (solution), then propo-
the computer service
consulting firn
bureau
4
sition is considered to be technical; if the split between mana- financial institute
gerial and technical falls in the center 40 percent, then we medical/legal service
conclude that the problem (solution) has both a strong mana- utilities
gerial and a strong technical character and cannot be character-
ized as either managerial or technical alone. The weighted
managerial average is defined to be and DoD officials, to legendary figures in the computing field,
and project managers and programmer/analysts on a software
NR MGT + 0.5 (NR BOTH) project. Many respondents publish regularly in leading tech-
NR MGT + NR TECH + NR BOTH nical journals and are highly sought-after speakers at confer-
ences in the the U.S. and abroad. The authors of well-known
where "NR MGT" is the number of surveyees that answered textbooks and leading researchers both within the university
part B (C) with "management," "NR TECH" is that number system and at R&D laboratories were well represented. Al-
that answered "technical,"' and "NR BOTH" is the number though the survey participants can be considered a cross section
that answered "both." of the computing community (see Table II), by design the em-
Two hundred and ninety-four survey replies were received phasis was to obtain those individuals who through visibility or
in all including those mailed to specific individuals and those position sway the opinions of many others working in software
returned from handouts at conferences and the classroom. engineering or other aspects of data processing management.
Approximately 25 percent of those specifically addressed to Of course, the individual names of the respondents are protected
an individual were returned. Respondents ranged from high by a promise of confidentiality.
ranking decision makers in computer science, senior corporate One of the more interesting possibilities in analyzing the
THAYER et al.: SOFTWARE ENGINEERING PROJECT MANAGEMENT 337

TABLE III
CATEGORIES OF PARTICIPANTS IN SURVEY
PARTICIPANT CATEGORY DEFINITION OF CATEGORY

Project managers project managers of a project

which include software line
managers, each of whom had a
first hand knowledge of the major
problems of SEPM.

Project individuals individual programmers/analysts

who worked on projects.

Technical leaders people who are in a high position

of influence in their company's
data processing function; highly
visible authors or speakers on
computer science, particularly
data processing management; and
authlors of texts, papers, or re-
ports on project management.

R?,D personnel people who through their business

or avocation were interested in
furthering the state-of-the-art
in project management, or in
perhaps a few cases, some other
aspect of computer science.
Educators usually university professors;
however, some non-university edu-
cators were included.

TABLE IV
SUMMARY OF PART A RESULTS-IMPORTANCE OF PROBLEM
INDEX ?IAJOR ISSUE RESULTS IN P,.RCENT PROBLEM BY PARTICIPANT CATEGORY
NR (Short Title) COlfPOSITE PROJ MGRS PROJ INDS TECH LDRS R&D EDUCAT0RS
NUSBER REPORTED 294 72 62 82 6C 77

1 plan requirements 97 100 97 96 9!, 93

2 plan success 82 75 86 83 SS 80
3 plan project 90 92 85 89 91 92
4 plan cost 88 92 86 84 8 95
5 plan schedule 94 97 93 91 92 95
6 plan design 72 71 67* 66X 72 75
7 plan test 79 85 77 73 69* 86
8 plan maintainability 67- 68* 64* 6:* 7-
9 plan warrantv 74 72 70 72 70 78
10 plan control 61* 54* 60* 54* 63* 69*
11 organize type 46* 51* 40* 42* 44* 48*
12 organize accountability 81 75 72 86 89 88
13 staff project manager 77 73. 82 72 70 77
14 direct techniques 59* 58* 50* 48* 57* 66*
15 control visibility 58* 65* 62* 71 77 74
16 control reliability 85 90 84 78 79 86
17 control maintainability 76 76 67* 71 82 82
18 control goodness 62* 60* 62* 60* 56* 65*
19 control programmers 78 77 70 78 81 84
20 control tracing 67* 67* 55* 70Q 69* 68*

* Results were inconclusive

survey results was to determine how different groups of people solutions to the problems in English prose, is presented in the
answered the individual questions. To do so, the participants next section.
were divided into the five categories shown in Table III. These For part "A" of the survey, each respondent was asked to
five groups were selected because of their potential for conflic- judge the relative importance of the hypothesized problem.
ting views. Do project managers view the major issues ofSEPM For all groups at least 30 percent believed each proposition to
differently than project individuals? Do R&D personnel hold be an important problem, so that none of the 20 issues can be
different beliefs than project managers? Do universities recog- discounted. For the six categories in Table IV there are a total
nize the major problems so they can direct their teaching and of 120 figures shown (6 columns X 20 figures per column). Of
research to those areas? And finally, what do the technical these 120, only six fall below 50 percent, and only 15 fall be-
leaders believe? low 60 percent. When all respondents are combined (COM-
The results of the survey are summarized in Tables IV-VI POSITE column), 13 of the 20 issues were classified as definite
along with the applications of the 30/40/30 rule. Each major problems, while seven were inconclusive. These numbers varied
survey group is scored separately so that differences of opinion somewhat across the five subgroups. Project managers also
between groups can be observed. Parts "A," "B," and "C" are classified 13 issues as definite problems. For project individuals
all shown separately. Part "D," in which respondents offered the number was 11, for technical leaders it was 14, for research
338 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-7, NO. 4, JULY 1981

TABLE V
SUMMARY OF PART B RESULTS-NATURE OF PROBLEM

INDEX MAJOR ISSUE RESULTS IN PERCENT MANAGEMENT PROBLEM BY PARTICIPANT CATEGORY

NR (Short Title) COMPOSITE PROJ MGRS PROJ ITEDS TECH LDRS R&D EDUCATORS

1 plan requirements 64 69 68 56 56 61
2 plan success 70* 74* 71* 69 67 59
3 plan project 87* 89* 83* 90* 86> 80*
4 plan cost 71* 76* 64 75* 73* 70*
5 plan schedule 68 71* 69 68 69 68
6 plan design 45 50 52 40 34 32
7 plan test 41 49 48 31 32 32
R plan meintainability 44 47 46 45 44 52
9 plan warranty 52 47 47 43 50 54
10 plan control 79* 86* 85* 80* 80* 66
11 organize type 92* 94* 95* 88* 92* 79*
12 organize accountability 94* 97* 97* 93* 94* 89*
13 staff project manager 95* 98* 96* 94* 94* 90*
14 direct techniques 89* 86* 89* 88* 90* 88*
15 control visibility 74* 78* 80* 73* 66 57
16 control reliability 23± 22'± 27-. 22t 18i 20`
17 control maintainability 36 42 43 32 30 38
18 control goodness 27' 30 23L 25. 25-± 33
19 control programmers 65 62 75* 64 66 60
20 control tracing 62 67 75* 60 59 50

* Management problem.
t Technical problem.

TABLE VI
SUMMARY OF PART C RESULTS-NATURE OF SOLUTION

INDEX tAJOR ISSUE RESULTS IN PERCENT MANAGEMENT SOLUTIONS BY PARTICIPANT CATEGORY

NR (Short Title) COIPOSITE PROJ MGRS PROJ INDS TECH LDRS R&D EDUCATORS

1 plan requirements 61 64 67 53 59 55
2 plan success 66 70* 65 66 62 59
3 plan project 84* 86* 87* 87* 81* 80*
4 plan cost 66 .67 66 65 65 65
5 plan schedule 64 70* 63 63 62 6S
6 plan design 45 50 47 40 35 25
7 plan test 40 48 42 36 33 33
8 plan maintainability 45 48 51 47 44 50
pS warranty 50 53 51 43 47 51
10 plan control 76* 81* 81* 78* 82* 68
11 organize type 91* 93* 95* 88* 91* 85*
12 organize accountability 92* 95* 96* 90* 91* 89*
13 staff project manager 96* 97* 96* 94* 96* 91*
14 direct techniques 83* 80* 80* 85* 86* 89*
15 control visibility 72* 78* 77* 70* 60 59
16 control reliability 24t 22t 33 23± 22+ 20±1
17 control maintainability 36 42 46 33 29± 39
18 control goodness 22- 26± 26, 21, 20±l 22±1
19 control programmers 61 61 66 59 61 59
20 control tracing 59 65 67 56 53 53

* Management solution.
T Technical solution.

and development personnel it was 14, and finally educators to be important problems. A full 15 of these issues were so
felt that 15 of the issues were actual problems. categorized by them. In no case were the educators unsure of
Project individuals overall categorized only 11 of the issues the importance of an issue for which the overall populace was
as definite problems versus 13 for the total set of survey parti- decided. Educators believed that Issues 8 (plan maintainability)
cipants. In no case did the programmers feel that an issue was and 15 (control visibility) were important problems even though
definitely an important problem when the overall populace was the general populace was inconclusive about them. In fact, the
unsure. Project individuals also inconclusively judged Ques- educators and R&D personnel were the only groups who did
tions 6 (plan design) and 17 (control maintainability) while believe that planning for maintenance is an important problem.
the overall populace labeled these as definite problems. We Perhaps this indicates why commercial software is so hard to
find it very intriguing that programmers who must live with maintain. Industrial personnel at all levels are not convinced
the consequences of poor designs and must maintain badly that planning for maintanence is an important problem. It is
developed software did not conclusively feel that these two interesting to note that instead of portraying an ivory tower
issues were important problems. We cannot, of course, with attitude of indifference to "real world" problems, academicians
the data obtained, account for these feelings. seem even more concerned with these issues than do the com-
Educators, more than any other group, believed these issues mercial software personnel who must live with them daily!
THAYER et al.: SOFTWARE ENGINEERING PROJECT MANAGEMENT 339

Perhaps the most interesting group is the project managers, TABLE VII
COMMON SOLUTIONS TO MAJOR PROBLEMS OF SOFTWARE ENGINEERING
those who presumably carry out SEPM functions on a daily PROJECT MANAGEMENT
basis. Their perspectives agree with the overall populace in
every case. INDlEX SOLUTIONS
Overall, planning activities seem to be almost universally seen A Use or enforce (c-xisting) . . . standards, procedures,
as being the most important problems. The first ten problems and documenLaLion.
all deal with planning activities. For seven of these, all groups B Define and conduct R&D in
C Use (existing) software engineering Lechiniques anid
unanimously agreed that they are important problems. Eight tools.
out of ten planning problems are accepted as important by the D Improve or insure requirement specifications to in-
overall populace. clude
In part "B" respondents indicated the nature of the problem, E Educate or train project managers in
F Use competent, experienced software development man-
managerial, technical or both, while in part "C" they indicated agers.
whether the solution would be managerial, technical, or both. C Analyze . data from prior software developmcniiLs Lu
determine best method of
Table V shows in detail how each group responded to the ques-
II Use (existing) software engineering project managemlent
tions of part "B," while Table VI shows the corresponding methods and tools.
detail for part "C." I Plan and manage or develop . . . as part of project
The overall populace thought that only nine of these prob- management plan . . . .
J Educate, inform, or involve (top) management in
lems were definitely management problems, thought that nine K Review or audit
mixed both management and technology, and classified the L Use extensive tesLt techniiques and tools.
other two as technical problems. For project managers this NI Select and define the correct (or best) organizational
changed to 10, 9, and 1, respectively, while for the other groups stLructure (or test team, or development team, or.
for project environment and Lecthniques.
the corresponding figures are: project individuals-10, 8, 2; N Involve user or increase communication between user
technical leaders-8, 10, 2; research and development-7, 11, and developer in . . .
2; and educators-6, 13, 1. Hence, it seems as while we are 0 Divide project or task into programs, miodules, or sub-
tasks and
correct in stating that these 20 hypothesized problems are in- P Educate or trairt software developers in
deed problems, the general populace and specific subgroups do Q Use a project control system to
not all agree that these problems are managerial. In fact, there K Use good descriptive documentation techniques.
was unanimous agreement among all groups that controlling S Use configuration management or change control pro-
reliability is a technological not a managerial problem, and cedures.
T Define or consider quality in terms of deliverables,
nearly unanimous agreement about controlling the "goodness" i.e., reliabiliLy, maintainability, etc.
of code. Hence, it seems that we have validated nine problems U Use competent, experienced software developers.
as being primarily managerial, nine problems as having mixed V Allow sufficient time for .
elements of both management and technology, and two prob- W Use existing, conimiercial software system to

lems as being technological rather than managerial. X Use anr (autLomatic) reporting or tracking system.
Y Use any method, thc resulLs are not sensitive to
Educators and R&D personnel, who develop technical solu- Other.
z
tions to problems as part of their normal job activities, seemed
more likely to see at least some technical aspects in these prob-
lems than the other groups. Project managers, on the other
Only for Question 15 (control visibility) was there disagree-
ment by more than one group.
hand, who deal with these problems from a managerial perspec-
tive, seemed more likely to view them as being managerial in V. PROPOSED SOLUTIONS
nature. However, for no group was the shift in opinion very Part "D" of the questionnaire asked respondents how they
dramatic. Interestingly enough, project individuals were the would or did solve the stated problem. Almost 4500 individual
only group who felt that controlling programmers is a mana- answers were given by the 294 people who returned the survey.
gerial problem. Because these answers are in free-format English, it is impos-
When we look at part "C" in which people expressed their sible (and pointless) to list all of them separately. Instead we
beliefs in whether these problems can be solved through im- created a number of categories and grouped answers into those
provements in management or technology, there is an analogous categories as shown in Tables VII and IX. The categories were
trend towards improvement in management. For the overall constructed from a careful examination of the answers, and
populace, seven problems were felt to be solvable by improve- were not predefmed or created independently of the survey.
ments in management, while 11 were mixed between manage- The survey question form left only a small amount of room
ment and technology. It was felt that' two problems could be for a response to part "D." Hence, all of the responses are, un-
solved by improvements in technology alone. Project managers fortunately, quite vague. However, we feel that they still pro-
had somewhat stronger beliefs that improvements in manage- vide insight into what many people believe are solutions for
ment would solve 9 of the 20 problems. This number drops to the 20 major issues of SEPM. Table VIII shows some statistics
seven for project individuals and technical leaders, to six foron the more common solutions proposed. The questions are
research and development personnel, and to just filve for edu- grouped into the five management acttivities of planning, or-
cators. The consensus of opinion across groups is strongest ganization, staffing, directing, and controlling, as was done in
here. For 14 of the 20 problems, all groups scored the same. Table I. The figure for a particular solution category and man-
340 30IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-7, NO. 4, JULY 1981
TABLE VIII TABLE IX
STATISTICS ON COMMON SOLUTIONS UNIQUE SOLUTIONS TO MAJOR PROBLEMS OF SOFTWARE ENGINEERING
PROJECT MANAGEMENT
SOLUTION PCT OF TOTAL SOLUTIONS PER MGT FUNCTION
INDEX PLAN ORGN STAFF DIR CONTR TOTAL
PR{OBLEM WEIGHT SOLUTION TO PROIBLEM
A 11.3 16.2 7.0 28.2 22.7 15.4
Plan 9 Use it.eration of the requirements with solu-
B 10.8 2.7 5.5 10.7 16.0 11.1 Requiremcnts tions.
C 6.6 9.0 6.0 Plami 3 liequirv management to define objectives in
O 8.6 3.4 s .s Success terms f4 qjuality desired
1 4.3 5.0 17.2 14.6 2.5 s .0 2 Establ :.sh success priority criteria
F 3.9 6.9 16.4 9.7 2.6 4.8 Plan 5 Develop a truithful, accurate cost accepted
Cost by management/customers and manage to it
G 3.9 10.7 5.8 4.1
2 Allow for contingencies
H 3.9 6.4 3.8
Plan 5 Develop a truthful, accurate schedule accep-
I 4.5 1.9 0.9 2.8 Schedule ted by management/customers and manage to it
J 3.6 13.3 v 0.1 2.6 5 Allow for contingencies
K 1.5 3.1 5.2 2.4 Plan 1 Performi alternative analysis to select best
L 3.3 o.6 1.9 Design techniques and tools
M 0.2 16.2 0.3 1.9 Plan Test 4 Design better software (to be tested)
N 2.8 0.8 0.1 i.6 Plan 4 Use follow-on maintenance support (include
O 1.6 2.2 1.4
Warranity agreeing on price before developing software)
1 Use fellow-on maintenance contract
P 2.0 1.0 1.4
1 Make it a legal problem not a software prob-
Q 1.4 2.3 0.1 1.0 lem
R 0.5 1.7 0.7 Organize 11 Use organizational structure directed by top
s 1.1 0.G Type management (or staff)
T 0.8 O.4 O.6 7 Use a project organization
U 1.0 0.5 6 Use a matrix organization
V 0.7 0.4 0.5 3 Use existing organizational structure
W 0.8 0.4 3 Let the project manager select the organiza-
tion type
X 0.3 0.7 0.4
I Give project manager full authority for pro-
Y 0.1 0.8 -- 2.9 0.1 0.3 ject
z 20.3 47.3 37.5 23.3 17.3 23.3 Organize 19 Make slpecific assignmlent of work to software
Aceouit devclopers (using WBS)
11 Define clearly each work package
agement activity indicates what percentage of the total solutions 1 Give project manager full authority for pro-
ject
presented for that activity fell into that solution category. For Staff Project 13 lave a selectLion process that selects good
example, 11 percent of the solutions for "planning" activities Manager and eliminates poor project managers
could all be classified as category A, i.e., "use or enforce (exist- 9 Use a selection criteria that is based on1
manageinent abilities, not technical abili-
ing). . . standards, procedures, and documentation." Many of ties (or availability)
the more interesting solutions which did not cross over several 4 Establish job performance standards
issues are shown in Table IX. 2 Promote software developers (who have denmoni-
strated competence as managers)
Most of the categories listed account for only a small percent- 2 Have available a pool of project managers to
age of the total answers. However, in a few cases, a sizeable sclect fromn
percentage of the total number of answers falls into one cate- Direct
Techniques
8 Use best judgment (individual problem)
gory. For example, 28 percent of the solutions related to 1 Promote successful software engineers to
project m-anagers
directing questions essentially said to use or enforce standards, Control 2 Just get the programs to work (reliabilitLy
procedures, and documentation. Twenty-three percent of the Reliability not imn ortant)
solutions for control problems fell into this same category. Control 4 Just get the programs to work, that is suf-
Goodness ficient
Fifteen percent of the solutions for problems of directing fell Control 5 Develop and use job performance standards
into category H, i.e., educating and training project managers. Programmers for software developers
Interestingly, category B, (define and conduct R&D in an area) 2 Measure the results of the -software develop-
ment
never exceeded 16 percent.
2 Keep records on productivity (eliminating
The general conclusion to be drawn from the multitude of low pr-oducers)
answers is that there is no consensus today as to how to solve
these problems, although a sizeable percentage of the respon-
dents occasionally supported a single solution for a single VI. UNIVERSITY PROGRAMS ON SEPM
managerial activity. The fact that there is no consensus may in To augment the data collected in the original survey, modified
part be due to the unstructured format in which respondents questionnaires were mailed to over 100 universities including all
stated their solutions. Perhaps if we had specifically listed a institutions which grant Ph.D. degrees in computer science, and
number of options, a stronger consensus might have been pos- all computer science departments in the U.S. These question-
sible. The survey does show that, at least without a list of aires sought the opinions of professors on the same twenty
candidate solutions specifically presented, software engineering issues, but more importantly attempted to determine how much
personnel do not generally agree on how to solve these major emphasis is placed on these issues in classroom instruction.
problems. The response rate was a somewhat disappointing 20 percent.
THAYER et al.: SOFTWARE ENGINEERING PROJECT MANAGEMENT 341

Follow-up phone calls to some of the schools which did not re- TABLE X
IMPORTANCE VERSUS COVERAGE
spond revealed that in some cases schools did not return the
questionaire because they do not offer any courses in which INDEX MAJOR ISSUE MEAN MEAN A - B
:Ni; (short title) "A " "&I
SEPM is covered. A further examination of course catalogs
for over two dozen universities which did not respond showed 1 plan requirements 1.4 1.8 -0.4
that in only a handful of cases do these universities offer any 2
3
plan success
plan project
1.9
1.6
2.9
2.1
-1.0
-0.5
courses which could be identified from the catalog descriptions 1.8 2.4 -0.6
4 plan cost
as pertaining to the principles of SEPM. Although this survey 5 plan schedule 1.9 2.4 -0.5
-0.9
is incomplete, it appears from the data gathered that few 6 plan design 2.2 3.1

schools offer courses on software engineering principles beyond 7

a
plan test
plan mainta-inability
1.9
1.8
2.1
1.8
-0.2
0.0
those of "structured programming." Less than 10 percent of 9 plan uarranty 1.6 2.8 -1.2
the schools offer any courses which present a substantial amount 10 plan contr-ol 2.2 3. 1 -0. 9
of material on SEPM. 11 organization type 2.6
2. 1
3.4
2.9
-0.6
-0.8
12 organization accountability
The questionaires were "course" oriented. Each department
-1.1
was asked to identify its software engineering courses. No def- 13
14
staff project manager
direct techniques
2.1
2.2
3.2
3.1 -0.9
inition of "software engineering" was provided. It turned out 15 control visibility 2.5 2.8 -0.3
that there was wide variation in how that term was interpreted. 16 control reliability 1.6 2.6 -1.0
-0.3
Some institutions interpreted virtually every software related 17
18
control naintainability
control goodness
1.9
2. 2
2.2
2. 1 +0. 1
course as being a software engineering course, beginning with 2. 1 3.0 -0. 9
19 control programmers
introductory programming. Other departments only classified 20 control tracing 2.5 3.0 -0.5
those advanced courses dealing with "software reliability,"
"software testing," and other more specialized topics as falling
under software engineering. a disparity exists between the believed importance of an issue
For each software engineering course, the course content was and the amount of coverage it receives in the classroom. While
identified, allowing us to determine whether any time was this sampling is small, we feel it provides some insight into the
spent discussing SEPM. Those classes in which SEPM was not major issues of SEPM. There were three primary reasons sited
addressed are ignored in the data shown here. Only courses for not covering these issues more:
which have identifiable components dealing with SEPM are 1) lack of expertise,
reported. Hence, courses dealing only in "structured pro- 2) lack of texts and other teaching materials, and
gramming practices" are not counted. 3) inappropriate for computer science departments.
For each software engineering course in a department's cur- Some of the professors stated that they had little or no
riculum, a separate questionnaire was completed which con- managerial experience. As such, they felt that they lacked the
tained information pertinent to that course. The 20 problems expertise required to teach about SEPM. Instead, they concen-
were identical to those sent to the original group. However, trated on the more technical issues which they were more
instead of being asked to answer four parts for each problem, comfortable with and in which they had industrial experience.
the professor answered only two. Part "A" was the same as Others cited a lack of suitable textbooks in this area, arguing
before. This allowed us to gauge the relative importance which that for undergraduate classes it is very difficult to teach from
the professor attributed to each issue. Part "B" specifically a collection of notes or papers, and that it is possibly more ef-
asked to what extent this problem was covered in the class. The fort on their part than it is worth to create such a collection.
four options were: "a great deal," "somewhat," "very little," The reason they felt there were no textbooks which adequately
and "not at all." Almost without exception, professors seemed handled the material was because the area is so new and because
to feel that the problem is more important than the coverage these issues indeed are problems so there are no solutions to
actually allocated to it in the classroom. If we accept the idea write about! Perhaps the most interesting reason cited for not
that critical problems should be covered a great deal in class, teaching more SEPM material is that some professors felt the
important problems covered somewhat, problems which are material was not appropriate for a computer science depart-
not important covered very little, and issues which are not ment. Because the material is managerial, these professors
problems should not be covered at all, then a natural corre- believed that it should be taught in business schools within the
spondence between the opinions of part "A" and the coverage university. When asked whether, in fact, this material was cur-
indicated in part "B"-emerges. This correspondence is shown rently being taught in the business school on their campus, the
in Table X. A negative number in the column labeled "A-B" answer was invariably no.
indicates that an issue is inadequately covered by the professor's
own admission. The more negative the number, the more in- VII. SUMMARY AND CONCLUSIONS
adequate the coverage. To compute the numbers shown, the Software engineering project management has been the focus
answers of part "A" were numbered from 1 (for critical) to 4 of much recent attention because of the enormous penalties
(for not a problem), while those of Part "B" were numbered incurred during software development and maintenance result-
similarly (1 = a great deal, 4 = not at all). If a professor felt ing from poor management. To date there had been no com-
the question was improperly stated, his response is not counted prehensive study performed to determine the most significant
in the table. issues of SEPM, their relative importance, or the research direc-
We phoned a number of the respondents to find out why such tions necessary to solve them. We conducted a major survey of
342 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-7, NO. 4, JULY 1981

individuals from all areas of the computer field to determine Interagency Committee on Automat. Data Processing, June 13-
the general consensus on SEPM problems. 14, 1978.
[14] P. F.W. Keen and E. M. Gerson, "A politics of software system
Twenty hypothesized problems were submitted to several design," Datamation, vol. 23, pp. 80-84, Nov. 1977.
hundred individuals for their opinions. We believe that the [15] S. P. Keider, "Why projects failed," Datamation, vol. 20, pp. 53-
experimental evidence supports the conclusion that at least 55, Dec. 1974.
13 of the 20 hypothesized problems actually are important [16] B. W. Kernighan and P. J. Plauger, The Elements of Programming
Style. New York: McGraw-Hill, 1974.
problems which face software engineering project managers. [17] -, Software Tools, Reading, MA: Addison-Wesley, 1976.
None of the other seven issues has been discounted, but opin- [18] P. J. Klass, "NORAD data system has 100% overrun," Aviation
Week Space Technol., vol. 109, pp. 61-63, Oct. 30, 1978.
ion is more mixed on their relative importance. [19] K. Kolence, "Software engineering management and methods"
A number of potential solutions or research directions were (Discussion), in Proc. NATO Conf., Software Eng. Concepts and
indicated by the respondents which, if followed, the respon- Techniques, P. Naur, B. Randell, and J. N. Buxton, Eds. New
York: Petrocelli/Charter, 1976, p. 13.
dents believe would lead to solutions for these problems. The [20] M. Lipow and T. A. Thayer, "Prediction of software failures," in
task facing industry, government, and the universities is to solve Proc., 1977 Annu. Reliability and Maintainability Symp., 1977,
these problems and to disseminate these solutions for the wide- pp. 1-6.
[21] J. H. Manley, "Embedded computer system software reliability,"
spread benefit of the computing field. Defense Management J., vol. 1 1, p. 13-18, Oct. 1975.
[22] R. McCarthy, "Applying the technique of configuration manage-
ACKNOWLEDGMENT ment to software," Defense Management J., vol. 11, pp. 23-28,
Oct. 1975.
Programmer and analyst support was provided by: B. J. [23] R. E. Merwin, "Software management: We must find a way"
Nieland, L. M. Hanger, R. D. Heckler, G. Collins, J. W. Robino, (Guest Editorial), IEEE Trans. Software Eng., vol. SE4, pp.
L. A. Morris, D. E. Sturdevant, and L. H. Jones. 307-308, July 1978.
[24] B. Miller, "Avionics problems bar debute of F15 with TAC,"
Secretarial, proofreading, and composing support was pro- Aviation Week Space Technol., vol. 103, pp. 23-25, Dec. 1975.
vided by numerous people: B. E. McPheeters, T. Hamilton- [25] P. Naur, B. Randell, and J. N. Buxton, Eds., Software Engineering:
Ricketts, M. L. Mueggenburg, W. Antwiler, and H. Antwiler. Concepts and Techniques. New York: Petrocelli/Charter.
[261 J. L. Ogdin, "Designing reliable software," Datamation, vol 18,
Editing, proofreading, and overall grammatical corrections pp. 71-78, July 1972.
were provided by M. Smith and S. R. Green. [27] A. J. Perlis, "Software engineering education" (Discussion), in
Proc. NA TO Conf., Software Eng. Concepts and Techniques, P.
REFERENCES Naur, B. Randell, and J. N. Buxton, Eds. New York: Petrocelli/
Charter, 1976, pp. 199-206.
[1] J. D. Aron, "Estimating resources for large programming systems," [28] Rome Air Development Center, unpublished R&D program in
in Proc. NA TO Conf. Software Eng. Concepts and Techniques, P. software cost reduction (FY), 1977.
Naur, B. Randell, and J. N. Buxton, Eds. New York: Petrocelli/ [29] S. R. Ruth, "What can the Navy learn from ALS?," unpublished
Charter, 1976, pp. 206-217. document, 1974.
[2] R. H. Austing, B. H. Barnes, D. T. Bonnette, G. L. Engel, and G. [30] Information Processing/Data Automation Implications of Air
Stokes, Eds, "Curriculum '78," Commun. Ass. Comput. Mach., Force Command and Control Requirements in the 1980's (CCIP-
vol. 22, Mar. 1979. 85), Highlights, vol. 1, SAMSO Tech. Rep. 72-141, Apr. 1972.
[3] Statement of work, "Reliable software," BMDATC, Huntsville, [31] J. I. Schwartz, "Analyzing large-scale system development," in
AL, Rep. SW-A-44-74, Sept. 25, 1974. Proc. NATO Conf, Software Eng. Concepts and Techniques,
[4] , "Data processing system requirements," BMDATC, Hunts- P. Naur, B. Randell, and J. N. Buxton, Eds. New York: Petrocelli/
ville, AL, Rep. SW-A-88-75, Dec. 9, 1974. Charter, 1976, pp. 260-275.
[5] B. W. Boehm, "Software engineering," IEEE Trans. Comput., [32] J. B. Slaughter, "Understanding the software problem," in Proc.
vol. C-25, pp. 1227-1230, Dec. 1976. Symp. High Cost of Software, J. Goldberg, Ed. Menlo Park, CA:
[6] -, "Software and its impact: A quantitative assessment," The Stanford Res. Inst., Sept. 17-19, 1973, pp. 41-52.
Rand Corp., Santa Monica, CA, Rep. TR-P-4947, Dec. 1972. [33] J. M. Spier, "Software malpractice-A distasteful experience,"
[7] B. W. Boehm, J. R. Brown, and M. Lipov, "Quantitative evalua- Software-Practice and Experience, vol. 6, pp. 293-299, 1976.
tion of software quality," in Proc. 2nd Int. Conf. Software Eng., [34] R. H. Thayer, 'The Rome Air Development Center R&D program
IEEE Comput. Soc., Oct. 1976, pp. 592-605. in computer languages and software engineering," Rep. RADC
[8] F. P. Brooks, Jr., 'The mythical man-month," Datamation, vol. TR 74-80, Apr. 1974.
20, Dec. 1974. [35] D. A. Walsh, "Structured testing," Datamation, vol. 23, pp. 111-
[9] J. D. Cooper, "Corporate level software management," IEEE 118, July 1977.
Trans. Software Eng., vol. SE-4, pp. 319-326, July 1978.
[10] B. C. DeRoze, "DOD defense system software management pro-
gram" (Letter), Office of the Assistant Secretary of Defense, Richard H. Thayer (S'61-M'63-SM'72-S'76-M'77-S'78-M'78), photo-
Washington, DC, Mar. 1, 1976. graph and biography not available at the time of publication.
[11] B. C. DeRoze and T. H. Nyman, "The software life cycle-A
management and technological challenge in the Department of
Defense," IEEE Trans. Software Eng., voL SE-4, pp. 309-318,
July 1978. Arthur B. Pyster (M'76), photograph and biography not available at the
[12] Findings and Recommendations of the Joint Logistics Com- time of publication.
manders Software Reliability Work Group, vol. 1, Executive
Summary, Nov. 1975.
[13] Invited workshop "Organizing ADP projects," cosponsored by: Roger C. Wood (M'68), photograph and biography not available at the
Nat. Bureau of Standards, IEEE Comput. Soc., and Federal time of publication.