Software Standards:

Their Evolution and Current State

Reed Sorensen
TRW

This article touches on the last 30 years of software standards development in the Department of
Defense, the purpose of and the difference between formal and de facto standards, the Ada expe-
rience, some current thinking regarding the documentation monster, and an update on J-STD-
016 status.

“Stuff Happens” — so do Standards • a more complete life cycle perspective with links to system
If you sit down and try to categorize all the types of standards, engineering
the list gets really long really fast [1]. Under each type you start • better consistency with modern development models
to come up with multiple entries. Standards seem to proliferate (incremental, evolutionary, reuse/re-engineering)
and infiltrate any aspect of human endeavor that involves any • improved accommodation of nonhierarchical design
degree of complexity. Standards are not inherently good or bad, methods
they just seem to inherently “be” — meaning that whether they • more flexibility with documentation formats and media
are developed formally by a big, slow-moving international • attention to reusability, security, safety, and risk manage-
committee or whether they grow spontaneously from an innova- ment
tive free market, standards are not going away. • improved references to metrics and indicators
While standards as a life-form will survive, evolving a spe- • more responsive in-process evaluation and review activities
cific species can be (and probably always is) difficult. Standards • significantly improved transition to software support and
have to be cared for, they have to be fed, they have to evolve or sustainment activities
they become extinct. Two examples:
1. Something as basic to the military as the ability of joint Meanwhile, IEEE/EIA 12207 provides a higher-level view,
task forces to share standardized location data, just “ain’t” with fewer specific requirements, and spans not only develop-
that easy [2]; ment, but acquisition, supply, operation, and maintenance.
2. J-STD-016 has been going through the revision and ballot
process for more than a year and should be undergoing
re-balloting as you read this, but delays in this process have
Table 1. Chronological list of some major software and software related stan-
threatened its existence. dards.

Software Standards Date Designator Topic

1968 MIL-STD-490 Specification Practices
Evolved Over 30 Years 1970 MIL-STD-483 Configuration Management Practices
Table 1 is a representative list of standards. Common school of 1978 DOD-STD-480A Configuration Control
1979 MIL-S-52779A Quality
thought has software standards evolving from the black ooze of 1983 MIL-STD-1815A Ada83 Language Reference Manual
hardware standards of the 1970s to early ’80s. Some hardware 1983 DOD-STD-7935 Documentation Standards
standards began including words, sometimes in the form of 1983 DOD-STD- Software Development
1679A
appendices, to deal with software, e.g. MIL-STD-483 [3] and 1984 DOD-STD- Trainer System Software Development
MIL-STD-490 [4]. Later, software-specific standards evolved 1644B
1985 MIL-STD-490A Specification Practices
that talked like software animals using software terminology, but 1985 MIL-STD-1521B Technical Reviews and Audits
walked liked hardware animals because they were direct hard- 1985 DOD-STD-2167 Defense System Software Development
ware descendants, e.g. DOD-STD-2167 [5]. Today, the soft- 1988 DOD-STD- Documentation Standards
7935A
ware community has troubled over the differences between soft- 1988 DOD-STD- Defense System Software Development
ware and hardware enough that software standards (EIA/IEEE 2167A
J-STD-016 [6], IEEE/EIA 12207 [7]) are genuine software 1988 DOD-STD-2168 Defense System Software Quality
1992 MIL-STD-973 Configuration Management
standards rather than feathered lizards. 1994 MIL SPEC REFORM
Both MIL-STD-498 and J-STD-016 promote a more flexi- 1994 MIL-STD-498 Software Development and Documentation
ble approach to defining the software process and collaboration 1995 J-STD-016- Software Life Cycle Processes, Software
1995 Development, Acquirer-Supplier Agreement
among all stakeholders, e.g. with joint technical and manage- 1998 IEEE/EIA 12207 Software Life Cycle Processes
ment reviews. Compared to older standards, MIL-STD-498 and 1998 EIA-649 Configuration Management
1999 J-STD-016 Software Life Cycle Processes, Software
J-STD-016 provide: Development

December 1999 C ROSSTALK The Journal of Defense Software Engineering 21

Evolution of Software

Standards Enable Communication process, then customize that process for individual develop-
Standards are about communication — communication ment projects. The Department of Defense (DoD) Single
between organizations, e.g. between the acquirer and developer, Process Initiative further encourages developers to define and
among developers, between the developer and the maintainer, utilize its own processes.
communication between two commercial-off-the-shelf software A grassroots approach to standards is observed in the devel-
products or two software units, and communication between a opment tools, platforms, and graphical user interfaces that pro-
fax machine built by Canon and another built by Sharp. liferate in the marketplace. Resulting de facto standards obvi-
The revolution in knowledge that accompanied the printing ously influence development. A software development organiza-
press was a communication revolution. The printing press pro- tion may be a “Windows shop” or a “Unix shop” or “Linux
vided the means to build upon the experience of those who shop.”
came before1. A standard is either a specialized document that
captures the experience of the many for the use of those who The Influence of the Ada Experiment
follow, or it is a groundswell that, by virtue of inertia, establish- The Ada experiment caused some to step back and consider the
es default experience. In the first case, many may meet in com- purpose of requirements and design. You do not just sit down
mittee and have a balloting process to gather experience from and start coding in Ada. It takes considerable work to get to a
the broadest base of useful knowledge. In the second, a compa- piece of Ada code that can be compiled, so you end up thinking
ny may flood the marketplace with its solution to a problem in a lot about the design. This thinking is indispensable in large
hopes that it will become a de facto standard — not that the systems.
solution is inherently the best but if the solution becomes Systems developed in Ada are arguably easier to maintain
omnipresent it will provide the most effective mechanism for than had they been developed in C or another language less ver-
most people to deal with most others (Microsoft Windows 3.1 bose than Ada. Because Ada is verbose it is easier for a main-
for example). tainer to read than many languages, such as C, which are
designed to be easy to write rather than easy to read. Ada is
Formal Standards Influence designed to be easy to read.
Software Development Did Ada influence the object-oriented community? While
it is hard to measure such an influence, clearly Ada was the first
Entire cottage industries pop up around formal standards. There
language that enforced rather than simply permitted encapsula-
are a lot of consulting organizations offering CMM® and ISO
tion and abstraction. You can argue that Java was influenced by
90002 expertise. Seminars are available and tools are marketed to
Ada by observing that Java is C++ syntax with Ada semantics.
support “498” [8] and “12207.” All of these kinds of free market
Ada’s contributions to software engineering aside, it was not a
ventures do their best to influence software development.
public relations success for the standards community. The 1986
But not everyone buys into the concept of formal stan-
mandate that Ada be used in new defense systems left a bad
dards. Formal standards tend to be imposed by management
taste for many software development organizations, an aftertaste
rather than because the programmers sign a petition demanding
that remained even when free Ada compilers, trained Ada pro-
that they want to be CMM Level 3, though life for the pro-
grammers, and Ada development tools became plentiful some-
grammer is often better at Level 3 than Level 1.
time later.
The Interplay of Formal Standards,
Data Item Descriptions and the “D” Word
De Facto Standards, and Best Practices With the announcement in 1994 by Secretary of Defense
Alex Polack of ATA, a vendor of software documentation tech- William Perry on Mil Spec Reform [10], the pendulum of com-
nology, sees standards as a way to canonize best practices. bined thought had reached the zenith of its swing regarding the
Whether canonized or not, some advocates for best practices see enforcement of formal standards. To a large extent, DoD got
them as a more natural and useful approach to improving soft- out of the expensive business of caring for and feeding stan-
ware development than are formal standards. In Rise and dards. As the pendulum has swung from rigid enforcement of
Resurrection [9], Edward Yourdon suggests: standards to total lack of enforcement3, Data Item Descriptions
“If the standards/methods procedures group wants to rename (DIDs) often disappear from the contractual setting. This leaves
itself as the best practices group, then it should realize that its job is many developers confused about how the data should be pre-
to act like social scientists visiting an alien race deep in the forest: sented. Acquirer and developer find themselves reinventing the
observing behavior, documenting existing practices, and offering wheel. It is a little like discarding all known languages and
advice on which practices have been observed to produce good developing your own, even though both parties speak English
results, e.g. higher levels of quality.” fluently. Like English, the DIDs provided a basis for communi-
In concert with evolving best practices, the more recent cation. Time need not be used reinventing nor rediscovering, if
high-level commercial standards, such as IEEE/EIA 12207, take acquirer and developer agree from the start to follow the J-
a process-oriented approach and expect a software development
organization to define and continuously improve its software The Capability Maturity Model and CMM are registered in the U.S. Patent
and Trademark office to Carnegie Mellon University.

22 C ROSSTALK The Journal of Defense Software Engineering December 1999

 Software Standards: Their Evolution and Current State

STD-016 DIDs (called Software Product Descriptions or Useable Obsolete DIDS

SPDs). Replacements
Often, the misapplication of 2167A resulted in docu- J-STD-016 & MIL-STD- DOD-STD-2167A DOD-STD-7935A DIDs
mentation — the “D” word — that took on a life of its own. 498 17 DIDs 11 DIDs Superceded by MIL-
22 Product Titles & DID#s Titles & DID#s 498 DIDs
Whereas the software standards evolved from the primordial Descriptions
goo at the fringe of the hardware disciplines, the emergence Titles &DID#sMIL-STD-498

of the associated software documentation was seemingly

from a lab in Dr. Frankenstein’s basement. It escaped to ter- Computer Programming
Manual (CPM)
Software Programmer’s
Manual DI-MCCR-
MCCR-80021A

rorize the DoD village, consuming lots of dollars, and caus- DI-IPSC-81447 80021A

ing general havoc among the defense populace. In an Data Base Design Database IPSC-80692; MCCR-
attempt to drive a wooden stake through the documentation Description (DBDD)
DI-IPSC-81437
Specification
DI-IPSC-80692
80305

monster’s heart, no DIDs appear in IEEE/EIA 12207 . 4

Interface Design IDD interface design from MCCR-80027A
The apparent lack of DIDs has been/is troublesome for Description (IDD) DI- DI-MCCR-80027A Unit Specification (US)
acquirers and developers who were accustomed to the IPSC-81436 DI-IPSC-80691

detailed guidance provided by DoD-STD-2167A and even Interface Requirements IRS interface requirements MCCR-80026A, 80303
Specification (IRS) DI-MCCR-80026A from US DI-IPSC-
by MIL-STD-498. Much like a communist society without DI-IPSC-81434 80691
an iron curtain, the software acquisition/development com-
Operational Concept systems concept from systems concept from IPSC-80689
munity is confused by the new-found freedom of no DIDs. Description (OCD) SSDD Functional Description
The developer is directed to provide “data,” the popular DI-IPSC-81430 (FD) DI-IPSC-80689

euphemism for the “D” word. But what is this data to look Software Design SDD design from US, MM MCCR-80012A, 80304,
like? Developers often want an example so they do not have Description (SDD) DI-
IPSC-81435
DI-MCCR-80012A 80306; IPSC-80691

to start from scratch. While IEEE/EIA 12207 does not leave

Software Development SDP section 7 of FD MCCR-80030A, 80297,
one to start from scratch, some developers almost feel that Plan (SDP) DI-IPSC- DI-MCCR-80030A 80298, 80299, 80300,
81427 80319
way. Tables 3 and 4 show samples of the level of guidance
provided by IEEE/EIA 12207. But the solution is available; Software Product SPS Maintenance Manual MCCR-80029A, 80317;
one can use the materials referenced by IEEE/EIA 12207.1- Specification
DI-IPSC-81441
(SPS) DI-MCCR-80029A
plus maintenance pro-
(MM) DI-IPSC-80696 IPSC-80696

1997 “Table 1 - Information item matrix,” which includes grammers' procedures

from CRISD
the J-STD-016 SPDs and other IEEE standards or one can
Software Requirements SRS DI-MCCR-80025A requirements from Unit MCCR-80025A, 80301
use J-STD-016, as shown in Table 2. Further, while MIL- Specification (SRS) Specification
STD-498 was cancelled in May 1998, the DIDs from MIL- DI-IPSC-81433 DI-IPSC-80691

STD-498 have not been cancelled, although they may be System/Subsystem S/Segment DD DI- design data from FD CMAN-80534; MCCR-
cancelled when the full use J-STD-016 is available. Design Description
(SSDD)
CMAN-80534 (without
OCD data)
and System/
Subsystem Spec (SS)
80302

The latest thinking on documentation is to use the nat- DI-IPSC-81432 DI-IPSC-80690

ural work products of the development process for today’s Software Transition Plan Computer Resources MM planning data MCCR-80024A
complex systems that are likely to need continuing evolution. (STrP) DI-IPSC-81429 Integrated Support
Document (CRISD)
Rather than requiring the developer to add extra steps to the DI-MCCR-80024A
(without maintenance
development processes to transform software development programmers' proce-
products into a prescribed standard format, the developer dures)
may generate the required information directly from the System/Subsystem S/Segment S requirements from CMAN-80008A; IPSC-
development environment — as long as the information is Specification (SSS) DI-CMAN-80008A FD, SS 80690
DI-IPSC-81431
usable by all stakeholders during initial and continued devel-
Software Test STD detailed portion of Test MCCR-80015A, 80310
opment and sustainment. Thus, documentation may be in Description (STD) DI- DI-MCCR-80015A Plan (PT)
the form of a requirements database, Computer-Aided IPSC-81439 DI-IPSC-80697

Software Engineering (CASE) tool data, software develop- Software Test Plan STP high level portion of IPSC-80697; MCCR-
ment folders, and other artifacts produced during software (STP)
DI-IPSC-81438
DI-MCCR-80014A PT 80014A, 80307, 80308,
80309
development. For these reasons, EIA/IEEE J-STD-016 and
MIL-STD-498 relaxed the requirements for documentation Software(STR) Test Report STR
DI-MCCR-80017A
Test Analysis Report
DI-IPSC-80698
MCCR-80017A, 80311;
IPSC-80698
form to concentrate on usable content, i.e. capturing engi- DI-IPSC-81440

neering and planning information in the form used for proj- Software User Manual SUM End User Manual MCCR-80019A, 80313,
(SUM) DI-MCCR-80019A DI-IPSC-80694 80314, 80315; IPSC-
ect management and development activities. DI-IPSC-81443 80694

Software Version Version Description MCCR-80013A, 80312

Table 2. A partial list of obsolete DIDs and usable replacements, excerpted Description (SVD) Document
DI-IPSC-81442 DI-MCCR-80013A
from Army Communications-Electronics Command Common
Equivalent (ECOM) Software Engineering Center Request for Proposal
Guidebook (www.sed. monmouth.army.mil/se), Operations, Strategic
Planning & Policy, RFP Guide.

December 1999 C ROSSTALK The Journal of Defense Software Engineering 23

Evolution of Software

Are Current Software Standards Used? Purpose: Describe a planned or actual function,
Yes. Gary Hebert, a civilian electronics engineer at Hill AFB, design, performance, or process.
says he used IEEE/EIA 12207 to understand the current think-
ing on documentation. He had come from a 2167A back- A description should include:
ground. He found that IEEE 12207 provided flexibility allow- a) date of issue and status
ing the use of electronic documents rather than hardcopy. Based
b) scope
on this direction, his organization uses Ives Development’s
c) issuing organization
“Team Studio Analyzer” to capture the design of a Lotus Notes
d) references
database system. In this case, the tool generates a record describ-
ing the attributes for each of the Lotus Notes components, e.g. e) context
forms, views, subforms, agents, scripts or subroutines. These f) notation for description
records are part of a documentation database. According to g) body
Nigel Cheshire, CEO for Ives Development, one of the advan- h) summary
tages of a database over a traditional collection of documents is i) glossary
the ability to generate reports against the database to verify that j) change history
the software design complies with an organization’s standards.
Table 3. Description — generic content guideline from IEEE/EIA 12207.
For example, you could generate a report to verify that each
user-editable field has help text attached as per your internal
ence functionality they were getting from MIL-STD-498, but
corporate standard.
with the benefit of the latest thinking.
And no. In speaking with some Air Force personnel, I find
With the DoD prohibition of putting standards on con-
that early this year they were using MIL-STD-498 as a reference
tract, the provisions in J-STD-016 for contractual use will be
document to recommend a series of software development doc-
changed to provide for use as a basis of agreement on the activi-
uments and also to develop a concept of operations using the
ties and work products of the development process, where the
Operational Concept Description Data Item Description
agreement may take any form, from a handshake to a formal
(DID). They were just barely aware of the commercial version
contract, and may be within an organization or between organi-
of the standard (J-STD-016) and were unaware of IEEE/EIA
zations. The full-use standard will be intended for project-spe-
12207.
cific application, on legacy systems and in organizations that
have legacy processes (i.e., processes based on the old DoD stan-
An Update on J-STD-016 dards), not as a basis of defining an organizational life cycle
While MIL-STD-498 was cancelled in May 1998, the J-STD- process. The expectation is that defense and commercial devel-
016 is still alive and well and is being updated. Those interested opers and acquirers will use international and professional stan-
can get the trial-use J-STD-016-1995 now, and will be able to dards such as IEEE/EIA 12207 as a basis for their long-term
get the updated J-STD-016 when it is available. Both the trial- organizational process definition.
use version and the full-use version will provide that same refer- The conversion of J-STD-016 to a full-use standard was

Table 4. Description — software interface design from IEEE/EIA 12207.

Purpose: Describe the interface characteristics of one or more system, subsystem, hardware item, soft-
ware item, manual operation, or other system component. May describe any number of interfaces.

IEEE/EIA 12207.0 reference 5.3.5.2, 5.3.6.2

Content: The software item interface design description should include:

a) generic description information (See Table 3)
b) external interface identification
c) software component identification
d) software unit identification
e) external-software item interface definition (e.g., source language, diagrams)
f) software item-software item interface definition (e.g. source language, diagrams)
g) software component-software component interface definition (e.g., source language, diagrams)

Characteristics: The software item interface design description should:

a) support the life cycle data characteristics from annex H of IEEE/EIA 12207.0 (see 4.2 of this guide)
b) define types of errors not specified in the software requirements and the handling of those errors

24 C ROSSTALK The Journal of Defense Software Engineering December 1999

 Software Standards: Their Evolution and Current State

delayed in Environmental Impact Analysis (EIA) balloting and 888 South 2000 East
has been further delayed in combined EIA and IEEE ballot res- Clearfield, Utah 84015-6216
Voice: 801-525-3357
olution due to a lack of resources to complete the resultant revi-
Fax: 801-525-3355
sions. Hopefully, by the time this article is published, re-ballot- E-mail: [email protected]
ing will be in progress and the schedule for its completion will
be available from the EIA and the IEEE.
References
1. Some sources of lists — http://www-library.itsi.disa.mil./,
EIA/IEEE J-STD-016 and MIL-STD-498 DIDs vs. http://standards.ieee.org/catalog/index.html,
DOD-STD-2167A and DOD-STD-7935A http://global.ihs.com/cgi-bin/litmus_test.cgi?FRIT
Table 2 provides usable product descriptions from J-STD-016 TER=134547&NODE=PC
and DIDs from MIL-STD-498. The DIDs from DOD-STD- 2. Polydys, M.L., “Operation Data Storm: Winning the
2167A and DOD-STD-7935A were harmonized to produce the Interoperability War through Data Element
DIDs of MIL-STD-498. The product descriptions in the Standardization,” CrossTalk, July 1999.
annexes of J-STD-016 are the commercial equivalents of the 3. MIL-STD-483, Configuration Management Practices,
DIDs in MIL-STD-498, and have the same names. 1970.
For each obsolete DID associated with DOD-STD-2167A 4. MIL-STD-490, Specification Practices, 1968.
5. DOD-STD-2167, Defense System Software Development,
and 7935A, the table shows the corresponding usable J-STD-
1985.
016 product description and MIL-STD-498 DID. The table
6. J-STD-016, Software Lifecycle Processes, 1999.
also shows the DIDs that each MIL-STD-498 DID officially
7. IEEE/EIA 12207, Industry Implementation of
superceded; some were from DOD-STD-2167A or 7935A and International Standard ISO/IEC 12207:1995 (ISO/IEC)
some were from other MIL-STDs. Standard for Information Technology, 1998.
(Note that 498 has been cancelled, but as of this writing, the 8. MIL-STD-498, Software Development and
DIDs have not been cancelled.) Documentation, 1994.
9. Yourdon, Edward, Rise and Resurrection of the American
Acknowledgements Programmer, Yourdon Press Computing Series, page 128.
I want to thank Marilyn Ginsberg-Finner for providing her com- 10. Perry,William J., Secretary of Defense, DoD Policy on the
ments, the J-STD-016 updates, and Table 2. Les Dupaix, Dr. David Future of MILSPEC, CrossTalk September 1994.
Cook, Gary Hebert, and Alex Polack, also contributed to this article.
Notes
About the Author 1. This came when the advent of written language by the
Reed Sorensen has more than 20 years experience press made it available on a scale of a magnitude greater
with TRW developing and maintaining software than handwriting provided.
and documentation of embedded and manage- 2. A series of international standards on quality.
ment information systems; providing systems 3. Alex Polack notes that from the mid-1980s to the mid-’90s,
engineering and technical assistance to program there was a tendency to use standards as a club to beat up or
offices, and consulting with many DoD organiza- monitor developers. Marilyn Ginsberg-Finner notes that
tions regarding their software configuration man- within current DoD acquisition reform policy, standards are
agement and documentation needs. He provides configuration man- primarily guidance to developers in defining their software
agement, software control, interface control, and systems require- process and standards are still essential as a basis for acquirers
ments analysis in support of intercontinental ballistic missile sustain- to evaluate the processes, activities, and work products that an
ment, modifications, and replacement programs. Sorensen has pub- offeror proposes and provides.
lished several articles in CrossTalk on various software related subjects. 4. IEEE/EIA 12207.1-1997 “Table 1 - Information item
matrix” references standards, including J-STD-016 that
TRW ICBM Systems have DID-like guidance.
Attn: Reed Sorensen N14GC

What is on the Web?

CrossTalk online!
Visit http://www.stsc.hill.af.mil to find:

— current and past issues (1994-present) — theme announcements (editorial calendar)

— author guidelines — send us an e-mail or letter to the editor
— subscription form (to subscribe or update — search for articles on many software engineering
reader contact information) topics

December 1999 C ROSSTALK The Journal of Defense Software Engineering 25