This article was downloaded by: [ECU Libraries]

On: 25 April 2015, At: 11:23

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal of Intelligence

and CounterIntelligence
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/ujic20

Managing intelligence analysis:

Strategies for playing the end game
a
Arthur S. Hulnick
a
Coordinator of Academic Affairs in the Public Affairs Office ,
Central Intelligence Agency , Washington
Published online: 09 Jan 2008.

To cite this article: Arthur S. Hulnick (1988) Managing intelligence analysis: Strategies for playing
the end game, International Journal of Intelligence and CounterIntelligence, 2:3, 321-343, DOI:
10.1080/08850608808435068

To link to this article: http://dx.doi.org/10.1080/08850608808435068

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
 ARTHUR S. HULNICK

Managing Intelligence Analysis:

Strategies for Playing the
End Game
Downloaded by [ECU Libraries] at 11:23 25 April 2015

INTRODUCTION
Events of recent years have raised a number of serious questions about the
management of the U.S. intelligence system. Among the most significant for
intelligence professionals is the question of the quality of the intelligence
product.
I circulated this article among the management of the Central Intelligence
Agency (CIA) to make certain that it reflects accurately the system of pro-
duction controls and management's attitudes about them. CIA's managers
now generally believe that the agency should try to explain more about this
system, including its strengths and weaknesses, as a way of encouraging an
informed dialogue about intelligence. We at CIA want scholars and the
public to be better informed about our work.
Producing the finished reports — the studies, the daily intelligence, the
maps and charts, the compendia of information — is perhaps the least
glamorous of all the processes in intelligence. No one sees James Bond
writing reports in his adventures. Not even in the motion picture "Three Days
of the Condor," starring Robert Redford, which was supposed to be about
researchers, did the audience see much writing and reporting.* Few students

*In his novel The Hunt for Red October, author Tom Clancy depicts one of the main characters as an
American intelligence analyst, but this character's exploits go well beyond anything seen in the real
world.

Arthur S. Hulnick is a 30-year veteran in the intelligence profession. Cur-

rently, he is Coordinator of Academic Affairs in the Public Affairs Office
of the Central Intelligence Agency in Washington. This article was derived
from a conference paper he delivered at the 1987 Annual Meeting of the
American Political Science Association.

321
 322 ARTHUR S. HULNICK

of intelligence, and even fewer alumni, write much about this aspect of
intelligence. And few publishers would probably be interested in a book
entitled My Life as an Intelligence Analyst. Nevertheless, the production
process is the end game of intelligence. Without intelligence production, the
collection of intelligence — the information gathered from spying and the
technically sophisticated gadgets that penetrate the secrets of our adversaries
— would be virtually useless. At issue, then, is how this process can be con-
trolled to insure that the product accurately reflects the needs of consumers
and judgments of analysts untainted by politicization, bias, or policy inter-
ference.
The neutrality of intelligence production is critical if the product is to be
both believable and useful. The founders of the U.S. intelligence system
Downloaded by [ECU Libraries] at 11:23 25 April 2015

clearly sought this kind of neutrality. To understand the problems related

to product control, we should study the period preceding the creation of the
CIA because the questions raised then still haunt us, although perhaps in
different form.

A LOOK AT HISTORY
In establishing the CIA, President Harry Truman wanted to divorce it not
only from politics, but from interdepartmental frictions as well. The strains
that had existed in World War II between the Office of Strategic Services
(OSS) and other intelligence units had demonstrated clearly the kinds of
problems that could be created. Ray Cline, who was in at the beginning of
the development of the OSS Research and Analysis Section, describes this
early history 1 as does Thomas Troy in his biography of General William
Donovan.2 A frequent problem arose from different intelligence elements
seeking to pressure analysts to reach preconceived conclusions.3 Thus was
born the dissenting footnote, which later led to increased emphasis on com-
petitive analysis.
The Directorate of Intelligence (DI) emerged after the creation of the
CIA. Although its roots were in the Research and Analysis section of the
OSS, it now faced a different world, where production decisions were some-
what less driven by events. Planning became increasingly important. Never-
theless, in its early days the DI devoted considerable effort to current intelli-
gence on the burning issues of immediate concern. Policy consumers, under
heavy pressure to "fix" some problem, spent little time on issues beyond the
horizon. Political appointees rarely stayed around long enough to take the
longer view.* Certainly, a great deal of research was being done, but in my

*Estimates vary, but observers seem to agree that the average senior appointed policy official remains
in office only 18 months to two years.

INTERNATIONAL JOURNAL OF
 MANAGING ANALYSIS STRATEGIES FOR PLAYING THE END GAME 323

experience, much of it was done to enable the analyst to speak authorita-

tively on a current issue, rather than for publication. Outside the National
Intelligence Study (NIS) program,* a long-range research program did not
exist. In fact, except for contributions to National Estimates, analysts had
few opportunities to write "think-pieces."
Over time, the structure of the CIA's analytic component was shifted
from heavy emphasis on current production to more detailed, thoughtful
and research-oriented products. The Office of Current Intelligence, essen-
tially the CIA's political analysis unit, was eventually transformed into an
office for political research paralleling similar offices devoted to economic,
biographic, and military products. In 1980-1981, the entire structure was
revamped into offices devoted to geographic areas. These offices combined
Downloaded by [ECU Libraries] at 11:23 25 April 2015

political, economic, military, and other kinds of research under "one roof,"
thus making more detailed analytic research feasible. After some grumbling,
the new offices began to turn out the in-depth products consumers had been
seeking, while still providing current intelligence materials.**

CONTROL POINTS IN PRODUCTION

To understand how control is actually applied in the production of intelli-
gence, we need to focus on various aspects of production, a process far more
complex than scholars believe. Some academics think that producing intelli-
gence analysis is no more difficult or complicated than producing the typical
academic research paper. In fact, producing analytic products encompasses
a wide range of activities with numerous control points.
At the 1985 American Political Science Association convention, I sug-
gested a matrix model of the analytic process.4 This model contains a num-
ber of semi-autonomous elements in intelligence analysis and production
(Figure I).
At first glance, this matrix apparently contains sequential processes. While
an individual piece of intelligence may indeed pass through each one of the
production elements in turn, the processes themselves are mostly autono-
mous. This matrix was developed as a substitute for the prevailing, and in
my opinion, erroneous theory of the Intelligence Cycle (Figure 2). 5 While
this original matrix model sought to explain the relationships between indi-
vidual elements of intelligence and consumers, it offers little help in thinking
about management and control. The matrix has best been conceptualized as

*The National Intelligence Studies were designed to be comprehensive, encyclopedic examinations of

all aspects of a country's history, politics, economics and social activity to provide a general under-
standing of its culture and behavior. These studies sometimes ran to 10 or more volumes, but were
later condensed into one book-length work.
**A detailed organizational chart of the Directorate of Intelligence is contained in the CIA Fact Book,
published from time to time by the CIA.

INTELLIGENCE AND COUNTERINTELLIGENCE

 324 ARTHUR S. HULNICK

Management
Requirements
Tasking
Analysis
Production
Dissemination
Evaluation
FIGURE 1. Production Matrix
Downloaded by [ECU Libraries] at 11:23 25 April 2015

a wheel rather than a box, with a central management element as the hub
from which each process element radiates as a spoke (Figure 2).
THE INTELLIGENCE PRODUCTION PROCESS: A MODEL

FIGURE 2. The Intelligence Production Process: A Model. Source: 'The intelligence cycle', from
Amos A. Jordan and William J. Taylor, American National Security Policy and Process (Baltimore and
London: The Johns Hopkins University Press, 1981).

Thus, we have two ways of looking at the groupings of the elements of the
Production Matrix. To understand how individual items of intelligence
become parts of a finished product, we can refer to the linear sequence
suggested in Figure 1. To understand the control elements, Figure 3 is more
appropriate because it isolates processes for separate investigation.
Evaluation Requirements

Dissemination Management* -Tasking

Production ^Analysis

FIGURE 3. Production Matrix II

INTERNATIONAL JOURNAL OF
 MANAGING ANALYSIS STRATEGIES FOR PLAYING THE END GAME 325

Control in the management of intelligence takes a variety of forms (Figure

4). To insure acceptable levels of work, management seeks to apply an
agreed standard to the operations under its jurisdiction. Who sets the stan-
dards, who monitors them, and who makes the necessary changes when
standards are not met? To answer these questions we might consider each
process in the matrix.
OFFICE DIRECTOR

Management
Level 1
Division A Division B Division C
Chief Chief Chief
Management
Downloaded by [ECU Libraries] at 11:23 25 April 2015

Level 2

Branch Chief Branch Chief Branch Chief

A B C
Management ~~~~~~~~"
Level 3

Analysts Analysts

FIGURE 4. A typical intelligence organization, managers existing at every level.

MANAGEMENT
The matrix contained in Figure 1 suggests that management is "on top,"
giving direction to each process below. In fact, Figure 3 more accurately
depicts how intelligence services work. In most cases, management does not
directly control the processes, but offers advice and guidance. In part an out-
growth of the compartmentation principle, but also because managers either
are not substantive experts or do not have the time to function in that
capacity, they must leave such details to working elements. Management can
also choose to be the final filter on product output. Clearly, the notion that
managers must concentrate on resource utilization rather than substance
conflicts with the desire of these same managers to be final arbiters of the
substantive output.
Resolution of this difficulty has taken various forms. Under some adminis-
trations, the managers styled themselves as substantive experts,* in others,
*As DCI, Admiral Stansfield Turner thought he had an obligation to take an active part in the process
of producing estimates, since he had to sign them. He once told the author that he saw them as "his
estimates."

INTELLIGENCE AND COUNTERINTELLIGENCE

 326 ARTHUR S. HULNICK

surrogates were placed "in the loop,"* and at times, substance was divested,
and each production element made its own decisions. None of these systems
has proved to be perfect. In fact, a reading of the intelligence product over
the years suggests that management positioning may not make much differ-
ence in terms of quality. The crunch comes only when some grievous failure
occurs, and each method has had its share of successes and failures.6
No one will argue that management should not give resource guidance.
But should it be giving substantive guidance as well? If so, of what kind? In
my experience over the past 20 years, I have determined that management
must be careful how it asks the questions it wants answered lest it intimidate
and influence the direction and emphasis of its own production elements.
Management should, of course, insure that users of intelligence make their
Downloaded by [ECU Libraries] at 11:23 25 April 2015

needs known to the production elements. Even if management merely

fosters producer-consumer relations, it makes a major contribution to the
process.
In Washington, management also has a continuing problem with pressures
on the most senior officers to make presentations or to appear at substan-
tive policy meetings, a possible outgrowth of the notion that all participants
must be of equal bureaucratic rank. This practice can lead to potentially
ludicrous situations. If one goes high enough, the participants, because of
their senior rank and the application of the need-to-know principle, may
well be the least informed about the subject under discussion, particularly
if the focus is on details.
Under those circumstances, management cannot restrict itself to resource
decisions; it must become substantively involved. Managers thus feel pressed
to concern themselves with the resolution of the final product, not merely
for control purposes, but to enable them to be thoroughly familiar with the
analysis contained in the products they approve. This may explain why,
with rare exceptions, the Deputy Director for Intelligence at CIA has come
up through the analytic ranks and hence is well-schooled in the production
process.

REQUIREMENTS
In the cycle theory of intelligence, requirements are somehow supposed to
come to the intelligence community in much the same way intelligence
flows to the policy system. In fact, policymakers rarely spend much time

*The development of the National Intelligence Council under DCI William Colby created a group
acting for the DCI in creating and approving estimates. The National Intelligence Officers who made
up the council were to serve as the DCI's substantive experts and could act for him in the estimates
process.

INTERNATIONAL JOURNAL OF
 MANAGING ANALYSIS STRATEGIES FOR PLAYING THE END GAME 327

detailing to intelligence producers what they need or want to know. Yet,

understanding the needs of policymakers is essential in determining produc-
tion requirements. How, then, do participants in the system know what to
do and when to do it?
Production is generated in at least four ways: First, policymakers them-
selves, of course, provide some clues to intelligence, although they some-
times ask the wrong questions. Moreover, they strongly tend to want infor-
mation not for enlightenment but to support their own policy initiatives.
Experienced intelligence managers know that they cannot wait for policy-
makers to articulate their requirements, but must actively solicit information
about what policymakers perceive as their needs and requirements. Unfor-
tunately, this articulation will inevitably come too late to be of much use
Downloaded by [ECU Libraries] at 11:23 25 April 2015

because policymakers will not focus on the issues early enough. Thus, intelli-
gence must establish some systematic way to prod the policy system to
provide product guidance.
Second, requirements are often self-generated by intelligence. To under-
stand what products might be most useful, analysts should be knowledgeable
about the activities of the policy community. They must ask themselves:
"What kinds of questions would I want answered, if I were the consumer?"
They must not merely consider what might be wanted, but what might be
unwanted as well. They must focus on needs, even in face of the possibility
that the product, when delivered, may be unpopular or unpleasant.
Third, requirements are often generated by world events. Usually, these
are activities unanticipated by either the policymakers or intelligence
officers. Obviously, having taken place, coverage — beyond the daily
reportage — must be undertaken. This kind of production tends to be ad hoc
or informal and under heavy time pressure. Later, production elements may
review the situation for more formal papers.
Fourth, production requirements may be generated by previous produc-
tion. This occurs because many intelligence inputs are steadily incremental in
nature in that they flow into policymakers as situations unfold, or as policy
initiatives are tried or modified. An understanding by intelligence managers
of previous inputs may significantly influence production decisions as events
progress.
Lags in intelligence production occur because of the time needed to find
information sources, to do research, and to sift and evaluate judgments in
terms of the varied demands of intelligence consumers. As a result, produc-
tion elements tend not to centralize their resources into a formal production
requirements organization. Instead the emphasis has been on building a
requirements element into each production unit. This has led, for example,
to the practice of having the research director of CIA production offices
organize at the office level at least some of the production decisions.

INTELLIGENCE AND COUNTERINTELLIGENCE

 328 ARTHUR S. HULNICK

Despite pressures to deal with current events, production elements must

always be thinking and planning for the long term. The key lies in establish-
ing and maintaining good relations with policy elements at the working level,
where officers tend to be non-political, career-oriented and well-tuned to the
activities of the senior officials they serve. They know their job and expect
to survive the political winds. As a result, they can often provide excellent
guidance about what kind of products might be useful, when they ought to
be delivered, and what form they should take. Staff assistants, executive
officers, and what Washington terms "horse holders" may be more forth-
coming than their bosses about their needs for intelligence.
From experience we know that this producer-consumer relationship has
to be established at the initiative of the intelligence community. Consumers
Downloaded by [ECU Libraries] at 11:23 25 April 2015

will not do it. We also know that the relationship has to be nurtured and sus-
tained to be really useful. Unfortunately, managers may try to avoid the
required expenditures of resources to maintain the tie. Managers are far more
comfortable in commanding analysts to stay at their desks and read their
mail rather than allowing them the relative freedom to establish and main-
tain a network of contacts in the policy community. Fortunately for the
system, the active policy of both CIA and DIA is to maintain good relations
at the working level.
In intelligence production, considerable evidence exists that the require-
ments process has led to products that are timely, relevant, and useful. In
my opinion, no significant activity has gone unnoticed, unreported, or un-
analyzed. Actually, the requirements determined by the production process
have led to an enormous body of finished intelligence that was only of
peripheral interest when completed but later on when refined proved highly
useful.*

TASKING
In the matrix scheme of production, tasking is quite different from require-
ments, although many writers consider tasking and requirements to be part
of one function. Tasking refers to management decisions regarding the
assignment of staff members to perform analytic work concerning who will
do what. In CIA's early days, this posed no problem. Analysts were organ-
ized according to function — political, military, or economic — and within
those functions by country desk. Thus, an analyst had a "country account"
and was expected to do whatever was needed to produce intelligence on that
country.

*Unfortunately, examples of the use of off-the-shelf material generally remain classified. The author
can testify, however, that some of his own obscure research later proved useful when events broke in
parts of the world where activity was unexpected.

INTERNATIONAL JOURNAL OF
 MANAGING ANALYSIS STRATEGIES FOR PLAYING THE END GAME 329

Today, tasking is more complex because CIA production elements are

organized not only geographically, but into overlapping units to facilitate
multi-disciplinary analysis. Within country accounts, several analysts may
deal with specific aspects of issues or problems. While the 1980-1981
geographical rearrangements have produced the intended analysis, tasking
decisions are no longer simple. Managers cannot now assume that the
country-desk officer will independently address the requirements problem as
generated. In dealing with long-term research, a team approach is often
necessary and desirable. While this helps to create multi-dimensional and
multi-disciplinary analysis, management must make it happen. Usually, the
control point for tasking lies at the division level (the second echelon) or at
the office level (third echelon). Where trans-national or multi-country issues
Downloaded by [ECU Libraries] at 11:23 25 April 2015

are at stake, inter-unit coordination must be developed.

Tasking may be highly structured bureaucratically — perhaps the Defense
Intelligence Agency and military intelligence tend to do this more neatly
than the civilian agencies — or it may be self-generated from the bottom up.
That is, analysts may decide who will do what, and just get it done. Some
negotiating may have to take place, either to ensure that specific units are
included in a project, or to make certain that they are not expected to do
anything.
In any event, personalistic elements certainly enter into tasking decisions
because no perfectly objective way exists to assign analytic work. The great
danger would be a system in which certain analysts were consciously
excluded from participating in a project because of their known views or
their anticipated inputs. In my experience, analysts at CIA have always
been encouraged to "put their two cents in." The free exchange of ideas is
welcomed and encouraged. One never knows when the key analytic point
will come from the sidelines rather than from the mainstream analysts.

ANALYSIS
The analytic process itself is worthy of research. The most intriguing aspect
of the analytic profession for those of us who have been engaged in intelli-
gence analysis) lies in questions relating to how the analyst develops a thesis
or reaches conclusions when his information is incomplete or fragmentary.
Furthermore, the accumulated data may be clouded by deception or dis-
information — false data intentionally planted by an adversary. Thus, the
intelligence analyst is faced by challenges well beyond those of the typical
academic scholar, who may not have all the desired information but who at
least is not usually worried about being intentionally misled or fooled.
Intelligence analysis involves a number of control decisions among which
are the following:

INTELLIGENCE AND COUNTERINTELLIGENCE

 330 ARTHUR S. HULNICK

• Determining that adequate data have been gathered so that analysis be-
comes more than just an educated guess based on "gut feelings." Analysts
must have some way to determine a reasonable basis for drawing conclu-
sions about the issue at hand.
• Determining the "terms of reference" to ensure that all the topics are
covered in ways that will prove useful to consumers of the final product.
• Deciding on the methodology to be used. Analysts previously relied on
their intuition and "seat-of-the-pants" techniques. Today, analysts know
how to use statistical inference, probabilistic models, computer-generated
expert systems, and other sophisticated techniques.
• Establishing a review procedure that will remove or suppress bias, personal
predilections, and prejudices while still encouraging new and unusual ideas
Downloaded by [ECU Libraries] at 11:23 25 April 2015

to flourish.

ARE THE DATA ADEQUATE?

In today's world, electronic message systems and other modern communica-
tions apparatus make certain that analysts receive incoming information on
a truly timely basis. In CIA's early days, incoming reports often arrived by
diplomatic pouches with the result that messages were usually delayed and
often misrouted or lost altogether. Classic stories (perhaps apocryphal)
include the vital message that only came to light several years later when the
pouch was finally emptied completely. Today, in Washington at least, an
analyst receiving a message can routinely expect that his colleagues in the
other agencies are receiving the same message in the same form at about the
same time.
Electronic communications have dramatically improved delivery but have
not solved all the problems. Some categories of material still do not auto-
matically come to the analyst. Analysts must personally hunt down elusive
documents such as memoranda of conversation, informal debriefing reports,
and other such material. They must also seek out certain categories of par-
ticularly sensitive reports that do not receive normal dissemination. The
question remains: When the material does not arrive routinely, just how does
an analyst know it exists? All that can be done is to establish and maintain
a network of contacts with intelligence collectors and other offices that con-
trol such documents in the hope that these materials can at least be obtained
to be read and digested if not retained.
When does the analyst have enough data to begin the process of sifting
data, comparing new information with the existing data base or plugging in
bits and pieces into some equation or algorithm? For analysts the answer is
simple: they never have enough data. Especially in regard to current or
warning intelligence, analysts always seem to be waiting for the next cable

INTERNATIONAL JOURNAL OF
 MANAGING ANALYSIS STRATEGIES FOR PLAYING THE END GAME 331

or message that will clarify a murky picture or pin down the data of some
predicted event. Sometimes analysts have to be pressed into writing, even
though they believe that they are not yet ready.*
Does an informational threshold exist that defines the sufficiency of
information to support analysis? No hard and fast answers are readily appar-
ent. Suffice to say that unless there is enough data on which to base con-
clusions, analysts will have a hard time going through a review process where,
at each level, increasingly senior officers pick the material apart, challenge
the conclusions, and play devil's advocate.

ARE THE TERMS OF REFERENCE CORRECT?

Terms of reference — essentially the questions to be addressed by intelli-
Downloaded by [ECU Libraries] at 11:23 25 April 2015

gence analysis — are extremely important in shaping the eventual end

product. Skewing the terms of reference may well change the nature of the
judgments that can be made. In my view, establishing the terms of reference
ought to be an interactive process, in which producers and consumers agree
early on about the nature of the analysis that will be done, as well as the
timing and format of the end product. Nonetheless, managers can easily
allow terms of reference to be created in a policy vacuum. While not particu-
larly important in current intelligence, this practice can make a difference in
basic research, and is critical in preparing estimates or other products that
are keyed directly to policy considerations.
Different policy-users frequently have different interests, so the terms of
reference must include questions or input from the segments of the policy-
making world, who may eventually become the end-users of the product.
Usually this process is informal because experience shows that policymakers
will not respond to formal efforts to query them on terms of reference. They
will, however, comment once the terms are drawn up. Circulating terms of
reference to end-users for preliminary comment will often reveal problems
that can be dealt with in analysis. If the problems are not discovered until
the product reaches the end stages, time may not be available to deal with
the unanswered questions, issues not covered, or data left unanalyzed.

IS THE METHODOLOGY EFFECTIVE?

Advances in analytic methodology seem to have grown remarkably in the
past decade. While a discussion of explicit methods goes beyond the scope
*When to stop collecting information in itself presents a knotty problem. Almost every decision is
accompanied by what Jon Elster calls a shadow decision, a decision when to stop collecting informa-
tion. (Note Rational Choice, edited by Jon Elster, Washington Square, New York, New York Univer-
sity Press, 1986, p. 14.) Normally we stop collecting information when time runs out, when we are
too tired to carry on, or when somehow we are convinced that the cost of collecting more informa-
tion surpasses the marginal value of the information.

INTELLIGENCE AND COUNTERINTELLIGENCE

 332 ARTHUR S. HULNICK

of this article — and the knowledge of the author — a wide variety of ways of
attacking a problem often exists. Managers of intelligence must assume that
analysts try different techniques, sometimes by giving the problem to
separate groups for analysis by different disciplines, sometimes by seeking
an outside expert to provide competing analysis.
The classic case of using competing groups was the famous "A-Team/B-
Team" exercise under DCI James Schlesinger in the mid-1970s.7 That was,
in my view, a good idea badly handled. The original notion that there might
be a benefit in creating two competing teams of comparable experts to
review an existing data base and see how their conclusions might differ was,
on its face, sensible. But in the end the political and institutional biases of
the B Team were so overwhelming that the results were hardly credible and
Downloaded by [ECU Libraries] at 11:23 25 April 2015

not particularly useful.

Informally, of course, competing analysis is ongoing in the intelligence
community. On most significant issues, analysts from CIA, State, and
Defense are almost certain to be working on similar issues and, if their lines
of communication are good, they will benefit from each other's analysis
without bureaucratic intervention.*

IS THE REVIEW PROCESS INSTRUCTIVE?

The review process, often quite painful for analysts, is the part of the an-
alytic process that differs most from academic research. The goal of intelli-
gence analysis is to produce an institutional product — one that the agency
itself supports. While the expertise and insights of the analysts involved are
important, policymakers really want to know if the views expressed are
supported by the Director of Central Intelligence. Thus, the analytic process
tends to play down individual expertise in favor of "group-think."* Rela-
tively new analysts have often told me that they find the review process
frustrating because, at each level, they think that substantive managers —
line supervisors, division and office chiefs — tend to "ruin" the carefully
crafted wording of the original analytic document and change not only its
content, but also its meaning. In fact, the process assists analysts by focusing
their prose, sharpening their insights, and providing expert advice not only
about substance but about form as well.

*During the period that the author served on the Intelligence Community Staff, a count was made of
the informal groups active in the Community; more than 125 were tallied.
**"Group-think" was coined by psychologist Irving L. Janis of Yale University. Difficulties arise
when concurrence-seeking becomes so dominant in a cohesive ingroup that it tends to override
realistic appraisal of alternative viewpoints. Group-think fosters "shared illusions and misjudg-
ments" which "interfere with critical thinking and reality testing." (Note: Victims of Groupthink
by Irving L. Janis: Boston: Houghton Mifflin Company, 1972.)

INTERNATIONAL JOURNAL OF
 MANAGING ANALYSIS STRATEGIES FOR PLAYING THE END GAME 333

Each level generally has the authority to change the wording, but will
explain to the author the reason for the changes. Line supervisors may give
the analyst support, assuming they agree with the draft that is working its
way upwards. Most analytic works move rapidly through the review process
without problem. But difficulties occasionally arise. Poorly written work or
faulty analysis will be detected rather quickly.
Analysis that reaches controversial conclusions, or conclusions that are
out of step with the "conventional wisdom," may also require the analyst
and supervisor to "defend" the work in the review process. Analysts should
not allow themselves to be intimidated by this process but should be pre-
pared to justify their conclusions as they move their work upward through
the various levels of review.
Downloaded by [ECU Libraries] at 11:23 25 April 2015

Some products may go through as many as six levels, but the review
process is designed to eliminate personal bias, political prejudice, and policy
advocacy. This system tends to play off each individual's own biases against
the scrutiny of the next higher level. This system will tend to "zero out"
personal views in favor of the institutional view. Of course, congruent biases
may possibly be found at several levels, but it seems highly unlikely that
similar biases would occur at so many levels.
In the end, although they would be loathe to admit it, analysts may be
grateful for a process that has saved them from errors, bad judgment, and
poor exposition — even though the cost may be "pride of authorship." No
standard exists for judging these issues very well. We have no certain way to
judge the effectiveness of the review process helping to avoid the politiciza-
tion of intelligence. Observers who have been reading the products over a
period of many years and several administrations necessarily admit that the
review process has created a certain sameness in style and content of the
judgments that are made, regardless of the political party of the administra-
tion in power. Consumers who are asked to evaluate the products often
remark about the unexciting language and "blandness" of the analysis. If
this indicates neutrality — and not intentional dullness — then the review
process probably has been successful.

PRODUCTION
A distinction must be drawn between analysis and production. Analysis
involves a judgmental process, while production describes a process of
"packaging" that has as much mechanical as intellectual content. In produc-
tion, a vehicle is created to enable intelligence analysis to be recognized,
easily absorbed, and readily used by consumers. Professionals often refer to
it as "finished intelligence," although the work of intelligence is certainly
not finished when production takes place.

INTELLIGENCE AND COUNTERINTELLIGENCE

 334 ARTHUR S. HULNICK

The production process is steeped in tradition. If policy consumers from

the 1940s were to be invited to review most "finished" intelligence today,
they would probably find it similar to what they were used to. At the
national level, a great deal of emphasis is placed on several categories of
formal products. They are instantly recognized for what they are — a kind of
product identification that marketing strategists will understand. Intelligence
products can be classified into several standardized categories, each with
its own particular style and with related control issues:

Alerting Intelligence
A term used to warn policymakers of crisis and is usually given them either
just before an event takes place — if intelligence has detected it soon enough
Downloaded by [ECU Libraries] at 11:23 25 April 2015

— or more likely as an event begins to break.

In theory, intelligence should be able to give the policymaker an intelli-
gence product outlining not only what will happen, but what the event will
mean for the United States. Events tend to take place in far time zones while
policymakers are asleep. This means waking some senior official for immedi-
ate briefing, with more detailed information to follow. Alerting intelligence
tends to be stylized into "alert memos," situation reports or similar short
analytic works designed to be read quickly. They are formatted in such a
way that the material can be put together quickly in the production office
itself.
The control issue for alerting or warning intelligence involves pre-position-
ing or planning so that the product can be put together without having to
worry about design, format, or packaging. Managers must have on hand the
necessary forms so that when a decision is made to issue a warning notice or
"alert memo," no time is lost. These products also tend to come from alert
centers or warning units that operate on a 24-hour per day basis. The various
teams — usually four or five teams are required to handle such rotations —
are each equally capable of handling the crisis from an intelligence point of
view.
Further, when the situation requires office briefings in advance of pub-
lished material, a production office must be prepared with maps, charts, and
other visual aids to reinforce the message being delivered. Alert centers are
usually well equipped with such materials so that even when an event takes
place in a remote part of the world, or involves forces not well known the
briefing officers will be able to put together enough materials to be credible.
Periodically, alert units must also rehearse their handling of crisis. This
requires pactice in the use of communications gear and other equipment to
make certain it will function when a crisis actually hits. In my experience,
military alert centers tend to practice a great deal and are capable of using
the machinery without problem. Civilian-run centers tend to be somewhat

INTERNATIONAL JOURNAL OF
 MANAGING ANALYSIS STRATEGIES FOR PLAYING THE END GAME 335

more reluctant to practice, particularly in waking up senior officials just for

rehearsal. As a result, they may be less familiar with crisis procedures, with
the inevitable problem that machinery may fail to operate just when it is
needed most.

Current Intelligence
A term referring to the daily flow of finished intelligence that policymakers
have come to expect with their morning coffee. It is essentially a journal-
istic effort that encounters many of the same problems encountered in news-
paper, radio, or television work. Formats are fixed (whether in a publication
or in a briefing), publishing and delivery times are regulated, and content
tends to reflect the same kinds of issues one finds in the daily press or on the
Downloaded by [ECU Libraries] at 11:23 25 April 2015

morning news shows. Control issues come down to what items to use and
how to display them within the fixed boundaries of the vehicles for current
intelligence. We know from various surveys and other data that senior
policymakers usually spend no more than 5 to 15 minutes a day in absorbing
current intelligence so the product will inevitably have to be somewhat
simplistic and cursory. The hope is that policymakers who need to know
more will seek additional information from their intelligence contacts or
will have more material delivered to flesh out their knowledge.
A danger in producing current intelligence is that managers may occasion-
ally seek to use these products to "go on the record." That is, they will
publish or disclose materials to show that they know what is going on, and
not necessarily to enlighten consumers. In fact, current intelligence will
rarely enlighten real experts in the policy community on the issues with
which they are most familiar. On issues of peripheral interest or for senior
officials who are true generalists, current intelligence can be enormously
helpful.

Basic Intelligence
The great bulk of intelligence production today refers to the encyclopedic
data and analysis demanded by policy consumers. Basic intelligence appears
in studies, research papers, detailed analytic manuscripts in far greater detail
and complexity than most consumers will want to use, but for the policy
officer enmeshed in the process of trying to construct a policy option or to
implement a policy decision, the details may be just what is needed. Since
no way exists to gauge exactly the needs of individual consumers — and since
there is also a requirement to provide information that is appropriate even if
not needed — basic intelligence is designed to be all things to all consumers.
Basic research papers can be set aside for future reference, used selectively
to excerpt data, or pored over by a select few.

INTELLIGENCE AND COUNTERINTELLIGENCE

 336 ARTHUR S. HULNICK

Packaging is a major issue in basic intelligence. A good deal of analytic

work can be dissipated by a failure to put the information into a format
consumers find comfortable and useful. Thus, consumers will see, in addition
to the standard ten- to twenty-page research paper, wall charts, foldouts,
detailed graphics, and the like. Sometimes, the best way to present research
is through detailed briefings or the production of video tapes. Whatever the
medium, it should enhance, not mask, the message.
Managers must ensure that these research works are introduced or sum-
marized so that they can be easily absorbed by senior officials, who may
have no additional exposure to the work. Sometimes, however, these sum-
maries tend to grow to the point that they themselves could use a summary.
While analysts are always reluctant to see their work compressed, the
Downloaded by [ECU Libraries] at 11:23 25 April 2015

evidence is quite strong that summaries effectively deliver the basic thrust
of the research they introduce.

Estimate Intelligence
This is perhaps the most traditional of intelligence products. In fact, when
journalists and academics write about "intelligence" they often mean "esti-
mates." Over the years estimates have become so stylized that old-timers
would find them the least changed of all product categories. In estimates, the
issue relates far more to content than to the package itself. Most analysis
involves some sort of estimate process, but the production of National
Estimates — the agreed judgments of the intelligence community acting in
concert — involves far more than merely trying to guess what the future
holds.
Producing estimates involves the critical application of each process that
I have described in the production matrix (Figure 3). Thus, intelligence
managers must put high priority on learning about consumers' needs, select-
ing the right persons to perform the analysis, making sure the terms of refer-
ence are correct, and packaging the end results in such a way that the
material can be readily absorbed. A variety of styles has been tried in
estimates, but some seem to work better than others. The scenario estimate
tries to outline the most likely paths future events may take and gives the
readers clues to enable them to tell when a particular path is being followed.
The expository estimate provides a detailed discussion of a future situation as
it can be devised from present data. The research estimate combines basic
intelligence with a forecast based on the information.
In the end, managers must recognize that the special power ascribed to
estimates by intelligence proponents is not necessarily recognized by policy-
makers. Unless policy officials are educated about the estimate process itself,
they may consider these products as merely another bit of the flow of intelli-
gence reaching their desks, and not the most interesting bit at that.

INTERNATIONAL JOURNAL OF
 MANAGING ANALYSIS STRATEGIES FOR PLAYING THE END GAME 337

Dissemination
Writers about earlier notions of intelligence — especially those who believe
in the "cycle" theory of intelligence — do not emphasize the importance of
this element in the production matrix. Yet, if policy consumers do not
actually receive the intelligence materials that are produced, the entire
intelligence system is wasting its time and the taxpayers' money. Intelligence
managers now recognize that the distribution of finished intelligence must
not be merely routine or mechanical. Managers now spend more time
ensuring that intelligence products actually reach the consumers for whom
they are intended.
Intelligence is still delivered in very traditional ways; for the most part,
trucks deliver packages to mail rooms, clerks put individual items in mail
Downloaded by [ECU Libraries] at 11:23 25 April 2015

boxes, and secretaries deliver the mail. A more effective way to distribute
intelligence products involves personal delivery by analysts, at least to key
consumers. While making their deliveries, analysts can provide additional
briefings or solicit feedback, thus garnering significant payback for their
delivery effort. Of course, the intelligence system, like private industry, is
beginning to use more sophisticated methods for distributing its product.
While that may be more efficient in terms of time, it may also depersonalize
or sever the links betwen producers and consumers. Should that happen,
both sides would suffer.
No matter how the material is delivered, managers must be concerned
with the distribution list. Intelligence producers tend to believe that giving
most materials — unless they are particularly sensitive — to as many readers
as possible is cost-effective. In my experience, this may provide material to
some people who either have no use for it, or do not understand why they
have received it. A flow of unwanted products can drown those specific
items consumers really ought to notice. Distribution of intelligence might be
more effective if it went to fewer customers. Managers should therefore be
required to pay attention to dissemination and tune their output accord-
ingly.

EVALUATION
The systematic effort to evaluate the intelligence product is a relatively
recent development in intelligence management. Evaluation is not part of the
traditional "intelligence cycle," and analysts have long complained about an
inability to tell "how they are doing." Evaluation involves more than feed-
back, although feedback is clearly important in any effort to evaluate pro-
duction. Intelligence suffers from the fact that there is usually more informa-
tion about a subject or event available during the evaluation period than
during the analytic period. Thus, judging yesterday's analysis based on

INTELLIGENCE AND COUNTERINTELLIGENCE

 338 ARTHUR S. HULNICK

today's knowledge provides a somewhat skewed view. Instead, a system

must be developed that will provide "lessons learned" rather than a "grade."
Another tradition in intelligence is the tendency to focus on questions of
intelligence "failure." Considerable evidence indicates that much could be
learned from evaluating success despite the lack of emphasis in the litera-
ture. 8 In any event, a systematic evaluation effort ought to take into
account three methodologies:

• An internal evaluation by intelligence managers to determine if the

product met the terms under which it was initiated, fulfilled the require-
ments for analysis, and proved effective as an instrument of intelligence.
• An evaluation by a "disinterested" unit created for that purpose, perhaps
Downloaded by [ECU Libraries] at 11:23 25 April 2015

a private group - scholars, or researchers outside the intelligence system —

who could provide an independent view.
• An evaluation by end-users, either policy consumers, or other users within
the intelligence system.

Evaluation of a single product may not be as useful as an evaluation of a

stream of production. Because the analytic and production processes are
autonomous in many ways and because analysis and production are continu-
ous processes, judging a stream of production on a subject, a region, or an
issue may provide more meaningful lessons. The stream of production may
also give a better picture of how well a unit handled a problem, added to
consumers' knowledge, or factored in new material as it became available.
Another scheme calls for the evaluation of the analytic effort before pro-
duction. While this approach might prevent poor analysis from reaching con-
sumers, it seems to focus on only one aspect of the process. Evaluation
ought to take into account not only the quality of the analysis, but the
nature of the product, as well as the effectiveness of delivery.
A thorny question for evaluators is how to make the lessons learned avail-
able to intelligence analysts and producers. While giving them feedback
would seem essential, evaluators might be nervous about serving in such units.
This is most likely to occur if, after providing negative information about a
stream of production, they had to return to the producing office for a future
assignment. On the other hand, making analysts into professional evaluators
may limit them in terms of career and their potential contribution to the
overall mission.
Probably no perfect way exists to evaluate intelligence production. None-
theless, experience shows that using the methodologies herein described
gives managers and analysts alike a more sensible view of the utility of their
work, and enables them to correct problems before serious mistakes are
made.

INTERNATIONAL JOURNAL OF
 MANAGING ANALYSIS STRATEGIES FOR PLAYING THE END GAME 339

MANAGING INTELLIGENCE ANALYSIS: THEORY VS. REALITY

Relating theory to reality is usually a problem for the theorist who has not
been involved in the actual workings of the process about which he or she is
theorizing. Complaints about the obtuseness or lack of economic theory, for
example, tends to put off the casual reader who is looking for explanation
about why the economic world behaves the way it does. Understanding why
the intelligence world behaves the way it does is even more difficult, because
interested scholars on the outside cannot — without submitting themselves
to heavy background checks and other security procedures — be let into the
world they wish to study. (Note Stafford Thomas's article in vol. 2, no. 2 of
IJIC.) Nevertheless, sufficient materials are now in the public domain to give
perceptive scholars insights into the actual workings of the intelligence pro-
Downloaded by [ECU Libraries] at 11:23 25 April 2015

duction process, even though some of this material may in itself display its
own brand of bias and politicization. Among these materials are works by
academics and former intelligence professionals, as well as the growing
series of evaluations that have appeared by a by-product of the Congressional
oversight process.
Perhaps the most cogent review of issues related to the control of the pro-
duction process comes from scholar Walter Laqueur whose A World of
Secrets discusses at some length recent examples of successes and failures in
intelligence analysis.9 Laqueur is somewhat critical of intelligence failures,
events about which he could easily obtain data. But for an outsider to learn
about successes is much more difficult since the successes rarely receive
publicity or are not documented in public. Nonetheless, Laqueur's discussion
of the issues is well worth reading.
Laqueur lays heavy emphasis on the competence and integrity of
analysts,10 as well as on the need to recruit and train them properly. 11
Further, he undertands quite well the kinds of political pressures that under-
lie the producer-consumer relationshp and the effect this can have on
analysis.12 The reader who seeks to obtain a balanced picture of the major
issues involved in the production of finished intelligence will be well served
by this book, even though it tends to paint a somewhat gloomy picture of
the capability of the analytic system.
An even more critical study is Allan Goodman's article, "Reforming U.S.
Intelligence." Goodman, a former CIA staff officer and now Associate Dean
of Georgetown University's School of Foreign Service, takes a swipe at the
analytic process13 by striking at several critical issues in the matter of intelli-
gence production.
Goodman claims that the intelligence system designed to support policy
has certain "major defects," and that the system itself has been "eroded by
abuses" stretching over many years.14 Goodman complains about a number

INTELLIGENCE AND COUNTERINTELLIGENCE

 340 ARTHUR S. HULNICK

of problems. Those involving the production process are particularly worthy

of our examination. Goodman believes that:

• Analytic elements in the U.S. Intelligence System have allowed themselves

to be intimidated and corrupted by policymakers and have thus lost their
ability to create independent analysis.
• Excessive competition among production elements in various agencies has
led to shoddy work, incomplete analysis, and a breakdown in the co-
ordination process.
• Pressure from policymakers on analysts has been exacerbated by intelli-
gence managers who have pressed for the "right answers" to satisfy senior
policy officials.
Downloaded by [ECU Libraries] at 11:23 25 April 2015

Is Goodman correct? He cites The 1987 Tower Commission Report and

several Congressional reports to support his contentions, but clearly his
views are colored by his experience, and by the events of this past year. In
fact, Goodman has long complained about inadequacies in production
management.15
The question of how closely intelligence officers should work with policy-
makers is much debated within the intelligence community. 16 Ready
answers on how the mythical line should be drawn do not exist. But over the
past decade, managers in two widely divergent administrations seem to have
agreed that a close working relationship is the best way to promote relevancy
and timeliness in intelligence analysis. Intelligence managers always worry
about serving the policy systems without becoming sycophants or hand-
maidens of policymakers. With some relatively rare exceptions — and these
are the ones that have been most publicized — their record is quite good:

• Intelligence analysts had no reluctance in telling the Carter administration

that prior analysis on the North Korean Order of Battle was incorrect,
thus throwing into question the president's plan to reduce U.S. forces on
the peninsula.
• Analysts were not reluctant to explain the pitfalls in President Reagan's
plan to thwart Soviet construction of a natural gas pipeline from Siberia to
Western Europe.
• Analysts have not been reticent about explaining the real strengths and
weaknesses of the insurgents in several Central American countries — both
those the administration favors and those it opposes.

Competition among production elements is one sure way to avoid overlook-

ing new or different ideas about a problem, as long as each competitor is
given the full range of raw data with which to work. Electronic delivery

INTERNATIONAL JOURNAL OF
 MANAGING ANALYSIS STRATEGIES FOR PLAYING THE END GAME 341

systems and the great range of informal ties among offices in different
agencies have now made it relatively rare for one office to have data that no
one else has.
Pressure from intelligence managers on their own "troops" to come up
with the "right" or acceptable answers depends on the nature of the man-
agers themselves. Other than the Director and Deputy Director in CIA, no
political appointees exist in the U.S. intelligence system. Senior managers
may indeed appoint other managers who hold similar views to their own, but
the nature of the review and coordination process makes it difficult to push
a particular line or analysis without generating alternative points of view. In
this area, professionalism and integrity are particularly important. Many
analysts consider it their highest duty to "fight" management when they
Downloaded by [ECU Libraries] at 11:23 25 April 2015

think their analytic honesty has been jeopardized, and to give "bad news,"
if necessary, to policymakers at whatever the cost.
Professor Goodman believes that the answer to the problems he outlines —
primarily on the operations and not the analytic side — lies in the develop-
ment of a legislative charter for the U.S. intelligence system.17 As he would
be quick to admit, previous efforts toward developing such a charter came to
nought because they were so- complex.18 But, can one legislate integrity,
honesty and professionalism? The strength of U.S. intelligence production
mechanisms lies in the quality of its personnel, not in the bureaucracy in
which they work.
The student of the intelligence production process has one other signifi-
cant source to research in trying to learn more about the real world. Both
the Senate Select Committee on Intelligence (SSCI) and the House Perma-
nent Select Committee on Intelligence (HPSCI) have issued, at the end of
each session of the Congress, a report on their activities. Each committee has
a sub-committee to evaluate the intelligence product, and some information
can be gleaned from a careful study of these reports. 19 Another instructive
congressional document comes from a series of hearings before the House
Committee on Foreign Affairs in 1980. 20 Many of the issues discussed then
are still valid, and the concerns remain. Yet, much has been done to
improve the intelligence product since those hearings took place.
Today, the products of the Central Intelligence Agency — and other pro-
duction elements of the intelligence community — reflect continued growth
in the expertise of the analysts, a broader and deeper range of subject
materials, and an increasingly sophisticated data base from which to work.
Coupled with advances in presentational methodologies, the intelligence
product is more useful, timely and relevant today than ever before. Quite
likely, a survey of consumers would reveal their general satisfaction with
much of the product they see. And perhaps equally likely to continue are
consumer complaints — especially at the most senior level — about intelli-

INTELLIGENCE AND COUNTERINTELLIGENCE

 342 ARTHUR S. HULNICK

gence reports that are unpleasant, unsettling, or unnerving. When policy-

makers stop complaining about unpleasant news from intelligence, then
intelligence managers should begin to worry about the process they control.
Perhaps their intelligence is merely reflecting the "prevailing wisdom" rather
than the way things really are or are likely to be.

REFERENCES
1
Cline, Ray S., 1976, Secrets, Spies and Scholars, Acropolis Books, Washington, D.C.
2
Troy, Thomas F., 1981, Donovan and the CIA, CIA Center for the Study of Intelli-
gence, Washington, D.C.
3
Cline, op. cit. pp. 119-120.
Downloaded by [ECU Libraries] at 11:23 25 April 2015

4
Hulnick, Arthur S., "The Intelligence Producer-Policy Consumer Linkage: A Theo-
retical Approach," in Intelligence and National Security, Vol. 1, No. 2, May 1986,
pp. 212-233.
5
Ibid. pp. 217-219.
6
See Richard K. Betts, "Analysis, War and Decision: Why Intelligence Failures Are
Inevitable," in World Politics, XXXI, 1 (October 1978) and David Morrison, "From
Iran to Trade to Soviet Intentions, Can Government Intelligence Officers Keep Their
Judgments Free of Politics," Government Executive, June 1987, pp. 52-56.
7
Morrison, op. cit. p. 54.
8
See Helene Boatner, "The Evaluation of Intelligence," a paper prepared for the 25th
Annual Convention of the International Studies Association, Atlanta, GA, March
1984.
9
Lacqueur, Walter, 1985, A World of Secrets, Basic Books Inc., New York.
10
Ibidp. 37.
11
Ibid p. 319.
12
Ibid p. 109.
13
Goodman, Allan, "Reforming U.S. Intelligence," in Foreign Policy, Number 67,
Summer, 1987, pp. 121-127.
14
Ibid p. 121.
15
See Allan Goodman, "Dateline Langley: Fixing the Intelligence Mess," Foreign Policy,
Number 57, (Winter 84/85), pp. 160-179.
16
Hulnick, Arthur S., and Deborah Brammer, 1980, "The Impact of Intelligence on the
Policy Review and Decision Process, Part II," monograph, Center for the Study of
Intelligence, CIA, Washington.
17
Goodman, "Reforming U.S. Intelligence," pp. 127-128.
18
Turner, Admiral Stansfield, 1985, Secrecy and Democracy, Houghton Mifflin Co.,
Boston, pp. 277-278.

INTERNATIONAL JOURNAL OF
 MANAGING ANALYSIS STRATEGIES FOR PLAYING THE END GAME 343

19
These reports have been issued periodically since 1977. The most recent are Report by
the Permanent Select Committee on Intelligence, HR 99-1033, 1987 and Report of the
Select Committee on Intelligence, United States Senate, SR 98-665, 1985.
20
See "The Role of Intelligence in the Foreign Policy Process," Hearings before the Sub-
committee on International Security and Scientific Affairs of the Committee on
Foreign Affairs, House of Representatives, 96th Congress, Jan-Feb. 1980.
Downloaded by [ECU Libraries] at 11:23 25 April 2015

INTELLIGENCE AND COUNTERINTELLIGENCE