Conducting and Coding Elite

Interviews
Introduction phenomena in the population of top adminis-
trators, both political appointees and high-
In real estate the maxim for picking a piece
level civil servants, and among high-level
of property is “location, location, location.” In
elected officials as well. This meant that we
elite interviewing, as in social science gener-
had to draw representative samples of mem-
ally, the maxim for the best way to design and
bers of these elites and use an interviewing
conduct a study is “purpose, purpose, pur-
technique that would enable us to gauge
pose.” It’s elementary that the primary ques-
subtle aspects of elite views of the world.
tion one must ask before designing a study is,
Drawing a sample of members of Congress
“What do I want to learn?” Appropriate meth-
was quite straightforward. Lists of members
ods flow from the answer. Interviewing is of-
are easily accessible and drawing them at
ten important if one needs to know what a set
random (stratified into two broad age groups
of people think, or how they interpret an event
in our case) was a simple process. Sampling
or series of events, or what they have done or
high-level administrators was quite another
are planning to do. (Interviews are not always
matter. We wanted to study officials who
necessary. Written records, for example, may
worked for federal agencies primarily con-
be more than adequate.) In a case study, re-
cerned with domestic policy (a requirement
spondents are selected on the basis of what
for a comparative aspect of the study), who
they might know to help the investigator fill in
were at a level where they might have a say
pieces of a puzzle or confirm the proper align-
in policymaking, and who, for convenience’
ment of pieces already in place. If one aims to
sake, worked in the general vicinity of
make inferences about a larger population,
Washington, DC. Further, we wanted to make
then one must draw a systematic sample. For
sure that we covered both political appointees
some kinds of information, highly structured
and career civil servants. To accomplish these
interviews using mainly or exclusively close-
goals, we had to compile lists of top adminis-
ended questions may
trators in each agency, determine who held
be an excellent way
top career positions in each hierarchy (our
by to proceed. If one
criterion was that civil servants had to hold
needs to probe for
Joel D. Aberbach, information and to
the top career positions in their administrative
units and report to a political appointee), and
UCLA give respondents
then sample in such a way that we had repre-
maximum flexibility
Bert A. Rockman, in structuring their
sentative groups of career and noncareer exec-
utives. We eventually drew people randomly
Ohio State University responses, then open-
from cabinet departments, regulatory agencies,
ended questions are
executive agencies, and independent agencies
the way to go.
in proportion to the number of executives in
In short, elite
each sampling classification within each
studies will vary a lot depending on what one
agency.
wants to learn, and elite interviewing must be
The good news is that bureaucratic elites
tailored to the purposes of the study. Our focus
are little studied by political scientists, so
here will be on the types of studies we have
response rates were very high (over 90% for
conducted as reported in Bureaucrats and
career civil servants). They were lower for
Politicians in Western Democracies (coauthored
members of Congress—in the high seventies
with Robert D. Putnam, 1981) and In the Web
in the first round of our study (1970–71) and
of Politics (2000)—studies of elite attitudes,
lower when we tried a second round in
values, and beliefs—but from time to time we
1986–87, so low in fact that we did not feel
will make reference to other types of studies as
it appropriate to use the second-round con-
well.
gressional interviews for anything but illustra-
tion. This points to an important problem
Designing the Study facing those wishing to interview elites. One
must get access, and it can be quite difficult
Our goals were to examine the political to secure interviews with busy officials who
thinking of American administrators and (in are widely sought after. It helps to have the
the first round of our study) members of imprimatur of a major and respected research
Congress. We were interested in their political house like the Brookings Institution, and it is
attitudes, values, and beliefs, not in particular important to be politely persistent. One should
events or individuals. A major aim was to not be too put off when told that your poten-
examine important parameters that guide elite’s tial respondent is too busy to see you when
definitions of problems and these responses to you call (after writing a letter) and call back
them. We wanted to generalize about these in an attempt to arrange the interview. One

PSOnline www.apsanet.org 673

should write a letter laying out the general purpose of the patterns and perceptions, but we did recognize the cost—the
study and be ready to repeat your “spiel” over the phone to kinds of data we collected made it more difficult to produce
the appointments secretary. Mention prestigious organizational an analytically elegant end product, at least if one uses statisti-
sponsors if you have them and mention some past experience cal elegance as the major criterion in evaluating analytical
in studying the area of interest if you have it. It sometimes elegance.
helps to mention what you’ve written, but do not expect re- A second consideration leading us to use an open-ended
spondents or those who schedule them to be impressed that approach was our desire to maximize response validity.
you have published in APSR Open-ended questions provide a greater oppor-
or its equivalents. They are tunity for respondents to organize their answers
more attuned to other types of within their own frameworks. This increases the
journals (like National Elites especially—but validity of the responses and is best for the
Journal) or to the press.
Getting in the door is im- other highly educated kind of exploratory and in-depth work we were
doing, but it makes coding and then analysis
portant, but what you do next
is even more important. We’ll
people as well—do not more difficult.
The third major consideration is the receptiv-
touch only briefly on the sug-
gestion that you refrain from
like being put in the ity of respondents. Elites especially—but other
highly educated people as well—do not like
spilling the coffee that may straightjacket of close- being put in the straightjacket of close-ended
be offered to you or that you questions. They prefer to articulate their views,
look reasonably presentable to ended questions. explaining why they think what they think.
the rather conservative Close-ended questions (and we did use some)
dressers in Washington, and often elicited questions in return about why we
get to the heart of the matter—what you ask the respondents used the response categories we used or why we framed the
and how you ask them. What you ask is, of course, a function questions the way we did. Parenthetically, in later rounds of
of what you want to know, but so also is how you ask the our longitudinal study, we used close-ended versions of some
questions. As noted, we wanted to examine the political think- of our earlier open-ended questions, but by then we had the
ing of American political and bureaucratic elites. We wanted benefit of great experience in ascertaining both the mind-sets
to know about their political attitudes, values, and beliefs. We of our respondents and the range of responses they would find
were not trying to predict discrete behavior, for example, their tolerable.
choices of particular policies; rather, we were interested in ex- We should stress again that there are costs in using open-
amining the parameters that guided their definition of problems ended questions. First, there are substantial costs in time spent
and their responses to them. in doing the interviews themselves, in transcribing them or
To accomplish our goals, we decided on an approach using otherwise preparing them for coding, and in the coding
mainly open-ended questions that allowed the respondents to process itself (see below). Second, there are the related costs
engage in wide-ranging discussions. One of our main aims in money. The process is slow and the costs mount in direct
was to get at the contextual nuance of response and to probe relation to the time spent. Third, as mentioned above, there
beneath the surface of a response to the reasoning and prem- are costs in analytic rigor, certainly in terms of limits on what
ises that underlie it. Consequently, we decided on a semi- one can do in data analysis. But, going back to the maxim on
structured interview in which the open-ended questions we purpose, answering the research questions one starts with in
mainly relied on gave the respondents latitude to articulate the most reliable way is more valuable than an analytically
fully their responses. This requires great attention from the in- rigorous treatment of less reliable and informative data.
terviewer since such an interview has a more conversational
quality to it than the typical highly structured interview and
questions may, therefore, be more easily broached in a manner
Conducting the Interviews
that does not follow the exact order of the original interview
instrument. There is an obvious cost here in terms of textbook We noted earlier some of the practical considerations in get-
advice on interviews—respondents may not necessarily have ting an interview. They include such things as writing a brief
been asked questions in the same order—but in our experience letter to respondents on the most prestigious, non-inflammatory
the advantages of conversational flow and depth of response letterhead you have access to, stating your purpose in a few
outweigh the disadvantages of inconsistent ordering. That sug- well-chosen sentences (no need to be too precise or certainly
gests a key principle of real-world research—sometimes one overly detailed); having a good “spiel” prepared for the ap-
does something that is not the ideal (in this case, vary the or- pointments secretary and later for the respondent prior to the
der of questions) because the less than ideal approach is better interview; and fending off questions about your hypotheses
than the alternative (in this case, a clumsy flow of conversa- until after the interview is over. That prevents contamination
tion that will inhibit in-depth ruminations on the issues of of the respondents and also puts this part of the conversation
interest). on the respondent’s “time” and not the time reserved for the
There are three major considerations in deciding on a interview. Obvious advice includes the need to be persistent
mainly open-ended approach rather than one using more close- and to insist firmly, but politely (and with a convincing expla-
ended questions. One is the degree of prior research on the nation) that no one but the person sampled, i.e., the principal,
subject of concern. The more that is known, the easier it is to will do for the interview.
define the questions and the response options with clarity, that It can be a major undertaking in time and effort to secure
is, to use close-ended questions. Our study explored a series the interview, but success there is only the beginning. We did
of rather abstract and complex issues in a relatively uncharted most of our interviews in the respondents’ offices, but you
area at the time, the styles of thinking as well as the actual should be prepared to do them where you can. Our most
views of American political and bureaucratic elites. Emphasiz- harrowing experience was interviewing an administrator as he
ing close-ended questions and tight structuring would not have drove to an appointment. He was very animated (and in a
served our major purpose, the exploration of elite value hurry) and nearly got himself and us killed as he weaved

674 PS December 2002

through Washington traffic late in the afternoon while present- actually a much more dynamic institution than stereotypes
ing his views in a passionate and often amusing style. might lead one to believe. Aside from reorganizations and the
We tape-recorded the interviews in order to facilitate use of creation of new administrative units, which were easily dealt
a conversational style and to minimize information loss. Few with when we constructed each successive sampling frame,
respondents refused to be taped, and almost all quickly lost the Civil Service Reform Act of 1978 created the Senior
any inhibitions the recorder might have induced. Starting with Executive Service (SES) to replace the system of Supergrades
innocuous questions about the person’s background facilitates that existed prior to the act. SES created a rank-in-the-person
this since people find talking about themselves about as fasci- system in place of a rank-in-the-position system and made us
nating as any subject they know. Our judgment (and the judg- reexamine our earlier criterion of interviewing the highest
ment of our coders) is that the interviewees were frank in civil servant in each hierarchy. We eventually decided to con-
their answers, especially because our questions focused on tinue sampling the highest civil servant in each hierarchy for
general views and not information that might jeopardize the purposes of continuity, but added a sample of other SES
respondents’ personal interests. executives. In the end, this proved substantively beneficial, as
readers of In the Web of Politics will see.
Second, by round two of the study we encountered a few
Coding Open-Ended Interviews executives who knew something of our earlier work published
Coding procedures assume paramount importance when, as on the basis of round one. We interviewed these people, but
in our studies, one employs open-ended interviewing tech- there may be some unknown effects of their familiarity with
niques to elicit subtle and rich responses and then uses this the project.
information in quantitative analyses. Particularly in elite inter- Third, we had to decide whether to repeat questions in later
viewing, where responses to questions are almost always co- rounds even though we knew we could ask better ones. Fol-
herent and well formulated, respondents can productively and lowing the advice of Philip Converse, we tended to repeat
effectively answer questions in their own ways and the analyst items, choosing to keep whatever measurement error the origi-
can then build a coding system that maintains the richness of nal item introduced over the great problem of comparing re-
individual responses but is sufficiently structured that the inter- sults from different questions.
views can be analyzed using quantitative techniques. The Fourth, as already mentioned, the costs of open-ended
wealth of material contained in the responses, in fact, allows a longitudinal studies are considerable. We dealt with this by
varied set of codes, some recording manifest responses to the shortening the instrument in subsequent rounds—retaining
questions asked and some probing deeper into the meaning of the questions we knew to be key as our understanding deep-
the responses. ened over time. In addition, we supplemented with more
We developed three basic types of codes to achieve the pur- close-ended questions in later years, and now have a basis
poses of our study (Aberbach, Chesney, and Rockman 1975, for comparing open and closed questions in certain areas.
14–16). Manifest coding items involved direct responses to There were also advantages beyond those we have already
particular questions (for example, whether differences between mentioned to doing a longitudinal, heavily open-ended study.
the parties are great, moderate or few). Latent coding items First, once we actually developed our codes in the first
were those where the characteristics of the response coded round of the study, the costs of coding dropped substantially in
were not explicitly called for by the questions themselves (for subsequent rounds. We did some refining of the codes, of
example, we coded variables dealing with positive and nega- course, but the costs here were minor compared to our original
tive references towards the role of conflict from questions investment.
about the nature of conflict in American society and the de- Second, as noted above, in each succeeding round, as we
gree to which it can be reconciled). Global coding items were developed a fuller understanding of what and how elites think,
those where coders formed judgments from the interview tran- we were able to use more close-ended questions to supplement
scripts about general traits and styles (for example, coding the open questions.
whether respondents employed a coherent political framework Third, our interviewing technique means that we have a raw
in responding to political questions). product that should be of great use to historians. We inter-
In the first round of the study we had two sets of coders viewed during a turbulent period in American administrative
independently code each interview and calculated inter-coder history (particularly during the Nixon and Reagan administra-
reliability coefficients for the various variables. Not surpris- tions) and we have transcripts of in-depth conversations with
ingly, on average, the manifest items were the most reliable, people who are historically important. Because of confidential-
followed by the latent items, and then the global items. We in- ity promises, these interviews will not be available until re-
creased reliability further (we hope) by having a study director spondents are deceased, but eventually the interviews should
reconcile any disagreements among the coders in conferences prove valuable in understanding the era when our respondents
with the coders. Our experience with coding taught us that wielded power.
simultaneous coding by the two coders with immediate recon-
ciliation yielded much more reliable coding than serial coding
where large numbers of interviews were coded prior to Conclusions
reconciliation meetings.
To reiterate the key point, studies must be designed with
purpose as the key criterion. Elite studies are no exception.
Some Problems and Advantages in We conducted our longitudinal study the way we did because
Doing a Longitudinal Elite Study of our desire to probe deeply elite attitudes, values, and be-
liefs, and also because of the state of prior research in the
We encountered a series of problems in doing a longitudinal area, our desire to maximize response validity, and our sense
study, most of which impacted both the interviews themselves that elites would be most receptive to the type of interview
and the coding. we conducted and would be well positioned to handle the
First, elite systems do not necessarily remain stable over types of questions we asked. While we would use more close-
time. This is particularly likely in the bureaucracy, which is ended questions in future research because of what we learned

PSOnline www.apsanet.org 675

(and we used more as time went on), the basic approach of a us to use clues from the most insightful respondents to sug-
semi-structured and largely open-ended interview still seems gest hypotheses for our analysis.
best to us. We learned a great deal from our subjects—and We close with a general observation about elite interviewing
about our subject—through these conversations. Using a sys- studies; they take a lot of persistence, time, and whatever
tematic coding procedure not only allowed us to employ quan- passes these days for shoe leather, but they are immense fun.
titative techniques in our later analyses, but also kept us from You’ll meet some of the most interesting people in the country
allowing the colorful interviewee or especially enjoyable story and learn a huge amount about political life and the workings
to dominate our view of the overall phenomena we were of political institutions. If you like both politics and political
studying. At the same time, the interviewing technique helped science, it’s one terrific way to spend your time.

References
Aberbach, Joel D., Robert D. Putnam, and Bert A. Rockman. 1981. Aberbach, Joel D., James D. Chesney, and Bert A. Rockman. 1975.
Bureaucrats and Politicians in Western Democracies. Cambridge: “Exploring Elite Political Attitudes: Some Methodological Lessons.”
Harvard University Press. Political Methodology. 2:1–27.
Aberbach, Joel D., and Bert A. Rockman. 2000. In the Web of Politics:
Three Decades of the U.S. Federal Executive. Washington, DC: The
Brookings Press.

676 PS December 2002